Increasingly, managers use optimization-based planning software to evaluate ‘what-if’ alternatives to mitigate or even overcome such risk. In doing so, however, we have to wonder:
- Can the choice of a specific mission-critical optimization software package itself be a long-term risk?
- What about the internal Operations Research (OR) team or consultants necessary to maintain the optimization models created?
- Can the long-term risk of relying on both software and key individuals be high enough to merit attention at the senior management level?
Yes — especially if the software impacts financials and/or if it’s embedded into mission critical processes like monthly Integrated Business Planning. Nevertheless the answer depends on multiple factors, including the class of software the package belongs to; the pool of trained resources to support the software; and the software vendor’s experience and ability to support the customer.
There's a Big Problem with the Way Optimization Software is Purchased
Let me state this clearly — I’m not questioning the accuracy of any commercial optimization-based software package, nor a company’s decision to hire an internal OR group or any outside consulting firm. Many thousands of optimization models currently help companies all over the world make important sales and operations planning, inventory planning, supply planning and many other decisions. Some industries, such as airlines, railroads and oil & gas, would not be able to satisfy demand and would be deeply unprofitable without these optimization models. Clearly, these companies are realizing value regardless of the technology or people involved.
But this is a discussion about risk. There is always an inherent long-term risk of relying on any software and the people who maintain it. This is certainly true of ERP, MRP, CRM and other enterprise software. I would argue that optimization software packages must be held to an even higher standard. Why would a company adopt such an important decision making tool and then assume there is little to no risk in the future? How and why do most companies make the initial purchase decision in the first place?
In my experience, most project sponsors are willing, albeit somewhat reluctantly, to allow internal OR professionals or outside consultants to choose (within a budget) the optimization platform and to develop the planning or scheduling models as they see fit. This important decision is often due to nothing else other than the OR professional had used a particular software package in a university class; knew how to use the Excel solver; or learned to code in Visual Basic, C++ or another 3rd-generation language and was familiar with the CPLEX or Gurobi callable library. At the time, and maybe in the years since, that decision made some sense. Now, maybe not so much?
Historically, optimization software tools have been based on highly specialized algebraic mathematical modeling languages that require writing custom code (e.g., equations, control programs, etc.) The languages, with acronyms like AIMMS, AMPL, GAMS, LINGO, MPL, started during Ronald Reagan’s first term as US President. Then, software tools were developed around these languages to allow users to code and debug models. These packages neared maturity about by the end of Bill Clinton’s second term—eons ago in the world of software.
This is actually a fairly simple equation. Imagine a model with dozens of such equations, maybe hundreds. Most non-OR people in organizations simply don’t have the education, experience or the patience to learn model logic at this level of detail, and instead must rely on solution output to be accurate and actionable. Consequently, full knowledge of each model’s logic remains concentrated within a very small team of people; sometimes only a single person! Let me repeat this last statement:
A mission-critical optimization model — often responsible for affecting P&L top- and bottom-line values — frequently has just one person to a small team of people who have complete understanding how it works!
I have proof of this because I used to be one of these people. It was great — people were completely dependent on me. From a business perspective, though, being dependent on someone or something is quite the risk.
‘Black box’ Optimization Models Equals Job Security
Full disclosure — from July 1991 to August 2000, I earned a living by creating custom planning software for the forest products industry. I worked for a small but full service software development and consulting firm in Portland, Oregon. By working closely with each customer—companies like International Paper, Georgia-Pacific, Weyerhaeuser, Canfor and others—I was involved as a pre-sales consultant and in model design, development, testing, deployment, training and, especially, support.
I sometimes refer to my experience as “cradle-to-grave” modeling. Most supply chain optimization models that I created, I also supported until either the client stopped using it or I no longer worked for my company. It didn’t matter if I was full-time on new projects and a customer using a model built many years prior needed assistance, it was my support issue to deal with. Occasionally, a colleague could step in to temporarily provide assistance, but normally it was difficult to completely transition any model that I built to anyone else, and vice versa.
In my case, the LP and MIP models were built using optimization software based on a FORTRAN-like language that demanded specific modeling syntax and structure. Each model required hundreds and sometimes thousands of lines of custom code. It did not matter much that my colleagues and I were trained to use the same software; had similar knowledge of the industry; and were all smart and intelligent people. And, while we could re-use parts of one model’s code for a new model — i.e., softwood plywood mills tend to be similar — no two model’s code was ever the same.
The only person who really knew each model completely was the author (me); everyone else was limited to a support role, and only when absolutely necessary. When long-time colleagues left the company, the remaining employees (sometimes me) were burdened with code they might not ever truly understand.
Time for a Reality Check
If you’ve never built a full-scale industrial optimization model using an old-style, 3rd or 4th-generation language, you might be surprised to learn my former situation was much closer to normal than you might realize. It’s true for most kinds of prescriptive analytics modeling; not just LP or mixed integer programming but simulation too. It’s true for most mission-critical and highly custom algorithms. I’m sure that NASA has many such algorithms that only a select few highly educated people fully understand. Heck, it’s true for complicated Excel workbooks with many sheets stuffed full of cross-tab references and calculations.
You have probably experienced this fact already, so I’m just reinforcing what you already knew but maybe never fully admitted — no one can ever fully support an existing model like the original author or team. From a risk standpoint, this means that a specific person or team’s continued employment or availability (if a consultant) cannot be easily dismissed.
Here's a Quick Risk Assessment Exercise for You
Having been an OR practitioner now for over 24 years and completed over 50 major projects, I am somewhat of an expert on the pitfalls of the custom, algebraic modeling language optimization approach. In today’s business world where all forms of risk are assessed, I simply do not believe that OR practitioners or their chosen tools should get a free pass.
If your organization is currently dependent on an optimization model for a critical business planning process, try answering these questions:
- Have you seen the model code, preferably inside its development environment?
- If the answer to #1 is yes, did it make sense?
- More importantly if the model impacts financial plans, has it been verified by your auditors?
- Are you certain the model, as constructed, can and will evolve as your organization changes?
- Are you certain the model builder(s) will still be working to support the model in 5 years?
Would your CFO be comfortable with these answers, especially if the answer to #3, #4 or #5 is no?
Remember, this is a risk exercise. No software can ever be risk free, but mission-critical software should never be allowed to cause a significant disruption and irreparable harm to operational and financial goals. Do not accept that the optimization model must remain a “black box” simply because you or other senior management do not understand it, for whatever reason. And, do not just hope the model builder(s) will continue to enhance and support the model for many years to come. No one will work forever, but well-constructed and understood optimization models can be used successfully for many years.
If any answer was ‘no’, I strongly recommend asking the CFO, Chief Risk Officer, or external auditors to do the following:
- Evaluate the current situation to better understand the tools used.
- Evaluate how the model is constructed and supported (to the best extent possible).
- Make recommendations to minimize long-term risk exposure to your organization.
While you might be surprised by what you find from your audit, if it helps reduce long-term risk, it's worth the trouble.
If your discoveries cause you to reconsider your current software, our optimziation software comparison infographic (below) can help guide you in your search.
