Software runs the world. As we strive to find ways to boost productivity and launch innovative solutions, there comes a time when an off the shelf product is not available to meet the need. When designing software solutions, there are elements of the process that are unpredictable. Custom solution development is like knowing there is a need for a service to provide personal transportation but not knowing if a car, bicycle, airplane, or Star Trek transporter is the best solution to deliver such service. Often in the technology industry, what we all thought was the right path turns out to be a dead end.
For the purpose of this blog post, we will assume that off the shelf products do not effectively meet the needs outlined, and a custom solution is the best path forward. In a future blog post, we will expand on a previous discussion; the challenges of selecting and using off the shelf solutions.
The most common method to deal with the unknowns of solution design is to front load all the design before any development or implementation takes place. This is commonly seen in software development as the waterfall method. While easier to manage, the waterfall method engineers out flexibility in favour of a more predictable development path. However, Agile development methodologies acknowledge that it is impossible to forecast all the variables a solution requires without experiencing application in the wild. Over the course of 20 years, more development is being led by Agile principles.
Defining the solution upfront is risky, but we often see potential clients define the solution upfront in totality. At MERAK, the first question we always ask is, “what problem are you looking to solve?” The second question we always ask is, “why?” In today’s world, our IT systems are interwoven with the organization’s strategy, business processes, IT systems, and people. While asking for and building bigger, better, faster systems might provide quick wins, the process of matching needs with a solution is far more nuanced. Innovative solutions require experimentation and iteration. True discoveries require an investment and commitment to understanding the problem and matching a solution to the organization’s purpose.
Experimentation and iteration when designing a solution requires answering the obvious questions (“known unknowns”) and discovering new questions (“unknown unknowns”). A known unknown might be how long it will take to update a platform version and perform the necessary quality assurance functions on a legacy system. Unknown unknowns are the surprises that pop up requiring a refactoring of assumptions and in extreme cases pivoting the direction of the project. If we look at the example of personal transportation, early users of the Star Trek transporter discovered that on the other side of the transporter; they are actually copies of themselves. With this knowledge they are now no longer comfortable using a transporter despite it being a far superior transportation service compared to traditional alternatives. Users experiencing an existential crisis post-transportation was an unknown unknown, not foreseeable without experimentation. While painful, discovering unknown unknowns, and dealing with them early provides massive savings long-term by minimizing the “unforeseeable” technical debt in the project.
Ok great, we have a great plan now. We also have a fixed budget, want it done by next week, and want all the bells and whistles. There are three dimensions to manage the design and delivery of a technology solution: budget, schedule, and scope. Waterfall methodologies are able to “de-risk” the variability of these dimensions at the cost of later technical debt.
At best we can only control for two dimensions at the expense of leaving one dimension open to float. For instance, if you have a fixed budget and schedule, it is realistic to expect that the scope of the project will need to be flexible. The greater flexibility given to this “floating” dimension the higher probability that unknown unknowns can be discovered up front.
Experimentation and iteration requires a change management process not only for the culture, but an understanding that a transformational vision is required to drive what is an incremental change process. An explorative design process can be scary, however, when using an appropriate management process, the explorative design allows the dynamic dimensions of solution design to be controlled. Just remember, a goal without a plan is a wish.