Managing Complexity in Today's World

Are business problems more complex today? That's the assumption I'm going to operate on - and I'm sure you'll find no shortage of sources on the web to back that up (empirical evidence aside). So let's assume that rhetoric for now in good faith and an appeal to common sense. Padding aside, that complexity flows through to the systems we create - the software included. In order to manage that complexity, we've become Agile.

I hear ya: "Not another article about Agile...". I know, I know.

But let me at least tell the story about what inspired this and then move on to the point. An IT professional (non-technical) and I started chatting and the conversation led to tools for automating business processes. I've developed on platforms and developed custom solutions for business process automation. Evaluating solutions can be a complex process in and of itself. The analysis process is fraught with uncertainty, broad (inaccurate) estimation, bias, and difficult to predict outcomes. Marketing (being what it is) definitely can skew the analysis. Heck, that's part of the duty of marketing! With proper marketing, sales is nothing but finalizing the deal that was already done through the marketing. So what does that have to do with Agile?

Agile is about being able to respond to change. In software, if we are truly following Agile principles and the practices set forth by the minds that solidified the movement, then we are designing and developing software that can respond to changes in the face of complexity. Lately, that seems to be the mantra of the business world as well - business needs to be Agile so that it can respond to changes in the complex market. As technology and automation have been evolving exponentially, so evolves the need for business to evolve to respond to those - among many other - changes.

Thinking about what this means in terms of managing change in the face of complexity and cost. In the software world, cloud computing has become all-the-rage for just those reasons. In the cloud, you pay only for what you use; changing and configuring resources is relatively easy; there are no data centers or servers for the business to manage. Good software design has been about responding to change for many many years, but now the infrastructure has caught up. It is a service rather than an internal resource.

Back to the topic of deciding on a platform...

Some platforms follow the cloud model - pay for what you use. Others - pay a big fee up front along with licensing and maintenance fees annually. Some platforms can respond to change - they are testable, debug-able, manageable. Others...not so much. Some platforms can handle some business situations easily. Usefulness is a function of cost v realization of value - as always, use the right tools for the job.

But whom should decide what the right tools are?

When you take your car to a mechanic - you may be paying for the service and it may be your car, but the mechanic knows which tools to use. Same with the surgeon, plumber, accountant, attorney, chef, construction company, or any other number of service professionals. Why should it be any different for software? Professionals know their craft. Business people know their business - and many know how to do software, woodworking, plumbing, some things legal, accounting, have dissected a frog - does that mean they should perform surgery? If you've replaced the float in the toilet, should you be replacing your own stack*? So who should decide which tools to apply in software development? Logically, it should be the professional software developer who knows their craft.

However, the situation changes when the cost exceeds the budget of the development department. Other levels of the business has to get involved in the decisions, there are politics at play, the tool may be oversold - then what? Now you've bought a tool without a use! So then you look for ways to use the tool to realize it's value...now it's the non-tool user calling the shots (since they've put their money and reputation on the line) on which tools to use to do the job. Then partitioning of tool users happens - you have experts in such platforms. Someone higher up the food chain now decides which tool to use and assigns the problem to the expert(s) of that tool. The expert dutifully applies said tool to the problem (which of course is a problem that tool was designed to handle because when you only know how to use hammer, every problem is a nail).

What's the better model? Maybe that's better left to another post, but I'll give you a hint - there's no 01001001 in the solution.



*A stack is an air vent that allows the toilet to flush properly among other things; without it the suction would be a serious problem. Perhaps it gets blocked by leaves or a birds nest and you have some issues. They used to make them out of cast iron which is some really heavy stuff that can maim you if you don't know what you're doing. They've also used galvanized steel which rusts over time. That rust can also clog the stack. Replacement should be done by a professional only (no user serviceable parts inside).