Tuesday, July 14, 2015
Long Post: Thoughts on Conway's Law
After pondering Conway's Law I've come to think of it like this:
A typical organization will have departments that specialize in certain activities. There may be a marketing department, a sales department, a finance department, another for human resources, and so on. The natural design of any system (a set of processes and workflows that are systematic and repeatable whether automated or not) will be organized along the same structure where communications occurs between departments.
In a typical IT system, where the system is defined as all of the software which communicates with other software by design, the programs/sub-systems will typically be organized by the same functional areas of specialization. It takes some specialized work to organize the system in a way that is more efficient.
The human brain, for example, is organized in a way that areas have functional specializations. In the brain there are areas that work together to solve math problems, other areas work together to understand and produce speech, there are areas specializing in regulation of the body. It seems that the lower-level functioning of our brain, that which is concerned with keeping the body in a state of living, are specialized in a more specific sense, while higher-order thinking is comprised of areas composed dynamically for some general purposes.
Math problems are so varied and dynamic that it may take the stitching together of more parts of our brain to solve some than it would for others. We commit some math problems to memory - what's 2 + 2? Most of us don't even have to think about it because we've created an area that specializes in 2 + 2 long ago and it has solidified in our mind. How about divining a proof for the quadratic equation? Prove the existence of the Higgs Boson? Those require a special organization of areas of our brain to produce the result (unless we are math experts of course).
Imagine if our business organizations were aligned this way...there might be a math department which performs all math functions, an email department which is responsible for writing all emails, a bad news department...well, you get the picture. This sort of organizational construct would not serve well in an environment where people have to communicate across those lines of specialization. Human communications are relatively inefficient when compared to communications between computers or between nerve synapses.
Given that human communication is the main inefficiency between people, we organize into specializations based on higher functioning areas of business instead of specific functional needs. Our communications protocols are based on the human language and are transmitted via blog, email, memos, telephones, texts, etc. We need contextual understanding in order to decipher the messages, which have specialized meanings in specific contexts. Our communications are Klumsy, Lame, Ugly, Dumb, But Good Enough and barely at that! If not for poor communications the world wouldn't need therapists!
What is the solution to removing the weak links in the chain? Automate, of course. We made calculators, and now no one needs to learn basic math or consult a math specialist through our various means of poor communications. Instead we each have a math wizard in our pockets and under our fingertips at all times. What's more is that these math wizards can use their mathematical prowess to present and communicate information (and mis-information) to humans at near-light speed.
Our ability to communicate as effectively in the whole pales in comparison with the ability of computers to communicate with each other. What's more, is that computers have software. With software, we can teach a computer how to understand nearly any communication that it can receive by simply installing software. For humans to do this would require years of specialized training for each individual. Computers require specific training by a few individuals who produce the software. It may take years to produce, but the outcome is that each computer takes minutes to learn how to communicate in this specialized way.
Thus far, I have been setting the stage for an argument for a concerted effort toward the design of IT systems that seeks to maximize efficiency in ways that humans cannot. Given that communications between contextual boundaries is the main barrier to human efficiencies, when one seeks to improve efficiency, then the way to produce maximum gains is to address the barriers between contexts.
Since computers can specialize in low level functioning and do not suffer the same communications barriers that people do, systems comprised of computer software can be organized in a way that humans cannot. They can be organized in a way that maximizes efficiencies. Be warned, however, that there are different types of efficiencies when it comes to software; some of which apply to maintainability and development. For sake of argument, lets call these indirect efficiencies. Processing efficiencies are one direct efficiency that comes to mind in regards to software. When some work is performed, how quickly will the system arrive at the result? This depends on a number of things including the architecture (organization of the system), the hardware in the system, and the software design. Two of these can be easy to change, one cannot. Hardware is frequently upgraded and so is software. The single most difficult change in a system is the architecture - how the pieces are organized. In fact, the ability to upgrade software and sometimes hardware are directly impacted by the architecture of the system.
An architecture can be defined in terms of the coupling of its components - a system can be tightly coupled or loosely coupled. A tightly coupled system is one in which each of the components depend on one another in a way that makes it difficult to change any of the functional areas without affecting one or more others. Loose coupling is when the functional areas of a system can be changed at will without much impact to other functional components. So far, the descriptions of coupling are arbitrary. To quantify, lets consider the following: If a system has n functional components (logical chunks of work that it performs), and each component is coupled to x other components such that a change to one will affect another in a way that causes the other to change in order to provide the same functional work, then the coupling can be quantified as the sum of all couplings divided by the number of components
we would strive for 0 so that any component can be exchanged or even moved to different hardware so that the overall efficiency of the system can be improved.
Another nuance in meaning of efficiency of a system is tied to how it improves efficiency of the work being done - be that assisting humans in performing their own tasks more efficiently, or by automating the tasks. Some tasks are automatable and some require the elasticity of the human brain and its ability to compose its pieces in order to solve a problem. This is the way we should view IT systems as well, except that the barrier in being able to do this as efficiently as humans is human communications.
Most computer software requires humans to translate some thought into a computer language. This effect is called programming and requires a specialized skill that takes years to learn and master. It requires human interactions in order to understand the problem. It suffers the same inefficiencies as other human interactions between groups of specialists. It takes time. Human brains can pull their pieces together very quickly, while it takes much longer to tell a computer how to solve a new problem.
Because it does take quite a bit of effort to tell a computer how to solve a problem, it is necessary to build the basic low level components that can be pulled together to solve some specific problem in an efficient way. The problem solving components will be higher level components in the system. All components will need to communicate efficiently. Each component should be able to change in implementation or in location with minimal impact on other components. If an architecture supports these goals, it can be changed to make the pieces more efficient.