Tuesday, September 6, 2016

Coordinator Pattern for Distributed Enterprise Applications

When building software for the enterprise, particularly where data access to one or more source will be shared across multiple application domains, and where distribution and reuse through distribution is a virtue, it would be helpful to use a Coordinator as the authority for the unit of work. A Coordinator is a service which accepts a single incoming command and issues one or many commands to resources (data access services). The resources are separated into read and write resources for each source. The read resources may be used by the presentation layer or the Coordination Layer, but the write layer (which specializes in executing commands) may only be used by the Coordination Layer.
A Coordinator is similar to a Unit of Work pattern. The key difference is that it must be hosted as a service and use data access as services.
The Coordinator owns the transaction. It must begin and either rollback or commit the transaction. The transaction may be distributed across many data sources (throughout the writer resources). The writer resources must be designed in a way that handles concurrency. For example, if using an ORM you may need to load then update or insert any specific data entities involved in the transaction. These writer resources should handle the entire transaction for the data source which it wraps (typically one database). The Coordinator should send commands to any resources and raise any events necessary upon executing the commands.

The Coordinator should listen for events (as in event based architecture) and respond to those events by issuing those specific commands to the resources which it consumes. Those resources may be shared amongst other Coordinators. Resource commands should be synchronous so that transactions can be used appropriately. The Coordinator is an architectural pattern which can be used to implement a Unit of Work pattern in a distributed, service-based architecture. Coordinators should be consumed asynchronously using event-driven mechanisms. Coordinators should respond to events raised by other Coordinators or by users via the presentation layer.

Through the use of Coordinators in conjunction with the other patterns expressed in this article, a distributed system can be built such that it is loosely-coupled, highly-cohesive and easily-maintainable. The system will be loosely-coupled in that the communication between Coordinators occurs via events. Zero, one, or many Coordinators can respond to an event. Any Coordinator can be changed and (if hosted independently) redeployed without impacting any other Coordinator. The level of cohesiveness depends on the scope built into each Coordinator. Each Coordinator could handle one Unit Of Work. In this case each one would have a single service method exposed. This method may perhaps share a contract and differ only by location. In this case the method definition would need to describe an event and the Coordinator would need to load the Event details it needs via resources. In this case, an Event Store would be needed. Else, the payload would be a serialized packet of all of the data relevant to the event. This is a deeper topic and will require more in-depth exploration in order to weigh in completely on the interface mechanisms involved. The maintainability is affected by the number of moving pieces, but also by having a standard mechanism of control over the various state changes involved in a complex system.

Tuesday, August 23, 2016

Thinking About Dynamic Forms Using Angular pt 2

In the last post I started thinking about some of the problems we encounter with using forms for inputting data into a system. There may be other ways to input data and a well-designed system will be capable of accepting input from other sources. However, the focus of this series is on using forms - specifically via a web browser.

Since a well-designed system will accept input in a platform agnostic way, this leaves us open to design the user interface in a number of ways. For purposes of this thought experiment I'm going to assume we have the capabilities in place to accept input into the system in some standard way - perhaps through a service or some RESTful api for example. It doesn't matter so long as we can stay focused on collecting data from the user and sending that data into the system.

One problem we may have is incremental changes. Users have experienced automatic saving and it's no longer a novelty - we can and should do this in order to meet user expectations.

Another problem we may face is logging changes to data. Additionally, we may face changes to the form itself. Let us also take into consideration a form in which a user can add fields as needed.

From what I've gathered so far, we can apply two commonly used technologies to this set of problems and cone up with a solution that meets these needs. Here's what I have in mind:

Using Angular (or even your favorite SPA technology) and a data object with this interface: { template, data, metadata } we can store each form instance in a document database and load it in one go.

The template property holds several bits of info: the template text, version, source, form name, id, ... , anything else? }

The data holds just that - it's complete dynamic and goes with the template.

Metadata holds things like userId, action, date, ip, client device info, whatever else is relevant.

Any action the client takes - save, submit, delete...are saved as a new version of the document. Some document databases have this built-in. I'm thinking of one in particular which can even save files (js, html template, etc) along with the data.

In Angular 1 we can use directives to enable the forms. Dynamic directive creation should be achievable since its all js. All we really have to do is create components or directives server side and serve up the js and template. The template is either assigns to template or templateUrl of the directive/component. I've become accustomed to components so I'll prefer those with '=' (two way binding) which allows them to update objects in parent scope which are linked.

The data in the template would be interpolated using a standard controllerAs: 'vm' so that all data can be accessed in the templates via vm.property.

That leaves behaviors...not sure about these yet. I'm going to think a bit about this one...until next time it's a good place to start.

Thinking About How to Build Dynamic Forms with Angular

There are many times when we need to use forms to collect user data. Make no mistake, this is not where the app begins or ends, but only a small part of an app. There are several platforms, frameworks, etc that we have to create forms. The "classical" approach is to present a form which handles CRUD operations on a database, updating the tables directly. In some cases this may be an adequate approach, but in time it falls short of an optimal solution in many applications. There are several reasons for the shortcomings and some inelegant solutions have been used to make up for them.

Often the form requirements can change over time. In the classical style, a developer makes some changes to the form and the database schema - sometimes altering existing data in the process by dropping or changing columns. The new Form is deployed and going forward we have our new data. But what about the historical data? Perhaps an archive is used, perhaps an additional database and app version. Three versions later, a monolithic app can get messy fast when supporting multiple versions. This is inelegant solution #1.

Sometimes it is beneficial to have dynamic forms, several CMS solutions exist for such things. Not being overly familiar with the current CMS landscape, I'm certain my proposed Angular solution will be fitting of the CMS domain, however I will continue regardless since I believe we can one-up a CMS solution by enabling the user to add fields at their own discretion rather than admin only. Potential inelegant solution #2 - locking part of the solution into a specific platform.

Pre-emptive rebuttal to using Angular platform and locking into that: I propose a more robust solution with no specific template engine, Angular is a fitting engine at the moment since the dynamic nature of JavaScript lends itself to generating arbitrary forms. Plus the benefits of a document database backing will be clear once more details are presented. But enough about solutioning for now...lets continue defining the problem next post...I'm outta time ATM.

Monday, August 22, 2016

Decentralization of Business Tasks to Build in Better SOC

In the typical business organization, there are departments which specialize in specific business areas - HR, IT, Marketing, etc. To me this seems to be a benefit due to the specialized nature of each. There may be individuals in each department who have a knowledge and passion of another area and there may be some crossover, but by and large the units are siloed in their work - especially at larger institutions. Since the structure of systems have a tendency to model the structure of the organizations that create them (see Conway's Law) and many systems that tend to the needs of these units will provide some similar capabilities, it is up to the system designers at the enterprise to go against the natural tendency to design the systems along the same lines as business structure and design the system along the lines of capabilities. There are several benefits to doing so as described in this writing.
Reason 1: DRY - Do Not Repeat Yourself. This is a mantra of software development. This practice of not repeating yourself is commonly declared as a time saver as well as a defect deterrent, especially when there is a change in requirements for a feature. Consider a number of systems who have users and a user admin feature. During the course of development of this feature for each application a considerable amount of time was spent gathering requirement, doing mockups, designing, implementing, testing for this feature in EACH system. And when a defect is found, or security requirements change each system must be updated in turn. However, if DRY were applied at the enterprise level, there would be one user admin capability. That user admin would have behaviors designed to provide common functionality with respect to administering users - add/remove, assign role, change role, reset passwords, etc. The immediate benefit is that the features wouldn't have to be developed for each application. Other benefits are centralizes administration of all applications for system admins. Additionally, this enables "One level up" features to be developed where users can have access to all their applications centralized. One thing to consider with this design is the impact on individual applications due to changes to core features. Additionally, the system must be robust but flexible enough to allow extension by other systems. For example, a canned(packaged) solution may offer an API to its user administration. An idealized core user admin would be able to adapt to it so that it could also be administered centrally.

Reason 2: User Efficiency. Users are able to access actions, tasks and tools from a common location. Potential for more efficient interaction with technology.

Reason 3: Change. Centrally managed change. Changes happen in one place. Easier to find what is impacted by a change.

Tuesday, May 24, 2016


Having recently delved into a world of conversational machines in the form of chat bots, I've learned a few things about that world. There are so many platforms with so many options to choose from, but there are some common threads.

One common thread is that there are different AI technologies that can plug into a chat bot. Two of those are language interpretation and psychological profiling. Language interpretation uses a series of mappings to determine the topic(s) of a user's statement. Psychological profiling uses data gathered from social media to guess what a person's personality traits are. Combine these with some other sources including a series of waterfall questions and you can tailor a bot to give personalized responses to the user.

Imagine having a bot app on your device or computer that monitors your activities and responds to your needs! Bot apps can be programmed to launch programs and run tasks. I'm not constructing a world where the bot takes commands and executes them, but a world where the bot truly learns your behaviors and serves your needs by executing tasks when you need them!

Do you open a certain program everytime a certain person calls? Do you need to dial in to a meeting when it begins? Do you spend time on Facebook every morning? How about composing emails? I'm not suggesting chat bots should do all this for you, rather that the technologies behind chat bots can and should automate some of the most routine and mundane aspects or your life so that rather than being a slave to the machine (email, launching programs, dialing phones, scheduling appointments) the machines can be your servant and assistant.

Monday, May 2, 2016

JavaScript Fun: Method of Wrapping JQuery getJSON to Handle Errors to Avoid Duplicated Code

Here's a great way to wrap JQuery and other libs in your own wrapper so you don't have to repeat yourself, repeat yourself. It's pretty basic and simple for now.

JS Bin on jsbin.com

JavaScript Fun: how to merge two objects, no recursion yet...

I like using objects in javascript. This is a basic and easy way to merge two of them. Good example is if you take in options into something.

JS Bin on jsbin.com