Friday, April 21, 2017

Beyond Kanban

In the last lesson learned, we saw that Kanban is easy. It's easy because it only describes how work flows - there's a whole depth to it with value stream mapping, prediction, planning and measurement but that's really not that complicated either as we'll see soon enough. What Kanban cannot solve is how the work is understood.

In manufacturing, the origins of the culture that spawned Kanban, the product is always the same...therefore (much like a McDonald's) there are finite, well defined steps that procude a specified outcome everytime (within some degree of acceptance). In software, there's the same sort of problem but the product is different every time...

Since the product is different every time (if it wasn't we wouldn't need to build something new) the "specified" differs every time as does the "acceptance" and the "degree". Stephen Connolly eloquently describes this issue in this blog post

As Stephen describes by way of the problem with the shields on the USS Enterprise, the understanding of the feature will need to be clarified with the upstream party. Add a button to the screen, make it do something. Also make sure shields can be activated in other ways - different interfaces. the upstream party - shield system engineers (Scotty?) needs to make sure those shields can be activated from multiple sources. Now the shield becomes downstream for the activation call. Shield engineer specifies interaction protocols maybe defines some security practices - who makes that decision? Now it might be a deeper matter of verifying who is calling to raise the it Borg Captain Picard? Real Captain Picard? What's the override protocol? Does it need to change for this? Now it gets complex...what was a button is now a security and safety concern since were opening up access to a critical safety system...

Guess what? This is what Lean Change Management is all about - managing this sort of change without invoking filibuster (or bike-shedding, if you will). Map impact of the change, radiate change info, talk to people about the change, make changes small so they succeed. But I digress...

Point is, when we go beyond the initial ask (the happy path) and we have different requirements for each change, as Stephen points out, things need to be explicit. Kanban doesn't specify how this is done in software, but other processes do - to a degree.

In TPS (Toyota Production System) all the details are worked out well before the manufacturing process starts. In Software, that's generally not a good way to work. I recommend taking a bit from Scrum and breaking down the details at the start of some Kanban there is no iteration, but there is a work week so why not use that? Start going through the feature stack each week start. Fill in details about the top features and break down the features into work stacks with details. If the stack starts running low mid-week, put in a kanban for feature breakdown. Stop the line, breakdown features and reiterate. This way there is continual flow.

One thing this does is force small changes. Small changes are critical to agility, they are mutually inclusive. You have to be able to ship and switch gears to be agile - by definition.

On a personal note, I've always found the Sprint (of the Scrum process) awkward due to the fixed time box. Because of that fixed time box, we're forced to jam ill-fitting work into that box. Often we over-scope because we're inherently optimistic - to a fault. It makes more sense to me to make landmarks out of shippable features or feature sets. Estimation comes from measurement of past sizing...instead of asking "how long will this take to develop" we use past data on "how long will something this size/complexity take to deliver", a more important prediction by far. It's also easier to measure - when did we start and when did we release it and actually start to realize its value (lead-time). That's what really matters anyways.

Thursday, April 20, 2017

How to Kanban

Kanban - unless you've been in a prison camp for the past few years, you've at least heard about it. It's origins are in manufacturing - where they have to repeat the same task over and over (major implications). It's been ported to software development. The basics of the entire TPS (Toyota Production System) are about reducing waste and Kanban is one small part of that entire principle - Kaizen (reducing waste/continuous improvement) is one of the other important principles that make the TPS successful. In order to truly port those principles into Software Development, some of those principles need to be applied differently (in a software environment sort of way) in order to optimize throughput and reduce waste (the main goals of the TPS).

The start is the all ways. Start building your process with the end in mind and with the realization that YOUR process will depend on YOUR environment. Perhaps your environment benefits greatly when there is waste or when things take longer to accomplish, IDK. I'm only taking the happy path into consideration here - where your goals are to reduce waste and increase throughput (without reducing quality or increasing costs).

Most commonly, in software, the application of MVP (Minimum Viable Product) is used to reduce waste and that is a good start. Sometimes, and probably not rarely, important parts (artifacts) are cut out of the process in the guise of reducing waste. It's been covered already so I won't go there. I will focus right now on the kanban itself...the card.

A kanban is a card that represents a work order from downstream. Kanban move upstream and results move downstream. The kanban is a visual, tangible representation of something that a downstream step needs in order to do their work to deliver working product. All the way downstream sits the consumer. In software, data flow one way, kanban flow the other way.

The Kanban board is a visual way to see what's going on, but its located centrally somewhere so everyone can see what work is in process. This works when the team is co-located, but can easily fall apart when distributed. Each area should generally have its own board. These boards represent requests for work. Downstream should have a prioritized queue of all the work, those cards should be maintained by whomever is directing or requesting the product (the program manager, "the customer", the product manager, the BA whatever). The thing is, kanban move upstream in a pull system...usually starting at the end of the dataflow...the end-result.

For each work request, you need more than one for the board and one for the person doing the work. The one for the board is so that everyone else knows where something is...when its being worked on, the person/pair/team doing the work is indicated by placing their card on the work card...they get a copy of the work card.

Lets take a quick walkthrough of this process in action -
Lets say were doing a User Management thing. First story is add users - "Add Users Page" kanban goes to UI team. "User Persistence" goes to the backend team just before the UI team really needs persistence. Integration happens, goes out for testing. QA almost immediately sends a card to the UI team "View Users" because they can't see the users they just added. UI team puts a card in to the backend team "Get Users", works on view while waiting for data. UI team gets back the "Get Users" card and integrates. .QA gets the "View Users" card and checks it out. Now someone has the idea to edit users so those cards flow up and the feature flows down. Disable user becomes a thing and so on and so forth - cards flow upstream, features flow downstream.

That's it. That's all Kanban is. It sort of solves some things, but really there's a lot more to the whole TPS system that is of GREAT value to software teams if one has the impetus to use them...continuous improvement, Gemba (go to where the work is)...Gemba is a good one. It's just walking over to someone's desk and seeing what they're doing and how they work - understand their work and challenges they face. Pretty easy right!

Here's a good one - Stop the Line! If something isn't working right, stop the line and get it fixed - crash the problem, meaning everyone get to it and anyone has helpful ideas please let them be known.

Continuous improvement - always look for little gains in efficiency and reducing waste. Maybe its wasteful to have 4 manual steps in some process that could be automated and would reduce process time 80%. If that process is repeated 3x per day that's a lot of waste to reduce.

The moral? Doing Kanban is about an entire cultural alignment around Lean practices. It's pretty easy to do with some basic principles...and some obvious benefits...once you understand the nature of it.

The next daily lesson learned goes deeper into some details about the details.

Wednesday, March 29, 2017

Talking 'bout that Functional Spec

What's a spec?

A spec is a reference for what the software should do.

Who uses the spec?

Everyone who is working on the product. Devs, QA, BAs, Managers, Marketing, etc.

How do I know it's good?

Here's a test - QA should be able to use scenarios in the test directly. They should be able to create accounts based on characters, follow through their routine activities and the non-routine activities as describes in the spec.

A sample

PizzaMeNow is an app for ordering pizza online. Users can order for pick-up or delivery. Users select how many people they're feeding and what toppings are good to go. They pay for their pies through the app (paypal, CC, maybe bitcoins or whatever else).

Scenario: Big Party.

Rick is having a party for the big game. He has 20 friends coming over and wants to serve up some great pies. He opens up PizzaMeNow and plugs in his phone number and address, fills in 20 for how many people and selects the "party varieties" toppings (which includes the usual suspects that you find at any pizza party - cheese pizzas, sausage pizzas, pepperoni pizzas, supreme). He's ordering ahead a couple hours, the party starts at 4 so he selects the 6:00 delivery time slot that same day. He sees the note that says it could get there +/- 15 mins of the selected time, he's ok with that and proceeds to checkout. He uses his PayPal account to checkout. 6:05, the party is on and the pizzas arrive - now it's time to really party hearty! 6 grande sized pizzas from the Pizza Shack, plenty to go around!

Scenario: Lonely Chad.

Chad is newly single and kind of nerdy. He is bit down and feels a bit like staying in for the weekend and watching all of the Hobbit and LOTR movies in order. It's going to take awhile and he doesn't want to have to break mid-action to make food. He logs into his favorite food app PizzaMeNow and orders pizza for lunch and dinner for both days. He places multiple orders and pays for them all at once. He wants it all fresh an hot, so he selects 1 person on each order and declines the "enough for leftovers" option. He prefers a bit of variety, so he puts in different ingredients with each order (somehow matching the theme of each movie that will be playing while he's eating each pizza). He confirms the four orders and checks out using his bitcoin account.

Scenario: Hungry Kids - "Pizza Again..."

Judy is a working mother with four kids and a hungry husband. She has the drill down for Friday game over pizza, babysitter and out. She uses PizzaMeNow regularly so she has a profile. She uses the "Reorder" option to place the same order as last Friday. She has the autopay option set up with her CC info so it just works. She is hoping one day that a simple text message from her phone will replace the bother to go into the app to place her order, but she is happy to be up and running so quickly (and get her order in while the car is stopped at a red light - maybe she is one-of-those-people or maybe she is in the passenger seat).

Each of these scenarios tells a user story that describes how the user will actually use the app. I'm using scenario here but User Story could work too, however there is app interaction involves and user stories typically involve the description of need without describing how they interact with the know the type - As a mother of four plus a husband, I need to order our usual order quickly so that I can get on with my life...Ok, so these scenarios are a bit more like use cases combined. Use Case 1 : payment. Use Case 1.2 : pay with paypal. Use Case 1.2 : pay with CC. Use Case 1.3.a : pay with Visa....and so forth.

We'd really like to have scenarios for fails too ...

Scenario: Angry Dad.

Jim's wife Deborah ordered a pizza through PizzaMeNow and it's over 15 minutes past the selected time. The app said +/- 15 mins and so now it's late and Jim is hungry...hangry even. He searches the app furiously for a contact us link. There is no link to be found. He goes online immediately to the app store to give the app a terrible review...just then the phone rings. The driver is lost trying to find the place, meanwhile Deborah receives a notification that their next order will receive a discount automatically along with an apology for the delay in receiving their order.

Scenario: Can't pay.

Voldo is a bit of a scatterbrain...he put in his order, but forgot that his CC was maxed beyond maxed when he used it to pay for his pizzas he ordered through PizzaMeNow. He was is such a hurry that he didn't even wait for confirmation before exiting the app. He receives a text message saying that his order was declined and that he should supply another form of payment. He taps the link to open the order in the app and is immediately brought the order confirmation page. He proceeds to payment and uses a different payment form. This time he gets the confirmation in the app and all is well in his world.

We can see based on these scenarios the various features that we will have to develop in order to deliver to our users (pun intended). As the fictional users exercise each feature, we can begin to understand how they will be using the application. While writing these scenarios we can imagine the user experience when say they can't find their CC to pay for something, so we know we would want them to be able to come back to their order and pay later when they've found it. As we develop our scenarios further we can go into more depth of detail with each feature - perhaps create scenarios involving only one feature (requires some context usually though). This sample is not a complete spec, but only describes some possible real world uses of the application. They can get everyone in the mindset of thinking about how to design, develop and test the app. Consider, for example, how we would test the "Angry Dad." scenario...would we want to wait the whole time for the delivery plus the 15 mins to test that feature? Or would we want some control over those configurations so that we can just FF to the late state and trigger the email and phone call (btw: did the app call the driver and the customer or was that all the driver?)

Monday, March 6, 2017

Functional Programming like an OOP-er?

So, I've been learning much about Functional Programming techniques. Elm-lang is a great budding FP language that transpiles to javascript and is optimized for the browser - haven't seen much in the way of server-side packages for it...but then again, I haven't really looked.

One key feature of the FP languages I wan't to riff on is Currying - basically it's where you create a new function that does the same thing but with less parameters and you set defaults.

In javascript, for example if you had the following function...

    function saySomething(message, person){

you could curry the message and create a new function to sayHello

    var sayHello = function(person) {
        return saySomething("Hello", person);

Well, that's all well and fine. Our sayHello function would only be able to say "Hello" as that would basically be set in stone for this "instance" of saySomething. I said instance didn't I? Couldn't think of a better OOP way to put it. It's also called a closure isn't it? Basically we are creating a function object that has a locally scoped value of "Hello".

It could also be written like this (which is a bit more re-usable):

    var sayHello = currySaySomething("Hello");

    function currySaySomething(message) {
        return function(person){
            return saySomething(message, person);

//or if you like

    function currySaySomething(message) {
        return function(person){
         var _message = message;
            return saySomething(_message, person);

And in this way, an instance of the function object "saySomething" is returned as a closure with the locally scoped variable "message". It can be done another way...this one is straightforward OOP in javascript.

    var helloSayer = new HelloSayer();

    function HelloSayer() {
        var _message = "Hello";
        return { 
            sayHello : sayHello

        function sayHello(person) {
            return saySomething(_message, person);

Well, that's an OOP way to do it, but it still meets the same goal of immutable state that's important to FP. But if you think about it - currying is really setting the internal "state" of a function. It's kind of like using a Factory, although in the example below it might be a bit overkill - could just use new Sayer("Hello"), but that's not the point of a Factory. If you need a Factory, there it is.

    var factory = new SayerFactory();
    var helloSayer ="Hello");

    function SayerFactory() {
        return { build : build };
        function build(message) {
            return new Sayer(message);

    function Sayer(message) {
        var _message = message;
        return { 
            sayHello : sayHello

        function sayHello(person) {
            return saySomething(_message, person);

I guess what it comes down to is how you use the objects. One could have objects that hold internal immutable state. Using an OO language it's often necessary to avoid overly-complex, unreadable code.

Wednesday, February 1, 2017

Fix Your Coding Style Cause it Causes Errors!

This enthralling academic paper titled "Using Redundancies to Find Errors", came out of Stanford in 2002. It shows that analysis of redundancies in code "seemed to flag confused or poor programmers who were prone to other error types."

Some examples of the kinds of redundancies mentioned in the paper include: unused variables, reassignment of variables for no purpose, mathematical redundancies such as x/x, x|x, etc., logical branch and case statements which either always eval to true or false or are never reached, dead code - statements that are never reached due to an early break or continue in a loop.

Most of the examples given in the paper are the result just plain sloppy and/or lazy programming style. You see it too many times - copy-paste code, terse conditionals, disconnected variable names, poor structure and flow. My advice to anyone seeking to get better at programming is to reread your code before you commit. Write it three times - once to make it work, a second time so you can read it and a third time so everyone else can read it. When your done with it, you should be able to show one of your QA engineers the code and they'll know what it does! Then you know what you should do? Rewrite it again - this time to make it more concise, you could be missing out on something important like an opportunity to reduce the size of the code base, crate some reusability, or remove some diabolical nested if statements (one of the worst sins of programmerkind).

Look, if you were writing a book, would you be allowed to write the first draft and sell it as soon as you've written the last word? Hell NO! No one would want to read it anyways. It would contain several grammatical errors as well as redundancies, illogic, contradictions, and shit that just plain don't make any sense! Why would you abuse any language in such a manner? Languages are made to communicate. Programming languages are no different.

Tuesday, January 31, 2017

Database Development The Iterative Way

As I recently became more involved in developing a data layer (including database) I've been learning a new style of database migration. Previously, I've written plain SQL for database migrations for each deployment. Those were handed off to a DBA to run in each environment.

These days, I've entered the realm of Liquibase. Liquibase is my first experience with this sort of tool that allows for source controlled, database versioning. It's written in declarative style and translates the declarations into the underlying SQL. It's cross-platform and supports multiple declarative languages (JSON, XML).

Here's how it's changed my development process: In the old days, I used to write the SQL and run each file or put it all in one file and run as a batch. Then write some queries to validate, interrogate the schema etc. I would do mostly up-front design of the database components. Most of the queries and commands were generated by EF or in code. Other factors have changed that as well.

Nowadays, I'm writing a lot more stored procedures with none of the sql in code or generated by code. I'm writing tests first. Then I'm writing stored Procs, then creating/modifying tables declaratively via Liquibase. Sometimes I don't care about the table of the column until it's really needed. Sometimes the name of a column morphs during an iteration (a short cycle of failing test-code-passing test on the order of minutes). Nowadays, no problem! It's relatively simple to change a column name.

The big trick/advantage is that everyone else can pull in those source controlled changes and have their local databases updated by running a batch file or using a git hook. It only applies changes that are new on the target database, so if they haven't updated in awhile it'll apply all those changes until they are up to date.

It's all good! I never really had a chance to dig into EF code-first migrations to this degree. If its the same or similar in efficiency, I would recommend it for sure! Or any tool that enabled rapid changes to the backing store which has acceptable trade-offs (here's looking at you noSQL).

Wednesday, January 18, 2017


I've been hearing this phrase a lot lately "words have meanings". I've also been reading more of Domain Driven Design by Eric Evans, in which he stresses the importance of Ubiquitous Language. When we communicate about our work, when we collaborate as social human beings (even the most antisocial rely on social networks for survival - see the NatGeo show Mick Dodge) we use language. It's what makes us human, its what makes us powerful. Our very fabric and existence is a product of our ability to communicate, coordinate, and work together.

So when we communicate, we are more effective when we do so effectively. We do so partially through word selection. Words do have meaning. And, they have meaning in context. In the past, I've been exposed to the usage of Unit Testing used where the word Functional Testing was more appropriate in my context. To me (and I assumed the rest of the world), Unit Testing is when a developer writes automated tests at the lowest level of code she owns. To the other person, it meant manually testing each functional unit as exposed to the user - which I know as Functional Testing.

Here was a case where we were working at cross purposes whenever we were talking about Unit Testing. She pulled rank and had the term redefined in her world so that context switching had to happen...for all developers on the team. When she uttered the phrase "do Unit Testing" our minds immediately went to thoughts about writing code with test doubles, scenarios, and defining test cases for each string parser and logical branch. But we had to go against what we know and live in this other reality for a bit - fighting our neurological pathways which were forged over the years.

Every conversation in which more than one term is used to describe the same thing and especially where one term requires special domain knowledge is going to see a reduction in effectiveness. Words have meaning in context. What do we do about this? Especially when the same word has meaning in different contexts...

I propose the following: Combine context with the subject so that ambiguous terms become less ambiguous. Instead of Employee say Payroll Employee. Instead of User say Blog User. Say Functional Unit Testing, Ad Campaign, War Campaign, Political Campaign, Shipping Container, Bank Account, User Account, User Role, etc. Use contexts as containers for the concepts in which the words have meaning.

One term came up recently...owner. Owner is such an abstract concept that it must be used with some subject to show context or there will be wholesale confusion, particularly when there is more than one owner context involved. Home Owner and Business Owner have very different connotations...think of how many just in terms of taxes. So what if taxes was the context? Might we say the Tax Homeowner or the Tax Business Owner? Or is it enough to imply tax is the context - depends.