Thursday, September 28, 2017

Cultural Evolution Part Two

Yesterday I posted about how I evolved the process by improving the work tracking and planning tools. Today I will tell you about what we did that brought people together into a more cohesive team with high morale.

One day, I got an awesome board game as a gift (thanks to my amazingly thoughtful wife). It was a fantastic little strategy game that anyone could play. Gameplay involved dynamic shifts in both political alliances and strategy. This game was called Quorridor. There is beauty in its simplicity. It's easy to pick up which makes it the perfect catalyst for bringing people together for some clean, competitive fun. We played over lunch maybe a couple times per week. Our manager joined us and we all had a good time together!

What inspired this? Something stuck with me from when I was in the live sound business. One day, while working a gig at the Cadillac Theater I saw the house crew playing chess backstage to pass the time between setup and performance. There's a lot of hurry-up-and-wait in that business and usually you'd try to get a little shut eye since the nights were late and the days were long. I recall many days of just sort of sitting there trying to get some rest or exploring; but playing chess was how I really wanted to pass the time. That was the only crew I ever saw passing time that way and it stayed with me.

When I started my development career, I was itching to play some strategy games with colleagues. We'd go out to lunch together and they even had a Foosball table in the lunchroom. But none of us really spent a lot of time in the lunchroom because it was on another floor. Game playing started small, maybe a couple of us. Then we had some regular players and swapped others into the fun.

It was great for morale and a fantastic way to take a break from the work and go back in with a fresh start. Another benefit was the way it brought people together. The benefits of brining teams together are enormous...companies will send their teams on weekend retreats for such things. They'll have weekend events with BBQs and after work parties. Those are all well and fine and maybe often a little forced - those mandatory extra-curricular activities that often cause disruption in busy personal schedules. On the other hand, game playing over lunch is easy, optional, and not disruptive.

Something like this has to be driven by the team! And it needs to be supported by management. There's a fine line there. Sometimes a manager can just be a person and this is the context for it. At the same time, managers can have a BIG influence in the success of such things by jumping in and especially by not shutting something like this down. Heck, take some time for these things during working hours! Maybe on that Sprint Wrap up day after you've shown off your accomplishments for the Sprint. What else would you do for the rest of the day? Try to jam in another feature? No way! Sharpen the saw! Do games, coding challenges, put together a jigsaw puzzle...whatever the team decides!

I know there's some counterintuitive thoughts around this (I've encountered them)...were not being paid to play games were being played to build features. That's true. Fortunately, teams that play together stay together and there's no better way to get more done than with great team cohesion and high morale. It's an investment in a more productive team. So keep on organizing the work at a nice steady pace and helping each other out (by working together) and you'll have the time anyways - its a positive feedback loop!

Next post I'll talk a bit about local optimization vs overall optimization as it applies to teamwork and workflow. Here's a hint - there's a class of algorithm involved.

Wednesday, September 27, 2017

Cultural Evolution

I've been thinking a lot lately about organizational culture and how it might evolve. Actually, I've always thought about this and since my entrée into the software world I've applied myself to evolving both culture and process. I've worked through the evolution of tools and practices that enable collaboration. Here is one way I've applied tool improvement to help along the first team I worked with.

This team used a SharePoint list to track work - each work request had an entry and was printed out and handed to the developers with screenshots and a bit of write up about what was to be done. The process worked ok because when you went to the list you would be able to filter it and find your work. There was room for improvement so I jumped right into it like this -

First, I created a view for developers (where assigned to = Me). I shopped this around to other developers and my manager. She saw the potential there and asked for a view for managers. We briefly discussed and I went off and created it.

At the same time the managers changed their practices a bit and started using the views to plan on a weekly basis with quick checkins each morning. Afterwards, it was much easier for everyone to know what they should be working on. Micromanagement wasn't an issue. Managers managed the work flow and distribution, not each individual's work. Having the right tools in place helped with that because they could see right away where things were - no need to hover or constantly interrupt to ask or waste time in meetings that should be used to discuss impediments.

pro tip: Managers need to know how the work is going - proactive updates will keep them informed. Imagine if your mom was in a long surgery and you were left for hours and hours wondering how things were coming along. Or maybe you have something in for repair for weeks with no word about how it's going. It can be troubling not knowing and they've got to answer for those things to their managers or to clients.

Hopefully sharing this experience helps to illuminate the value in having the right tools to track and communicate about how the work is going. This form of passive communication enables anyone to check in on the work without bugging or micromanaging (both counterproductive activities).

Wednesday, September 20, 2017

Amazon Cognito?

I'm looking into interfacing with Amazon Cognito for user account management. I'm leaning toward abstracting it rather than using it directly in the web code via their JS SDK. Rather maybe use a server-side SDK. I could just use what ASP.NET has built in, seems like it might be a bit easier...the documentation for Cognito isn't straightforward - as it is with all their docs.

Wednesday, August 23, 2017

AWS Elasticache, Memcached, and Enyim

I've been knee deep in caching lately since I've been looking into an issue revolving around AWS Elasticache and Enyim. I've learned a lot so I'll share some things. Let's start at the beginning -

Enyim is a client for memcached which is a cache server - basically an in-memory key value store with a timer on each value so it can be expired after a certain amount of time to preserve memory.

memcached itself doesn't support clustering but Enyim can be configured to allow clusters. So you can have 2 or more memcached instances on one or more servers and use Enyim to talk to them all as if they were one cache...sort of.

Since abstractions leak, Enyim as an abstraction of multiple cache nodes, leaks in its own way. When a node goes down, you get nothing back in the client. Well that's not true, you get a Success == false and StatusCode == null that's what you get. And that's the default values for the Result object. And when you do get that, it means a cache node went down in your cluster - but not to worry!

Interesting thing about those cache clusters and how Enyim manages to use them. Enyim uses a hashing algorithm (actually several you can select from,) which shards based on the key and the number of nodes in the cluster. It picks a different one based on the key. Additionally, provided you have the servers are added in the same order to the cache client, it will be the same no matter where you are calling from. You could be in any number of producers and consumers of the cache and it will pick the right node.

Let's say you've got 4 nodes and you have a key 'my data key'.

Let's say you use Enyim to Store("my data key", someData, ...) and hypothetically that key hashes to a match for node 3. 

Now when your consumer of that data - could be on a different server in a different part of world for all it cares - calls Get("my data key"), that key will also hash to match server 3 and you'll get 'someData' in the response!

Internally, Enyim calls those nodes servers. It marks each server as IsActive when it makes a successful socket connection to the cache instance. When you use the Enyim CacheClient to make a call to the cache and the socket cannot connect, it marks the server/node as inactive (IsActive = false) and adds it to the dead server queue.

Don't worry though, it tries to reconnect to the servers in the dead queue. When the connection is re-established it automatically becomes available in the cache cluster again. There is a caveat when it fails though. When your client calls and the node is dead, the node isn't marked dead BEFORE the call, but after. It's up to you to test for those conditions (null StatusCode) and retry the call so that it WILL use the next node in the hashring.

In the same scenario above, let's say node 3 is unreachable. Well you could see how network connectivity could be an issue if they aren't co-located. In that case it could be down for only some clients but not others. Let's ignore that issue for this scenario and say the server is being rebooted for patching or something.

Here's what will happen...your consumer will make the first call and the key will resolve to node 3. Enyim will attempt to establish a connection to that node. When the socket connection times out it will mark the server as "dead". It will return Success == false and not set a StatusCode in the response object. The VERY next call will resolve to the next server for that cache key while the node is down.

With Elasticache in the mix, there's another layer of abstraction in play. Elasticache offers its own grouping and node discovery so that you don't have to specify each node address manually. This means you can add nodes without re-configuring the client. When using this option, you should definitely use Amazon's cluster configuration for Enyim, it handles the node discovery for you meaning you only configure 1 endpoint per cluster. This is a good thing, but what they haven't put in the mix is handling node failures on their end. Would be nice if they did, but they don't. So just think about that...

Still, when you are using cache clusters in Elasticache, the best way to handle a node reboot is to simply make the call and the amazing Enyim will do its alchemy and switch to a live node until the dead one is back up and running again. Works the same with adding and removing nodes too. It's all automagic!

Summary -

When using Enyim, just put some retry logic in your code - preferably in your own wrapper class so you aren't repeating yourself everywhere and forget to do it somewhere.

Wednesday, August 16, 2017

Where to Begin the Agile Project

So, your a programmer and you get a new project to work on. And let's assume this project involves some data source(s), probably some persistence, and user interaction. How do we approach the construction of this system?

In the first place, we'll want to understand what the system needs to do and what it will generally "look like". When I say look like, I mean how it will generally be architected - how does the data flow? What layers? What components? Where do they interface? This involves some basic design and there are several patterns to draw from that would get the job done.

Either way you design the general structure of the system, you'll need to understand the data flow. You'll be presenting some data to a user and that data will come from somewhere. It seems obvious that the first step is to begin where the data flow starts...begin at the beginning. However, I say NO!

Through years of trials and errors of beginning at the beginning, I've found that you generally have to change things near the data source at least a few times throughout the life of an "Agile" project.

I've spent much time modeling databases and building data layers for applications only find that once the front-end was built on top, the data model had some fundamental flaws. The main trouble is that those flaws only came out when the business stakeholders were able to interact with the system - when they were working with something concrete. Only then did the details truly solidify. But by then, there were layers built up below the presentation layer - interfaces, implementations, tests - that all needed to change to support this new understanding. This leads to much rework!

If we can have a better understanding of the system before there is a full foundation baked into the system, then we can reduce the amount of wasteful rework on a project.

To solve for this problem, we have some options. We can build wireframes and prototypes in a Design Sprint. This sort of design-up-front is useful in as much as our understanding of the system is complete at the beginning of the project. If there is a good general understanding by the requester, it will be relatively more useful to do mockups - it may trigger insight and understanding earlier in the "Agile" project. It's a way of saying "if I were to build this, would it solve your problem?" I'm for it and it does a lot of good, but it's not where we should end the game!

Another technique I like to use, and I'm urging you to do the same, is to "start at the end"! By starting at the end - meaning the desired outcome of the feature we're building this cycle - we can set some concrete earlier in the sprint (preferably on Day 1 - Sprint Planning). If you're building an analysis system, start with the graphs and charts. Mock up the data that feeds them. Then build backwards from there. It's best to find out how the system will provide value. Is it an invoicing system? Start with the invoice itself! Work back to entering the billing info.

When we do this - and it doesn't have to look perfectly pretty yet - we give the business stakeholder something to interact with BEFORE we build up all the supporting layers underneath it and put on the fresh coat of paint. It's like building a custom bike starting with the seat and handlebars first...then you put on the wheels and a temporary engine to test the rake of the front forks, footpegs, shifter, grips, brake levers, etc...the user interface. Then they know it's built to what they need - after all it is a custom!

But software is a bit different than building a hog. Users can't see or feel the engine directly. They interact through whatever UI you put on there - there can be mockups and design drawings and diagrams, but it's the concretion that will have the most meaning. Besides, mockups are throwaway - waste. You can build a mockup with the actual code, it's easy these days to get a basic UI up and running quickly. Better to build the real thing then iterate from there one little manageable decision at a time. Better to have to refactor early in one layer than have to chase a change up the whole stack!

Friday, August 11, 2017

Friday Challenge #6: Which Way Should We Go?

Lets say you've got some decisions to make. You have a bunch of stuff to get done but you aren't so good at managing your you create a system to keep your priorities in order.

Ok that's cool. But how do you prioritize those taks? Maybe you want to take care of the one that's the latest first - if there are any that are late. Or how about the most important? The one closest to due date that isn't late?

Make a Priorities class with a method called GetHighest that gives back the highest priority item depending on the aforementioned conditions.

Lets try this two ways:

1: Write an algorithm that takes into account all those ways to prioritize using 'if' statements.

2: Use a class for each way you want to prioritize and pass it into the GetHighest method. The class should have a Rank method that takes the array (or whatever) and gives back the ordered array (or whatever).
You should have 1 class for each way and pick the way in a different method.

Now go back to way 1 and add a new condition and ranking. And do the same for way 2.

Ok so maybe you have to add and if statement either way or use a map of conditions to class for way 2. But how about this...combine conditions. Or maybe chain filters...make filter classes to remove those that are late and those with low value...combine those two filters in different ways with the soonest due ranking. Now you can create different filter and ranking strategies based on different conditions...

Now if you have 2 hours to spare and the top priority task takes 4 hours, you can apply a strategy that takes into account how much time you've got, how many people are available, what skills are required, or whatever else you that with procedural code and you'll wind up with a twisted mess to sort through anytime you need to add a way to do rank or filter! Go ahead...try it with way 1!

In this challenge you've exercised the strategy pattern and possibly the factory pattern if you've used a factory to retrieve the appropriate strategy.

A few things to try with the factory - use contextual information from the real world - time of day, number of tasks to prioritize, time available, etc. Re-use the filters in the factory to get percentages of tasks that are late and chose a certain ranking strategy based on the can see how various strategies can be combined to create new strategies and even reused in other ways that a procedure cannot. This also works in functional programming, in fact it's how functional languages work, filter, etc. The strategy is the map function you pass into map or the filter function you pass to filter, etc.

Friday, August 4, 2017

Friday Challenge #5: Fixed Gear or 3-speed?

When I first encountered an OO language (Java) there was a tutorial that used bicycles to explain polymorphism. All bikes have some behaviors in common - you pedal them to move forward, you turn one of the two wheels using the handlebars to steer. While there are exceptions - pedal-less bikes for kids, sit down bikes with levers to steer, pedal assist (actually a moped) - the common case serves as great fodder for object oriented programming. With that, let's start the next challenge!

You don't have to use Java to take on this challenge, there are many many object oriented languages and many that have support for classes. Use whatever you would like to use - learn a new lang maybe!

Given the following types of bikes:

3-speed cruiser, id=1, type=MultiSpeedBicycle
10-speed mountain bike, id=2, type=MultiSpeedBicycle
BMX - fixed gear, id=3, type=SingleSpeedBicycle

Write a method that takes in a bike id and returns the bike. The method signature should have the base/parent type as the return type.

Bicycle getBicycle(int id);

Bicycle is an abstract type.

You can either make a static method on Bicycle if your language allows it, or make a BicycleFactory to house the method.

The basic bike (the abstract Bicycle type) has 3 methods - pedal(int pressure), applyBrakes(int pressure), steer(float angle).

A bike with gears EXTENDEDS the basic bike with two additional methods: shiftUp(), shiftDown().

*If you'd like, you can add some additional methods like getCurrentGear(), getMaxGear()...or you could return a bool from shiftUp and shiftDown to say if it changed or not...or return the resultant gear of the action. The more the internals of the object are protected the better, the consumer should not know what gear the bike is in, only that they can shift gears up or down.

Whatever you use to call getBicycle will eventually want to check what type that bicycle far as it knows it only has the basic bicycle methods. Until it does know, shifting isn't an option.

Let's say we have a UI for this bicycle. Actually we would want a different UI for each type since users will interact differently with each type. The shiftable bicycles will have some way to shift gears in the UI - maybe a certain keypress or tapping the screen a certain about pressing shift levers?

Fetch the proper UI for the type inside a method that has this signature...

BikeView getView(Bicycle bike); 

Inside that method, interrogate the type of bicycle and return the view for that type. Put that method into a BicycleViewFactory - do you think the constructor for it takes in all the views or maybe something that knows how to load all the views? Just stub out something for now - create the various view classes and have the method return the right one for the type of bicycle. Maybe you really only need two types of bicycles, maybe you want three - that's entirely up to you and far you want to go with the details.

So that's the challenge...write up those classes and put it altogether with some way to exercise (pun intended) the methods and verify that the right View is returned in the end. The View doesn't actually need to have anything in it, neither do the methods on Bicycle...stubs will work just fine. The lesson here is in how getBicycle and getView are implemented.

Oh! I almost forgot! There's one more thing you can do - you'll want something to load the bike and return the proper views...a BicycleController! First it needs to load up an input view to receive the bike id (maybe list them all - listBicycles() and have the user select one if you'd like). Once an id is sent to getBicycle, that method should call your _bicycleFactory.getBicycle() method to load up the bike. Pass that instance to the ViewFactory to get the view...if you got back a BicycleView return it to the user - else maybe return an ErrorView...the BicycleController should take in both the BicycleFactory and the BicycleViewFactory and hold internal references to those...they should both be set up and ready to use before they are passed into the controller.

Feel free to implement the UI in any way you see fit - heck try going for multiple UIs! Everything should be re-usable at this point...the only things that should change based on the UI you want to implement are the Views!

Let's take that a bit further... Consider different UIs. Maybe the user has certain preferences. Or maybe it's a different view per screen type. Perhaps there's a VR view with actual pedals and steering wheel as input...a stationary bike! Got it!? Those would have different Views for different user interfaces.

Alright, have fun! Happy Coding!

This challenge is all about polymorphism, factory pattern, MVC pattern, encapsulation, Dependency Injection.