Thursday, December 31, 2015

Lambda, Lambda, Lambda - How To Use a Lambda Expression

Many programming languages use lambda expressions: Python, Ruby, C#, JavaScript, and GO to name a few modern examples. Lambdas can be extremely expressive and save several lines of code when you just need a function without the need for reuse. I and others use Lambdas to create greater readability. On the flip side, if you don't understand how they work l, the syntax can be confusing and the code will be impossible to understand. My goal in this post is to help you understand what Lambdas are all about so you can understand them next time you see one and even add Lambdas to your tool set to improve your ability to be more expressive.


Here are some examples of adding 1 to x using Lambdas in various languages:

Python:
lambda x: x + 1;

Ruby:
lambda { |x| x + 1 }

C#:
x => x + 1;

js (ES6):
x => x + 1;

GO and js (<ES6):
This is where it gets interesting. These languages treat functions as first class and so a function can be passed as an argument to a function, which is really the point of lambda. This works the same way in PHP too. The syntax really doesn't matter in terms of understanding what a lambda is. Syntax is just to make things cleaner and gives you a way to declare the Lambda inline.

Let me explain by example:


Lets say we have a js function that expects a function:


function foo(bar){
    return bar();
}


This is a root form of the lambda.


In a strongly typed language like C#, it will look something like this:


int Foo(Func<int> bar)
{
    return bar():
}

Notice how the input is a Func? A Func is an anonymous method that returns a value (specified by the type in angled brackets <int>. There's also an Action which expects void.

Here is a more useful example:


public int OperateOnX(Func<int, int> expression)
{
    return expression(this._x);
}

and we call the method like this:


var a = intance.OperateOnX(x => x+1);
//or
var b = instance.OperateOnX((x) => x*x);


or if we use the C# Linq library:


var c = list.Where(x => x > 3);


In this example, the Func returns int and expects int as the first argument. Mulitple arguments can be specified as Func<int, int, string> (Action<int> expects one int argument, and Action<int, string> expects an int and a string argument):

var b = instance.OperateOnXY((x, y) => x*y); //Func<long, int, int>


The Where extension method iterates over the list and passes each item to your Lambda and returns items for which the expressions evaluates to true. The Lambda Expression is a filter.

For more advanced operations, you can encapsulate like this:



var c = list.Select(x => 
{
    var y = x.Name;
    if(y == "")
    {
        x.Name = "Phil";
    }
    return x;
});


Lambdas can take multiple arguments, after all they should behave just like a regular method or function. In fact you can pass a method to a Linq Where like this:


var d = list.Where(Filters.GreatFilter);


so long as GreatFilter matches the expected delegate type - its signature should look like this:


bool GreatFilter(ListItem item);


Since Where is an extension method in C# Linq (it Extends IEnumerable<T>) the input argument to the generic function will be T, or in this case whatever type "list" items are.


Hopefully, you understand what Lambdas are all about by now, if not feel free to ask and I will do my best to explain a bit further.


Wednesday, December 30, 2015

Extension Methods in C#

If you've ever used Extension methods in C# but don't really understand what they are about this post will hopefully help you understand them a bit and even how to create your own. If you haven't used them this should serve as a good foray into expanding your toolset.



Here's what I've learned:

Make a static class and add a static method (this is the extension method).

The extension method takes in the object it is extending as the first argument in the following format:

public static class BakerExtensions
{
    public static Cake MakeCake(this Baker baker)
    {
        //make that cake
    }
}

If we have a baker instance we would call the extension method as if it belonged to the Baker class.

Baker baker = new Baker();
Cake mmmCake = baker.MakeCake();

I recommend keeping your extension classes at a 1:1 ratio to the classes they are extending. If you have a Baker and a CandleStickMaker you should have a BakerExtensions and a CandleStickMakerExtensions class. This is not required, but it will help with organization.

You may also want to namespace your extension classes the same as the classes they extend, but if you do remember that the extensions will automatically be brought in with the class if it is in the same namespace. You could go with <original namespace>.Extensions if you want more control over when to bring them in and when not to, but it may end up adding confusion in the end.

So why on earth would you want to use extension methods? Why not create subclasses and just add the methods to those?

Ok, so let's say you have some existing code with the Baker class all over the place. You might have some subclasses already and you don't want to change the base class OR you want to add functionality to built-in types like string or int.

public static class StringExtensions
{
    public static string Shuffle(this string inputString)
    {
         //shuffle that string
    }
}


var shuffled = "something".Shuffle();

//shuffled  == "emogsinht"

It's a pretty neat way to keep in line with the Open/Closed Principle if you don't need to access instance members.

Word of caution: be careful about side effects when it comes to extensions. Side effects are a code smell in any case. Thogh it may be tempting to have them in extension methods, you should treat them just like you would if they we a normal class method.

There are a couple of drawbacks - You wouldn't get access to internal members of the class; and your extension methods won't be called if they have the same name and signature of a class method.

Here's an msdn article on the topic for more detail: https://msdn.microsoft.com/en-us/library/bb383977.aspx

Tuesday, December 29, 2015

How To: Run a Program in Administrator Mode

Often I need to run a program in administrator mode when using Windows. There are several ways to run a program in administrator mode on Windows 7. The way I've found to be the mode efficient is to pin it to the taskbar and set that pinned shortcut to always run as administrator. When you do this, it's a 2-click process - one click to launch it and another to accept the dialog.

Here's how to do that once you've pinned the program, this example is using Visual Studio 2015 since often it needs to run in admin mode to debug against local IIS.

Right-click any shortcut and click Properties. Here I am right-clicking the shortcut that is in the pinned program in the Taskbar. You can access these properties from any shortcut - on the desktop, or the start menu/screen to name a few.
In the Properties dialog, click Advanced*.
This dialog should appear and all you have to do is check the box and click "ok" and "ok" again and you are all set to run in admin every time.
*If you are right-clicking an icon for an exe you will find run as admin in the Compatibility tab of the Properties dialog.

Monday, December 28, 2015

Leadership: Gateway to a new YOU!

Lately, I've spent a lot of time reading up on leadership. I've been discussing the topic at length with friends, family, mentors and colleagues as well. It seems there is a few common threads that run through it all. And if you want to really be successful and happy in your life, you'd be better off to get this right.


Live by a core value system. Communicate with empathy. Build trust and by doing do so grow influence. Work together with others and delegate to maximize results.

Friday, December 18, 2015

Analyze, Trust, or Buy-In?


Do I really have to question everything? So I'm waiting at a train depot and of course there is a sign that advertises how much greener it is to ride the train than to drive a car on a daily commute. I'm going over-analytically submit a usual "it depends" sort of answer.


What might it depend on? Well...how much energy and what sources of energy are consumed with daily operations of the rail system? What is the waste of maintaining and building tracks vs roads? What is the total waste of producing and maintaining trains vs autos? What distance are you travelling? What additional energy and waste do you consume in your commute?


If we really want to prove the statement, we would have to analyze the data. Or we can just trust that the train company did this. Or perhaps some trusted analysis entity can verify the facts for us. I guess if I want to verify it I would either have to find a trusted authority (who knows if they really did a proper analysis), find data and do my own analysis (can I trust the data?), or just buy in.

Wednesday, December 16, 2015

The Importance of Understanding Expectations

A recent acquisition of a set of new chairs shed some light on how a solution can meet requirements while at the same time fall short of expectations. This caused that sudden realization to solidify in my gray matter. The upshot is that meeting requirements is a science while meeting expectations is an art. A friend, mentor and colleague has repeated so many times that software development is as much art as it is science - the majority of the chorus being sung in regards to choices of patterns and styles when coding. I want to come back to that though, but for now let me share a bit of the story of the chairs.


We were in the process of getting a new office space and one thing that came into play was the choice of new chairs for a communal/meeting/lounging area. The idea was that the chairs could be circled up to facilitate discussions. The first option had feet with pads, a bucket design, and a little desk top attached. Well, it would have been nice to have wheels and if the desk was removable so that it wouldn't be in the way if not needed. This feedback was given and the next sample chair that arrived with casters and a removable tray.


A few days later a sample arrived that fit those requirements. At first sight was a bulky eyesore of a white leather chair, with dirty handprints all over the back to boot. Then I rolled it around and the casters felt as if they would fall off. The removable tray was on the left - only on the left. It was nearly useless as it was genuinely too far away for comfort given the size of the chair and it was generally too small to support a laptop comfortably.


What is the lesson here? You get what you ask for! But seriously, think about how those requirements created solutioning constraints. What we needed was a modular place to gather comfortably and talk face-to-face for short durations. We needed the space to be able to double as a viewing space for presenting ideas, plans, apps, whatever. This is the art - understand the users needs, environment, and their expectations. Once you understand the users in this way, then you should have a better understanding of what solution to provide.


Now lets return to the code for a minute. Choosing design and style of architecture and code is an art. In order to deliver to the needs and the expectations, we must first understand the users of the code and deployment. Sometimes, you are the user and sometimes other developers. Ops teams could be users. What goals will users have in the future? There will be a need to change the code - its code of course so it is always changeable. The code will have to be changed in ways so that unexpected things do not happen. Other developers will want to understand the code, and without a whole lot of pain. You and others may want to be able to write unit tests against the code, or other forms of automated tests. Sure you could obfuscate everything and keep the code as compact as possible, keep it super-efficient, keep it obscure so that only the mightiest of developers can understand it. var a=g();a++;if((b===x&&y!==l)||(z|1)&&((((((m())+5/c)&3)|(l-5)))){return g===h?(6*f)>=9?'v'::''.....
but why would you put future you through that? It's like looking at a Jackson Pollack piece, except that someone will have to change it someday. Imagine what kind of mess you'd end up with if over time several other painters of varying skill level changed one of those paintings...it would probably end up a muddy looking mess before you know it! Instead its a nice clean mess.


Sometimes we inherit a Pollack. And sometimes we are asked to change a Pollack. Oh where to begin! In this way, the art of software can be lost in the science. The art of software is more like a mural on an endless wall...you can add on without destroying the original, especially if the original lends itself to extension and to change. Picture a mural on a wall that begins to the left and continues endlessly to the right. If the first part has lines that lead the eye to the right, we have something to build on. If it has white-space we can add to it by filling in the space. As our mural is continued over time to grow to the right, we may decide to change a part somewhere to the left of the current one. Themes can be continued, but maybe some offensive part will be changed or something later causes an earlier work to be inadequate and that part would need to be revitalized lest it bring the whole down.


When applied to software, we must think of the art of it as well as the science and leave our designs and styles open to change and before that to be understood.

Tuesday, December 8, 2015

Javascript Constructor Pattern to Ensure new is Called

Here is a pattern that will ensure "new" is called when defining an object constructor. When calling a JavaScript constructor function, if "new" is not used, then the variable being assigned will be undefined unless the constructor function returns something else.


function Griffen(name){
  this.name = name;
  if(this instanceof Griffen){
    return;
  }
  return new Griffen(name)
}
console.log(Griffen("Pete"));

Wednesday, December 2, 2015

JavaScript Constructors

I learned a lot about creation patterns from this http://shichuan.github.io/javascript-patterns/ today. I was even inspired to do my own experiments on some creation patterns not shown there.

JS Bin on jsbin.com
go there

Wednesday, November 18, 2015

Data Modeling - The True Meaning of Entities

When modeling data an entity refers to a unit that has properties which we are modeling. Some of those properties of the entity are intrinsic (natural properties) and some are extrinsic (externally applied/relative properties). If we model our entities such that the intrinsic properties belong to a root model (Person) and the extrinsic properties belong to subclasses of the root model (Employee) we can establish a proper representation of actual relationships of the entities in our model.

To illustrate, let's take a common example of modeling a person who is an employee at a specific company.

The person has some natural properties: Name, DOB, Gender, etc. These properties describe the entity in its base state. As an employee, the person may have additional properties that relate specifically to their state as an employee: id, email address, extension, location, title, etc. This person will also have some personal properties that may be interesting to model: email(s), phone number(s), address(es), family, etc.

If we are modeling the data as a relational model, none of the columns in the data tables should be nullable. Nullable columns are a clear indicator that the data can be further normalized and that it does not likely represent the entities accurately.

To achieve the goal of creating our model, we need to be able to create the proper relationships:

Person - Email (1:*)
Person - Address (*:*)

These relationships bring up interesting questions (and leave ambiguity in the model).

If someone has more than one address and one primary address is a requirement (assuming we are not modeling homeless), then how does a many-to-many relationship enforce the many-to-one relationship that must exist to model the persons primary residence?  Address may be either of two types: primary xor non-primary. We could set a bit flag to identify primary. In order to enforce the *:1 (an address can be primary for more than one person) constraint with the primary, we would need an FK on the Person table and a reference to the address table. In the *:* relationship above, how can that FK be constrained to the PrimaryAddress type? The answer lay in table-per-type (TPT). We want to get to this:

 ...
Person - Address (*:*)
Pesrson - PrimaryAddress (*:1)

To implement the TPT for this model, we would need an Address table a PrimaryAddress table and a AdditionalAddress table. The Address would contain the details of the address and the PK. But each typed table would contain an FK to the Address table which is also its own PK. The Person record would contain a FK ref to the PrimaryAddress. An additional xref table would be needed to link the Address table to the Person table in general, this would be a *:* relationship. This PersonAddress table can better represent the fact that more than one Person can live at the same Address and a Person may have more than one Address. The FK to the PrimaryAddress will enforce and show that the Person must have a PrimaryAddress and the 1:1 relationship between PrimaryAddress and Address will establish which Address is that Person's primary Address.


It is important to recognize the boundaries of the entity since any entity may serve as the root. It is equally important to establish distinctions between different types/classifications of entities. Here are a few examples of these relationships:

Person (base type) -> [Employee, Parent, Volunteer, Student, Teacher] (sub-types)
Contact (base type) -> [Email, Phone -> [Internal], Skype, Signal -> [Tree, Smoke, Call]] (sub-types)
Relationship (base type) -> [Friend -> [Best, Acquaintance], Family -> [Sibling -> [Brother, Sister], Parent -> [Mother, Father], Cousin, ...] (sub-types)

In this way we can establish meaningful relationships in the data model, while accurately representing the entities we are modeling.

Saturday, November 7, 2015

ASP.NET MVC RenderAction, Action

In attempting to render some popup boxes with data driven select lists into a layout (_Layout) I came across an issue. The layout was being called recursively each time the action was being called by RenderAction or Action html helper methods.


I moved them to a partial outside the layout and called RenderPartial from layout. This did not help. Nor did calling the Partial helper. After a bit of research, I found that I was returning a View which uses a layout page according to how the app was configured.


The fix was to return a PartialView from the GetDialog action. For sake of brevity, I set the return type of the method to a PartialViewResult. GetDialog looks like this:



public PartialViewResult GetDialog(string dialogName)

{
    var dialog = _dialogFactory.GetDialog(dialogName);

    return PartialViewResult(dialog.TemplateName, dialog.Model);

}

And the layout template like this


...
    @Html.Action("GetDialog", "Menu", new { dialogName = "NameFilter" })
...

Which renders a JQuery UI dialog to the output stream without the recursive layout calls.

Wednesday, October 21, 2015

Who is the Decider?

When it comes to decisions regarding software, it is generally the business who decides what needs to be done, when to do it, what will not be done, and even how to do it. There are so many decisions to be made, some requiring intimate knowledge of the business at hand and others requiring technical expertise.


When the development group is external to the business, and clear objectives are established in what is to be delivered, the lines between decisions that the business makes and the development company makes can be made clear by contract and/or convention. Conversely, when the technical team resides within the business, those lines can be a bit more blurry.
Through working closely with those in the business areas, who are intimately familiar with the working of the business, some technologists (particularly those working on applications used by the business) gain a deep understanding of the processes and value streams of the business. They can, therefore, be well informed partners in maximizing value through applying technological solutions in ways that enable efficiency, productivity, and even opening new revenue streams.


Careful and proper application of technology can help achieve these. Improper application of technology can hinder flow, agility and efficiency. It may end up costing the business dearly. Therefore, it is imperative to successful application of technology that technology experts are included in decision making, especially regarding matters involving application of technological solutions.


It may be beneficial even to have a few key technological resources involved in other decision making processes since there would be opportunities that evolve as a result of having technologically adept minds in the mix. Put another way, a certain portion of a board, committee, or other decision making body should be representative of technology (perhaps even representing different sectors of technology depending on the decisions and the business).

Saturday, October 17, 2015

Kanban, Can it Work Anywhere?

Kanban is a system in which cards are used to visualize a workflow through a value stream. Sure it finds its roots in manufacturing, and yes it has wound it's way into software development. Let's ponder for a bit whether this system would work in other contexts.


First, here is what we know about Kanban:


  • Not a process itself.
  • Supports a pull based system.
  • Enables visualization of work through a value stream.
  • Visualization enables measurement, which enables Kaizen (reducing waste).
  • Waste is downtime in the value stream.


Being a person who is either impatient, or who values efficiency - one activity that I recognize could use Kanban is the deli counter at a supermarket. I'll take a stab at applying Kanban and suggest how it might improve the process.


If you visit a supermarket deli, typically you would take a number and wait around for the next worker to help you with your order. You'll be asked what you need and they'll keep it in their head and usually have to ask a few times per order item to clarify. Most deli counters will take you a sample and ask if the thickness is ok. Perhaps they will work on one thing at a time and ask again what's next after each one until your transaction is complete.


If we introduce Kanban into this system, the first thing we need are order cards and customer cards. The orders would come in from the customer on an order card. This would contain information about each order item - item type, brand, quantity, thickness, etc. The order card would be taken by the worker one at a time and the customer would receive a corresponding card with a number on it. When the order is ready, it is delivered to the customer at the counter, the customer exchanges their numbered card for their order items.


But wait, I'm sort of getting ahead of myself here. We should not change the process just yet, but only use kanban cards to represent exchanges of information and goods in the current process. Let's try again -

First - customer grabs number kanban and waits until number is called. Once number is called, customer gives number to worker.

Worker passes customer an order kanban for the first item and a kanban for a sample. Customer gives worker first order item kanban and an order item sample kanban. Worker makes a slice of the first order item and exchanges it for the sample kanban. Now the customer either approves of the order, or declines and either changes the order or leaves.

If the customer approves (we'll look only at the happy path for sake of brevity), then the worker completes the order item. Once the item is complete, it is exchanged for the order item kanban. If the customer wants anything else, repeat.

It's easy for us to see that there are multiple opportunities for waste in this process. This is evident because the number of transitions at best are 5n + 2 where n is the number of items. For three items there would be at least 17 transitions. The number of times between transitions that the customer is idle are 2n + 1. The worker is never idle, however they are wasting a lot in transitioning.

One improvement would be to have the customer place the whole order, then pick it up later after they've had a chance to shop for other items in the store. Or the customer can have the whole order on some card with standard thicknesses and indicate whether or not they want a sample. There can be special instructions if needed. This would reduce waste by the worker and reduce lead time (where the customer is waiting).

In an even better world, where Kaizen and Just in Time is valued even more, the customer would receive their order JIT as they are checking out!  I would suggest placing the deli between the registers and entrance. Customers would stop by the Deli, get samples and place their order. When checking out, the deli items would be paid for by the customer, then delivered JIT bagged and ready to go into their cart along with other items.

This example serves to show how Kanban and other principles of TPS can be used to improve existence - no more wasting time at the deli counter. But this can be used to improve development processes as well as other IT processes. Wonder if it could improve the lead time on government initiatives?

Monday, September 28, 2015

Classify Work Items for Queue Order

http://www.pluralsight.com/courses/kanban-fundamentals
by Steve Smith
taught me about a simple classification method which uses quadrants composed of urgent and important to identify how work items should be addressed.
quadrant

Thursday, September 17, 2015

Good, Clean Code Helps With Error Resolution

Stack dumps can contain useful information about an exception. They can also be vague and misleading, especially if we have bloated classes and/or methods.


A stack trace can only go so far as to show which methods were called on which classes and in what order. Sometimes there is a line number but if not and we have a lot of lines in a specific method it can take a lot of guessing to find the culprit.


In .NET, for example, private methods are not included in the trace. Protected and up methods are. So, following good practices an improve usefulness of a stack trace.

Away With You Sprints!

Let's face it, building software is more like a marathon or a journey than it is a race. There are checkpoints, people need breaks to refuel (their brains and bodies), and working as fast as possible - all the time - will cause more defects and puts us all in a position to compromise (aka cut corners).


We're sending the wrong message with "Sprint". Let's stop calling an iteration a Sprint, and start calling it something more sensible. If we want to truly convey how we need to build software, I strongly suggest choosing appropriate metaphors in our processes. Sometimes we are on a journey or a tour or a mission.


We have a destination chosen before we leave. What way markers do we need to pass? Will we set certain checkpoints? We make a map. We have a mapmaker and a planning committee. They plan the journey and make the map. One of the committee is familiar with the terrain and can act as a guide. We have a guide. The guide may go forth on the journey or prepare the travelers before they go and guide as needed during the journey.


We have a few roles so far, and there will be more - we don't have as many people as roles, so some people will serve in many roles. They will be prepared to fulfill the duties of each role, lest the journey be a tragic one.


We move forward together as a group in a troop. We have a troop leader. The leader makes decisions that affect the group, but must leave others to learn as they go so that they too can be leaders. The leader must know how to make balanced decisions. The leader must let others lead in their own rights.


Occasionally, we need to send a scout ahead or down some path to the side to see if there is anything of value. We can huddle up and plan our next steps, then continue forward towards our destination or alter our course based on what is learned by the scout. As part of altering the course, we must consider the destination and how we will still arrive in the end.


We update the map as we learn. The navigator updates the map. It is helpful if the navigator is the scout and the mapmaker but not necessary.


We have a journal and a handbook. The journal is used to track daily activities and for lessons learned. The journal is updated daily. The leader updates the journal others may contribute. Each member of the group is encouraged to keep their own journal.


The handbook is a set of guides, principles, and best practices that we consult when confronted by a situation or an obstruction. It is created by or is a compendium of works by those who have experience in navigating many types of terrain. We use it every day, we learn from it, we train with it, every member must have a handbook and know its contents. It describes how to organize, how to plan, what to watch for, how to best deal with most situations. It is the most valuable commodity the troop can own.


The handbook is updated after each journey, or as needed. The handbook has information about how to update it. The handbook may be a subset of a full guide, in this case it would be a field guide. The full guide may be consulted during planning activities and the field guide/handbook during journeys.


The guide books may serve to help define and answer some of the most important questions. What criteria do we use to know if we need to abort the mission early? What hazards might we face? What if we come across an obstruction, can we pull together to clear it or do we need to call in a ranger? If we lose our way, we will need a ranger to help us back on track. Who is the ranger, who are the scouts, who has the map? Who does what? How do we...


There are rangers. There are different types of rangers. There are forest rangers, they are others within the organization who keep the forest healthy and the paths clear so that we may navigate. Occasionally, we may need to call in a special ranger. Special rangers are from outside the organization, they are called in to solve a specific problem that cannot be resolved by a forest ranger.


Training is second to only the guide book. We have training sessions, training missions, should we have badges? Each member must have the appropriate training before taking on a role. Each role must have its training requirements defined. Some journeys may be too hazardous for some members of the pack. The journey should be appropriate for the member.


 We have principles that must be learned, those principles are sensible and important to success, we proudly learn and teach those because they are integral to our way - it is an honor.


We stick together, we help each other, we help others, we uphold our principles,...

Friday, September 11, 2015

How to Load Text into Html from a File

When creating a "static" page with some informational text, it may be helpful to enable admins to replace the text without editing the main html page.


This can be done by using an Ajax request to load a text file or preformatted html file into the innerHTML of an element.


JQuery.ajax can be used to fetch the file then write the contents to the page.



$.ajax({
    url:'textFile.html',
    type:'html',
    success:function(data){
        $('#elementId')[0].innerHTML = '<pre>'+ data + '</pre>';
    }
});


with this approach, the textFile can be marked up with plain html tags and it will work without the need to escape them.


One thing to note about using this approach is that the file should be available via the web server and at a path relative to the host site, this would avoid additional configurations - just let the same web server serve up the textFile.


Basically, in this way you can have a wrapper and put the content into the wrapper however. Sort of a way to do an SPA.


I'd you'd like, set the pre CSS to wrap the lines. As an alternate to appending pre tags, you could style the parent element.


More information about that -
http://www.w3schools.com/cssref/pr_text_white-space.asp

Friday, September 4, 2015

What is Void?

We cannot define void in terms of what it is, only in terms of what it is not. Void is that which is not everything else. If everything else exists, then so must void. Void and everything else are like the Yin and the Yang. Void is not a number, it is not a char, it is not the air or the sun, it is not made up of the stuff that makes everything else, however void exists because know it. Perhaps it only exists in our minds.

Speaking of void and everything else, what is everything else really? Is it that which we see, hear, taste, experience? Or that which we can experience? We see it through a small window as it flashes past. Or perhaps we can think of a computer screen on which everything scrolls by. We cannot control the scrolling, only what we choose to witness. The void is there all along, it is where we do not see.

Friday, August 28, 2015

Battle of the Languages

One great thing about the challenges offered on CodeEval.com is that they can be solved in so many languages - e.g. choose the language of your choice and go at it! As a side benefit, one can easily compare run-times and memory footprints of languages by solving a challenge in many languages and viewing the results. As an experiment I did just that with the following challenge.

Here is the description of the challenge, as it can be seen on the site:



And the basic solution, which I implemented using the same basic algo in five languages (JavaScript, Java, PHP, C#, and C).

The test data is supplied via a file passed to args during execution, the same file is used in every execution run. JavaScript is run on Node. Execution takes place server side on a Linux based server (more details on the CodeEval site).

Basic algo is: for each line, split line on comma, parse each val to an int, run computation, print results to standard output.



Drum roll please....









As expected, C has the smallest mem usage and the fastest run-time, and by far at that. But I didn't expect Java to be that bad. Granted this is a trivial example. It would be interesting to see if the gap closes when the C program needs more supporting code. Educated guess is yes but not by much. The Java code for is below, I wonder if I'm doing anything wrong here or if it really is that less efficient in all ways.


import java.io.*;
public class Main {
    public static void main (String[] args) throws IOException {
        File file = new File(args[0]);
        BufferedReader buffer = new BufferedReader(new FileReader(file));
        String line;
        while ((line = buffer.readLine()) != null) {
            line = line.trim();
            String[] vals = line.split(",");
            int x = Integer.parseInt(vals[0]);
            int n = Integer.parseInt(vals[1]);
            compute(x,n);
        }
    }
    
    static void compute(int x, int n)
    {
        int m = n;
        while(m<x){
            m+=n;
        }
        System.out.print(m+"\n");
    }
}

I plan on doing this same test in Ruby, Python, and GO to do an even broader comparison, when I have some time. Results will follow if I do.

Thursday, August 27, 2015

EF natural keys and duplicated properties

While working with a VS solution that has been around for about 6 years, an issue was brought to my attention regarding duplicate records from a query. The solution uses and old version of Entity Framework and ties the results of an imported stored procedure to a web forms control (data grid, repeater, or similar). The entity type is based on and mapped to the returned data table from the stored procedure.


A more concrete example would be a Teacher entity with three properties [Name, Classroom, School].


The results of the stored procedure represented below:


June Ward, 102, James Woods High School;
Fiedora Vilma, 233, MLK High School;
Fiedora Vilma, 126, MLK High School;


Were showing up in the app as:


June Ward, 102, James Woods High School;
Fiedora Vilma, 233, MLK High School;
Fiedora Vilma, 233, MLK High School;


Notice that 233 is repeated for the last 2 rows.


Upon investigation of the Teacher entity via the model designer, I saw that only the FullName property was designated as the entity key. But it seems as if the natural key in this set would need to be the FullName and ClassroomNumber.


> Highlight FullName, properties, set entity key false.
> Highlight FullName AND Classroom, properties, entity key true.


now the combo key is shown. Rebuild solution, check to see and the classrooms are no longer repeated.


Wonder if this was "fixed" in some later version of either EF or ASP.NET. Anyways, the solution is fixed for now.

Wednesday, August 19, 2015

Here's a Challenge!

If you haven't done so already, join Code Eval https://www.codeeval.com
it's a great way to brush up or learn a new language. There are challenges which can be solved using the language of your choice. Its kind of like a game, you earn points and get ranked. There are sponsored challenges and potential for finding jobs. When you do this, find me "philn5d".


There's another site called Top Coder with challenges. This one has payouts for sponsored challenges, but also pure algo challenges. The sponsored challenges are companies paying to have some work POCd or perhaps actual implementations. Given that, the payouts are actually really low considering what they would pay a contractor and you'd have to place top 2 or 3 for a payout. But, there are really interesting challenges, most of which are technology constrained. Give it a try, you may like it and may earn a few dollars in the meantime it is free to join. http://www.topcoder.com/
I'm on there too.

For a Good RDRR try LOLCODE

Seriously, if this isn't old news to you, "go to bing and type g o o g l e then execute the search. Click the link that takes you directly to google.com and search for" LOLCODE. Wikipedia has all the dirt.


If, for some strange reason, you would like to drive yourself mad for a bit - go to http://www.tutorialspoint.com/codingground.htm
this is the best thing since mouse pads. Seriously, they have TONS of online editors that really don't suck too badly. I've found quirks in some execution engines, but really this is a great way to learn or POC from anywhere!


Btw, the quote is modified from the famous "how I met your girlfriend" video.

msdn xand something exciting

First, I'd like to gripe for just a minute. I've been receiving MSDN magazine in the mail for a few years now due to it's part of the subscription package that my current place of employment maintains. Cool. I get it. Here's our latest thing we are doing and how to use it. My gripe is that it's more or less the same thing now - here's how to do some basic shit using Azure. Use Azure. Here's some awesome pattern, and btw here's how to use it on Azure. It's basically a monthly sales pitch for Azure. Kudos for the advertising tactics, MS is really going hard into this one and it makes sense. It's just getting a bit old is all. I've demoed Azure and its not bad (when it isn't completely frustrating due to something being down or taking forever or not responding). Oh I generally do the free thing or the low-cost option, and yeah its lower in the priority queues for processing. But hey, its my experience so far - not making me want to fully commit though.


^TLDR




Then there's at least one article that is just pure and clean. This month's is about 3D Web Games using JavaScript (Three.js). Awesome! I've had it from other sources that a guy in the Netherlands or thereabouts (Anders Evenrud) built A FUNCTIONING OS IN JAVASCRIPT! So... Anyways check that out at http://os.js.org/
and contemplate that for a bit. And he even has a badass name! This guy is my hero.

Thursday, August 6, 2015

What is the Future of Software Development

With advances like TDD, BDD, IBM's Watson, and Microsoft Cortana what will the future hold for software development?


Today, we can write automated tests to validate design and implementation. If we're doing this in an optimal way, then we are writing the automated tests to specify the system. What if the next level is to write only the tests and not the code? What would that look like?


I would imaging that the tests would have to be written very specifically to define every aspect of the system including environmental considerations. It would certainly take the saying "the tests are the requirements, the requirements are the tests" into another form. Something like "the requirements are the system, the system is the requirements."


This would require some clever sort of meta-programming model that could take a ubiquitous definition of the the system defined by example and translate it into whatever language. Some sort of translator/code generator/compiler would take the BDD style requirements and, ultimately, turn them into runnable applications.


First, the spec reader would need to parse the requirements into a standard baseline. With that, any BDD style format could be used as input, given the proper translations.


Next, something would need to analyze the baseline and decide how to efficiently process each set of rules. Perhaps some decisions would be made with respect to which language to use to maximize system efficiency and conform to the defined standards.


In order to meet those goals, certain meta-information would be needed about the system itself and about each requirement/feature. For example, one may define the security requirements of the system as a whole, or one particular feature or function. The data retention policy would be another aspect of the system. How about accessibility or internationalization? How about defining the environment constraints? Will there be a multiple environments? Or only one? Will it need to scale out?


Obvious complexities aside, I believe we have the technology and the know-how necessary to build such a system today. However, I also believe such a system designed this way would be fraught with the same problem that today's systems have and that's about the requirements themselves.


Let's assume, for sake of retaining optimism, that we provide accurate statements about every aspect of the system - what it runs on, how it will be available, what data it will use and persist, what transactions, how many users, which locations are the users, what roles, features, integrations. We define every aspect in our acceptance tests. Then those are translated, processed, and some machine or IL code is produced as a result - it's installed or deployed and we're up an running without writing a single line of code. We've only written pre-conditions, actions, and expected results. For a Given Condition, When some Action, Then some Result.


What would the first requirement look like? Would we define system constraints first? Would we define outputs first? Inputs? When would those be compiled? We wouldn't exactly have code to verify with the tests, so perhaps it would be real time, but the analysis process would take so much time especially at first.


Let's assume we would define the environment first. Given a growing population of users, And the users are geographically distributed, When users need to use the system, Then they would use a web browser. But should we even need to define that? Perhaps the compiler would know which platform would be best based on how we define the users, so we'd start there.


Given a globally distributed population of users, When the users access the application, Then they should be able to access the application from any location. With this spec in mind, the system may decide that a distributed system with multiple types of clients would be the best design. There may be decisions to make about caching on clients due to lack of specification about internet connectivity, so the only design must support with and without internet. Perhaps the compiler would need to make certain assumptions or respond with prompts. Perhaps certain parts of the design are better served as wizard-like or key-value pair configurations.


How about data, how would data be defined? In Cucumber BDD, input data is defined in either a Given statement or a Setup for a feature. But multiple features may share the same entities, so we wouldn't want to have too many definitions of them. Perhaps some tag could be used and all those related bits of data can be correlated by that.


In any case we can see that this is possible, but the details are fairly complex. It will be slow and cumbersome at first, but in time will be the usual way of building systems.

Monday, July 20, 2015

WCF Extensibility - message call-chain tracing

I'm nearing closer to implementing a correlation id through an entire WCF call chain. It was working for single threaded calls. Then my parallel calls test executed and the result was not what I needed.


Here's what I mean by all of this:


Suppose we have three services - Sa, Sb, and Sc. And a single client - Ca.


When Ca calls Sa and Sa calls Sb and Sc in the same call, then responds to Ca I want to be able to trace all of those related calls in a call log so that I know which calls contributed to an event (perhaps an error or a specific data update).


Currently, we have all calls being logged in a database via a custom operation invoker which implements IOperationInvoker. That implementation sends a one-way call to a singleton logger service which logs the pre- and post-invoke including operation, call time, who called, etc.


I'm adding messageId from the IncomingMessage built in header property "MessageId" (only the Guid part though). I'm also adding a CorrelationId to the header and this is where I'm currently sort of in the weeds.


CorrelationId is a custom header value that will be logged to the call log for each call in all services. For all calls related to a single origin call, it will have the same value. For each origin call, it will have a different value. An origin call is a call to a service which begins a call-chain. It would originate from outside the system (collection of services), typically from a UI or a service outside the system.


The service context that handles the first incoming call would need to create the Guid before the call is logged. It would need to be passed via a message header before the call is made to the service. Each service would need a way to know if it is the first service in the call chain. The absence of the correlation id in the header should suffice.


Implementing an IClientMessageInspector should be perfect for this. The BeforeSendRequest method has the message and adding the header value is trivial.


I created a correlationId member on the class that implements this interface. The only thing left is initializing this Guid when the CorrelationId messgaeheader is missing and add it to the headers. Hopefully that will wrap up the implementation. Then its just a matter of adding the extension to each service.

Various Types of Fun!


I've been busy lately, busy reading up on MS SQL Server query optimizations, WCF header message ids for logging call chains, and Disciplined Agile Delivery.


Here are a few nuggets about each topic:


SQL - I got here because I was looking up Hash Searching, http://blogs.msdn.com/b/craigfr/archive/tags/Joins/
Craig has some really deep explanations about all sorts of SQL Server related topics. If you really want to improve your SQL chops, you gotta read his stuff. The blog is written in a "we" context, because he worked on SQL Server at the time - got it? A quick search of the net shows his imprint.


WCF - when you have a service oriented architecture or a polyglot architecture which includes services (micro-services, queues, whatever) you will want a running log of the calls. Then you will want to correlate calls within a call chain and you will want to know what happened and when. Fortunately, if you are using WCF as your message-based framework of choice, you can use the message headers to pass this information from call to call. Additionally, you can do this in a AOP type of way so you don't have to alter every contract to add traceability. Here's a link with some of that info http://www.brendanwhelan.net/2013/wcf-custom-authentication-headers


DAD - no it's not about dads, its about process for software delivery. More specifically, its a framework to guide design of a process for software delivery. Or just guess and hope you are right... Here's the entry point for making better choices http://www.disciplinedagiledelivery.com/start-here/


But wait, there's more...


Mel Conway - after catching a glimpse of Conway's Law (how system designs will follow the same patterns as organizational communications), I delved deeper into Mel's experiences. It turns out that he has a lot to say on the public education system. After teaching 10th grade geometry in one of the poorest schools in Massachusetts, he shares his thoughts and experiences here http://melconway.com/Home/Home.html


IBM's cloud - after joining TopCoder (which is a thing where coders sign up for challenges and build real software for prizes) I started receiving newsletters. One newsletter lead me to Watson (IBM's famous Jeopardy champion) Services. Apparently even Watson is on the cloud. Check it out http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/visual-recognition.html
I'd really like to dig in on that someday.


I finally installed a Linux based OS - Ubuntu. I did this on a VM running on Hyper-V on Windows Server 2012. I know, right?!? Anyways, I did it this way because I wanted to explore and it seems like the best way to experiment. Oh, first I tried this in the Azure cloud on 2 VMs - one with Ubuntu another with the Oracle's RedHat clone. I'd really like to do more with these (MEAN stack).

Tuesday, July 14, 2015

Long Post: Thoughts on Conway's Law


After pondering Conway's Law I've come to think of it like this:


A typical organization will have departments that specialize in certain activities. There may be a marketing department, a sales department, a finance department, another for human resources, and so on. The natural design of any system (a set of processes and workflows that are systematic and repeatable whether automated or not) will be organized along the same structure where communications occurs between departments.


In a typical IT system, where the system is defined as all of the software which communicates with other software by design, the programs/sub-systems will typically be organized by the same functional areas of specialization. It takes some specialized work to organize the system in a way that is more efficient.


The human brain, for example, is organized in a way that areas have functional specializations. In the brain there are areas that work together to solve math problems, other areas work together to understand and produce speech, there are areas specializing in regulation of the body. It seems that the lower-level functioning of our brain, that which is concerned with keeping the body in a state of living, are specialized in a more specific sense, while higher-order thinking is comprised of areas composed dynamically for some general purposes.


Math problems are so varied and dynamic that it may take the stitching together of more parts of our brain to solve some than it would for others. We commit some math problems to memory - what's 2 + 2? Most of us don't even have to think about it because we've created an area that specializes in 2 + 2 long ago and it has solidified in our mind. How about divining a proof for the quadratic equation? Prove the existence of the Higgs Boson? Those require a special organization of areas of our brain to produce the result (unless we are math experts of course).


Imagine if our business organizations were aligned this way...there might be a math department which performs all math functions, an email department which is responsible for writing all emails, a bad news department...well, you get the picture. This sort of organizational construct would not serve well in an environment where people have to communicate across those lines of specialization. Human communications are relatively inefficient when compared to communications between computers or between nerve synapses.


Given that human communication is the main inefficiency between people, we organize into specializations based on higher functioning areas of business instead of specific functional needs. Our communications protocols are based on the human language and are transmitted via blog, email, memos, telephones, texts, etc. We need contextual understanding in order to decipher the messages, which have specialized meanings in specific contexts. Our communications are Klumsy, Lame, Ugly, Dumb, But Good Enough and barely at that! If not for poor communications the world wouldn't need therapists!


What is the solution to removing the weak links in the chain? Automate, of course. We made calculators, and now no one needs to learn basic math or consult a math specialist through our various means of poor communications. Instead we each have a math wizard in our pockets and under our fingertips at all times. What's more is that these math wizards can use their mathematical prowess to present and communicate information (and mis-information) to humans at near-light speed.


Our ability to communicate as effectively in the whole pales in comparison with the ability of computers to communicate with each other. What's more, is that computers have software. With software, we can teach a computer how to understand nearly any communication that it can receive by simply installing software. For humans to do this would require years of specialized training for each individual. Computers require specific training by a few individuals who produce the software. It may take years to produce, but the outcome is that each computer takes minutes to learn how to communicate in this specialized way.


Thus far, I have been setting the stage for an argument for a concerted effort toward the design of IT systems that seeks to maximize efficiency in ways that humans cannot. Given that communications between contextual boundaries is the main barrier to human efficiencies, when one seeks to improve efficiency, then the way to produce maximum gains is to address the barriers between contexts.


Since computers can specialize in low level functioning and do not suffer the same communications barriers that people do, systems comprised of computer software can be organized in a way that humans cannot. They can be organized in a way that maximizes efficiencies. Be warned, however, that there are different types of efficiencies when it comes to software; some of which apply to maintainability and development. For sake of argument, lets call these indirect efficiencies. Processing efficiencies are one direct efficiency that comes to mind in regards to software. When some work is performed, how quickly will the system arrive at the result? This depends on a number of things including the architecture (organization of the system), the hardware in the system, and the software design. Two of these can be easy to change, one cannot. Hardware is frequently upgraded and so is software. The single most difficult change in a system is the architecture - how the pieces are organized. In fact, the ability to upgrade software and sometimes hardware are directly impacted by the architecture of the system.


An architecture can be defined in terms of the coupling of its components - a system can be tightly coupled or loosely coupled. A tightly coupled system is one in which each of the components depend on one another in a way that makes it difficult to change any of the functional areas without affecting one or more others. Loose coupling is when the functional areas of a system can be changed at will without much impact to other functional components. So far, the descriptions of coupling are arbitrary. To quantify, lets consider the following: If a system has n functional components (logical chunks of work that it performs), and each component is coupled to x other components such that a change to one will affect another in a way that causes the other to change in order to provide the same functional work, then the coupling can be quantified as the sum of all couplings divided by the number of components


SUM(x0,...,xn)/n


we would strive for 0 so that any component can be exchanged or even moved to different hardware so that the overall efficiency of the system can be improved.


Another nuance in meaning of efficiency of a system is tied to how it improves efficiency of the work being done - be that assisting humans in performing their own tasks more efficiently, or by automating the tasks. Some tasks are automatable and some require the elasticity of the human brain and its ability to compose its pieces in order to solve a problem. This is the way we should view IT systems as well, except that the barrier in being able to do this as efficiently as humans is human communications.


Most computer software requires humans to translate some thought into a computer language. This effect is called programming and requires a specialized skill that takes years to learn and master. It requires human interactions in order to understand the problem. It suffers the same inefficiencies as other human interactions between groups of specialists. It takes time. Human brains can pull their pieces together very quickly, while it takes much longer to tell a computer how to solve a new problem.


Because it does take quite a bit of effort to tell a computer how to solve a problem, it is necessary to build the basic low level components that can be pulled together to solve some specific problem in an efficient way. The problem solving components will be higher level components in the system. All components will need to communicate efficiently. Each component should be able to change in implementation or in location with minimal impact on other components. If an architecture supports these goals, it can be changed to make the pieces more efficient.

Monday, July 6, 2015

Learn Web Development

Starting with a list of essentials, here is a guide to starting out with web development:


HTTP verbs
HTML
Client-Server model
CSS
JavaScript
JQuery
OWASP Top 10
1 or more Server Side languages/platforms (PHP, Ruby, nodejs, Go, etc)
RESTful services, how to build and interact with
SOLID, OOP (Java tutorial, GoF design patterns)
TDD
SQL
at least 1 noSQL
some html templating language (mustache, handlebars, underscorejs, etc)
mvc/mvvm patterns and some SPA implementation of this (Angularjs, Emberjs, Knockoutjs)


I would suggest reading this entire article then diving into the links getting an overview of each topic, then diving in a bit deeper. If you get to a point where you feel like you are blocked from understanding a topic, take a step back and dive deeper into a previous link. I tried my best to list them in order of dependencies - HTML, CSS, and JavaScript being first and security deep dive requires an understanding of so many topics that it appears later.






A good place to start is http://www.w3schools.com/
that site has tutorials with an option to try each concept as you go. It's not a fully interactive tutorial, which is great because it also serves as a good point of reference.


Of course, the real heart of the web that we all know is based on HTTP. Here's a great way to understand the model. http://stackoverflow.com/questions/2001773/understanding-rest-verbs-error-codes-and-authentication
I once read a fantastic article about HTTP, but can't seem to locate it. That post on stack overflow has some good info. https://en.m.wikipedia.org/wiki/Client%E2%80%93server_model is sufficient for now to intro client-server.


StackOverflow is one of the best community-driven resources on the web. I'll say no more.




Everyone should know about the World Wide Web Consortium(W3C) http://www.w3.org/, they set and publish standards such as the html spec, JavaScript spec, CSS spec ao if you really want to learn the standards that's where to go.


There are some great tools available for editing html and other types of code. I've found that Notepad++ works well for several languages. It's lightweight, and works for a variety of languages including semantic markups. https://notepad-plus-plus.org/

it is free and "easy to use".


All of the most popular web browsers have debuggers built in for debugging JavaScript. Usually pressing f12 gets you there. Debugging is a sort of dark art that takes time to do well. It takes practice. All of that being said, anyone can start debugging on almost any full blown browser. Look for a future post on debugging tips and tricks.


Client-side debugging is one thing, but when you own the server-side code, you will need to be able to debug that too. In that case you will need an editor capable of debugging. Without going too much detail, this is basically where the editor is hooked into the server-side hosting (typically on your local machine, not a remote host) and any breakpoint you set in code will result in halted execution. You will be able to control the progress of execution in a number of ways - continue, step-through, step-into. While debugging, you can examine flow of control and variables that are in scope. A common free tool for Java is Eclipse https://eclipse.org/ide/. This IDE(Integrated Development Environment) also functions for C/C++ and PHP. Definitely put this on your dev box and learn this tool. Also, while you are at it, learn Object Oriented Programming - perhaps by way of Java. PHP has an object model, so does C++, JavaScript, and most other common languages.


What's Object Oriented Programming all about? Check it out on this Java tutorial https://docs.oracle.com/javase/tutorial/java/concepts/object.html
I cut to the chase, feel free to navigate back to the root and read the intro. Vest in Java or don't, but learn what they have to teach.


OWASP is a community focused on web security. They publish a wealth of information about how to protect your applications from being hacked - mitigate risk rather. You can find the top 10 threats and a wealth of other info listed on their site https://www.owasp.org/index.php/Main_Page


PluralSight is a hugely influential training site that has videos covering a range of IT related topics. They have some free videos, but a monthly membership gets you open access to all. This is a great way for more experienced developers to grow their knowledge base.

Tuesday, June 23, 2015

Highlighting in Visual Studio

I found my old friend Lime Green Highlighting again in Visual Studio today. I've been working with some pretty hairy code and found that Lime Green really helps when finding the end brace of a 4000 line if statement...ho boy.




Tools > Options > Environment > Fonts and Colors > ....


From here you get a dialog with a variety of options for changing colors and fonts (Captain Obvious). I use Brace Highlighting (the one with a background setting) and change the background color to Lime Green. I can really tell where the statement begins and ends with that one! Especially since I have a slightly grey bg to save my eyes.


Here's the weak reference from MSDN
https://msdn.microsoft.com/en-us/library/ms165337.aspx

Monday, June 22, 2015

When NOT to use a noSQL

...or more precisely, when not to use an eventually consistent concurrency model.


I've been reading a lot lately about trying to fit a square peg into a round hole. What I mean by that is they are trying to force noSQL databases, particularly ones with an eventual consistency model, into a transactional operation.


I've been reading about two step patterns and other types of hacks and second generation noSQL with built-in transactions. I want to just state an obvious point - how about don't use eventual consistency when you need a transaction. When you need a transaction, use a model that supports one - its plain and simple! When you don't need a trans and you need a dynamic model and/or high throughput use noSQL.


It seems to me that some architects are not quite understanding how to pick the right tool for the job. It can cost a fortune to make a mistake as epic as allowing "sophisticated users" to post multiple concurrent requests to transfer assets and have them all go through due to lack of single concurrency on an entity. Search the web for bit-coin heist or similar and you will find out how many millions of dollars it can cost.




I've had a concurrency issue manifest at a very small scale, one that did not involve transfer of assets and did not cost a fortune. It did introduce a seemingly random defect and it was not a noSQL backing store.




 I admittedly did a read-then-insert-or-update without wrapping the entire op in a transaction. I did this to simplify the API so that a Save method was exposed instead of separate insert and update statements. The client makes calls asynchronously and perhaps a user clicked five time or there was some other technical issue which resulted in five records being added to the collection. This illustration further illustrates the need for transactions (and for properly using them).


So why not consider polyglot persistence and apply the right persistence tool for the job? There is a lot of good information on how to design a system that can take advantage of different backing stores and completely abstract the data store from the application. With such a design, your system can use transactions when it needs to and eventual consistency and distributed data when it needs it.



Thursday, June 18, 2015

A Quick Funny Thing

I've been letting my hair grow just a bit lately. It's about 10-13 cm now. At a gathering today someone asked me if my hair was new. I, of course, said "only the hair closest to the scalp." Another person in the group kindly reminded her that I am a software developer.

Wednesday, June 17, 2015

BDD - Tagging Features in SpecFlow

I've been working with SpecFlow for a couple months now on one passion project and I'm starting to get a handle on how to manage features and how to use tags for better organization.


Without further ado, I'll dive into an example:


Suppose you have a requirement with some business rules to determine if a project can start. These rules determine canStart, a boolean. There's another feature that sends some notifications on the project's scheduled start date. The recipients and the contents of the email are to be determined by project.canStart? value. Other logo is also driven by canStart including UI elements and some unicorns and trolls.


How can we organize these features and still keep track of every feature that uses canStart?


Firstly, whatever rules determine can start should be defined in its own feature since that sounds like it could be really complex logic that is highly volatile.


That feature could be tagged as @canStart, @businessLigic, @volatile, whatever else.


The notify start date feature would be in it's own .feature file. It would have several scenarios each tagged with several tags including @canStart since that is a dependency for the notifications.


When you need to change some logic that depends on @canStart, you can just do a tag search and voila! You now know all the dependencies!

Tuesday, June 16, 2015

Scoping a Project - To Size Right, Start at the End

Today's lesson learned is about how to scope a project under resource constraints in order to keep it as lean and beneficial as possible. This is largely about determining the minimum viable product (MVP).


The MVP of software is not the person who writes the most SLOC, or has the lowest defect defect density. It's certainly not the person who stayed late for weeks to deliver a feature. To the contrary! The MVP of software is the one who, through Socratic Methods or whatever it takes, gets the size of the product whittled down to the core. To the minimum deliverable product without any wasted efforts; with only useful features; with the lowest cost, highest quality, and most business value. (see this previous post for more about the importance of quality)


Joking aside, here is some seriousness about MVP - Minimum Viable Product (we really need a better TLA for that).


Identifying the MVP is critical to keeping down costs and/or impact to other efforts. It will result in earlier delivery of value, higher quality due to narrowed focus, and decreased initial efforts. All of these attributes are linked in a tight Tango dance.


In order to achieve the MVP we start at the end game. What is the most important output of the project? Which feature or set of features delivers the most value and how can that be delivered first? Gold plating is a plague on software, we've all seen it in those features we never use, some of us ;) have even been guilty of adding gold plating ourselves (I confess).


So, raise your right hand, place your left hand on the screen, and repeat after me:


From this point in time and into the future, <state your name> will be sure to ask - "what is the one most important deliverable?", "What can be deferred?", "Is this feature feasible from a business perspective (cam the business support it)?" I will refuse to accept "all of it" as an answer unless backed by sound reason. I will not add features which I deem "useful to the users", nor will I suggest such features get added without proper analysis. If they become necessary, they will be implemented later. I will determine the end game before I start.


This is not an easy feat, politics will likely be at play. The boss's offspring, a valuable client, a controlling interest may have something grand and glorious in their heads. They may insist. They may not be easily swayed. This is why the Most Valuable Player is the one who can keep things under control and still be a hero in everyone's eyes.

Sunday, June 14, 2015

Security Feature or Imposition?

I'm torn. I can't decide. In my experience it hasn't worked the way it is supposed to. At least 3 times I've been blocked from using my debit at a checkout due to "fraud protection". It's frustrating, it's embarrassing, and it's infuriating!


Yet there was a time when my credit card number was stolen and duplicated. The duplicated card was being used all over a city I've never been to. The bastards were running all over making small purchases. I noticed this and reported the issue. They didn't catch it, I did.


This last time I was prohibited from accessing my funds, I was at a grocery/everything store that I frequent. I'm there once/week or perhaps every other. We've been making some extra purchases lately - birthdays, spring cleaning, etc. And so the bill is a bit higher, bit not much.


I actually have had no say in this "protection" they just block a purchase, inconvenience everyone around, then call and demand my identity and annoyingly have me verify purchases or something else. This last time I got a text at 1am while I was asleep - long after the attempted use. I replied when I was awake next day and saw the text. Why did it take 5 hours to get the text?


I've had several frustrating phone calls with the fraud people. I'm almost ready to call it quits and use cash. Is it worth the inconvenience for some added possibility of security? Is this really a feature or an imposition on how I can use my money? Even if it is making things more secure, is it worth the cost? It would be great if I had some measure of control over this "fraud protection", then I might be willing to call it a feature. Right now its just some other folks sitting around looking at data deciding how and when they can block my purchases. We as humans can do better, I'm sure.


Looks like I'll be changing banks soon at least and try my luck somewhere else.

Thursday, June 11, 2015

Can You Name 3 Threats on the OWASP Top 10?

I've been asking this question and not getting good results. I've asked software developers who are making apps. I've asked students. It seems that security awareness just isn't on the agenda.




A security engineer commented on this to me today and gave me some insight. Basically its either you like to know how things work and you eventually are interested in SQL Injection, XSS, security mis-configuration etc or not. If you are, you're likely to be very interested in checking out the OWASP site.

Tuesday, June 9, 2015

Builder Pattern - a Real Life Example

Here is a real world example of using a pattern to set up dependencies that need user input in order to function.


Given an application that contacts a server for state information. Lets use a console interface for this example.


The app consumes an interface that defines a method to get server state. The user interface layer should know nothing about the implementation details, but needs to pass the server name to the implementation. We want to use the same instance per each server we interface with. If we connect to five servers, we should have five objects.


Those objects can have different methods and for efficiency, they should be reusable. Each instance has internal state about the server name, which is an implementation detail, and can only be used for that server (think multi-threading).


We should avoid constructing the object in the user interface code because it would know too much detail about the implementation and therefore would be too tightly coupled to the specific implementation details. Additionally, we would not be able to pass a mock object for testing purposes so we need some way to pass the information to the code that processes the user request.


Instead of passing that we can define an interface that build the dependency based on user provided data. The Build method accepts a server name, constructs the server object passing the name to the constructor, and returns the server instance to the caller.


The caller uses the returned object that implements the server state interface to retrieve information about the server it knows about.


In C# code form:



interface IServerInformationReader
{
    Services GetServices();
}
interface IServerInformationLogic
{
    bool IsServiceRunning(string serviceName);
}
interface IServerInformationLogicBuilder
{
    IServerInformationLogic BuildReaderForServer(string serverName);
}

namespace N5D.ServerManagement.Builders
{
    class ServerInformationBuilder : IServerInformationLogicBuilder
    {
        public IServerInformationLogic BuildReaderForServer(string serverName)
        {
            return new ServerInfoLogic(new ServerInfoReader(serverName));
        }
    }
}

namespace N5D.ServerManagement.Tests.Builders
{
    class ServerInformationFakeBuilder : IServerInformationLogicBuilder
    {
        public IServerInformationLogic Build(string serverName)
        {
            return new ServerInfoLogic(new ServerInfoReaderFake(serverName));

        }
    }
}


This allows us to create the object in multiple ways depending on our use and the implementation details. Those will not be known to the consuming code but will be abstracted by the builder.

class ApplicationCode
{
    IServerInformationLogicBuilder _builder;
    IServerInformationLogic _serverInfo;

    ApplicationLogic(IServerInformationLogicBuilder builder)
    {
        _builder = builder;
    }

    void DefineServer()
    {
        string serverName = GetServerNameFromUserInput();
        serverInfo = _builder.Build(serverName);
        
    }

    void IsServiceRunning(string serviceName)
    {
        Console.WriteLine(_serverInfo.IsServiceRunning(serviceName));
    }

}




This enables us to pass a different builder to the app code depending on whether we are testing or writing production code. The component that fetches the info from the server has that sole purpose while the component that computes IsServiceRunning is a logic detail. In this way we can return known data with a fake reader and test the logic.


The logic in this example is clearly simple, but illustrates a situation when a builder is appropriate. The builder can be injected with some IoC container in each context. The reader must be made at runtime. That's when you need a builder.

Monday, June 8, 2015

Measuring Success

A team works for months on a software project which is nearing completion. The PM is pushing toward the goal of getting the beast into the wild. Finally, the day has come...it's up and an all out celebration ensues. The PM declares the project a success and everyone does shots and pats each other on the back.




Fast-forward 2 years. The product needs some changes. Several Bugs need patching, some features need modification, and some new functionality is desperately needed in order to make the product live up to its promise.


The core team is gone. The PM has moved on with the momentum of a successful project propelling him/her upward at another company. No one asked the questions about the long outlook and now its here staring the current team starkly in the face.


From an ROI perspective, this product is a complete failure since it was projected to return costs in 18 months and instead has costed more in terms of construction AND ongoing costs. Plus there were indirect costs that not a single person measured.


This fictional scenario may or may not sound familiar to you or someone you know. If so, try the following...take deep breaths and count to 10 slowly. Feel better? If not recurse, else read on.


Lets consider a few parameters of a project that is rationally based on cost savings (as opposed to increasing income). What parameters might one expect? Lets put those into some sort of cost-benefit model:


costs: Building the Beast, Maintaining the Beast, Running the Beast, User Impact, Morale Impact


benefits: Lower Insurance Premiums, Lower Risks, Increased Productivity, Staff Reduction


Some of these properties are one-time such as build costs; others are recurring like licensing, user impact, and lower premiums; some are readily calculated, others not so much (looking at you morale). For the not so quantifiable properties, lets consider a fuzz factor that modifies a cost value. Costs are negative numbers, benefits are positive numbers. Fuzz is +/-. For estimation make the numbers relative.


Building app : -40, Fuzz 10. Range -30 to -50.
Increased Productivity: 3/month. 2. 1 to 5.
etc...


I won't bore the reader with details here, rather leave those up to the reader as an exercise. In the end, compute the median, best and worst case scenarios once fuzz factors are applied to all parameters of one-time and ongoing costs and benefits. Take the high range of costs and low range of benefits for worst case and vice-versa for best case.


Once all the ranges are established we can see that an ROI in the worst case can never be achieved due to negative ongoing impacts. The main impact drivers of long-term costs are quality related. The important qualities that affect long-term costs are user experience, technical design, and technical debt. Some costs are hard to measure but easy to explain.


Suppose users of a system have had regular interactions with other people, but as a course of automation those interactions have been reduced significantly. There is certainly a benefit to having face-to-face interactions with certain people. Take the automated phone tree as a prime example.


While I'm certain automation has reduced direct costs of having staff to answer phone calls, some of the best companies have turned back towards having real people on the other side in order to elevate brand experience. Why? It increases brand value...sure there is a cost, but there is also a cost of automation - frustrated customers who feel no loyalty. We've all been there.


How much loss has there been as a result of automated phone trees? Fuzzy right? Lets say worst case you lose 10/yr at a cost of 300 for implementation. Your ROI is never achieved and in fact returns are more negative over time as a result of loss which is more costly ground to recover than customer retention. Short-term gain, long-term loss. Fuzz factor - 5/yr. Range 5 to 15. What if loss is 15 due to implementing automation? That a rapid fail!


Now lets say 2 employees leave as a direct result of product support frustrations. Now you have loss of productivity and cost of replacement which is considerable collateral damage. Not only that, but lets say other projects are directly impacted as a result of resource drain to support the fruits of the project. Would you still deem this project a success after it has been truly tested in the wild? Now the shots become draws to sleep at night.


Lets reconsider the project parameters...


Long-term success should be defined up front and measured throughout the life of the product. The recurring parameters are the most important factors throughout the life of the product. Consider the recent air-bag recalls - how much has that cost in the long-term? Would a higher initial cost have prevented the damage? Apply that mindset to your projects.


Spending more up front can reduce long-term costs, so long as the spending focuses on the qualities that will drive those costs. Plan defensively!

Choose Wisely What You Build

"...software is hard."

- Knuth, Donald (2002). ref
13 years later and software is still hard. It's hard for a number of reasons including people, communications, and people communicating. In any case, it's also costly. Because of the costs (initial and ongoing) any feature and any new product should be considered carefully against those costs.


Adding up the costs:



People are likely going to cost more than equipment, licensing, and incidental costs. Good people are hard to find and expensive to keep. People are needed in order to build and maintain software. Software is soft, so it will change. People are needed to change software. Software is hard so it will be hard to build and maintain the software. It will be hard so it will be expensive. If it is not expensive to build, it will be poor quality and it will either be hard to maintain or it will not be valuable.


Measuring value:



Software is valuable when it either produces income or reduces costs. Predicting cost savings might be the easiest thing to do, still hard, but not as hard as predicting the cost of building it or the increase in income. There are too many uncontrollable values in predicting income producing software (market conditions, competition, advertising success, etc). Therefore it is more difficult to measure cost benefit.

Cost benefit of cost saving software will depend on usability and quality in addition to initial and cost of ownership. How can one measure the success criteria in terms of usability and quality? If there is some predefined criteria of gained efficiency that can be quantified, taken against the loss of some personal experience of the users and the cost of building and maintaining we can arrive at some ROI. I would propose that the loss of something by the end users is not easily understood or quantifiable. Whether the loss of personal experience, the human touch, or responding to change it could have an adverse impact.


What we can do:



Analysts considering a new feature or product should consider all of the costs as well as have a thorough understanding of the benefits before beginning a project. However, much of the factors are not easily quantifiable and will remain fuzzy. There must be ranges given where this is so and certain usability thresholds must be achieved before considering a project a success.

A template can be used with certain values in cost and value categories and assign estimated ranges in order to perform valuation. Properties such as analysis cost, construction cost, maintenance cost, end user loss costs, and incidental costs can be weighed against properties such as productivity gains, increased income, and loss reduction. When considering a product, each feature may have its own values so that they may be added incrementally. Break the work down into smaller pieces to reduce risk. Start with the features that are more valuable or less fuzzy, and stop when appropriate.

Thursday, May 28, 2015

TDD - Test-Driven Design

TDD, TFD, BDD, ATDD...all of those should be read x-y Design and not x-y Development. It would clarify the real value in doing development this way. However, it may seem if this is true, then the only tests that really matter are the BDD and ATDD tests. This would violate the testing pyramid where most tests should be Unit Tests. All systems do one thing, respond to input in some way. The response usually results in some state change and/or some other actions that also result in a state change. Example: User completes some task, task should be marked complete. Upon completion a communication should be sent to various recipients. That example contains so many assumptions that when one begins to design the feature, it would be impossible because the design is not yet specified. Enter BDD. First step is to set up the pre-conditions: User has a task that is in state InProgress. User completed form. Form passed validation. There are subscribers to the completion event. The subscribers are listed as follows: Josepha in accounting via email j.doe@example.com The Budgeting Dept via some other application Etc etc etc. Next is to describe the action that causes a change. User marks task complete. Last is to verify the result. Task status is complete. Email sent. Budget app data sent. Etc etc etc In this way the requirements are not ambiguous and all the scenarios are easier to understand. We have a better starting point to design our system.

Tuesday, May 26, 2015

SPA Framework Templates Have No Reuse

For all the comparisons of SPA Frameworks I haven't seen much focus on the templating. There's a common complaint that all templating systems have drawbacks. One that I've not seen considered is that the templates are very specific to the framework. Wouldn't it be nice if the templating engine was truly a separate concern? Then when you build an app with a template you can keep all your markup when things change. Each framework does templating in its own way. Angular has you creating your own tags, Ember uses Handlebars markup, React seems to mix JavaScript and html-as-a-strig, Knockout uses data- attributes. All of these have their drawbacks, however consider if you or the developers of the framework are going to change the M/C/VM engine or perhaps go to the next framework. What amount of rework does that introduce? It would be nice to grab your favorite templating engine off the shelf, hook that into your favorite router, then add your controllers with a bit of DI and config everything works! Whether you like the ng-whatever syntax, the purity of data-bind, or the separation that {{item.name}} provides, or how about Jade with its minimalist markup? Then there's Express, Mongoose, Jade how about that! Grab a template engine (Jade), a router/controller (Express), and a model engine (Mongoose), run them all on Node and hit go! I have to have a better look at this!

Thursday, May 21, 2015

Change is Good, as Long as its Not Big Changes all at Once

Today's lesson is about change. When we make big impactful changes, they may backfire in the long run especially if they do not leave options on the table. If change is desired, find a way to make a small change that can be built upon in a number of ways. View change as growth that happens gradually without notice. In that way, it happens without too many growing pains which can result in change viewed in a negative way. For example, if you are changing to introduce a new technology that will help productivity, try to find a way to roll it out and have a few champions work with it for a bit and spread out the word gradually rather than shoving it into the hands of everyone all at once. In the long term, acceptance and usage will be much better.

Monday, May 18, 2015

Budget as a Team!?

Would it work? Can the team manage a budget as a team? Nothing says empowerment like giving responsibility of managing money. So why not give the team a budget to manage? If course things could really run afoul and it's easy to imagine how, but lets focus for a bit on the potential benefits. A sure benefit is that the team would have full exposure to the workings of the budgeting aspect of the business. How often are asks denied or ignored because they would "cost too much"? Of course, the team may have some great ideas about how to improve things and they may carry a cost. Often enough, denial of perfectly good ideas can be detrimental to morale. In this case, the improvements would be weighed against the budget and more innovative approaches may emerge. More importantly, morale crushing would be based on reality instead of "because I said so". Approve budget increases and supplements. There's no sense in implementing a strategy of giving the team a budget without allowing for necessary increases and approvals. Of course the team would have to show why this is necessary, but that's what managers would need to do anyways. If the team has to ask the manager first, unless the manager is fully ingrained in the team and understands the needs, there may be much lost in translation. Who knows best about how a team can function better than the team? That is, unless you have a bunch that just doesn't love what they do. In that case it's time to revamp. If you have the right people, they will be more than willing and plenty able to make decisions responsibly given the information and proper authority. The role of the manager in this case would be to understand the priorities and set the goals. Imagine that your job is to plan a weekend trip to build a team. The trip would include challenges that the team would have to overcome by working together. What challenges would you devise for them? Who would emerge as leaders? Who would glue the team together? Who would provide moral support? So much of the team dynamic would emerge from such a weekend! What if that dynamic could exist every day! One way to challenge a team regularly in the office could be through continual planning in the area of improvement! This can be even more of a challenge if the team is responsible for the budget allocated toward improvement. As far as implementing this Budget as a Team approach, a manager may be inclined to give this a trial run with a portion of the budget...kudos! Just keep at it for consecutive budget seasons, allow for some mistakes, and celebrate successes! When things start going well, increase application. Set the course. What if you notice that there is a lack of knowledge in certain areas within your team? How can you address this through setting up a challenge for the team? Why not put it to the team? Set a budget, have the team work on the problem, be open to them asking for more if needed, they will justify it. Even if the justification seems outrageous. If the funds are not available or cannot be allocated, show them why. Perhaps they will be able to find other means of obtaining funding, or they will modify the solution. I would love to hear from anyone who has done this. Did it succeed? Share your success stories! Did it crash and burn? What was the root cause of failure? How can others do better? Is this strategy just simply bound for failure? Why? Why? Why? Why? Why?