Tuesday, December 4, 2018

New Page With External Blog Posts

Some folks have requested links to the blog work I've done for clients. I added a new page with some of my work. I'll update it maybe each quarter. In lieu of that, I'm thinking about cross-posting the work I do here as well so look for that in the near future. In the meantime, check out the posts I've added to the new page!

https://blog.thedailylessonlearned.com/p/blog-page.html



Wednesday, November 28, 2018

OOP Concepts: The Aggregate Class

One fundamental principle in object-oriented programming (OOP) is hiding data. The idea is that you use a class to encapsulate data associated with some entity in the business model. All data access and manipulation should be performed by methods on the class.

This principle also applies to collections. In this post, I'll cover some details of a specific pattern for hiding collections. This pattern is called the aggregate class. An Aggregate Class follows the same principles as any other class: data hiding, separation of concerns, etc. It also has the same benefits: reuse, contained changes, centralized business logic.

The Problem With Functional Style in OO Languages


With so many developers using ORM's and Linq (and other lambda-based operations in other languages besides C#), we see a trend toward functional style programming in OO languages such as C# and Java. We should still be careful to protect the data since OO languages do not work the same way as functional programming languages.

In OO languages, collections such as lists and arrays still allow you to change the state of each element. Even if the collection itself is read-only, it's only a pointer to the underlying objects. You could, perhaps, use structs in C#. But then again, you might not have control, and still, you'd have to go the extra mile to properly handle state changes.

If you're really doing functional programming, you won't mutate the state of a record. Some OO concepts have bled into functional languages like Lisp and F#. Basically, the lines have blurred on both sides. When it comes to OO languages, classes are mutable by default. The whole point is to mutate state within a class instance! So, we need to be careful to protect the state of collections from corruption by external manipulation.

Enter the Aggregate

The Aggregate Class is just the thing to keep our data within collections protected. The aggregate class helps by containing changes to the collection. Also, it gives us a centralized place to organize our collection logic. That's really important too!

As an example, let's consider an application for managing people within a school system. There are several types of people in a school. Major classifications are students, teachers, staff, and administrators. There may be some database table containing basic information of all persons: name, phone number, etc. But, indeed, there will be various information for each type of person. 

Specifically, a teacher may be tenured. Say a teacher earns tenure after five years. You can write a filter using Linq to get all tenured teachers like this:


var tenuredTeachers = persons.Where(
    person => person.Role == "Teacher" && 
    (DateTime.Now - person.StartDate).Days > (365 * 5)
);

As you can see, it can be a little messy. And of course, this encourages copy-paste coding because you might need the logic elsewhere. How do we fix this?

First of all, we'll make a Teacher class and put an IsTenured property on that to contain the logic.

var tenuredTeachers = persons.Where(
    person => person is Teacher && person.IsTenured
);

Already looking cleaner!

Next, let's do away with the "person is Teacher" by using OfType<Teacher>:

var tenuredTeachers = persons.OfType<Teacher>().Where(
    person => person.IsTenured
);

Even better! OfType will filter the collection down to the specified sub-type. But must we repeat this code every time we need teachers? No! We can use the Aggregate Class. Let's make one like this:

class Persons
{
   public IEnumerable<Teacher> Teachers { get { return _persons.OfType<Teacher>(); } }
}

And we can use that to get teachers wherever we need them. But we can go a bit further and create a Teachers Aggregate Class too.

class Persons
{
   public Teachers Teachers { get { return Teachers.Get(_persons); } }
}

class Teachers
{
    // factory method
    public static Teachers Get(IEnumerable<Person> persons)
    {
        return new Teachers(persons.OfType<Teacher>());
    }

    private Teachers(IEnumerable<Teacher> teachers)
    {
        _teachers = teachers;
    }

    public Teachers TenuredTeachers { get { return _teachers.Where(t => t.IsTenured); } }
}

And with that, we can return only the tenured teachers and contain the logic. Now in our business objects, controllers, or wherever we need to get tenured teachers, we can use this Teachers Aggregate.

var tenuredTeachers = persons.Teachers.TenuredTeachers;

And there we have some nice clean code!

Applying Functions to Aggregates

We want to protect the data within the collections. There's a pattern we can use to pass functions to the collections within the Aggregate Class. It's basically the visitor pattern, and it goes a little like this:

persons.Teachers.Apply(t => notification.Notify(t) );

// or

persons.Teachers.Apply( notification.Notify );

In this case, we're passing the "Notify" function to the Teachers Aggregate. The Teachers Aggregate will handle passing it along to all the teachers.

Even with the Apply function, we aren't really protecting the data unless the Teacher class protects its own data. A typical pattern you'll see is that the Teacher or Person class is a POCO—meaning it just has public properties. POCOs are really just DTOs. They're intended for transferring data, but you shouldn't really manipulate the data everywhere in your code. This is where you have a real divergence between the intention of OO and how we often see the languages used in practice.

On the flip-side, we see functional programming operating on records. But when that happens, the default is to create new records as a result of the application of a function. A Map method in a Teachers class would look like this:


public IEnumerable<Teacher> Map( Func<Teacher, Teacher> map )
{
    return _teachers.Select( t => map( t.Clone() ) );
}

In this Map method, the elements of the internal collection are copied then passed to the map delegate given to Map. Surely, this is an odd mix of OO concepts and functional principles. The Aggregate Class shouldn't really return a collection of the internal data even if it's a copy. This is really a generic type of method that has use in low levels of your application stack. It isn't beneficial. Think about it this way...what business function does "Map" perform? None. Which brings us to putting business methods in Aggregate Classes, which is the proper way to do things in the OO paradigm.

Business Methods in Aggregate Classes

What do you actually need to do with Teachers? We've already seen a case for "notify tenured teachers." We can expose other useful subsets of teachers like TeachersWithAbsences. But if you really feel the need to present a way to apply arbitrary filtering, the items in the set should prevent modification of internal data AND the set should prevent altering the items in the set. You can run into trouble with filters if they allow the collection the change:


// Dangerous Notify method...
public async IEnumerable<NotifyResult> NotifyAync( Func<Teachers, Teachers> filter )
{
    foreach( teacher in filter( _teachers ) )
        yield await teacher.NotifyAsync( notification );
}

// Dangerous call to Dangerous Notify method...

... await teachers.NotifyAsync( teachers => teachers.Where(t => t.IsTenured ? t : null ) );

Here, the programmer is returning null when a teacher isn't tenured. The implementation of NotifyAsync, while intended to be as flexible as possible, invites danger. A better implementation prevents modification to the internal collection as follows:

// Better Notify method...
public async IEnumerable<NotifyResult> NotifyAync( Func<Teacher, bool> filter )
{
    foreach( teacher in _teachers.Where(filter) )
        yield await teacher.NotifyAsync( notification );
}

// Dangerous call to Better Notify method...

... await teachers.NotifyAsync( teacher => 
  { 
    teacher.Email = null;
    return teacher.IsTenured; 
  } );

Here, the collection can't be changed, so it's better. But still, unless the underlying items are adequately protected, they can even be modified in ways that are dangerous. This example is a bit obvious, but similar trouble can occur when the underlying data is allowed to be manipulated when it should not.

We can go all the way to protect the underlying data by either denying access to the underlying items altogether or by limiting exposure to the underlying items.

// Best Notify method...
public async IEnumerable<NotifyTeacherResult> NotifyAync( NotifyTeachersFilter filter )
{
    foreach( teacher in filter.GetFiltered(_teachers) )
        yield await teacher.NotifyAsync( notification );
}

public class NotifyTeachersFilter
{
    public bool? IsTenured { set; private get; }

    internal IEnumerable<Teacher> GetFiltered( IEnumerable<Teacher> teachers )
    {
        return teachers.Where(t => IsTenured != null && t.IsTenured == this.IsTenured);
    }
}

With the filter type, we've entirely walled off access to the underlying items in the collection. This is a simple example of how to use filter types with an Aggregate Class. You can go further by passing a collection of filters or even an ordered collection.

Final Thoughts

I want to conclude by saying that using Linq is not precisely the same as Functional Programming. Sure, you can and should bring some of the concepts of FP into an OO language. But, remember that the language itself is built for OO programming. The principles won't translate 100%, and you can end up shooting yourself in the foot quickly by trying to do FP in an OO language. It's better to switch to a functional language like F# so you get the full support and benefits. When using FP concepts in OO languages, keep in mind the OO principles and use the Functional Programming practices with a grain of salt. Keep in mind that they can help but use with caution!



Friday, October 26, 2018

Taming Events: How to Use SoC to Organize Events.

If you're using any kind of eventing system in JavaScript, whether the built-in events or something more, you'll want to give some thought to how you organize the events. In the grain of self-documenting code, SoC, and containing risk I offer some suggestions on how to pull it off without pain.

What's an Eventing System?

Basically, an eventing system is a way to raise an event in one context (such as in a function or class) and listed for events in one or more different contexts. It's a great way to decouple your code, but you should be careful not to paint yourself into a corner. Here's an example of an event and a listener (you've no doubt seen this kind of thing before):

$input.on('change', handleInputChange);

This is an example of a JQuery listener. It listens to DOM events on an input that's represented by the variable `$input`. When the `change` event is raised by the DOM, the `handleInputChange` function is called in response. This is about as basic as it gets. Let's see how to raise an event programmatically.

$input.trigger('change');

This is the JQuery way to trigger an event. Any listeners such as `handleInputChange` will be called in order once the event loop comes around to them in turn.
Mozilla lists a boatload of native events on their developer site.
Some eventing systems, such as Backbone Radio (part of Marionette) or Redux, are built for handling custom events. You define the events, raise them in code, and listen to them. For example, Backbone Radio works a bit like this:

Backbone.Radio.channel('my-channel').once('my-event', handleMyEvent);

// elsewhere

Backbone.Radio.channel('my-channel').trigger('my-event', ..args);

And this will work just fine functionally. The event will trigger and the handler will handle it. You'll add more channels and more events. One day, you or a future developer will need to find out what channels are out there. One problem is, you can accidentally duplicate channels and events. Who knows what's listening to all those events. Organization to the rescue!

Organizing Events

In an event-driven system, you've got to be organized or you're system will essentially be running wild. By "organized", I mean three things:

  1. Use well-defined conventions. 
  2. Keep things in logical places. 
  3. Know where and how to find something easily.

Failure to organize will result in great difficulty resolving issues. For example, you might need to hunt and peck your way around the code to find what events are raised in the first place. Or, you may need to run the application in order to figure it out. Worse you might not be able to run the application in its current state. There are ways to solve these problems!

Make Channels/Queues Explicit

You can put all of the channels into one file. Then, simple require or include this file wherever you need to interact with events (trigger or listen to). This advice isn't just for Backbone Radio or even JS frameworks, it's true for any given event-driven system. If you're using a message queue via ruby or python, to name a few, you still want to separate where your queues are connected from where you're using them!

Here's an example using C# for sending messages to a queue:


// somewhere in a method 
 this._messageQueues.UserMessageQueue.QueueEvent('user-update', eventData);

OK, I admit that example is a bit generic. But, you can see how easy it is to raise an event with this system of organization.An instance of the class that holds the message queues is injected into the class through the constructor (not shown). Then the instance is used to get the specific message queue which is used to queue the event. This beats the alternative:

// in the same method
  SomeSpecificMessageQueueClient queue = new SomeSpecificMessageQueueClient('user');
  queue.Initialize();
  var topic = queue.GetTopic('user-update');
  topic.SetConfiguration(...)...more boilerplate code...
  
  ...finally, 30 lines of config code later...

  topic.QueueMessage(messageToQueue);

Alright, I concede once again to making this more complicated than necessary. We could have something like Queues.get('user') right? Of course we could! That's less boilerplate which is good. However, the problem still exists that all queues are adhoc in a string. Better to be explicit so that you have self-documenting code!
I still don't really like the fact that we're defining the event keys as strings. I'd like to be more explicit about that too so we know which events are raised throughout our code. In large code-bases this gets to be really important. Smaller code-bases can benefit too. We can make the events explicit too.

Make Events Explicit

By making the events explicit, we can easily see what events are raised and listened to in a system. We might be raising events for no reason, for example. When they're explicit, as in a method or function rather than a string, we can see how they're being listened to. Remember the `once` listener in one of the early examples? That registers the listener to listen one time to the event, then deregisters it after that first time. In this case, we would have an explicit method for the event such as `MyChannel.listenToMyEventOnce(listener)` and no methods on `MyChannel` named `listenToMyEvent`. From the class itself, we can easily see that there are no perpetual listeners to `MyEvent`. Whereas, if we have `once('my-event')` scattered throughout the codebase, we would have to search everywhere to find out. That's a long process. Besides, you might not have all the consumers of your events in the whole codebase when you're using something like a message queue. In fact, that's the whole benefit of message queues in the first place! Here's a more comprehensive code sample of what I'm advising here:

class UserEvents {
  private const BEFORE_SAVE = 'user-before-save';
  private _queue;

  constructor(Queue){
    super();
    this._queue = new Queue('user');
  }

  raiseBeforeSave(user) {
     this._queue.raise(BEFORE_SAVE, user);
  }
  listenToBeforeSave(callback) {
     this._queue.on(BEFORE_SAVE, callback);
  }
}

In this class (which is sort of Typemock-ish), I've defined all of the user events for now. The queue type is injected into the class so it can be swapped out for testing (or even adapted to use a different queue system). The consumers of UserEvents has no idea how it implements or even interacts with the queue. We've got all that detail contained within the domain-specific class "UserEvents".

Finally...

I just want to leave you with a final thought...this is all about self-documentation. It makes for easier programming and issue resolution without having to strap on a debugger or hunt through mounds of code. One addional benefit to containing all the eventing logic...you can see how easy it would be to add logging to the Queue right? Just pass in an instance of your custom LoggingQueue class that has the same methods but which logs each event. "raise" would first call the logger, it can easily log the handlers attached. And now you have a nice way to see the chain of asyncronous events that are always so much trouble to understand in an event-driven system! Happy Coding!

Monday, July 16, 2018

C# Logging: Where To Setup The Logger

Applications need logging. In C#, you will typically log various types of events like exceptions, debug, and informational. Your logs will contain valuable information which you'll use to tune your application and understand user behavior.

It's easiest to either pass an enum value to a single Log method or call a specific method to write different types of events. It's typical to write to different logs depending on the context. The problem is, where and how do we access the logging API from our methods?

Dependency Injection


Dependency Injection is one possible approach. If you go this route, you have to pass around a logger to all your classes as a dependency. You can use constructor injection and take the logger in as a constructor parameter. For a User class, you would pass it in along with the UserRepository as follows:


public class User
{
  private readonly IRepository<UserData> _userRepo;
  private readonly ILogger _logger;

  public User(IRepository<UserData> userRepo, ILogger logger)
  {
    _userRepo = userRepo;
    _logger = logger;
  }
}


The User class takes in the logger along with any other dependencies.

Via IoC Container

Even if you use an IoC container like Ninject or Unity, you have to add the logging interface to all the constructors and make a class member. With the IoC container approach, you can also ask the container for the implementation.

Container.Get<ILogger>()?.LogError(...);

Results and specific approach may vary according to the container, but this will at least allow you to fetch the logger whenever you need it without having to pass it around.

The .NET Core Way

In .NET Core 2.1, there are two ways to go about it as written by Luke Latham. Luke recommends using the LoggerMessage class over the LoggerExtensions for high-performance applications.

The Problem With DI

But let's suppose that you're not using core and you don't like the idea of adding the logger dependency EVERYWHERE! It isn't a dependency of the class that has anything to do with the business logic of the class. It's an implied dependency of all classes in the application. If you first design the class without the logging dependency, then you have to add it when you need to use logging.

IoC containers make this easier, but what if you have static methods from which you need to log (it happens)? In that case, you don't have constructor variables. You could pass the logger in the method params, but then you're passing around a logger. That's not a particularly elegant pattern! For one thing, you end up with more parameters. Another, you get into the habit of passing things around you may or may not need.

As Global

If it's a dependency everywhere, it's a global dependency. I don't like globals for many things. They are for configuration and that's really about it. It's beneficial to have a global logger too!

I also don't recommend statics for most things. Well, the logger could be a static global so long as it doesn't have mutable state. In other words, so long as you set the logging configuration once and leave it along throughout the entire application lifetime.

Log.LogError(...);

And in your implementation of the static method, you can get it from the container or some other global context.


public static void LogError(params object[] logParams)
{
  Container.Get<ILogger>()?.LogError(logParams);
}

This is actually a better practice since you only have to change your Log class if you change IoC containers and the interface doesn't match. It's better separation of concerns (SOC). The place to change the logging is in the Log class rather than everywhere else!

Concluding


Most of the methods discussed here are fine and dandy. I'm just not a huge of fan of having to pass things around in every class. I wouldn't extend this idea to something like a repository though, those are business logic specific dependencies that actually have meaning to the class at hand. A User class taking a UserRepository is meaningful. Every class taking an ILogger has no meaning and actually distracts from the purpose of the classes.

In sum, global is the place for the logger!

Book Review: Astrophysics for People in a Hurry

While I was shopping for a new Chromebook, Amazon recommended some books for me. One of those books was Neil deGrasse Tyson's "Astrophysics for People in a Hurry" (if you're interested, please use this affiliate link or the one at the end to purchase your copy - I'll get a bit of commission at no extra cost to you). My wife got me this book as a gift in 2017 as a matter of fact. I have to say, it was a fun read!


I've read a lot of pop-physics books over the years. I had already been exposed to most of what is in this book. Still, it was a fun read! Neil just has an entertaining personality and it shines through in the book as much as it does over audio or on TV.

Contents

Just to give you an idea of what's in this book, some of my favorite topics are:

  • Dark Matter
  • Dark Energy
  • Invisible Light


Neil starts the book at the beginning. THAT beginning. The Big-Bang. He also has a chapter called "On Being Round."

On Being Round


Well, it's not about being overweight so much as it is about being massive. As Neil says, any mass in space will experience surface tensions which force it into a round shape. Spinning objects will have a more-or-less flattened round shape. Pepper that with anecdotes about spherical cereal boxes and a whole wealth of other information and you have a pretty good idea about one of my favorite chapters in the book!

Final Thoughts

I don't want to spoil anything else for you, but let's just say there are some pretty spooky things in this book! One of the latter chapters speaks of things that will leave you awestruck! And this is why we love Neil deGrasse Tyson, isn't it! He can give us the heebie-jeebies and leave us awestruck at the same time!

If you're into this kind of thing and you want a fun, easy read by one of the best minds in the physics world, read Astrophysics for People in a Hurry! That affiliate link will take you right to the order page on Amazon so you can order it now.


Sunday, July 15, 2018

Bought a Netbook

Last night I did some comparison shopping for a new netbook. I posted the details in this blog along with some affiliate links. I figured, since I'm going to buy a netbook anyway, I might as well share the experience.

A Software Developer's Rationale for Affiliating


I'm trying the Amazon Affiliates program for this experience. There's a benefit to both of us. If I'm doing some comparison shopping, we will gain maximum advantage through re-use if you (dear reader) were in the market for a netbook. And in return, you'd be helping me out by buying your netbook through one of my affiliate links. It costs the same for you, and I get a little commish for having posted my experience here.

What I really want is to share some good things with my readers. For example, I applied to be an affiliate for PluralSight. I love their selection of training videos, and I hope that happens because I believe in what they have to offer! But, I digress.

And the Winner Is...


Getting back to the topic. I went with the ASUS Chromebook C202SA-YS04 11.6" (that's the affiliate link in case you'd like to follow suit and get a great little netbook for yourself or a loved one). It seemed like the best deal on the market. With Prime Day coming very soon, you might want to keep an eye out for a good deal on it. It really came down to having the most/best reviews in the price and spec range.

Basic Specs

Here are the primary technical specs for this model.
  1. 4 GB RAM - I'd like more, but I don't know that I would need more.
  2. 11.6-inch anti-glare LED-Lit HD monitor - I'll be using this on the train a lot and sometimes that sun shines right into the windows. Anti-glare is a huge plus!
  3. 32 GB internal storage - I wanted the extra storage, wouldn't hurt whereas running out of space would!
  4. Chrome OS - I'm giving it a try. I use a lot of their apps and services, so why not. If I don't like it at all, I'm going to see about installing a Linux distro on it. Maybe Ubuntu. Ubuntu on a netbook...yeah!
  5. 2.65 pounds - this is the main thing. I needed something lightweight because I'll be slinging it on my back. I have a lot in my pack, and the laptop is just too much to lug around.
  6. Dual processor - A lot of the less expensive netbooks have a single core processor. That means only executing one thread at a time. I prefer to have my processor multi-task so that I can keep working while background processes run. I know it depends on the app and processor architecture, but 2 cores are usually better than 1.
So those are the basic specs. And that's part of why I went with this model.

Primary Deciding Factor

It really came down to the reviews. This one had the specs I liked, and it had the most reviews and a comparable rating to other netbooks in its class. To me, this says that enough people bought it and liked it enough to give a favorable rating. Statistically, a set of over a thousand reviews is more significant than a set of under one hundred. So I went with the popular vote.

I'll post some pics when I open it and do a quick review once I get a feel for Chrome OS. Thanks for hanging with me tonight! Have a happy Prime Day!

Saturday, July 14, 2018

Buying a Chromebook

Ok, so I have a Lenovo Notebook that I use for everything right now. It's decent enough to do what I do. I'm using it right now, in fact. I can use the internet. It works for programming. Sure, it isn't a top-of-the-line model, but it's getting the job done!



Except, when I take it with me on my daily bike or train ride and it's hanging out in my backpack adding 5 or 6 pounds to the load. Then it sucks! I need something lighter. I'm considering a Chromebook but I don't know which one. Maybe it doesn't matter since they're priced so low that it's not like it would be a total loss if it wasn't that great. But then again I don't want to be so annoyed that I can't be productive. I plan to blog a lot on it and need it to be portable.

Here are a few that I have in mind (I really am buying a netbook, but I want to mention that all the links are affiliate links, so if you would like to purchase any of them I get the benefit of a commision at no cost to you...and a big thank you if you do!):

ASUS

The ASUS Chromebook C202SA-YS02 11.6" has 4GB RAM, 11.6-inch monitor, and only weighs 2.2 lbs! I like the lightweight.

But then again, it's only got a 16 GB hard drive and the next level up has a 32 GB. The ASUS Chromebook C202SA-YS04 11.6" is the same but with the bigger hard drive. Apparently, 16 GB extra of hard drive adds 0.4 lbs.

The only thing is they come in dark blue. I'm not too concerned with color so long as it's not a standout color, but I would prefer something dark-grey or black.

Samsung

I could go with a Samsung Chromebook 3 XE500C13-K03US. It has a 4-GB RAM and a 32-GB Hard Drive. And, oh look! 2.54 lbs! And what else? A Dual-Core Intel. Hmm...Samsung, should I give you a shot or keep looking? There's only one left, but I am a sucker for familiar brands. I haven't had a Samsung laptop. I know other folks have had some problems with other types of Samsung products. 4 stars for this though. Let's see what else:

Acer

This Acer Chromebook 11 is looking mighty nice in Clamshell White. Not really my taste. I would get one for my daughter though. Ok, so it does come in black too.

HP

I do have experience with HP products - mixed but generally good on the hardware side. So there's the newest HP 11.6” HD IPS Touchscreen Chromebook. Touchscreen, eh? I never really worked with a touchscreen before. I might give it a shot! It's pretty lightweight at 2.5 lbs. Says the battery will last 12 hours on average! Most of the others were around 9, so that seems pretty good! 2 processor cores - another plus. The 16 GB hard drive is a bit of a drawback.

Decisions, Decisions...

In the end, I think I'll go with the masses. That second ASUS with the 32 GB has the most reviews and a four-star rating at the moment. Seems like it would be the best all-around. It's just a little more pricey than some of the others, but you get what you pay for right?

Friday, July 13, 2018

Introducing: "SWARMing"


Hello all! I popped into Dan North's blog to see what he's been writing about lately. Dan North introduced BDD (Behavior-Driven Development) to the world which bridged a huge gap between the Customer and the Team.




His latest post "In Praise Of SWARMing" grabbed my attention. I thought it was going to be about "swarming" a problem as in Kanban. But it's actually a different-ish approach to adopting Agile at Scale. SWARMing is Scaling Without A Religious Methodology.






His ideas really hit home with some great points! There are some harsh criticisms of competing methodologies. Those are somewhat tasteful. And I have to warn you, its a bit wordy. At over 4200 words, it's quite a bit more than your average "browsing the internet" post.


Here are some things that really jumped out at me.

The Good Parts



I particularly like the contrast between "moving the work to the people" and "moving the people to the work." This translates to reorganizing. The term "self-organizing teams" comes to mind.


I do prefer the flat structure of "every part of the org is geared in delivering value" vs the slanted structure of "sales makes the money and everyone else spends it." It allows businesses to utilize all their assets in focus of value delivery.




I read once that value is expressed as the benefit for the cost. A "good value" doesn't necessarily mean inexpensive. It means you actually got a return on your investment, monetary or otherwise. I wonder if looking at your organization from a "good value" perspective would make a positive difference.


Speaking if value: There are a couple terms worth following up on. OKRs are a relatively new way to set and measure goals. I learned about Risk-Adjusted Return on Investment, which is your profit plus or minus risk.


The Bad Parts

The post is long. It has quite a few run-on sentences. The upshot is that it's not an easy read. My concerns are that you (dear reader) won't see through to the beneficial parts. Please press on, it's worth it!


You've also got to see past the sales-y aspects. He's pretty tough on competing methods of implementing Agile at scale. He's right with those points, but it drags the article and makes for a slightly bitter taste. Sorry.


I get it, he's selling consulting services and differentiating from his competitors. But that wouldn't really be necessary if the most valuable points were laid out without the cruft. Maybe do those parts in a future post dedicated to a comparison.


The last "bad part" is the focus on hard numbers. These days, organizational psychology says to keep your focus on doing good for your customers. But that depends on perspective I suppose. Those with the pocketbook will care about the revenue aspect, especially when they're being told to rethink how they allocate funds!

The Rest




Somewhere past halfway, Dan iterates over eight points about how to be SWARMing. Some of those go into depth with definitions of types of leaders: servant-leader and leader-leader. This section has some practical advice for hiring services to help with your transformation process. The successful transformation will be a long and investment-intensive road, so buckle up!

Conclusion

I sent Dan an email asking if he had a more concise description of SWARMing. One that, hopefully, lays it out without the heavy padding. Those things are valuable to support the idea, no doubt! But I can't exactly expect busy execs to read such a lengthy argument all at once. Especially when it's a new idea which asks them to rethink their organization from top-to-bottom, front-to-back, and side-to-side.


All in all, I'd say it's worth taking the time to read his post. With the right packaging SWARMing could be a catalyst for much needed change. I hope it gets that with a bow on top.





Thursday, July 5, 2018

Book Review: To Sell is Human, by Daniel Pink

I've never posted a book review on this blog, so here goes!

I've been inspired in the past by Daniel Pink when I read his bestselling book Drive, I subscribe to his newsletter, I've written him a couple emails (to which he responded). You might call me a fan. Maybe I am. I did, after all, read another of his books - To Sell Is Human.

Here are some things you might like about it (I know I did)...

What's It About?


In this book, Daniel Pink points out that we all sell. He makes a distinction between sales and non-sales selling. Sales selling is the traditional kind that makes you think of a used-car salesman from the 70's. Non-sales selling is the kind we all do - all the time!

Daniel takes you on a sales call with Norman Hall, the last remaining Fuller Brush Man. Norman is a "door-to-door" salesman. He is resilient, friendly, and has specific characteristics that make him a perfect seller. We should emulate Norman!

We go on other journeys to find out more about non-sales selling. Daniel takes us along to a training session with VPs, CEOs, and other business folks who are sharpening their selling saws. This book is packed with practical advice and results of studies to substantiate his claims.

So, Why Does Traditional Sales Suck?

The book shows us how we generally think of traditional selling in a negative light. Then it opens our eyes as to why. Spoiler Alert! The sales we think of - used car sales from years ago - is an unbalanced transaction. The sellers have all the power. That power was so often abused that the decent folk got pushed out. Why did they have all the power and what changed?

I'd recommend that you read the book and find out for yourself. But, since there are so many goodies in To Sell Is Human, I don't feel sorry about giving away this bit. The equation changes when buyers are informed. For example, we can find out everything we need to know and more about a used car before we buy. And we're not as limited in our options. We have the internet. The "information superhighway" for those of you old enough to remember the TV commercials.

Traditional sales used to suck, but it doesn't anymore. It's been transformed into more of a partnership. The role is more about discovering problems and applying solutions than it is about tricking uninformed consumers. Sure, those types of salespeople are still around feeding off the bottom. But your real sales jobs are much more elevated. Think about this...who is running the company you work for? Do they know how to sell?

How Do I Sell?

And then there's non-sales selling. You already do this. Whether you sell the dentist to your kids, a vacation to your spouse, or a new procedure to your boss you're selling! Daniel gives us plenty of practical advice we can all use in everyday life to make the sell.

He presents the new ABCs of selling (Attunement, Buoyancy, and Clarity) in an entertaining and digestible way. There's a lot to learn for those of you who have a hard time selling your ideas. There's some for those of you who think you have it all figured out too!

Besides those ABCs of how to be, he shows you what to do. Pitch, Improvise and Serve.

But, I'm an Introvert! I can't Sell...

This may come as a shocker, but extroverts aren't much better at selling than introverts! It turns out that ambiverts, those who are in the middle of the spectrum between intro- and extro-, are the best sellers. And chances are, you're more in the middle than you care to admit!

"vertedness" isn't binary any more than "brainedness" is right or left. Both are a spectrum, and most people are somewhere in the middle. Find your middle ground and become better at selling! Too much extro- leads to pushiness and not listening. Too much intro- leaves you under-assertive and too quick to walk away. Balance is better, despite what folklore says.

What Can I Take With Me?

This book is a perspective changer. It has shocking revelations! It has practical advice! It's fun to read! To use one of the techniques in the book: To Sell Is Human is a great read, you'll be pleased indeed!

Friday, June 8, 2018

The Big Five + Johari Window: Creating the Holy Grail of Personality Tests

As for personality tests, there are many options to choose from but which will guide us to the truth about ourselves? Meyers-Briggs? What kind of Ice-Cream are you?

Grouping and Sorting


We like to group and sort. At a young age, we learn this skill. But it doesn't mean that the world fits the models we create so nicely! And that's just it—we create models as substitutes when reality is beyond our comprehension.


Models help us communicate more easily with others. Imagine explaining a bird, for example, by iterating the entire set of species within the bird family. We didn't really get the classification of planets right until we found such an outlier that we had to redefine what it means to be a planet!

Binary Choices

We seen to like binary choices - 'A' or 'B'. A-B testing is the common paradigm for proofing a new feature design in application development. We default to two political parties in the US almost to a fault! I've noticed that my children respond more readily with binary choices. It's just easier to reason about!


The trouble is, binary choices are mis-leading. If you have two compasses, and they're slightly different, which do you follow? Either you need a third compass to prove out the faulty one, or you need to just pick one and go with God!


Is it accurate? When you move beyond the fervor of political campaigning, can you thoughtfully agree with everything on one platform vs another? Am I an INTP or an ENTJ? I've come out with both! Sometimes I align with E and sometimes I. Sometimes I'm Perceiving other times Judging. Seems logical that I straddle the line on those two factors.


Finally, with Autism they've done away with binaries. It isn't as if one is either autistic or not autistic. It really that we're all autistic to varying degrees. It's just another way of thinking 😔.

Beyond Binary Lies a Continuum

As it is with many things, our personalities lie on a continuum. If we take something like the Big Five, and rate each of the five traits on a continuum we will have a closer model of reality.
There is, however, a specific problem I want to address in how we collect the data. Self-selected ratings are prone to bias. Therefore, those surveys you take for yourself are highly prone to error. They may tell you more about how you perceive yourself or how you'd like to perceive yourself than how you actually are. And what good does that do you? After all you know yourself anyways, right??

Crowdsourcing

Besides the self-affirming nature of those questionnaires, the sample size is way too small—it's one! Thankfully we have social networks of friends who are generally willing to participate in social games. If only we can make it enough of a joy to participate in the game, that they'll readily participate. 50-100 question surveys aren't very rewarding! Enter Johari.

Making it a Game

The Johari Window model is kind of like a game. The subject and his/her friends, relatives, and colleagues choose adjectives that best describe the subject. The intersection of those choices fall into four quadrants: Arena, Façade, Blind Spot, Unknown. They're classified by whether or not the adjectives are chosen by the self and others.

The adjectives in a Johari Window are generally good traits such as "cooperative", "intelligent", and "friendly". There's an inverse called the Nohari Window which uses negative traits like "Stubborn", "Quarrelsome", and "Dense".

Combined, the Johari and Nohari Windows can give you a pretty decent view of how you perceive yourself compared to how others perceive you. The tricky part is to get enough participation to get a well-rounded view.

If you're interested in doing your own Johari Window, this one at Kevan.org is pretty darn straightforward. There are some other fun things at Kevan.org including this personality test.

 #   |.
###u#+.
     |)
If I were a NetHack monster, I would be a unicorn. Most people are only after one thing - I try to maintain a quiet and respectful distance until I feel sure that I can trust someone.
Which NetHack Monster Are You?

How about that...I'm a Unicorn after all! And here I was all along thinking I was a Bridge Troll.

And... if you haven't lolled off into Kevan-land by now, I'll be getting to the point soon.

Participation

Getting participation for something unfamiliar or that's going too much out of one's way is challenging. As previously mentioned, social media can help with this. It has familiarity, where an unfamiliar and poorly designed website can make others standoffish. You've got to expend some social capital on getting folks to participate.

That's not good. We want to build social capital with these exercises, rather than spend it!

Combining FTW

I'm thinking of combining concepts from the Big Five (or six, or whatever) with concepts from the Johari/Nohari Window.

This will work like this:

  • Use traits from the Big Five or HEXACO or some other number of traits
  • Present adjectives that fit with each trait (positively and negatively correlated)
  • The subject participates and asks for participation from acquaintances
  • From each trait group, participants choose 3 adjectives to describe the subject
  • There are two questions about the participant's relationship with the subject
    1. What type of relationship (choose all that apply): business, friend, family, acquaintance
    2. Scale of 1-10, how well do you know the subject
  • The subject answers the same 2 questions about each participant.

Additional Setup Details

Adjectives in each trait group vary in scale. For example, for the trait "Openness" some adjectives might be as follows:
  • accepting
  • progressive
  • open-minded
  • close-minded
  • conservative
  • curious
  • dull
  • intolerant
  • tolerant
Another option is to use emoji or some other visual indicator which is more culturally neutral to represent the adjectives.

Scoring

The strength and number of times each adjective is chosen are combined to give the rating scale for each trait. For example, if the traits given for Openness were selected as follows:


adjectivestrengthweight
close-minded-55
dull-72.5
intolerant-42
conservative-17

The subject would be considered low in Openness (-57.5).

Besides the ratings scale, the quadrants of the Johari Window can be brought into the model to provide more useful information to the subject. The concept of "known to self" and "known to others" is powerful in realizing how well we know ourselves and how we present ourselves to others.

In our example of Openness, we can also see that the subject did not pick any adjectives which are positive indicators of Openness. Therefore, the subject is not blind to this trait. This is the Arena quadrant of the Johari Window.

Anonymity and Sample Size

Two must haves in order for this to achieve useful accuracy are anonymity of the participants and large enough sample size of participants. They support each other. Large sample size secures anonymity. Anonymity allows more people to participate without fear. Additionally anonymity allows one to be more candid with their responses.

Diversity

A diverse sample is also important to get a more holistic view. The subject will be known in different ways by different people. This is the nature of relationships.

Conclusion

By combining concepts from the Big Five and the Johari Window, a better personality test can be created. What's more, is that this type of test will find more willing participants because of the fun nature of choosing a few adjectives rather than using something like a Likert scale (strongly agree, agree, neutral, etc.). This test is not strictly self-reporting, therefore not as subject to bias.

Wednesday, May 23, 2018

You Need to Know: Shadow IT

Troy Hunt just posted a new free Pluralsight video about "Shadow IT." That term sounds nefarious, but it's actually quite innocent. It's someone creates or uses software or a resource that hasn't been documented in the IT inventory and approved for use inside the organization. Because it hasn't been through the on-boarding process for IT resources, it also hasn't passed security checks.

Some examples include: A Google Drive or a One Drive to store or share files. A share drive with open access. Cloud services on Azure, AWS, Google Could, IBM Bluemix, etc.

What Are the Issues?

It's not that using these resources are an issue in and of itself; it's that they present potential security and management issues.

Security

Because the security of "Shadow IT Resources" are unknown to the organization, they could open security holes. Those security holes can be either external (exposing information outside the organization) or internal (exposing information to unintended people inside). It may not always be a problem per-se, but either scenario could really cause problems for the organization. Those problems can result in loss of business, legal proceedings, and even cause the business to fail.

Web app services on Cloud platforms are designed to be open to the world be default. They can be secured by deploying them inside a VPC (Virtual Private Cloud) which is accessible from within the network only. This same concept applies to many other Cloud services.

Besides Cloud services, there are countless tools, games, and application that are easily accessible to anyone with an internet connection. Security problems unknown, these could contain malicious code which is designed to leak information

Cost

Besides the costs of recovering from an information leak, another potential cost concern is an unplanned expenditure. Particularly with cloud services since its relatively easy to create a new resource on a cloud platform. Cloud services are pay as you go so it would be a slow-burn rather than a fast explosion that leaked information would present.

This kind of issue is easier to resolve since all activities are logged and can therefore be monitored easily. Services like Alert Logic and Stackify give you insight into activities on the Cloud.

Scaling is another source of cost. Cloud resources are made to scale -  meaning new servers or service handlers are created to handle increased traffic. Configure scaling appropriately and set limits to ensure that a DDOS (Distributed Denial of Service) attack doesn't end up costing you a fortune overnight. For example: the cost difference between a single small AWS server and many XXXL servers is in orders of magnitude of 100x the cost.

Goldilocks

Despite the aforementioned concerns, it's not worthwhile to be too restrictive when it comes to using the tools available. The trick is to find a path that's just right.

The Tale of Goldilocks According to Me

In the classic Goldilocks fable, Goldilocks happens upon a cottage in the woods. The cottage is the residence of three bears (papa, mama, baby). She "innocently" does a B&E (Breaking and Entering). Besides the unauthorized entry into the abode, she eats their food; sampling the porridge of each until she finds the one that's not too hot and not too cold, but just right! After that she samples the chairs. Baby's chair is just the right size, but she breaks it. Then she proceeds upstairs to the bedroom and tries all the beds: papa's is too hard, mama's is too soft, but baby's is just right. She falls asleep only to be awakened by the angry bear family returned from their morning walk ready to maul her. She barely escapes with her life after her little crime spree.

Lock-Down?

Besides the rampant crime in the story, Goldilocks has to try what's available until she finds what's right for her. Follow this practice, starting with most restrictive. However, do be open about the strategy so that those in the organization aren't taken aback by the sudden lock-down! Some of what exists in Shadow IT-land may be business critical! In that case a total lock-down would cause serious business disruption. Consider that they do lock-downs in prison when a fight breaks out...

Stay Calm and Keep Innovating

Another extremely important factor in applying the right level and doing so with care to respect the autonomy of individuals is the innovation factor. Theodore Henderson of the Forbes Coaches Council notes that "Innovation Is Crucial To Your Organization's Long-Term Success." He cites many success stories of innovative products that have lead to serious growth of organizations. One such example is GMail, which is the fruit of Paul Buchheit's 20% time according to Time.com (free time given for the purpose of innovation).

Disallowing the use of applications and services can seriously stifle innovation. It can do so in two ways:

1. Denying access to tools that can make people more productive.
2. Making employees feel less autonomous.

Autonomy is important to innovation which stems from motivation. Going into total lock-down mode can make people like they're under total external control which stifles their innovations and productivity. As a business model, that isn't going to go well unless you're business is 20th century line assembly.

Concluding

While it may be natural to knee-jerk and enter into total lock-down, it's important to find the right level of control. The right level of control means keeping Shadow IT to a minimum and plugging security holes while keeping all employees on the same side as Info Sec and Governance.

Read Troy Hunt's post here: https://www.tyroyhunt.com/new-pluralsight-course-the-role-of-shadow-it-and-how-to-bring-it-out-of-the-darkness/

Friday, March 9, 2018

Do The Gemba Walk

As a developer or analyst, you should sit in your users' seats so that you fully understand how to meet their needs. Interviewing is merely an introduction to those needs. In Kanban, they do a Gemba walk, which means going to where the work is done. We call it management by walking around. This is fine for management, but for actually creating something that helps the users or the business, one needs to actually do the work to comprehend the actual problems in order to solve them in the best way
While doing so, keep in mind the user's technical skill and framework. You may find that your applications have more than one persona using the application. A persona is different from a role. You can have many personas in each role. Let's say a Legal Assistant is a role. Those users may or may not be tech savvy. Consider that in your UX design!

Wednesday, February 21, 2018

What a Story! An Agile Story


You've heard of this thing called a User Story right? Perhaps you've even seen the template:

"As a <type of user>,
 I want <some feature>
 So that <some goal>."

But perhaps you've wondered how to put a feature like "read the users name from the database and put the value in txtbxUserName" into that format.

Occasionally, I write about Agile on this blog. In that entry, I wrote about a broader view of Agile from a developer perspective. In this one, I made a case for leaving the work up to the pros. This time I'm focusing on something much narrower — the User Story.

The Wrong Way



Perhaps not "wrong" way, just not something that's going to align very well with the benefits of Agile.

"As the one telling you how to do your job, I need you to write code that reads the username from the Users table and put that value in the txtbxUserName field so that it shows up on the page".


Or perhaps "as the project sponsor, I want a check box there so the users have to check the box before they can submit the page."

It can be a bit awkward to put those kinds of instructions into User Story format...especially when the goal is not for the user but for the project sponsor or manager to tell a developer what to do. You can succeed at writing good user stories if you frame them in more general terms — don't think about implementation details. A good test is — if it's awkward it isn't right.

Maybe Better

"As a user with access to multiple user accounts, I want to know who I'm logged in as so I know which account I'm using at any given time."

"As the company's legal counsel, I want the user to accept responsibility for using our services so that we have a leg to stand on if something goes wrong." That's the "I have read and understand the 30,000 words of legalese" checkbox.

Real Life Example

For another look, let's imagine were doing some work for a burger joint where their customers expect one thing — getting their food quickly. They want to order quickly and they want everyone else to do the same so the whole thing can flow like clockwork and they can be on their way to consuming those cals in under 5 minutes.

What does that story look like? Actually it may be helpful to have multiple User Stories since there are multiple user types or personas. Let's see about defining those now:

Regular Customer - knows what they want and orders the same thing all the time.

Infrequent Customer - didn't visit much and needs a minute.

Bulk-Orderer (team mom) - is ordering for the office or a party and has a bunch of items to order.

A story from each persona might look like this:

"As a regular customer, I want to place my usual order and get on my way so that I don't have to hassle with getting my food."

"As an infrequent customer, I want to take my time browsing the menu so that I can figure out what I want to order."

"As a bulk-orderer, I want to place my order without confusion so everyone gets what they wanted."

We're going to need a lot of cheeseburgers to feed that many Air-Force Cadets!
Who ordered no onions?


Next Steps

Now that we see the user stories in a more "user-need-goal" format, we can start to think through different ways to resolve the issue. That part of the process is a conversation. A conversation between the team and the customer.

That's a Wrap...(for now)

In this entry, we've seen how we van write User Stories from the perspective of the users, through different user personas. I haven't captured all of them and that's inevitable. The magic is that as we start rolling out features to support the users based on their stories, related stories will filter in.

You may have heard a little about different roles such as Team Member and Customer — especially about who plays which roles. We'll take a look at how that works in the next entry — there are some things to think about depending on your organization.

Tuesday, February 6, 2018

Web Basics - TLS/SSL https

We've been looking at the basics of the internet. If you've been wondering about how it all works or are interested in web programming, you need to know the things in this series of posts.

Today's topic is TLS - Transport Layer Security. The transport layer is essentially the connection itself. The web can be divided into a model with 4 layers - two of which we've been talking about: application (HTTP) and transport (TCP and UDP). The other two are "network goo" that we really don't interact with directly. They're important, don't get me wrong, just not important to this series.

As we saw in the last post on TCP, your information is flying around the world at light speed. With the right equipment and wrongful intent, someone looking to make a buck could easily tap into your data in transit (that's what we call it when its on the move) and sell your information (usually a big batch of information) to someone who will exploit it to steal money. That is, unless its scrambled before it's sent, then unscrambled on the receiving side. Enter encryption.


Encryption



The newest big business buzz of currency - crypto-currency -  is all possible because of encryption (that's the crypto- part). It's built on the premise of uniquely encoding a "block-chain" and adding that to the existing chain to make it more valuable.

Encryption took off during WWII because radio transmissions were used by all of the militaries participating in that war. As we know, anyone can tune into radio frequencies and listen in (we can also listen to the radio transmissions of the cosmos - all the way back to the beginning of our universe!). Unless you can send a message in a way that only the receiver knows how to understand, you're toast! Every one of your moves will be known. It would be like playing chess while thinking your whole strategy out loud - you just can't win that way!

So they encoded the messages. With the messages encoded only those listeners with the decoding sequence would be able to understand. The U.S. got really good at cracking the code - which was one of the main reasons why the Allies won. Another was the perseverance and sacrifice of millions of lives of Russian soldiers. And the third was massive industrialization in the U.S. - both automated and manual industry.

History lesson aside, encryption has been used to protect privacy long before the internet. In modern times, it is used to protect data both in transit and at rest (in a database or on a hard-drive). TLS represents encryption in transit. SSL (Secure Sockets Layer) is the outdated predecessor to TLS - it's been deprecated by the authorities on internet security (the IETF*) as of June 2015.

TCP establishes a connection to communicate between two servers. TLS secures that connection by ensuring that all information transmitted through it is encrypted. The mechanisms for applying this encryption involve a certificate.

Certificates



Certificates operate on a trust basis. There are companies that issue certificates (issuer). Those companies are called certificate authorities (CA) and your computer has their root certificate pre-installed. If you are securing your server, you would purchase a certificate from one of those companies. Your URL would then be registered to that certificate. You install the certificate on your web server. When https requests are made to your server, the requester gets a copy of your certificate. Your certificate is used to establish your authenticity. It's kind of like a driver's license, passport, or other form of id.



If you are the requestor, your browser will check the certificate's signature against the signature you have on the root certificate of the issuer. The domain in the URL also has to match the domain name on the certificate you receive from the server. If there is a match, the server has been Authenticated. Once the Authenticity of the server has been established, your computer and the server will generate an encryption key for the session. All of the information shared between you and server will be encrypted and decrypted on either end using that key.

Issues

This site is not https, but it's readonly - you don't exchange any sensitive information. Be careful when you have sites that require sending sensitive info and there is no https or it is mis-configured.
This site is configured for https.


This is how most of your information is secured on the internet today - provided you and the server are using https properly. Often we see misconfigurations on servers or servers that still support unencrypted http connections (http without TLS). There are also different versions of TLS which creates more configuration issues. The best you can do is pay attention to what your browser is telling you and think a bit about what kind of information you are willing to compromise - and remember some hackers are fairly sophisticated and can piece information together from multiple sources if you are a specific target (e.g. have a lot of money or power or work for a target organization/industry).

TLS works well to protect us when configured properly, but we should still remain vigilant. It can be easy to think that https solves all of our internet security problems, but there are other ways that hackers will try to pwn you.

Continued Learning



Encryption is a vast subject in and of itself. It comes in many flavors and varieties. There are one-way and two-way hashing algorithms, asymmetric and symmetric keys, private-private and private-public keys. And it all involves some pretty intense mathematics. Crypto-masters are a rare breed but the work they do is vital to our lifeblood - secure data!


IETF - Internet Task Force



OWASP is the go-to for internet security - they have great info about TLS



Some certificate authorities along with more details are listed on WikiPedia here:


Wikipedia has a lot on the subject of TLS in general:



Thursday, February 1, 2018

Web Basics - The Internet - TCP/IP

Yesterday, I posted about the basics of HTTP. Of all the message protocols, it is the most common one used on the web. If you are in IT or use the internet - basically if you have a pulse - and you are curious about how the internet works...or, I don't know...want to learn web programming, then you should also learn about TCP/IP. That's TCP over IP.

But First - UDP


Our journey though the internet begins at your computer. Let's imagine that it's only connected to one other computer through a wire. That's the simplest kind of network. Your computer is networked with another. Both computers have a network interface - which is a card or microchip or something on your computer that knows how to connect to the outside world. Think of this device as a telephone.


The telephone allows you to connect with another person at a distance. You can talk and listen. A network interface card (NIC) can send and receive data (as packets) over a network. Your computer talks into its NIC, and its NIC uses the network to talk to other computers...just like your phone!

With the 2 computer network, the NICs in both computers will need to know how to exchange data. The simplest way is to just send the data to the other side. This is called UDP (User Datagram Protocol). This would work well in a simple network like two computers over a single wire. But lets imagine those computers are very far apart - one in New York and another in Los Angeles. And let's imagine they're connected over a telephone wire. And let's imagine they're sending data over that wire.

TCP


Since those old phone wires weren't designed for data - they were designed for talking - some of the data might get lost along the way. Luckily there's TCP (Transmission Control Protocol). TCP is a network protocol where the receiver tells the sender that it got the message and how much of the message it got. If it didn't get all of it, the sender sends the missing pieces. Sending this way takes longer, but it makes sure the right message was received.

Protocols are there to make sure we get the right message. They exist in everyday life, we call them manners - the pleases and thank you that ensure clear communications. The army has protocols for sending messages over radio. Imagine if a platoon heard the wrong thing over the radio - it could march straight into enemy hands! There are even communications protocols for Presidents of the United States of America! Those protocols keep the people of the world informed without going into a frenzied panic - or marching straight into enemy hands!

IP


The internet is a wide network - it spans the globe. Even into near-earth orbit. The ISS and its occupants tweet from the space station! This web of computers is very complex - it has smaller networks within and it has branched off networks. It exists over wires and radio signals. Each connection has an address. The address schema (a schema is a way to organize something) is called IP. Don't confuse this IP with Intellectual Property - also technology related. This IP is Internet Protocol.

IP is how the systems that run the network know how to find your computer. It's how your computer knows how to find NASA.gov. It's how your computer sometimes knows how to find your printer...and sometimes when it doesn't.

Your computer can use IP to talk to itself. There's a special address for this: 127.0.0.1 there's also a special name for it (called an alias): localhost. Your computer is also configured with an IP address for the internet - the network gateway. That IP address is where your NIC sends all of your requests that are bound for the internet. There could be other devices connected to your network - like your printer, other computers, servers, storage, your home, TV, etc. Those have their own IP address on your network. They can communicate within your internal network using that IP address. They can also communicate on the internet through the same internet gateway as your computer!

Ports


Along with the IP address, computers listen for incoming calls on Ports. Ports allow one computer to have many "conversations" at the same time. Ports can be numbered 0 to 65535 (that's 16-bits unsigned). Some common ports are:
80 - http
443 - https
There are many other common ports for things like ftp, ssh, and mail.
When your NIC sends a message, it opens up a port. It uses that port to send and receive messages to and from the other side.
What the receiver sees is your internet gateway - your router. Actually the last router in your network before your message gets sent into the great beyond of the internet. After that, it gets passed around the internet routers to your destination - if it can be found.

Continued Learning


The internet may be a fascinating and complex web of networks that span the globe, but almost every bit of it operates on a few core technologies. TCP/IP is how connections exchange data over a not-so-reliable network and HTTP is how two computers know how to interpret what's sent over the networks.

There is a glaring issue with all of this - if all this data is flying over the network and it really is possible to tap into those messages, how do we keep others from prying in on our messages? The answer is encryption! TSL to be specific - that's https to you. Well see how that works and how your computer knows the IP addresses of websites in upcoming posts in this series.

More information on UDP, TCP and IP is available in Wikipedia:

https://en.m.wikipedia.org/wiki/User_Datagram_Protocol
https://en.m.wikipedia.org/wiki/Transmission_Control_Protocol

Wednesday, January 31, 2018

Web Basics - HTTP

I occasionally get asked to mentor by those interested in learning to code. My response and approach will vary depending on the situation. I've done this in a pair programming context, over the wire, and by simply offering advice from time to time. I enjoy mentoring, especially when the mentee shows commitment and drive. This blog was originally started on the basis of providing mentorship to my readers. With that, let me get back to my roots for this post.

If you want to learn web programming, you should probably start with the basics - HTTP. If you are mildly in IT (or have a pulse) you should understand the HTTP protocol to some extent. I'll start you on your journey to understanding this lifeblood of the internet.

http logo

What is it?

HTTP is a protocol for communication between two computers. In fact, it is an acronym for Hypertext Transfer Protocol. In HTTP, there is a request sent by a sender (client) to a receiver (server). And the server should send back a response.

The request has two parts, the headers and the body. The headers has information about the request and the body is where you put data you want to send to the server.

Verbs


There are different types of requests. Each type uses a different HTTP Verb (also called Method). Verbs are an important part of HTTP. They tell the server what you (the client) would like to do.

Verbs can be split into two categories - queries and commands. The most common are GET and POST.

GET is used to get a web page or some data. It is a query.

POST is widely used to do something, such as send an email or submit an order form. It is a command.

PUT is another command. PUT is meant for updating something. You might POST a file and later need to update the file's contents so you would PUT an update. There's also a DELETE command.

Another common query is OPTIONS. This is a request foe the server to tell you what you can do. Often servers will only accept certain verbs. OPTIONS can tell you which ones.

There are many others verbs. You can find more at the official w3 site at the end of this post.

Responses

When a request is made, the server can respond. One thing included in a response is a response code. A response code has 3 digits and ranges from 100-599. Each hundred has a different meaning. 100's are reserved, 200 level is success, 300 is content changed, 400 is a client error, 500 is server error.

Some common codes are 200 - OK, 204 - OK with no data returned, 302 - temporary redirect, 404 - not found, 500 - server error.

In addition to the response code, there may be other headers and content returned from the server. Some of the headers describe the content. For example, Content-Type and Content-Length. These tell your client (browser for instance) how to handle the content in the response. Should it display the content in a browser? Save it to a file? Open it in a plugin?

The Network

When we use the web, we are primarily using HTTP over TCP/IP (your next topic :) ). TCP is the transport mechanism/protocol (how) and IP is the addressing schema (where).

When you send an HTTP request, the server will need to know where and how to respond. Some of that information is in the request headers. And some of it is handled by the networking systems (everything that moves your request over the internet, from your client to the server).

Continued Learning

Most programming languages have HTTP clients that you can use is your programs. This is so common nowadays, that you cannot do programming for long without using an HTTP client. And when you do web programming, its at the core!

For example, web apps you use all the time use AJAX (Asynchronous JavaScript and XML) to make HTTP requests using the JavaScript code that the app runs in your browser! These requests do all sorts of things like fetch data for part of the page and send data to the server. When you see part of page spinning while it loads, that's usually the app waiting for a server to respond to an AJAX request!

You can find version 1.1 of the official protocol at rfc2616 

Learning HTTP is the start of your journey to understanding how the internet works. We'll explore TCP/IP in the future...its another core technology behind what the world runs on today.