Tuesday, March 17, 2015

When Your Test is Defective

It helps to verify and design the test with the domain expert. This will save much time downstream.


I had a defect in a system that I partly wrote last year. The defect was related to how we were fetching data from another system. It boiled down to a misinterpretation of the requirements. When it came into my work queue, I decided to take a pseudo- BDD approach.


The business analyst joined me at my desk and we began writing the acceptance criteria together in Gherkin syntax. Next, I created a Unit test via MsTest with the help of Moq framework. Then I implemented the fix.


 I had to break out the logic to run the test properly first, the original is more like a big run-on chapter with several too many methods. It could be a facade, except that it's all implemented in one fell class.


After making my logic testable, I created some fixtures right there in the test class. I needed two fixtures, one for each entity involved in the source data. In reality, each comes from a separate source, but are related. Similar to how client and assigned employees would be related.


With my fixtures in place, I set up fakes for the data access classes which now return my fixtures instead. The reason for using fixtures is so that the tests can be deterministic in analyzing the results. Fixture data does not change by some external force like data from a database can.


Next, I executed my method that gets the filtered data. This method grabs the data from both sources, joins it together to return the data I need, then passes it through a filter. That filtering is where the business rule rubber meets the road. And I got it wrong on the first pass...


You see what happened was...


Three things. First, we didn't have a complete setup in our fixture because we didn't learn that lesson yet - we are new at this BDD thing. Second, I setup the fixtures wrong. And thirdly, I implemented the initial query predicate wrong.


The first and second were resolved when the BA and I worked together to go through the test more thoroughly. The third was resolved after I did some testing against real test data via the data access API. Turns out I misinterpreted the meaning of one of the filter values.


Ultimately, the lessons learned here are several.


Many forms of testing are ultimately needed to produce real working code.


Acceptance criteria defined fixtures should be passed to the tests in an automated way to avoid that kind of copying errors.


The last point would also have forced me to write a domain specific object that was an aggregate of the two entities with the properties I really needed for the filtering. I took some shortcuts there.


Domain experts and code experts working together...priceless.

No comments:

Post a Comment