Monday, June 22, 2015

When NOT to use a noSQL

...or more precisely, when not to use an eventually consistent concurrency model.


I've been reading a lot lately about trying to fit a square peg into a round hole. What I mean by that is they are trying to force noSQL databases, particularly ones with an eventual consistency model, into a transactional operation.


I've been reading about two step patterns and other types of hacks and second generation noSQL with built-in transactions. I want to just state an obvious point - how about don't use eventual consistency when you need a transaction. When you need a transaction, use a model that supports one - its plain and simple! When you don't need a trans and you need a dynamic model and/or high throughput use noSQL.


It seems to me that some architects are not quite understanding how to pick the right tool for the job. It can cost a fortune to make a mistake as epic as allowing "sophisticated users" to post multiple concurrent requests to transfer assets and have them all go through due to lack of single concurrency on an entity. Search the web for bit-coin heist or similar and you will find out how many millions of dollars it can cost.




I've had a concurrency issue manifest at a very small scale, one that did not involve transfer of assets and did not cost a fortune. It did introduce a seemingly random defect and it was not a noSQL backing store.




 I admittedly did a read-then-insert-or-update without wrapping the entire op in a transaction. I did this to simplify the API so that a Save method was exposed instead of separate insert and update statements. The client makes calls asynchronously and perhaps a user clicked five time or there was some other technical issue which resulted in five records being added to the collection. This illustration further illustrates the need for transactions (and for properly using them).


So why not consider polyglot persistence and apply the right persistence tool for the job? There is a lot of good information on how to design a system that can take advantage of different backing stores and completely abstract the data store from the application. With such a design, your system can use transactions when it needs to and eventual consistency and distributed data when it needs it.



1 comment: