If you’re here for my experiments in culinary science move along swiftly, this post isn’t for you. This is all about enterprise architecture versus cloud native architecture.

RDBMS is a meatball

Enterprises use (or at least have used) Relational Database Management Systems (RDBMS), and such things have become deeply embedded into the organisation and culture around maintaining ‘books and records’ of the firm. Something I’ve previously labelled the ‘cult of the DBA'[1].

Enterprise meatballs don’t scale

RDBMS are generally limited by ‘the biggest box money can buy’. That’s not entirely true since the advent of Oracle Real Application Clusters (RAC), but by then much of the norms of RDBMS use were well established. The story goes roughly like this.

You might have a business need for some of my data, but you can’t use my database because my application is already running the biggest box money can buy to the ragged edge of its performance envelope[2]. Get your own RDBMS with its own box, and I’ll give you a copy of the data with an Extract Transform Load (ETL) job

and so we get spaghetti

RDBMS to ETL to RDBMS to ETL to RDBMS to you get the drift… Meatballs and spaghetti, spaghetti and meatballs.

It quickly gets messy. The worst part is the T in ETL, because the shape and naming keeps changing as data gets re-purposed for different uses.

What changes in the cloud?

Sticking with the metaphor, the cloud has infinitely large meatballs[3], so no need for spaghetti any more.

‘Cloud scale’ architecture liberates us from ‘the biggest box money can buy’ because the clouders found ways to scale horizontally. This is largely achieved by throwing off the shackles of ‘relational’, though we get to keep that if it’s really needed, and we can still use SQL too if that’s useful.

This simplifies things greatly. Every app can rally to the same source of truth, and ‘master data management’ boils down to the good management of one giant database rather than the cat herding exercise of figuring out how you got 122 different ways of describing ‘yield curve’.

This does not map well to present enterprise organisations

Meatballs, and the monoliths built on top of them, fit super snugly into traditional organisation structures (the purpose, boundaries and budgets for each siloed function). The spaghetti that wired everything together then became a cultural norm (how we do things around here).

Enterprise adoption of cloud native data management might hold the promise of greatly simplifying everything, but will be fought every step of the way as it cuts across the organisation structure and culture that evolved around it.

If (as Adrian Cockcroft says) ‘DevOps is a reorg’ then this is the same. Somehow ‘cloud data management is a reorg’ sounds less catchy. It should probably happen alongside the DevOps reorg anyway.

Notes

[1] See NoSQL as a governance arbitrage
[2] This is usually somewhere between a small fib and a massive lie. The biggest box that money can buy has been bought in the anticipation of many things that might affect capacity management over time, including how long it takes to get approval to buy anything. But the lie is told anyway because who wants to worry about another group’s capacity needs (or worse still setting up an internal charge back for their usage)?
[3] Not actually true, but in the real world you’ll run out of money before they run out of capacity.