Chances are that most of the transactional data in your organization are still stored in relational databases like SQL Server, Oracle and DB2. When modernizing your application landscape this will rapidly change. Why? And what does that mean?

The problem with relational databases

Modern software solutions are service oriented and very distributed, where the systems of record are still hardly having a user interface but do have good, mediated APIs that expose business logic as services. And the tailormade solutions are created through low-code and integration on top of these APIs. The composition of these services exposed through APIs is taking place on a higher level. That is where the differentation takes place. That’s where you use small apps that do exactly what you need, fully integrated with your application landscape. The underlying systems of record will in the meantime also be modernized by the suppliers, where these will also move away from the silo approach and relational databases. Mark my words.

There are three great problems with relational databases:

1. Relations are hardcoded, to ensure referential integrity. This cannot be used anymore in a software architecture based on microservices and apps.
2. Transactions are executed in the context of this one database. That no longer works in a distributed application landscape. Commit and rollback are phenomenons of the 80’s that cannot be used any longer.
3. And, not to forget, the costs of relational databases are way to high, because most of the times they are end-user based.

In practice, lots of relational databases are misused to store non-relational data. Just because the thing is already there and operated. And because it easily goes with the regular backups. That’s a wrong strategy. It should change. Now.

How to prepare?

Now is the time to slowly say goodbye to relational databases in your tailormade solutions. For every need of data storage, try to find out how relations with other data can be created. These days, there are lots of different storage methods, especially in the cloud. To name a few:

• NoSQL or Document database – here you can for example store JSON objects and search easily and flexibly. These databases don’t use schemas.
• Graph database – this is where you can store all kinds of information and dynamically create n:m relations. Ideal to create profiles and have the data that is interesting for you actually come to you.
• Data lake – here you can store all kinds of varieties, volumes and volocities of data and do big data analyses. This will often be machine generated data, such as with IoT scenarios.

In a (micro)service architecture it is important to cater for eventual consistency. The database cannot take care of that for you anymore, simply because that cannot be done overarching multiple databases. How you organize your services and APIs will become more and more important, including the data architecture belonging with that.

So, what about business intelligence and analytics? These solutions will have to change rapidly as well. More and more data that is needed to show in dashboards or reports will come from non relational databases. Traditional ETL cannot be used anymore. Cubes cannot be created that easily anymore. Artificial intelligence as part of your analytics solutions, even for the traditional hindsight applications (“wat happened?”), cannot be avoided. Let alone if you want predictive (“what will happen?”) or prescriptive (“what do I have to do to realize this?”) solutions. You’ll have to start thinking about these kind of things and adjust your architecture to it. Waiting for “the hype to blow over” is not a good strategy. The relational database is already clinically dead IMHO. You can throw your SQL skills in the bin in no time!

Cheers, Gijs