Serverless bitten in the ass by relational databases

Thomas Erl wrote this wonderful book on the SOA’s 8 principles of service design quite a while back. Together with Hophe and Woolf’s Enterprise Integration Patterns that’s basically almost all you need to know by heart, when starting out as a rookie (integration) developer.

No matter what software architecture used, the things learned from these books should always be applied. So, also in this new serverless era, the same applies.

But, the hardest part of every software architecture is the data architecture. You can fantastically modularize functionalities and integratie them at will, but if the data behind the modules is one big blur of relational mess, good luck with your APIs, principles, patterns and Devops then!

Is SQL Server (or any other relational database) actually witholding us from creating true serverless, reusable and autonomous APIs? Will the Graph or DocumentDB or CosmosDB help us out? Is big data coming to the rescue? Schema on read? CQRS? Blockchain based transaction ledgers?

Probably 80% of data in any given relational database is not relational. So why use a SQL Server database then? Why shoot ourselves in the foot right at the start?

If we want to become true serverless, not only do we need to adhere to the principles of service design and integration patterns, but also need to rethink our data architecture.

Eventual consistency looks like a nice concept, but in a real-time world quite hard to implement and make successful. But maybe it’s our only option? In any case, the use of relational data storage will in my opinion decline quickly. SQL Server will *have* to morph into something new, with less emphasis on the relational aspect. I know Microsoft have already started that transition.

Getting the right architecture guidance on serverless in combination with data storage is going to be crucial in the coming time.

Cheers, Gijs


Integration in the SaaS and API era

A lot is happening in the IT space these days. Moore’s law is outdated, not relevant anymore. We need a new law. Gijs’ law (LOL) would sound something like this:

“IT paradigms will change every other day, but the underlying principles will remain more or less the same. Unfortunately we forget that each and every time.”.

So many new initiatives happen today and can become successful so quickly, that large IT companies with lots of smart developers, patents and market share can become irrelevant within months or even weeks.

Especially when it’s about social media tooling, market shares can very quickly become significant and a startup with 5 developers and a savvy marketing / business development person can beat billion dollar software companies so fast that they don’t even have a clue what’s happening to them.

Most of this is happening in the consumer space so far. But is the same going to happen in the enterprise space as well? Will that be possible? Or are enterprises more conservative?

Have a look at, and Just a couple of examples of smart initiatives that quickly become popular and even leaders in their space. Attracting loads of smart investor money in some cases. And lots of users. Most of the times, inventions and innovations are just smart combinations of already existing stuff. Just nobody cared enough or was smart enough to combine them until just then.

The paradigms used by these startups are very interesting not only for consumers but also for businesses. Consumerization of IT can actually happen in more places (not only the device or app you use) than we can imagine.

Isn’t for example IFTTT an interesting concept for automating business integration? “If my post gets a Like then do add this person to my CRM”. “If new person added to my CRM then do send him the latest newsletter.”. “If temperature is trending up then do adjust airco setting.”. Just some examples.

What these kind of consumer products miss is the mission critical “stamp”. But on the other hand, if the service doesn’t provide the right service levels, consumers will go elsewhere (and take the migration effort for granted). They *have* to provide the right service levels. Not matter what tooling they use. Otherwise they’ll loose their market share and become irrelevant quickly.

For enterprise users it is different. They need to know up front what kind of service levels are provided. Otherwise they don’t sign up for the service. No risks can be taken. And if they get dissappointed a couple of times, they get mad with the vendor. And maybe get a couple dollars compensation. But it’s a hell of a job to “exit” from that vendor’s solutions and services most of the times. Especially if the underlying products are not based on open standards. Migration is most often a nightmare.

The larger vendors will problaby simply take over some of these new technology companies (for billions of dollars) and integrate them in their software stack. In that way quickly adapting to the changing market needs and giving the products the “enterprise treatment”. The same will probably happen in the API, SaaS and Micrososervices integration space. Some of these new and real easy to use consumer SaaS solutions will quickly make it to the enterprise world. That’s my vision at least.

What we should not forget though, is that things like mangeability, auditability, supportability are always going to be important in the enterprise market. The problem is that new products *never* start with these pain-in-the-neck “-ability” requirements in mind. And that not only applies to consumer products but also to enterprise products, unfortunately! Larger software and SaaS vendors are shamelessly releasing services into production, without having catered (enough) for these “-abilities”. And what also remains important, no matter what new IT paradigm we come up with: the underlying principles remain the same. When exposing an API or Microservice, make sure it ticks all the known and proven service design check boxes. That way the chance that end-to-end solutions built with those services or APIs will actually become manageable, auditable and supportable are much bigger.

Why do we each and every time forget about these simple software design principles and standard patterns? Developers are either too lazy (the happy flow works; check-in; done) or too modest (nobody’s going to use this piece of software). Maybe it helps if the folks creating and posting sample code actually post enterprise-grade samples.  None of this is of course gooing to change, and hence my law will remain applicable to infinity and beyond. My endless loop, free of charge 🙂

Cheers, Gijs