Agile

Integration is just one of the skills needed

gindafieldLeave a comment

In modern enterprises, business solutions are built by agile teams. Agile teams are by design multi-disciplinary. The Product Owner is responsible for the product backlog and the team(s) build and implement the stuff needed. In these modern enterprises, the need for an innovation & differentiation layer on top of systems of record is an absolute necessity to enable shorter time-to-market of these solutions and, in some cases, digital transformation.

In the Microsoft world this innovation & differentiation layer is basically provided by Microsoft Azure and Office 365. The first one is needed for the (big) data, business intelligence, integration, process execution & monitoring and web & mobile UX capabilities, and the latter one for the collaboration and document handling capabilities. In Microsoft-centric application landscapes, Dynamics 365 will be the way to go for your systems of record for customer interaction (CRM) and resource planning (ERP). In the future, whenever you’re ready as an enterprise, the whole two-speed or bi-modal approach will be a thing of the past, and the cloud infrastructure will enable just one-speed IT; full speed! But that will still take several years before it has become a reality that most enterprises can deal with. Because it’s not only an IT thing, but more an organizational thing (how do you keep on adopting these agile solutions; won’t we become tech-fatigue?).

Many skills are needed in the agile teams we deploy today. And when scaling the teams, a layer on top of the teams is needed to manage the portfolio and program aspects in a better way. The Scaled Agile Framework (SAFe) is a good example (I personally have experience with; I’m a SAFe Agilist :-)). Quite some enterprises have implemented this, or an alternative to this like LeSS (Large-Scale Scrum). This also facilitates DevOps at a larger scale.

Even in agile environments, enterprise architecture is still needed 😉 He or she is responsible for the overall architecture of the solutions that get built. Business architecture, information architecture and technical architecture are all still very important things. It defines the frameworks within which the solutions should be built. The agile teams work within these frameworks.

We also more and more see that agile teams focus on certain business domains. And that common practice within microservices architecture is that we don’t try to build stuff that can be reused by other teams as well. Unless we have one or two specific teams working on re-usable, enterprise wide functionality. It’s all about business value first.

Is the integration competence center something that is helpful in such an environment? Well, the role of the ICC will change. The ICC (just like any other CC) will not be involved as a “sleeping policeman” (pun intended) anymore. The ICC will basically be part of the enterprise architecture role, focusing specifically on the frameworks (to which the agile teams should adhere; comply or explain) and the enterprise wide functionality. Everything else will be solved by the agile teams. For specific domains. That way we are way more flexible and we still build re-usable enterprise wide stuff where absolutely needed.

Do integration projects still exist in the future?

I think not, or only in a very limited number of cases. Integration is just part of business solutions and should also be treated like that. The API developer, the UX developer, the DBA, the security expert, etc. They’re all integration capable team members. The commodity integration needs can be fully handled by the agile teams like that. And for the integration specials, like re-usable building blocks and patterns or the more difficult one-offs, we just involve the ICC, which probably is a separate agile team in the larger organizations. Their domain is cross enterprise. But they will not be roadblocks anymore, that used to slow down business solution development.

Will this change the way system integrators work? Absolutely. The more we can do to deliver complete agile teams (including big data, UX and collaboration folks), the better we can serve our customers! To become agile too, and shorten their time-to-market and maybe even transform and redefine their business models. As an integration specialist or integration team alone, you can’t do that. The folks that understand and can implement the whole Microsoft platform to deliver real business solutions are going to be the ones that enterprises will turn to…

Cheers, Gijs

Atomic Integration

gindafieldLeave a comment

Atomic Integration, I like the term. And I like the concept. Of course it has disadvantages, but don’t the advantages outweigh these? Let’s explore.

When the Integration Monday session “The fall of the BizTalk architect” (by Mikael Sand) was announced I was immediately triggered. It was the combination of the title and the presenter (Mikael often rocks) that made me create a note to self to watch it later on. And finally I did. And it triggered me to write this blog post.

For years we have been telling our customers that spaghetti is bad and lasagna is good. And we’ve also been telling them that re-use is the holy grail. And that by putting a service bus in the middle that everything becomes more manageable, integrations have shorter time to market and the whole application landscape becomes more reliable.

But at the same time, we see that re-use is very hard to accomplish, there are so many dependencies between solutions and that realizing this in an agile manner is a nightmare if not managed meticulously. Especially if we need to deliver business value quickly, talking about createing stuff for later re-use and having depencies on other teams is a hard sell to the product owner and thus the business.

Another thing that is really hard with any integration archtitecture and the accompanying middleware with its frameworks that have been built on top of the actual middleware (and thus becoming a part of that middleware) is ownership. And that is a two-headed beast. First of all, ownership of the frameworks that run on top of the middleware, and second the ownership of the actual integrations.

The first one is already a hard nut to crack. Who pays for maintaining these frameworks? Does it get financed by projects? Very hard to sell. Does it have a separate (business) owner and funds? Never seen that before and it probably wouldn’t work because the guy that pays most gets his way and it doesn’t necessarily mean that the framework will be the best usable framework of all times.

The second one is even harder to manage. Who is typically the owner of an integration. The subscriber and thus the receiver of information? Or is it the sender a.k.a. the publisher of the information? Who pays for it? Is that the owner? And will he be managing it? And what happens if the ingration makes use of all kinds of other, re-usable custom features that get changed over time and in which the actual owner of the ingration is not interested at all?

Why indeed not do more copy-and-paste instead of inherit or re-use? The owner of the specific integration is completely responsible for it and can change, fork and version whatever he likes. And as Mikael says in his presentation: TFS or Visual Studio Online is great for finding out who uses certain code and inform them if a bug has been solved in some copied code (segment). And of course we still design and build integrations according to the well known integration patterns that have become our best friends. Only, we don’t worry that much about optimization anymore, because the platform will take care of that. Just like we had to get used to garbage collectors and developers not knowing what memfree() actually means, we need to get used to computing power and cheap storage galoring and therefore don’t need to bother about redundancy in certain steps in an integration anymore.

With the arrival of cloud computing (more specifically PaaS) and big data, I think we now get into an era when this actually is becoming possible and at the same time manageable. The PaaS offerings by Microsoft, specifically Azure App Service, are becoming an interesting environment quickly. Combined with big data (which to me in this scenario is: Just save a much information about integration runs as possible, because we have cheap storage anyway and we’ll see what we’ll do with this data later), runtime insight, correlation and debugging capabilities are a breeze. Runtime governance: Check. We don’t need frameworks anymore and thus we don’t need an owner anymore.

Azure App Service, in combination with the big data facilities and the analytics services are all we need. And it is a perfect fit for Atomic Integration.

Cheers, Gijs