APIs

The El Niño of the digital ecosystems

gindafield

After digitizing current processes we started optimizing them. All purely focused on efficiency. Saving costs. Improving margins. Happier customers. Now is the time to start the real digital transformation. A revolution is needed.

Many organizations are working on process optimization. They are looking at eliminating human interaction by means of digitization, integration and robotization. Organizations working LEAN will go a bit further and will continuously try to improve the efficiency of processes. Mostly, this is all based on current revenue streams and business models. Also, it is assumed that the organization has a relevant function and will keep that function in the ecosystem.

What’s happening around us?

Every organization is part of an ecosystem. Ecosystems that become more and more digital. By exchanging information digitally between parties in such an ecosystem, these ecosystems will become more and more efficient. First we used EDI and today, in modern application landscapes we use APIs more and more. APIs that are becoming smarter and smarter and can find eachother more or less automagically. APIs that are a direct interface to an existing application. An application that most of the times is a system of record. Not a very “interesting” application most of the times, only capable of handling basic administrative needs such as handling orders and ship notices.

The middleman in crisis

There are only a few systems of record that really deliver true value add for an organization. The real value add of an organization usually is in the uniqe position in an ecosystem and the processes it delivers by means of a combination of all kinds of software systems, on-premises and in the cloud. But what happens when your combination of services suddenly is not relevant anymore? Because ecosystems have found a way to solve the isssues in another, much better way? Your organization actually doesn’t have a function anymore? Because you are actually the middleman? Crisis!

We are seeing everyday that new ways of working together in ecosystems evolve rapidly. The most important unique selling point of blockchain is to cut out the middleman. This is possible because the system enables collaboration without having trust in place. You don’t have to trust eachother. The trust has been digitized. By means of smart contracts. Look around you. Blockchains that enable peer-to-peer lending, that make wholesale and banks obsolete, that replace marketplaces and trading systems, etc. etc.

Blockchain is a revolution

Blockchain is the El Niño of the digital ecosystems; it changes traditional collaborations between companies and disturbs the balance. We will probably see an Al Gore like person in the near future who will warn us of the dangers. But, just like in nature, it will take a while but a new balance will evolve. After which the landscape will be thoroughly changed. There will be many victims, but sometimes that is needed to make the next step, as a whole.

If your organization will become a victim depends on how much value add you deliver. Oftentimes this depends on the need for physical assets or infrastructure. For example: A supplier of green energy who does not have assets but only handles the trade and contracts can be replaced by a blockchain. The supplier that also owns the wind- and solar farms and the infrastructure for homes and factories will be much more difficult to replace. Until the time that you’ll be able to generate 100% of your own energy need. But if energy surplus needs to go back into the net, you’ll be needing infra again. Uber can easily be replaced by a blockchain. Uber doesn’t have assets. And really no value add. Blockchain is a revolution. In every boardroom this should be on the agenda permanently. Make sure you understand the technology and its impact. Don’t be an American member of congres who doesn’t understand Mark Zuckerberg’s business model and technology. Work out a solution and really start innovating!

Cheers, Gijs

 

Integration is just one of the skills needed

gindafieldLeave a comment

In modern enterprises, business solutions are built by agile teams. Agile teams are by design multi-disciplinary. The Product Owner is responsible for the product backlog and the team(s) build and implement the stuff needed. In these modern enterprises, the need for an innovation & differentiation layer on top of systems of record is an absolute necessity to enable shorter time-to-market of these solutions and, in some cases, digital transformation.

In the Microsoft world this innovation & differentiation layer is basically provided by Microsoft Azure and Office 365. The first one is needed for the (big) data, business intelligence, integration, process execution & monitoring and web & mobile UX capabilities, and the latter one for the collaboration and document handling capabilities. In Microsoft-centric application landscapes, Dynamics 365 will be the way to go for your systems of record for customer interaction (CRM) and resource planning (ERP). In the future, whenever you’re ready as an enterprise, the whole two-speed or bi-modal approach will be a thing of the past, and the cloud infrastructure will enable just one-speed IT; full speed! But that will still take several years before it has become a reality that most enterprises can deal with. Because it’s not only an IT thing, but more an organizational thing (how do you keep on adopting these agile solutions; won’t we become tech-fatigue?).

Many skills are needed in the agile teams we deploy today. And when scaling the teams, a layer on top of the teams is needed to manage the portfolio and program aspects in a better way. The Scaled Agile Framework (SAFe) is a good example (I personally have experience with; I’m a SAFe Agilist :-)). Quite some enterprises have implemented this, or an alternative to this like LeSS (Large-Scale Scrum). This also facilitates DevOps at a larger scale.

Even in agile environments, enterprise architecture is still needed 😉 He or she is responsible for the overall architecture of the solutions that get built. Business architecture, information architecture and technical architecture are all still very important things. It defines the frameworks within which the solutions should be built. The agile teams work within these frameworks.

We also more and more see that agile teams focus on certain business domains. And that common practice within microservices architecture is that we don’t try to build stuff that can be reused by other teams as well. Unless we have one or two specific teams working on re-usable, enterprise wide functionality. It’s all about business value first.

Is the integration competence center something that is helpful in such an environment? Well, the role of the ICC will change. The ICC (just like any other CC) will not be involved as a “sleeping policeman” (pun intended) anymore. The ICC will basically be part of the enterprise architecture role, focusing specifically on the frameworks (to which the agile teams should adhere; comply or explain) and the enterprise wide functionality. Everything else will be solved by the agile teams. For specific domains. That way we are way more flexible and we still build re-usable enterprise wide stuff where absolutely needed.

Do integration projects still exist in the future?

I think not, or only in a very limited number of cases. Integration is just part of business solutions and should also be treated like that. The API developer, the UX developer, the DBA, the security expert, etc. They’re all integration capable team members. The commodity integration needs can be fully handled by the agile teams like that. And for the integration specials, like re-usable building blocks and patterns or the more difficult one-offs, we just involve the ICC, which probably is a separate agile team in the larger organizations. Their domain is cross enterprise. But they will not be roadblocks anymore, that used to slow down business solution development.

Will this change the way system integrators work? Absolutely. The more we can do to deliver complete agile teams (including big data, UX and collaboration folks), the better we can serve our customers! To become agile too, and shorten their time-to-market and maybe even transform and redefine their business models. As an integration specialist or integration team alone, you can’t do that. The folks that understand and can implement the whole Microsoft platform to deliver real business solutions are going to be the ones that enterprises will turn to…

Cheers, Gijs

BizTalk Server, Microservices & APIs

gindafield3 Comments

This week I attended Integrate 2014 (a.k.a. The BizTalk Summit). It was great meeting up again with lots of folks I’ve got to know in the past 14 years (veterans like Tom Canter, Brian Loesgen, Bill Chesnut, Richard Broida and the “younger generation” like Mikael Sand, Saravana Kumar, Michael Stephenson, Kent Weare and many more, including lots of Dutchies and Belgians), since starting to work with BizTalk Server in 2000. It was also nice to meet some of the Microsoft product team architects (online and offline, Guru Venkataraman and, offline Evgeny Popov) getting to understand their vision.

What keeps on surprising me however, is how bad Microsoft is at positioning their great stuff and how good they are at confusing us all, including the integration specialists and customers. The community will need to help them with bringing the right message. Customers deserve this. They need to be comforted. After all, they spend hundreds of thousands or even millions of dollars and euros on implementing, supporting and migrating their middleware solutions and their businesses are relying on it.

I think it’s quite simple. In integration, we have a couple of things to take care of, now and in the future:

  • Message based A2A integration
  • Message based B2B integration
  • API based B2B and B2C integration
  • The underlying SOA architecture. Because folks, that’s what it is. At least the SOA as I have always understood and applied it.

Thinking in (orchestrated) task, entity and utility services is crucial. How to expose and compose these services and how granular they are (micro, nano or macro), is basically not that important. But it depends on the consumers of these services.

BizTalk Server is integration middleware that you can use to create an on-prem SOA architecture and use for message based A2A integration. It can also take care of “traditional”, message based B2B integration. We use SOAP, flat files, EDI and XML to exchange stuff in a very reliable and manageable way. We include MDM and other data services to take care of (master) data. And in order to expose (business) interfaces as APIs, we use the REST/JSON adapter. That way, creating hybrid buses using a federated bus architecture.

BizTalk Services (MABS) is the newer, cloud based integration offering. It is basically a B2B (EDI) gateway. A very logical place to have that in the cloud, since the cloud is a big DMZ and B2B is about structured communication with customers and partners outside your own firewalls. It’s a pitty though that this has not built on further since its first release 2 years ago (but we now know why: the team has shifted focus to the new microservices architecture and tooling).

Now we have two more things: APIs and Microservices.

APIs expose your business services to Apps and Portals  in a managed way. Mashing up services. With granular security. And you can even monitize them that way, exposing them to affiliates and other partners. For an indepth (IMO) view of API management and how it relates to this, please check this article.

Microservices are there to easily create solutions by combining functionalities exposed by these services. But again, these services are juse plain old utility, entity and task services. But now exposed as lightweight JSON/REST. And probably more granular. So they can more easily be versioned and deployed independently. Check out Martin Fowler’s vision on microservices here (also read his comment on SOA vs. Microservices). They can also be exposed through a gateway. See this article.

Service (micro, nano or macro, don’t care) composition is something you can do by means of integration patterns, implemented through some sort of pre-defined proces, being an itinerary, orchestration or workflow (but probably in the not to near future, composition is also something that is taken care of by using NoProcess, based on AI: the system just automatically finds out what to do best and which services it will need doing that, and will constantly finetune that automatically).

What’s going to make or break your (hybrid) architecture is design- and runtime governance of these services. In order to be able to compose services into solutions, you’ll have to know what services are available and how you can use them. In order to be able to manage and monitor your solutions, they have to be instrumented in a standard way, feeding management and monitoring solutions in a real-time fashion. API Management (by Azure API Management and Nevatech Sentinet) is doing that for APIs. SOA governance (by Nevatech Sentinet and SOA Software) is doing that for integration middleware. These somehow have to be tied together to provide end-to-end runtime- and design time governance (including ALM), tying in with your development environment. If I were Microsoft, I’d just buy the best solution and integrate it in the platform. Just like they did with Apiphany.

Now let’s hope that Microsoft:

  • Keeps the BizTalk Server brand for on-prem A2A and B2B integration and building a SOA.
  • Keeps the BizTalk Services brand for cloud based B2B integration.
  • Comes up with a new brand for the cloud based (micro)services architecture and tooling.
  • Versions and roadmaps these offerings separately
  • Clearly depicts usage scenarios, including hybrid solutions. A couple of good architecture pictures should suffice.

On a final note, check out Next Generation SOA. Thomas Erl and his co-writers have done a great job on simply explaining it and it all fully applies to modern architectures including APIs and microservices. It’ll really help you think clearly about complex, services based solutions.

Cheers, Gijs