Logic Apps

BizTalk open source: a win win?

gindafield

Yesterday, Microsoft announced that its on-premises integration middleware product BizTalk Server will become partly open source. First step is that all the 10K+ schemas (mostly B2B, EDI) have been released and are now available on Github.

Next step will be to make it possible for the community to contribute to the adapters, pipelines and tools.

My take on all this, is that it has the potential to become a win win for both Microsoft, partners and customers, provided that a number of things are executed well. Let me try to explain how:

  1. Microsoft BizTalk Server is rapidly turning into the on-premises LOB proxy (or gateway) that makes it possible to bridge legacy on-premises applications to the Azure iPaaS (mainly Logic Apps and API Management, plus Service Bus, Event Grid, etc.). This is how Microsoft IT has positioned it during Integrate 2017 in London. Bottom line in this (great!) architecture: BizTalk = legacy gateway and iPaaS = all the logic and modern integrations.
  2. Becoming (partly) open source, means that the community can contribute to the quality and number of schemas, adapters, pipelines and tools. This makes the role of BizTalk as an on-prem LOB proxy even more relevant, enabling even more legacy applications to bridge the gap to the public cloud. BizTalk basically has the potential to become an even greater on-ramp to the public Azure cloud.
  3.  Microsoft will remain focused on making sure the core BizTalk engine remains relevant and can run on the latest versions of their own platform (Windows, SQL Server, Visual Studio, .Net) and provides a terrific bridge to the public Azure cloud. This includes the non-functionals like end-to-end hybrid monitoring and management.
  4. The community has to be supported and made enthusiastic about contributing to the, what we can basically call “on-premises LOB adapters”. This is going to be the hard part of this open source endeavor, in my opinion. But, as we have seen in the past with ISVs leveraging the popularity of BizTalk to position and sell their adapters and basically “using” BizTalk to become more successful themselves, open sourcing the adapters will potentially have the same impact. But this time it’s not about leveraging BizTalk, but leveraging the hybrid integration stack. Time will tell. In the meantime, Microsoft can stay focused on the core and the bridge to the public cloud and in the meantime probably can transfer a couple of engineers to the iPaaS teams.

My $.02 only.

Cheers, Gijs

The can-you-do-that-guys

gindafield

Here at the #Integrate2017 event in London (26-28th June), I loved the keynote today by Jim Harrer (Microsoft Pro Integration Group PM). During the last 5 minutes of his presentation, he nailed it!

As I wrote before in another blog post (“Integration is just one of the skills needed”), iPaaS is not about just integration, it’s about creating business apps. Integration is part of the multi-disciplinary teams that build solutions. And these solutions are more and more built by using the 80+ Azure PaaS building blocks (see my most recent blog post “iPaaS, what else?”). These building blocks are not just about moving information from one location to another (including from and to hundreds of SaaS apps), but more and more also include Big Data and AI (artificial intelligence) capabilities. Making it possible to integrate things like cognitive services, machine learning, etc. Creating real end-to-end business apps that the business wants, now! With technology that until a year ago, was just not available (at a reasonable cost) to smaller sized companies.

Being an integration guy, you do have a special role in the teams. You are the guy that connects the building blocks and makes sure that the business app actually is resilient. And that you can properly monitor and manage the solution.
The time-to-market for these apps is phenomenal. Instead of weeks or months, you can create value in hours or days! And the speed at which Microsoft is adding not only the functionalities but, more importantly, the non-functional features is amazing. They build the platform, we build the solutions!

During the conference we’ll of course learn about new features that have just been released or will be released in the very near future. But to me, that is not the most important part anymore.

The IT world to me is clear now: integration folks have to become “the can you do that guys“.

We need to show our customers what is actually possible by assembling all these great building blocks into very valuable business solutions. Just do a PoC or Pilot and show the customer you’re at what you can build in such a short time. Sooner or later, your customer will also be saying “iPaaS, what else!“. Our customers are all becoming software companies. We can help them do just that!

Cheers, Gijs

iPaaS, what else?

gindafieldLeave a comment

Integration is more important than ever, in all the systems of innovation we create.

With multi-disciplinary DevOps teams, we build and operate solutions that encompass all tiers; from UX to back-end and everything in between. Middleware specialists are juse part of the team and share their knowledge on integration specifics through cross team guilds. More and more, we build those solutions based on the service paradigm. If we don’t need endless scale (if we’re not doing B2C basically), we just build it SOA style using PaaS technology. If we do need enormous scale and continuous innovation (because we don’t want to loose the consumers that get bored easily), we’ll do it microservices style. But we also still have to cope with existing systems. They can be legacy, but they can also be SaaS. Especially in a SaaS before PaaS before IaaS environment, migrating commodity applications to their SaaS counterparts are quick wins. But, these SaaS solutions are still most of the times silo applications that we have to cope with. For these applications, on-prem legacy or more modern SaaS, we need to create wrappers, so they can expose their task- and entity services or API’s so we can compose them into greater solutions. As a whole, we’re building agile solutions with a mix of silo applications, (micro)services applications and a set of building blocks provided by the iPaaS and aPaaS platforms we use. And in the Microsoft world, we’re talking about Azure then.

In those environments, to me it does not make sense anymore to build new solutions with legacy middleware. To me, implementing a BizTalk Server on-premises (or in IaaS for that matter) just does not make sense anymore, in a greenfield environment. Of course, when you have already invested lots of time and money in an on-prem BizTalk based ESB, you should leverage that. And those environments will keep on running for years to come. But when it comes to new integrations, why use BizTalk? I can really only come up with 1 reason, and that is: “I really don’t want my integration middleware to be running in the cloud” (for whatever reason). But that decision does not have anything to do with technical capabilities.

Let me try and clarify that.

First of all, what we often hear is that it doesn’t make sense to use cloud integration middleware (iPaaS) if most of your systems run on-prem. Uh, why not? With an On-prem Data Gateway and an ExpressRoute connection, latency is not an issue. And talking about latency, the biggest latency in any integration built with BizTalk Server is the freakin’ MessageBox hops!

Second, integrating SaaS applications is not something else (from a location point of view) than integrating on-prem applications. Most SaaS applications don’t run on Azure. Even Office 365 doesn’t run on Azure. So, what is “the cloud” anyway? From an integration perspective, integrating Salesforce from within Logic Apps is the same as integrating an on-prem SAP system when it comes to location dependencies or preferences!

Third, legacy integration is not really that much easier with “out-of-the-box adapters”. It’s fairly easy to either use the On-prem Data Gateway or create a custom API app to talk to the legacy application. Most of the times, the number of interfaces is fairly limited. And, in a modern API based integration world, broad transaction scopes are not used anymore, so relying on idempotency and compensation logic is much more the norm. These type of interfaces are really easy to build as an API app.

Apart from these three very important reasons, the last important point I want to make is this: We have been building monolithic ESBs. It is very hard to deploy individual orchestrated task services because of all the dependencies in BizTalk Server. BizTalk Server has in fact become a monolith (or makes it very hard not to implement monolithic solutions) which in most cases is very hard to manage and monitor. iPaaS makes it much easier to deploy, manage and monitor integration solutions. With ARM, Application Insights, Azure Monitor and of course the Azure portal, it has become really manageable. And as soon as Service Map also makes it to PaaS, we’ve really evolved into a much more mature iPaaS than BizTalk Server has every been. I’m really a fan and don’t see any reason anymore why we should implement new integrations on-prem!

iPaaS what else

Cheers, Gijs

 

 

iPaaS should not become your Trojan Horse

gindafieldLeave a comment

I’m currently involved as a cloud architect at an insurance company, working on the hybrid cloud reference architecture. They are making a move from a hosted environment to a hybrid cloud. And cloud engineers are working on the detailed designs for networking, storage, subscriptions, identity & access and integration. Integration between the private cloud and the public cloud as well as enterprise integration and B2B integration.

The customer is currently using BizTalk Server for enterprise- and B2B integration. The hosted environment is managed by a 3rd party as well, which means that the customer’s IT department only focuses on the functional management of applications. This also applies to BizTalk Server. New services are provisioned by filling out a form, waiting a couple of days or weeks and getting access to the new service. All is well so far, except for time-to-market, cost and being able to make use of the latest and greatest technologies.

Hybrid cloud is needed to shorten the time-to-market and innovating business processes and at the same time decrease IT spend on infrastructure. The whole thing about (hybrid) cloud is the on-demand characteristics and also being able to move away from traditional solution development to agile solution development, deployment and operations. Because provisioning is so fast, solution development can become much faster as well.

This is not an easy endeavor I can tell you.

Apart from the organizational aspects (devops is a topic on its own; is your organization ready for it?), the constantly evolving cloud and at the same time the constantly applied pressure from the business (“hey, we now have shorter time-to-market; let’s see it!”) makes life not really easy for the IT folks. We’re working on the reference architecture, but in the meantime several projects are underway to deliver cloud based solutions. Some SaaS, some PaaS and some we-don’t-really-know-what-kind-of-aaS. Every day we run into issues with regard to security and governance aspects. You’re really going to store customer senstive data in a NoSQL database running on a public facing linux box? Let’s think a little bit more about that. Refactoring the (hybrid) cloud solutions that have been delivered so far is the first thing we have to do once the reference architecture and the detailed hybrid cloud designs are done. That, and making sure the organization can actually cope with hybrid cloud deployments, management and governance.

In the “good old hosting days”, security was designed, applied and governed. Processes were in place and everything worked fine. Today however, checking or unchecking a single box in the Azure portal can have quite the impact. Suddenly, data leaks are possible. And we thought all was covered. Not.

Infrastructure as Code (which not only applies to IaaS but also to PaaS), is a mandatory thing. Clicking in a portal should be avoided. Cloud resources have to be deployed and managed by means of code. Versioned code. Code that has been reviewed and tested. Since full DTAP environments are rapidly becoming a thing of the past, your DTAP process has to be in place pretty well in order to prevent screw-ups with production data.

Why is the title of this blog post about iPaaS (Azure Logic Apps, API Apps, Functions, API Management, Service Bus) and a Trojan Horse specifically? Integration is at the core of everything when it comes to hybrid cloud. All aspects of the hybrid cloud architecture are touched here. Before you know it, things are tried out and put in production with all the potential risks as a result. Let’s protect ourselves from that.

Four things are important here:

  1. Have a reference architecture for hybrid cloud. New rule apply here! Hybrid cloud is not private cloud. This should at the very least contain the architecture principles and high level requirements that apply to all hybrid cloud solutions deployed. Example principle “Passwords should be stored in a safe place”. High level requirement resulting from that “Passwords used in scripts and code should be stored in Azure Key Vault”.
  2. Document the Solution Building Blocks. Azure is box of legos. Make sure that you know how to use all those building blocks and make sure that everybody knows the rules about how to use them in which scenario. Solution Building Blocks are not evil, but necessary artifacts. When do you use SQL Database, Blob, DocumentDB? How does security relate to these choices?
  3. Hybrid cloud needs hybrid service management. Make sure your IT service management sees your private cloud, hosted cloud and public cloud as one hybrid cloud and is able to manage that.
  4. Design and apply the right level of governance. Architectures, principles, requirements and solution building blocks are completely worthless if you don’t make sure they are actually used (in the right way). Peer reviews. Signed-off solution designs. Random inspections. These are all necessary things that you should cater for in your devops teams.

And remember, these things apply to your organization, but also to your IaaS, PaaS and SaaS solution vendors.

Let’s keep the Trojans out of your hybrid cloud!

Cheers, Gijs

Integration is just one of the skills needed

gindafieldLeave a comment

In modern enterprises, business solutions are built by agile teams. Agile teams are by design multi-disciplinary. The Product Owner is responsible for the product backlog and the team(s) build and implement the stuff needed. In these modern enterprises, the need for an innovation & differentiation layer on top of systems of record is an absolute necessity to enable shorter time-to-market of these solutions and, in some cases, digital transformation.

In the Microsoft world this innovation & differentiation layer is basically provided by Microsoft Azure and Office 365. The first one is needed for the (big) data, business intelligence, integration, process execution & monitoring and web & mobile UX capabilities, and the latter one for the collaboration and document handling capabilities. In Microsoft-centric application landscapes, Dynamics 365 will be the way to go for your systems of record for customer interaction (CRM) and resource planning (ERP). In the future, whenever you’re ready as an enterprise, the whole two-speed or bi-modal approach will be a thing of the past, and the cloud infrastructure will enable just one-speed IT; full speed! But that will still take several years before it has become a reality that most enterprises can deal with. Because it’s not only an IT thing, but more an organizational thing (how do you keep on adopting these agile solutions; won’t we become tech-fatigue?).

Many skills are needed in the agile teams we deploy today. And when scaling the teams, a layer on top of the teams is needed to manage the portfolio and program aspects in a better way. The Scaled Agile Framework (SAFe) is a good example (I personally have experience with; I’m a SAFe Agilist :-)). Quite some enterprises have implemented this, or an alternative to this like LeSS (Large-Scale Scrum). This also facilitates DevOps at a larger scale.

Even in agile environments, enterprise architecture is still needed 😉 He or she is responsible for the overall architecture of the solutions that get built. Business architecture, information architecture and technical architecture are all still very important things. It defines the frameworks within which the solutions should be built. The agile teams work within these frameworks.

We also more and more see that agile teams focus on certain business domains. And that common practice within microservices architecture is that we don’t try to build stuff that can be reused by other teams as well. Unless we have one or two specific teams working on re-usable, enterprise wide functionality. It’s all about business value first.

Is the integration competence center something that is helpful in such an environment? Well, the role of the ICC will change. The ICC (just like any other CC) will not be involved as a “sleeping policeman” (pun intended) anymore. The ICC will basically be part of the enterprise architecture role, focusing specifically on the frameworks (to which the agile teams should adhere; comply or explain) and the enterprise wide functionality. Everything else will be solved by the agile teams. For specific domains. That way we are way more flexible and we still build re-usable enterprise wide stuff where absolutely needed.

Do integration projects still exist in the future?

I think not, or only in a very limited number of cases. Integration is just part of business solutions and should also be treated like that. The API developer, the UX developer, the DBA, the security expert, etc. They’re all integration capable team members. The commodity integration needs can be fully handled by the agile teams like that. And for the integration specials, like re-usable building blocks and patterns or the more difficult one-offs, we just involve the ICC, which probably is a separate agile team in the larger organizations. Their domain is cross enterprise. But they will not be roadblocks anymore, that used to slow down business solution development.

Will this change the way system integrators work? Absolutely. The more we can do to deliver complete agile teams (including big data, UX and collaboration folks), the better we can serve our customers! To become agile too, and shorten their time-to-market and maybe even transform and redefine their business models. As an integration specialist or integration team alone, you can’t do that. The folks that understand and can implement the whole Microsoft platform to deliver real business solutions are going to be the ones that enterprises will turn to…

Cheers, Gijs

The business value of Microsoft Azure Logic Apps

gindafieldLeave a comment

I’ve written a paper on the business value of Microsoft Azure Logic Apps for Integration. Mainly useful for CIO’s and IT managers considering Azure PaaS services for their Integration needs.

It describes the components of Microsoft Azure, and then drills down into aPaaS and iPaaS to position Logic Apps, API Apps and API Management. Furthermore it describes common integration needs in complex application landscapes such as keeping data in sync, creating composite apps, involving supply and demand chains and integrating apps and portals.

Next it describes the real business value, so that you can explain it to your business stakeholders as well. This includes creating value add on top of commodity SaaS apps, leveraging investments in legacy applications (your Systems of Record), decreasing time-to-market, channel renewal, agility: basically digital transformation using Gartner’s pace-layer application model.

Lastly it describes the different integration themes that Logic Apps can help you with.

Enjoy!

Please share as you like.

Cheers, Gijs

My take on the Gartner iPaaS MQ 2016

gindafieldLeave a comment

Yesterday, Gartner released its Magic Quadrant (MQ) on Enterprise Integration Platform as a Service 2016.

The strategic planning assumption this is based on reads “By 2019, iPaaS will be the integration platform of choice for new integration projects overtaking the annual revenue growth of traditional application integration suites on the way”.

I think they’re right.

Microsoft did not make the Leaders quadrant this time. This is mainly because of the fact that the 2016 MQ is based on cloud services that are generally available (GA) and the only service available from Microsoft today in this regard is Microsoft Azure BizTalk Services (MABS). Which is of course far from complete as we all know. And it is based on an architecture which has been rendered obsolete by now, with the arrival of Azure App Service.

The relatively good news is that Microsoft did make it to the Visionaries quadrant, but they still need to let IBM, Oracle and SAP in front of them. That’s not so good.

My take on all this is

  • Gartner correctly positioned Microsoft in this years MQ for iPaaS based on what’s actually available;
  • We should quickly forget about MABS and start looking forward (Microsoft can’t afford to make another architecture and delivery screw-up like with MABS);
  • Microsoft needs to quickly release to the public a stable first version App Service. I really hope Q2 will indeed be GA time for App Service with Logic Apps and that it functionally delivers on its promise;
  • Microsoft needs to strongly position App Service as an Application Platform as a Service but at the same time strongly position Logic Apps with API Apps and the Enterprise Integration Pack (or whatever it will be called at GA time) as the Enterprise Integration Platform as a Service. Customers see them as two different things (although I think that will change in the future, see my earlier post on this).

I strongly believe in App Service, now let’s make sure that Microsoft and System Integrators nail the Ability to Execute as quickly as possible and kick some Mule, Dell Boomi, Informatica and SnapLogic @ss. The strong Azure (Integration) community should use its forces not only to make sure that the world knows about Azure and how to use it, but should also keep on providing the best real-world feedback to the product teams so that they continually make the right choices with regard to backlog prioritizing. I want this to be in the upper right corner in the iPaaS MQ for 2017 and it should be, with all the efforts put into it right now.

We all know CIO’s take Gartner seriously, so Microsoft (and System Integrators) should take Gartner seriously as well.

Cheers, Gijs