INTEGRATE 2018: Highlights from Another Great Event

34777644_920192494824567_1394121413341741056_o

Photo by Tariq Sheikh

Two weeks ago I had not only the privilege to attend the sixth INTEGRATE event in London, but also the great honour of speaking for the second time. These events always provide a wealth of information and insight as well as opportunities to meet face-to-face with the greatest minds in the enterprise integration space. This year was no exception, with at least 24 sessions featuring as many speakers from both the Microsoft and the MVP community.

As usual, the first half of the 3-day conference was devoted to the Microsoft product team, with presentations from Jon Fancy (who also gave the keynote), Kevin Lam, Derek Li, Jeff Hollan, Paul Larsen, Valerie Robb, Vladimir Vinogradsky, Miao Jiang, Clemens Vasters, Dan Rosanova, Divya Swarnkar, Kent Weare, Amit Kumar Dua, and Matt Farmer. For me, the highlights of these sessions were: Read more of this post

Integration Down Under… is UP & RUNNING!

For years now Integration Monday has been faithfully giving us webinars almost every week. There have been some outstanding sessions from international leaders in the integration space including MVPs, members of the Microsoft product team, and other community members. For the Asia Pacific community, however, it has always been a challenge to participate in the live sessions due to the unfriendly time zone.  (I certainly know what a struggle it was to present my own session last October at 4:30am!)  Even from the listener’s perspective, it is usually nicer to be able to join a live webinar and ask questions rather than to consume the recordings afterwards.

Thanks to the initiative of veteran MVP Bill Chesnut (aka “BizTalk Bill”) and the sponsorship of his employer SixPivot, we now have a brand new webinar series starting up in a friendlier time slot for our APAC community! Integration Down Under is launching its inaugural webinar session on Thursday, 8th February at 7:00pm AEST. You can register for this free event here.

This initial session will introduce the leaders and allow each of us to present as very short talk on a chosen topic:

    There are already more than twenty registrations even though the link has been live for only a few days. I hope that this is a good sign of the interest within the community!

Feeling really fortunate to be part of this initiative, and looking forward to delivering my intro to Event Grid talk! It will be a slightly scaled down version of what I presented at the Sydney Tech Summit back in November. Hope to see you there!

Great Experience at INTEGRATE 2017 in London!

Last week I had the privilege not only of attending the INTEGRATE 2017 conference in London, but presenting as well. A huge thanks to Saravana Kumar and BizTalk360 for inviting me as a speaker – what a tremendous honour and thrill to stand in front of nearly 400 integration enthusiasts from around the world and talk about Hybrid Connectivity! Also, a big thanks to Mexia for generously funding my trip. 14-DMT_NickHhauenstein

With 380+ attendees from 52 countries around the globe, this is by far the biggest Microsoft integration event of the year. Of those 380, only four of us that I know of came from APAC: fellow MVP speakers Martin Abbott from Perth and Wagner Silveira from Auckland NZ, as well as Cameron Shackell from Brisbane who manned his ActiveADAPTER sponsor stand. Wagner would have to take the prize for the furthest travelled with his 30+ hour journey!

After publishing one blog post summarising my take on the messages delivered by Microsoft (which accounted for half of the sessions at the event),  I followed on shortly after with another similar post highlighting the MVP community presentations. In addition to BizTalk Server, Logic Apps, and other traditional integration topics these also spanned into the new areas of Bots, IoT and PowerApps.

iPhone Import 156Aside from the main event, Saravana and his team also arranged for a few social events as well, including networking drinks after the first day, a dinner at Nando’s for the speakers, and another social evening for the BizTalk360 partners. They also presented each of the BizTalk360 product specialists with a beautiful award – an unexpected treat!

You have to hand it to Saravana and his team – everything went like clockwork, even keeping the speakers on schedule. And I thought it was a really nice touch that each speaker was introduced by a BizTalk360 team member. Not only did it make the speakers feel special, but it provided an opportunity to highlight the people behind the scenes who not only work to make BizTalk360 a great product but also ensure events like these come off. I hope all of them had a good rest this week!

As with all of these events, one of the things I treasure the most is the opportunity to catch up with my friends from around the globe who share my passion for integration, as well as meeting new friends. In my talk, I commented about how strong our community is, and that we not only integrate as professionals but integrate well as people too.

01-SundayNight-Mikael  21-Pizza_TomCanter

Arriving a day and a half before the three day event, I had hoped to conquer most of the jet-lag early on. But alas, the proximity to the solstice in a country so far North meant the sun didn’t set until past 10:30pm while rising just before 4:30am – which is the time I would involuntarily wake up each day no matter how late I stayed up the night before! Still, adrenalin kept me going and the engaging content kept me awake for every session.

And no matter what.., there was always time for a beer or two! Smile

22-PostPizza_TomCanter

I look forward to the next time I get to meet up with my integration friends! If you missed the event in London, you’ll have a second chance at INTEGRATE 2017 USA which will be held in Redmond on October 25-27. And of course, if you keep your eyes on the website, the videos and slides should be published soon.

(Photos by Nick Hauenstein, Dan Toomey, Mikael Sand, and Tom Canter)

Great Week at Ignite Australia!

TLGsLast week I had the opportunity to attend Microsoft Ignite on the Gold Coast, Australia. Even better – I had a free ticket on account of agreeing to serve as a Technical Learning Guide (TLG) in the hands-on labs. This opportunity is only open to Microsoft Certified Trainers (MCTs) and competition was evidently keen this year – so I am glad to have been chosen. Catching up with fellow MCTs like Mark Daunt and meeting up with new ones such as Michael Schmitz was a real pleasure. Of course the down side was that I missed quite a few breakout sessions during the times I was rostered. Nevertheless, I still got to see some of the most important sessions to me, particularly those that centred around Azure and integration technologies. Please have a read of my summary of these on my employer’s blog.

By and far this was my best Australian Ignite/Tech-Ed event experience for many reasons, including:

  1. The Pro-Integration team from Redmond came all the way out to Australia show everyone what the product group is doing with Logic Apps, Flow, Service Bus, and BizTalk Server
  2. I was chosen to present an Instructor-Led Lab in Service Fabric – my first ever speaking engagement at Ignite
  3. I had the rare opportunity to catch up with some fellow MVPs from Perth and Europe.

It was truly phenomenal to see enterprise integration properly represented at an Australian conference, as it is typically overlooked at these events. In addition to at least four breakout sessions on hybrid integration, Scott Guthrie actually performed a live demo of Logic Apps in his keynote! This was a good shout-out to the product team that has worked so hard to bring this technology up to the usability level it now enjoys. I’m glad that Jim Harrer, Jeff Holland, Jon Fancey and Kevin Lam were there to see it!

iPhone Import 015Teaching the lab in Service Fabric was a thrilling experience, but not without some challenges. The lab itself was broken and required a re-write of the second half, which I had pre-prepared and uploaded to One-Drive here so the students could progress. The main lab content is only available to Ignite attendees, however if you want to have a go at a similar lab you can try these ones available from Microsoft:

Despite the frustration that some attendees expressed about the lab errata and the poor performance of the environment, I was pleased that all the submitted feedback relating to the speaker was very positive! Smile

iPhone Import 050iPhone Import 037 (2)Finally, perhaps the best part of events like these is the ability to catch up with old friends and meet some new ones. It was a pleasure to hang out with Azure MVP Martin Abbott from Perth and meet a few of his iPhone Import 047colleagues. It was also great to see Elder Grootenboer and Steef-Jan Wiggers from the Netherlands, who happened to travel to Australia this month on holidays and to speak at some events. Steef-Jan also took time to include me in a V-Log series he’s been working on with various integration MVPs, recording his 3-minute interview with me at the top of Mount Coot-tha on a sunny Brisbane Saturday! And Mexia’s CEO Dean Robertson & myself got to enjoy a nice dinner out with the Microsoft product group and the MVPs.

All good things must come to an end, but it was definitely a memorable week! Now it’s time to start getting ready for the Brisbane edition of the Global Integration Bootcamp on Saturday, 25th March, to be followed not long after by the Global Azure Bootcamp on Saturday 22nd April! I’ve got a few demos and presentations to prepare – but now with plenty of inspiration from Ignite!

New Online API Mapping Tool

One of the many challenges with an integration project is typically the mapping of messages from one API to another. The difficulty most often lies not with the technical implementation (although some former projects mapping SAP iDocs to EDI X12 are still giving me nightmares), but rather with forming the specification of the mapping itself, including understanding the semantical meaning behind each element. This is difficult because it requires expert knowledge of both the source and target system, as well as an analysts who can correct “draw the connecting line” between the two. The correct end result is only achieved through significant collaboration amongst the relevant parties.

The BizTalk Mapper goes a long way to facilitating this task with it’s graphical mapping interface. Aside from providing the developer a means of rapidly implementing a transformation, it also servers as a visual representation of the mapping that can be understood by a business analyst (if not too complex):

biztalkmap

(image courtesy of MSDN)

There are two problems with this approach, however:

  1. It requires BizTalk Server, which is not only expensive, but also may be overkill for a solution that can easily be implemented in WCF, REST, or another platform;
  2. The mapping must be implemented by a developer before it can be shown to analysts and business users for discussion and validation. This usually entails a number of iterative cycles until the mapping is correct.

Enter api-map.com – a new free online tool created by my colleague Joseph Cooney specifically to address these particular challenges. api-map provides a medium to formulate, display and share mapping documentation which can eventually be handed over to a developer for implementation on any chosen platform.

Read more of this post

Integrate 2016 – What an Event!

Last week I had the privilege of attending the world’s largest integration event this year, Integrate 2016 in London. A big thanks to my employer Mexia for sending me. As is typical for events organised by BizTalk360, it was on an especially grand scale (27 sessions with 25+ speakers) and did not disappoint in the content presented by members of the Microsoft product team and the MVP community.

Day 1 of the three day event featured a number of announcements from Microsoft that clarified their vision and direction for integration, even more so than the Integration Roadmap delivered at the end of last year. Showing their commitment to BizTalk Server as the on-premises integration platform and Logic Apps as the cloud platform provided some much-needed reassurance and comfort to the community. “BizTalk and Logic Apps better together” is the mantra underpinned by the addition of a Logic Apps adapter in the upcoming BizTalk 2016 CTP2 release and the new BizTalk Connector soon to be introduced in Logic Apps.

Without explicitly stating it, it also became rather apparent as to what is “on the outs” in the integration space:

    • Microsoft Azure BizTalk Services (MABS) is likely to be deprecated as both the VETER pipelines and the EDI/B2B functionality moves into Logic Apps by way of the Enterprise Integration Pack;
    • Azure Stack is no longer being touted as the on-premises integration platform; rather BizTalk Server will continue to be king of that domain.

I’ve already posted an article on Mexia’s blog giving my rundown on all the sessions presented by Microsoft and the  significant announcements. Soon after I followed up with a summary of the many MVP sessions that rounded out the conference.  In addition, there are plenty of other blog posts from the community giving their thoughts and recaps of the event; here are just a few:

Besides Microsoft’s clear roadmap message and the excellent presentations, perhaps the best thing about this conference was the opportunity to catch up with colleagues and friends from around the world – and meet new ones as well!

Kickoff Dinner
(photo by Thomas Canter)

 

Saravana&Dan
(photo courtesy of BizTalk360)

 

GreenwichKitchen
(photo by Tara Motevalli)

Dinner_with_MVPs
(photo by Steef-Jan Wiggers)

Kudos again to Saravana Kumar, BizTalk360, Microsoft and all the sponsors for making this such an outstanding event! Looking forward to Integrate 2017!

Azure Service Bus Relays, SAS tokens and BizTalk Server

Great article by Mark Brimble. SAS support is now available across all WCF adapters in BizTalk Server 2016 CTP1!

Connected Pawns

Many people have written about Azure Service Bus Relays in the past and a summary can be found here. Dan Rosanova recently tweeted “….We’re trying to discourage ACS for security. SAS is our preferred model.”. The ACS security pattern is described here and the SAS pattern is described here. This article attempts to summarise BizTalk adapter support for using SAS tokens.

Most BizTalk Server examples use ACS tokens rather than SAS tokens, probably because the BizTalk Adapters only allowed configuration with ACS tokens when service bus relays were first released with BizTalk 2013. BizTalk 2013 R2 has limited support for configuration of SAS tokens and most adapters only allow use of ACS tokens out of the box (OOTB). If you want to use a SAS token you have to be very inventive. I hope that BizTalk vNext will add SAS token support for all WCF adapters.

View original post 411 more words

"Flush failed to run" SQL error with BAM API

Today I encountered an unusual error when executing a pipeline component that utilises the EventStream API to write to BAM. The failure that showed up in the event log looked something like this:

A message received by adapter "WCF-SQL" on receive location "MyReceivePort" with URI "mssql://MyDatabaseInstance/MyDatabase?InboundId=Employee" is suspended.
Error details: There was a failure executing the receive pipeline: "MyReceivePipeline, MyAssembly, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1a2b345c67d89e0f" Source: "Log Message To BAM Receive" Receive Port: "MyReceivePort" URI: "mssql://MyDatabaseInstance/MyDatabase?InboundId=Employee" Reason: Flush failed to run. 

A quick Google search pulled up this helpful post by Yossi Dahan, which pointed me in the right direction. I knew that the connection string was all right, and I was using the BufferedEventStream rather than the DirectEventStream that Yossi referred to. (Incidentally, when using the BufferedEventStream your connection string actually points to the BizTalkMsgBoxDb database rather than the BAMPrimaryImport database.)

However, the clue was really in the second part of Yossi’s suggestion (and also in an anonymous comment), "…and related permissions…".  I could see that the BizTalk Application Users domain group had been assigned all of the appropriate roles in SQL Server, and I knew that all the host accounts had been dutifully added to this group when BizTalk was installed.

Er… hang on a moment. I double checked and found that the SQL Adapter was running under a new dedicated host account that had been created specifically for the data warehouse. A simple check on the account using the "net user /domain" command prompt unveiled the culprit. This account had not be granted membership in the BizTalk Application Users domain group.

Once that was accomplished, everything worked smoothly.

It would be nice to actually see an error that hinted towards permission issues. Perhaps the detail was buried somewhere in an inner exception, but the logging does not go past the first level.

BizTalk SB-Messaging Receive Adapter Suspends Brokered Messages Without a Body

When it comes to processing zero-byte messages, the built-in receive adapters in BizTalk Server are somewhat inconsistent (see this recent post by Mark Brimble for more information). However, it seems that most receive adapters do not successfully process messages without body content. For example, the File adapter will delete an empty file and kindly put a notification to that effect in the event log. The HTTP adapter will reject a POST request with no content and return a 500 “Internal Server” error. So it probably isn’t any real surprise that the Azure Service Bus Messaging adapter introduced in BizTalk Server 2013 also obstructs bodiless messages. The difference here though is that the message will be successfully received from the queue or topic (and therefore removed from Service Bus), but will immediately be suspended with an error like the following:

A message received by adapter “SB-Messaging” on receive location “SB-ReceivePort_Queue_SB” with URI “sb://<namespace>.servicebus.windows.net/TestQueue” is suspended.
Error details: There was a failure executing the receive pipeline: “Microsoft.BizTalk.DefaultPipelines.PassThruReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35” Source: “Pipeline ” Receive Port: “SB-ReceivePort” URI: “sb://<namespace>.servicebus.windows.net/TestQueue” Reason: The Messaging Engine encountered an error while reading the message stream.

Read more of this post

Why Did My BizTalk Services Stop Working? Check Expired ACS Credentials…

So… let’s say you were one of the early adopters of Microsoft Azure BizTalk Services (MABS) and actually put a solution in production. Everything goes swimmingly for about a year. Then one day your interfaces stop working. Moreover, you will see no error entries in tracking (actually, no entries at all for that interface), although you might see some errors from the client trying to send messages to MABS.

Although the error messages you see may not be very helpful, the length of time that the service has been deployed and running should lead you to suspect some expired credentials. As it turns out, there are multiple levels of credentials and places that they are managed. The SSL certificate might be an obvious one, but many folks forget about the ACS credentials behind the service. Read more of this post

John Glisson - Geek of the Cloth

Thoughts on integration, technology and what-not...

Prashant BizTalk And Azure Integration Blogs

My Integration Experiences - BizTalk And Azure Integration

The CRUCIBLE

THINK: It's not illegal....yet.....

Abdul Rafay's BizTalk Blog

My experiences with BizTalk related to architecture, development and performance in my enterprise.

BizTalk musings

Issues, patterns and useful tips for BizTalk development

EAI Guy.net

Enterprise Applicaiton Integration and SOA 2.0

Connected Pawns

Mainly BizTalk & Little Chess

Adventures inside the Message Box

BizTalk, Azure, and other tools in the Microsoft stack - Johann Cooper

Biz(Talk)2

Talk, talk and more talk about BizTalk

Richard Seroter's Architecture Musings

Blog Featuring Code, Thoughts, and Experiences with Software and Services

Sandro Pereira BizTalk Blog

My notes about BizTalk Server 2004, 2006, 2006 R2, 2009, 2010, 2013 and now also Windows Azure BizTalk Services.

BizTalk Events

Calendar of BizTalk events all over the world!

Mind Over Messaging

Musings on BizTalk, Azure, and Enterprise Integration

WordPress.com News

The latest news on WordPress.com and the WordPress community.