Integrate 2016 – What an Event!

Last week I had the privilege of attending the world’s largest integration event this year, Integrate 2016 in London. A big thanks to my employer Mexia for sending me. As is typical for events organised by BizTalk360, it was on an especially grand scale (27 sessions with 25+ speakers) and did not disappoint in the content presented by members of the Microsoft product team and the MVP community.

Day 1 of the three day event featured a number of announcements from Microsoft that clarified their vision and direction for integration, even more so than the Integration Roadmap delivered at the end of last year. Showing their commitment to BizTalk Server as the on-premises integration platform and Logic Apps as the cloud platform provided some much-needed reassurance and comfort to the community. “BizTalk and Logic Apps better together” is the mantra underpinned by the addition of a Logic Apps adapter in the upcoming BizTalk 2016 CTP2 release and the new BizTalk Connector soon to be introduced in Logic Apps.

Without explicitly stating it, it also became rather apparent as to what is “on the outs” in the integration space:

    • Microsoft Azure BizTalk Services (MABS) is likely to be deprecated as both the VETER pipelines and the EDI/B2B functionality moves into Logic Apps by way of the Enterprise Integration Pack;
    • Azure Stack is no longer being touted as the on-premises integration platform; rather BizTalk Server will continue to be king of that domain.

I’ve already posted an article on Mexia’s blog giving my rundown on all the sessions presented by Microsoft and the  significant announcements. Soon after I followed up with a summary of the many MVP sessions that rounded out the conference.  In addition, there are plenty of other blog posts from the community giving their thoughts and recaps of the event; here are just a few:

Besides Microsoft’s clear roadmap message and the excellent presentations, perhaps the best thing about this conference was the opportunity to catch up with colleagues and friends from around the world – and meet new ones as well!

Kickoff Dinner
(photo by Thomas Canter)

 

Saravana&Dan
(photo courtesy of BizTalk360)

 

GreenwichKitchen
(photo by Tara Motevalli)

Dinner_with_MVPs
(photo by Steef-Jan Wiggers)

Kudos again to Saravana Kumar, BizTalk360, Microsoft and all the sponsors for making this such an outstanding event! Looking forward to Integrate 2017!

Azure Service Bus Relays, SAS tokens and BizTalk Server

Great article by Mark Brimble. SAS support is now available across all WCF adapters in BizTalk Server 2016 CTP1!

Connected Pawns

Many people have written about Azure Service Bus Relays in the past and a summary can be found here. Dan Rosanova recently tweeted “….We’re trying to discourage ACS for security. SAS is our preferred model.”. The ACS security pattern is described here and the SAS pattern is described here. This article attempts to summarise BizTalk adapter support for using SAS tokens.

Most BizTalk Server examples use ACS tokens rather than SAS tokens, probably because the BizTalk Adapters only allowed configuration with ACS tokens when service bus relays were first released with BizTalk 2013. BizTalk 2013 R2 has limited support for configuration of SAS tokens and most adapters only allow use of ACS tokens out of the box (OOTB). If you want to use a SAS token you have to be very inventive. I hope that BizTalk vNext will add SAS token support for all WCF adapters.

View original post 411 more words

"Flush failed to run" SQL error with BAM API

Today I encountered an unusual error when executing a pipeline component that utilises the EventStream API to write to BAM. The failure that showed up in the event log looked something like this:

A message received by adapter "WCF-SQL" on receive location "MyReceivePort" with URI "mssql://MyDatabaseInstance/MyDatabase?InboundId=Employee" is suspended.
Error details: There was a failure executing the receive pipeline: "MyReceivePipeline, MyAssembly, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1a2b345c67d89e0f" Source: "Log Message To BAM Receive" Receive Port: "MyReceivePort" URI: "mssql://MyDatabaseInstance/MyDatabase?InboundId=Employee" Reason: Flush failed to run. 

A quick Google search pulled up this helpful post by Yossi Dahan, which pointed me in the right direction. I knew that the connection string was all right, and I was using the BufferedEventStream rather than the DirectEventStream that Yossi referred to. (Incidentally, when using the BufferedEventStream your connection string actually points to the BizTalkMsgBoxDb database rather than the BAMPrimaryImport database.)

However, the clue was really in the second part of Yossi’s suggestion (and also in an anonymous comment), "…and related permissions…".  I could see that the BizTalk Application Users domain group had been assigned all of the appropriate roles in SQL Server, and I knew that all the host accounts had been dutifully added to this group when BizTalk was installed.

Er… hang on a moment. I double checked and found that the SQL Adapter was running under a new dedicated host account that had been created specifically for the data warehouse. A simple check on the account using the "net user /domain" command prompt unveiled the culprit. This account had not be granted membership in the BizTalk Application Users domain group.

Once that was accomplished, everything worked smoothly.

It would be nice to actually see an error that hinted towards permission issues. Perhaps the detail was buried somewhere in an inner exception, but the logging does not go past the first level.

Following the Roadmap to Microsoft Integration

Microsoft has just released a document detailing their roadmap to integration. With all of the recent activity in the cloud around integration – including the release of Microsoft Azure BizTalk Services two years ago, followed by a seemingly different change of direction with Azure App Service announcement earlier this year – there has been much confusion about where Microsoft was headed in the integration space. This has been challenging for partners and customers who want to ensure that they invest in the “right” technology when building out their enterprise integration capability.

I am pleased to say that this document finally delivers some much-needed clarification in this respect. Aside from reinforcing that “BizTalk is not dead” and confirming some key new features in the much-anticipated BizTalk Server 2016 release, it also shows how Microsoft is aiming to close the gap between traditional on-premises integration afforded by the server product and the modern API-based approach offered in Azure:

convergence

Read more of this post

Collection of Microsoft Integration Stencils for Visio 2013

Great new set of Microsoft hybrid integration Visio stencils from Sandro Pereira – includes BizTalk, BizTalk Services, Azure App Service, PowerBI and more!

Sandro Pereira BizTalk Blog

In my two previous editions (here and here) I published some shapes to represent BizTalk Server physical architectures, Integration architectures (Cloud or Hybrid scenarios) or solutions.

In this resource I change the name of my Visio 2013 Stencils resource, now is called “Microsoft Integration Stencils for Visio 2013”, to include almost the Microsoft Integration Stack.

This package contains a set of symbols/icons (258 shapes) to that will help you visually represent Integration architectures (On-premise, Cloud or Hybrid scenarios) and solutions diagrams in Visio 2013. It will provide symbols/icons to visually represent features, systems, processes and architectures that use BizTalk Server, Microsoft Azure and related technologies.

  • BizTalk Server
  • Microsoft Azure
    • BizTalk Services
    • Azure App Service
    • Event Hubs
    • and so on
  • PowerBI and devices
  • Infrastructure
  • And many more…
BizTalk Server

BizTalk-Server-Stencils-Visio-2013-01

BizTalk Services

BizTalk-Services-Stencils-Visio-2013-02

Azure App Service

Azure-App-Service-Stencils-Visio-2013-02

Azure

Azure-Stencils-Visio-2013-04

Infrastructure

Infraestructure-Stencils-Visio-2013-05

PowerBI and Devices

PowerBI-Devices-Stencils-Visio-2013-06

That you can use and resize without losing…

View original post 124 more words

Testing for Empty/Nil/Missing Source Nodes in BizTalk Maps

Re-posted from another blog – original publish date 27 Aug 2012

Scenario: You are mapping an optional node in your source schema to an optional node in your target schema. But… the target schema has stricter validation rules and cannot accept empty or “nil” nodes, whereas the source schema can.

By default, if the source node doesn’t exist, it won’t be output in your target. No problems there. But… what if the source node is empty or null?

Let’s look at this mapping example:

BadMap

Read more of this post

Gotchas When Accessing an IaaS SQL Server in an Azure Cloud Service

It’s not that uncommon a scenario… You implement a spectacular BizTalk Server integration solution with well-designed Business Activity Monitoring, and you want to expose all that BAM goodness out through a custom-designed Web application built with HTML5, etc.

What might make this scenario slightly less common is when you’re hosting the BizTalk Server infrastructure in Azure. In this case, your custom Web application needs to be hosted within the same virtual network in order to access the BAM databases – unless of course you want to expose your SQL Server to the world by establishing a TCP endpoint on a port like 1433 (usually not a great idea).

Establishing that connectivity should be trivial – and it probably is if you get past a few potential stumbling blocks. I thought I might list some of them here in hopes that it may be helpful to others who try this.

Read more of this post

First Pluralsight Course Published!

PluralsightIn case anyone is wondering why I’ve been out of the blogging and social media scene for the past few months, it’s because I’ve had my head down trying to complete the authoring of my first Pluralsight course!

Well, I’m pleased to say that the course is now live!! It’s been a long journey and resulted in three scope changes, but is finally published. Also feel very privileged to have joined the ranks of the Pluralsight author team, comprised of so many exceptional people whose wisdom I’ve been following throughout my career.

The course is entitled Using Functoids in BizTalk Server 2013, and it features a deep-dive exploration of the eighty built-in functoids within the BizTalk Mapper toolbox, as well a brief look at creating custom functoids and hosting custom XSLT files. Despite the title, it is equally relevant to BizTalk Server 2010 users as the Mapper has not changed much between those two versions. (Come to think of it, the functoids themselves haven’t changed all that much since BTS 2006, even though the user interface was overhauled to great improvement in 2010.)

Read more of this post

BizTalk HTTP receive IIS Internal Server 500 Error

A very helpful post that got me out of trouble today! The constant HTTP 500 error responses aren’t very helpful…

BizTalk SQL Agent Jobs – Configuration Errors

In a previous post, I talked about how to configure the BizTalk SQL Agent jobs responsible for backing up the databases and purging the tracking data. These jobs are not configured by default, but their operation is essential for keeping your BizTalk system running smoothly.

Today, I inherited a developer VM at a new client with a warning that it was “running a bit slow…”.  Upon investigation, I discovered that the Backup BizTalk Server job had not been configured, so I promptly set about doing that task. However, I uncovered an error once I tried to run the job the first time:

Executed as user: NT AUTHORITY\SYSTEM. The statement BACKUP LOG is not allowed while the recovery model is SIMPLE. Use BACKUP DATABASE or change the recovery model using ALTER DATABASE. [SQLSTATE 42000] (Error 4208)  BACKUP LOG is terminating abnormally. [SQLSTATE 42000] (Error 3013).  The step failed.

Read more of this post

John Glisson - Geek of the Cloth

Thoughts on integration, technology and what-not...

Prashant BizTalk And Azure Integration Blogs

My Integration Experiences - BizTalk And Azure Integration

The CRUCIBLE

THINK: It's not illegal....yet.....

Abdul Rafay's BizTalk Blog

My experiences with BizTalk related to architecture, development and performance in my enterprise.

BizTalk musings

Issues, patterns and useful tips for BizTalk development

EAI Guy.net

Enterprise Applicaiton Integration and SOA 2.0

Connected Pawns

Mainly BizTalk & Little Chess

Adventures inside the Message Box

BizTalk, Azure, and other tools in the Microsoft stack - Johann Cooper

Biz(Talk)2

Talk, talk and more talk about BizTalk

Richard Seroter's Architecture Musings

Blog Featuring Code, Thoughts, and Experiences with Software and Services

Sandro Pereira BizTalk Blog

My notes about BizTalk Server 2004, 2006, 2006 R2, 2009, 2010, 2013 and now also Windows Azure BizTalk Services.

BizTalk Events

Calendar of BizTalk events all over the world!

Mind Over Messaging

Musings on BizTalk, Azure, and Enterprise Integration

WordPress.com News

The latest news on WordPress.com and the WordPress community.