WEBCAST: Integrating BizTalk 2013 with Service Bus Queues & Topics

I’ve just uploaded my latest production on the Brisbane BizTalk YouTube Channel, bringing our library up to four webcasts now! This presentation shows how easy it is to integrate BizTalk Server 2013 with Windows Azure Service Bus queues & topics, facilitating the creation of powerful hybrid applications:

Integrating BizTalk with Windows Azure Service Bus Queues & Topics[ http://youtu.be/jQefUBvc4Dk ]

You can also download the slide deck (complete with notes) from SlideShare [ http://www.slideshare.net/dtoomey/integrating-biztalk-2013-with-windows-azure-server-bus-queues-topics ]

Really loving my new RØDE Podcaster USB microphone, and also Camtasia 8.1 which makes editing these videos so easy!

WEBCAST: Schema Validation in BizTalk 2010

Just posted a new webcast on how to use out-of-the-box pipeline components to validate XML schemas for incoming messages:

http://youtu.be/4-Q9Ddy7-Tg

This is important because by default, BizTalk does not perform schema level validation on incoming messages, or even in map execution. This webcast shows two different ways that validation can be enabled, though.

Note that although BizTalk 2010 is used to demo the methodologies,  both approaches will work in all versions of BizTalk Server from 2006 onwards.

The slide deck for this webcast can be found on my SlideShare page.

Keep an eye on the Brisbane BizTalk User Group YouTube channel as more webcasts will follow!

Solving a Map Reference to a Deleted Schema

Recently a client got into a bit of “hot water” in their BizTalk integration environment because changes to a Common BizTalk application were not deployed prior to one of the dependent applications. This resulted in an error whenever trying to manage the environment’s applications via the BizTalk Admin Console (even when we followed the hint to refresh the console view):

Schema referenced by Map ‘Test.DependentApp.DependentSchema_to_CommonSchema’ has been deleted. The local, cached version of the BizTalk Server group configuration is out of date. You must refresh the BizTalk Server group configuration before making further changes.
(Microsoft.BizTalk.Administration.SnapIn)

The issue was that a new schema contained in the updated Common project was missing. Easy enough to understand, and the steps to resolve it would appear to be simple enough:

  1. Undeploy the dependent application
  2. Redeploy the Common application (using the new version)
  3. Redeploy the dependent application

However, the error above thwarted us at the very first step: we could not delete the dependent application either through the Admin Console or via BTSTask! Read more of this post

Removing the BOM from Outgoing BizTalk Files

Today we had an issue with sending a file to an external SFTP client – they did not like the Byte Order Mark (BOM) that was automatically pre-pended to the file by the BizTalk file adapter.

A BOM is a short set of invisible characters that is added to the beginning of a UTF-16 or UTF-8 file to indicate the endianess (or byte order) of a text stream. Among other uses, it helps consuming systems determine which encoding (UTF-8 or UTF-16) has been used in creating the stream. The BOM is optional in most cases, and you typically will not see it unless you use a special text editor that can display ISO-8859-1 or CP1252 characters (WinDiff appears to work well for this).

Here is some text that starts with a UTF-8 byte order mark (BOM); notice the three initial odd-looking characters? Represents the sequence 0xEF 0xBB 0xB in hexadecimal, or 239 187 191 in CP-1252 decimal. You won’t see these non-printable characters in most text editors.

Most modern systems can cope with the presence or non-presence of the BOM (especially in UTF-8 where the BOM is really superfluous), but this client clearly had issues with it. So it was off to Dr. Google to find the quickest solution that hopefully involved no custom code (since we are nearly in the Acceptance Testing phase).

Microsoft’s documentation partially came to the rescue, although some claims in this article do not present all the facts (at least not for BizTalk 2010). For example, the statement “If you use a PassThruReceive pipeline or a PassThruTransmit pipeline, a byte order mark is not appended to a message” is questionable, since we were in fact using the PassThruTransmit pipeline and the BOM was definitely present in the output. In addition, the article exclusively suggests creating a custom pipeline as the solution – but this is only necessary if you are not outputting XML.

Our solution? Simply swap the PassThruTransmit pipeline for another OOTB option, the XmlTransmit pipeline. Doing this gives you access to a pipeline PreserveBOM instance property that will strip the BOM from the message for you if you change it from its default setting of “True” to “False”:

BOM_XmlTransmit

That was it! No code changes, no recompilation, not even a restart – just a simple binding update and the troublesome BOM is now history. Note the difference between the before & after output messages compared here in WinDiff:

BOM_WinDiff

BOM’s away!    Winking smile

Duplicate Namespace Issue with Schema Validation in Pipeline Components

Today I was tasked with enabling XML schema validation for an existing WCF receive location in BizTalk. The client was already using the out-of-the-box XmlReceive pipeline, so I figured this would be short work, finished by lunchtime after all the regression tests had been run. Little did I suspect that I would run into a snag that would enlighten me to a new BizTalk “feature”.

Because of a compressed time schedule, we wanted to avoid having to add a new custom pipeline to the solution, even one as simple as containing a standard XmlValidator component. So instead, I opted to use port-based configuration by enabling validation within the XmlDissembler component in the existing XmlReceive pipeline. Of course, this does require specifying the list of specific schemas in the DocumentSpecNames property on the port instance:

SchemaValidation-Settings

The trick here, of course, is to ensure you get these specifications correct, as they need to be fully qualified assembly (FQN) references. Moreover, if you are expecting multiple message types, you need to separate the entries with a pipeline (“|”) delimiter, something not so obvious if you don’t read the Microsoft documentation. Even less obvious is how to deal with multipart schemas that contain multiple root nodes (at least this blog post sheds some light there).

Read more of this post

GOTCHA: Changing the Namespace on an Envelope Schema

Today I had to change the target namespace for an envelope schema to resolve a conflict that caused XML validation issues in the pipeline. I also had a few other changes to make and for various reasons I did not use the BizTalk Schema Editor tool, but instead opened the schema in the standard Visual Studio XML(Text) Editor mode.

After having dutifully made the change, tested all the maps, ensured the project compiled properly, and successfully run all the unit tests, I deployed the solution.

However, the first integration test failed because on receipt of the message with a pipeline error similar to this one:

There was a failure executing the receive pipeline: “Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35” Source: “XML disassembler” Receive Port: “XmlValidationReceivePort” URI: “C:\BizTalk\Folders\XML_VAL_IN\*.xml” Reason: This Disassembler cannot retrieve body nodes using this XPath: “/*[local-name()=’MyEnvelopeSchema’ and namespace-uri()=’http://XmlValidationTest.EnvelopeSchema’%5D/*%5Blocal-name()=’ChildSchemas’ and namespace-uri()=’http://XmlValidationTest.EnvelopeSchema’%5D”.
/*[local-name()=’MyEnvelopeSchema’ and namespace-uri()=’http://XmlValidationTest.EnvelopeSchema’%5D/*%5Blocal-name()=’ChildSchemas’ and namespace-uri()=’http://XmlValidationTest.EnvelopeSchema’%5D

Only then did I realise that in updating the target namespace of the schema, I forgot to update the BodyXPath value – which of course has the namespace all through it!

EnvelopeSchema-BodyXPath

Solution is to refresh the BodyXPath property by clicking the ellipses (…) button on the property grid and reselecting the node just above the repeating node you want to debatch:

EnvelopeSchema-Debatch

The good news is that when you use the Schema Editor to change the target namespace, this synchronisation is taken care of for you automatically. And if fact, you cannot change the BodyXPath property by typing in the property grid – you have to open the Body XPath dialog as shown above.

Moral of the story is: beware of circumventing the built-in editors and tools unless you are really, really careful. The tools are usually designed to help keep you out of trouble. 😉

A Cloudy Day for BizTalk (…but what a sunny forecast!)

Recently, Microsoft announced the release of a plethora of new features in Windows Azure, not the least of which was the preview version of BizTalk Services. A re-vamped version of the former Azure Service Bus EAI/EDI Labs which was released last year and taken down earlier this year, Windows Azure BizTalk Services boasts Business-to-Business (B2B) and Enterprise Application Integration (EAI) capabilities for cloud and hybrid integration solutions, including:

  • Built-in support for managing EDI relationships between partners
  • EAI bridges with on-premises assets (including built-in support for integration with SAP, SQL Server, Oracle and Siebel)
  • Optional integration between Windows Azure BizTalk Services and on-premises BizTalk Server deployments
    Like most Azure offerings, BizTalk Services runs on a secure, dedicated per tenant environment that can be provisioned on demand within minutes.  It does not require any upfront license, and supports a pay only for what you use billing model.

Learn more about how to setup and start using the Windows Azure BizTalk Services preview.

BizTalk360 Presentation to Brisbane User Group

Tonight I set out to present a 45-minute discussion and demonstration of BizTalk360 to the Brisbane BizTalk User Group. An hour and a half later, I was still answering questions and displaying in-depth features of the product! We’re a pretty small & intimate group, so there was plenty of opportunity for discussion and catering for specific requests (“Can I see how … works?”), which for me as a presenter is awesome. I love interaction because you know you are really reaching your audience (small or large) and addressing their interests & concerns.

BTS360The truth is, they were truly impressed with the power and flexibility of the product, despite the limited scenarios that I was able to present to them. Saravana and his team have really hit the mark with this release, addressing a critical void in the BizTalk operational support area. Looking forward to Version 7.0!

Only downside was that I barely had time left to present my second session for the night, new features of BizTalk 2013! Oh well, I don’t think Microsoft needs too much help in selling that product anyway! Winking smile

Next BrizTalk Meeting: BizTalk 2013 & BizTalk360 Demos!

I’m really looking forward to presenting a double-header at our next Brisbane BizTalk User Group meeting on Wednesday:

What’s New in BizTalk Server 2013

Finally released to manufacturing, this long awaited edition hosts a variety of new features that make integrating your on-premises system with the cloud so much easier. The new REST and Azure Service Bus adapters provide quick & simple connectivity for building powerful, flexible and highly scalable hybrid solutions with BizTalk & Azure.

We’ll see an example of both in action during this part of the presentation.

Monitoring with BizTalk360

Now in Version 6.0, Saravana Kumar and his team have released the best-ever edition of BizTalk360, an easy to install web-based application that helps manage the complexity of supporting your BizTalk environments. In addition to the standard features of monitoring, alerting, administrating and auditing, the new capabilities of this release include:

  • Search & Action on Artefacts (Receive locations, send ports, applications etc.)
  • ESB Exception Management Portal
  • Custom SQL Query Complete Rewrite
  • Database Query Monitoring
  • SQL Agent Job Outcome Monitoring
  • Threshold Monitoring Window
  • Scheduled Monitoring Downtime
  • Support for SSL

I’ll present a demo of an active installation so you can witness first-hand the simplicity with which this awesome tool provides value!

If you’re in the Brisbane area, drop by Microsoft at 400 George St, Lvl 28 at 5:30pm on Wednesday, 29th May for some pizza and a great session! All i ask is that you register first so that I know how many pizzas to order!

BizTalk 2013 RTM VM Now Available in Window Azure Gallery

With Scott Guthrie’s announcement the other day concerning General Availability (GA) of Windows Azure Infrastructure Services, we now have available a selection of new BizTalk Server 2013 RTM virtual machine images in the Windows Azure Gallery:

  • BizTalk Server 2013 Enterprise
  • BizTalk Server 2013 Standard
  • BizTalk Server 2013 Evaluation

The Evaluation image will help you get started with the evaluation bits of BizTalk Server 2013 and SQL Server 2012.

This IaaS availability is a key aspect of the BizTalk Server 2013 release. Now you can quickly and easily spin up pre-configured BizTalk servers in the cloud without relying on the time, expense and other overheads of on-premises acquisitions. Imagine being able to provision an entire testing environment in minutes/hours instead of days/weeks/months!  And the best part is that, like all Azure services, you only pay for what you use. 🙂

The pricing details for these VMs can be found here.  Remember that there is also a (very small) storage cost for the disk images as well.

More information can be found on the BizTalk Server Team Blog.

John Glisson - Geek of the Cloth

Thoughts on integration, technology and what-not...

prashantbiztalkblogs

My BizTalk Experiences

The CRUCIBLE

THINK: It's not illegal....yet.....

Abdul Rafay's BizTalk Blog

My experiences with BizTalk related to architecture, development and performance in my enterprise.

Mike Diiorio

Connected Systems and other thoughts

BizTalk musings

Issues, patterns and useful tips for BizTalk development

EAI Guy.net

Enterprise Applicaiton Integration and SOA 2.0

Connected Pawns

Mainly BizTalk & Little Chess

Adventures inside the Message Box

BizTalk, Azure, and other tools in the Microsoft stack - Johann Cooper

Biz(Talk)2

Talk, talk and more talk about BizTalk

Richard Seroter's Architecture Musings

Blog Featuring Code, Thoughts, and Experiences with Software and Services

Sandro Pereira BizTalk Blog

My notes about BizTalk Server 2004, 2006, 2006 R2, 2009, 2010, 2013 and now also Windows Azure BizTalk Services.

BizTalk Events

Calendar of BizTalk events all over the world!

Mind Over Messaging

Musings on BizTalk, Azure, and Enterprise Integration

The WordPress.com Blog

The latest news on WordPress.com and the WordPress community.