New Online API Mapping Tool

One of the many challenges with an integration project is typically the mapping of messages from one API to another. The difficulty most often lies not with the technical implementation (although some former projects mapping SAP iDocs to EDI X12 are still giving me nightmares), but rather with forming the specification of the mapping itself, including understanding the semantical meaning behind each element. This is difficult because it requires expert knowledge of both the source and target system, as well as an analysts who can correct “draw the connecting line” between the two. The correct end result is only achieved through significant collaboration amongst the relevant parties.

The BizTalk Mapper goes a long way to facilitating this task with it’s graphical mapping interface. Aside from providing the developer a means of rapidly implementing a transformation, it also servers as a visual representation of the mapping that can be understood by a business analyst (if not too complex):

biztalkmap

(image courtesy of MSDN)

There are two problems with this approach, however:

  1. It requires BizTalk Server, which is not only expensive, but also may be overkill for a solution that can easily be implemented in WCF, REST, or another platform;
  2. The mapping must be implemented by a developer before it can be shown to analysts and business users for discussion and validation. This usually entails a number of iterative cycles until the mapping is correct.

Enter api-map.com – a new free online tool created by my colleague Joseph Cooney specifically to address these particular challenges. api-map provides a medium to formulate, display and share mapping documentation which can eventually be handed over to a developer for implementation on any chosen platform.

Read more of this post

Mapping a Newline within a String

Today a colleague asked me how best to insert newlines into a concatenated set of address lines within a map. It struck me that despite spending eleven years as a BizTalk developer, creating hundreds if not thousands of maps,  and even authoring a Pluralsight course on the Mapper,  I’ve never actually attempted something as simple as inserting a newline into a string output!

Like most programmers with C# .NET experience, the first instinct was to try inserting “\n” or “\r\n” into the string. But of course that wouldn’t work – the output of a non-scripted mapper link is XSLT, not .NET code.

So that leads to the assumption that an XML entity character is required; surely that will work? Since the string “&#13&#10” is the XML entity representation of a carriage return followed by a line feed (CR LF), we should be able to just insert that as a constant parameter within the String Concatenate functoid:

01

But then it becomes evident that the mapper escapes all characters that might otherwise be unintentionally interpreted as markup:

02

Hmmm… not turning out to be as easy as we thought. Surely we don’t need to resort to inline XSLT just to insert a newline into text?

Read more of this post

First Pluralsight Course Published!

PluralsightIn case anyone is wondering why I’ve been out of the blogging and social media scene for the past few months, it’s because I’ve had my head down trying to complete the authoring of my first Pluralsight course!

Well, I’m pleased to say that the course is now live!! It’s been a long journey and resulted in three scope changes, but is finally published. Also feel very privileged to have joined the ranks of the Pluralsight author team, comprised of so many exceptional people whose wisdom I’ve been following throughout my career.

The course is entitled Using Functoids in BizTalk Server 2013, and it features a deep-dive exploration of the eighty built-in functoids within the BizTalk Mapper toolbox, as well a brief look at creating custom functoids and hosting custom XSLT files. Despite the title, it is equally relevant to BizTalk Server 2010 users as the Mapper has not changed much between those two versions. (Come to think of it, the functoids themselves haven’t changed all that much since BTS 2006, even though the user interface was overhauled to great improvement in 2010.)

Read more of this post

WEBCAST: Schema Validation in BizTalk 2010

Just posted a new webcast on how to use out-of-the-box pipeline components to validate XML schemas for incoming messages:

http://youtu.be/4-Q9Ddy7-Tg

This is important because by default, BizTalk does not perform schema level validation on incoming messages, or even in map execution. This webcast shows two different ways that validation can be enabled, though.

Note that although BizTalk 2010 is used to demo the methodologies,  both approaches will work in all versions of BizTalk Server from 2006 onwards.

The slide deck for this webcast can be found on my SlideShare page.

Keep an eye on the Brisbane BizTalk User Group YouTube channel as more webcasts will follow!

Duplicate Namespace Issue with Schema Validation in Pipeline Components

Today I was tasked with enabling XML schema validation for an existing WCF receive location in BizTalk. The client was already using the out-of-the-box XmlReceive pipeline, so I figured this would be short work, finished by lunchtime after all the regression tests had been run. Little did I suspect that I would run into a snag that would enlighten me to a new BizTalk “feature”.

Because of a compressed time schedule, we wanted to avoid having to add a new custom pipeline to the solution, even one as simple as containing a standard XmlValidator component. So instead, I opted to use port-based configuration by enabling validation within the XmlDissembler component in the existing XmlReceive pipeline. Of course, this does require specifying the list of specific schemas in the DocumentSpecNames property on the port instance:

SchemaValidation-Settings

The trick here, of course, is to ensure you get these specifications correct, as they need to be fully qualified assembly (FQN) references. Moreover, if you are expecting multiple message types, you need to separate the entries with a pipeline (“|”) delimiter, something not so obvious if you don’t read the Microsoft documentation. Even less obvious is how to deal with multipart schemas that contain multiple root nodes (at least this blog post sheds some light there).

Read more of this post

GOTCHA: Changing the Namespace on an Envelope Schema

Today I had to change the target namespace for an envelope schema to resolve a conflict that caused XML validation issues in the pipeline. I also had a few other changes to make and for various reasons I did not use the BizTalk Schema Editor tool, but instead opened the schema in the standard Visual Studio XML(Text) Editor mode.

After having dutifully made the change, tested all the maps, ensured the project compiled properly, and successfully run all the unit tests, I deployed the solution.

However, the first integration test failed because on receipt of the message with a pipeline error similar to this one:

There was a failure executing the receive pipeline: “Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35” Source: “XML disassembler” Receive Port: “XmlValidationReceivePort” URI: “C:\BizTalk\Folders\XML_VAL_IN\*.xml” Reason: This Disassembler cannot retrieve body nodes using this XPath: “/*[local-name()=’MyEnvelopeSchema’ and namespace-uri()=’http://XmlValidationTest.EnvelopeSchema’%5D/*%5Blocal-name()=’ChildSchemas’ and namespace-uri()=’http://XmlValidationTest.EnvelopeSchema’%5D”.
/*[local-name()=’MyEnvelopeSchema’ and namespace-uri()=’http://XmlValidationTest.EnvelopeSchema’%5D/*%5Blocal-name()=’ChildSchemas’ and namespace-uri()=’http://XmlValidationTest.EnvelopeSchema’%5D

Only then did I realise that in updating the target namespace of the schema, I forgot to update the BodyXPath value – which of course has the namespace all through it!

EnvelopeSchema-BodyXPath

Solution is to refresh the BodyXPath property by clicking the ellipses (…) button on the property grid and reselecting the node just above the repeating node you want to debatch:

EnvelopeSchema-Debatch

The good news is that when you use the Schema Editor to change the target namespace, this synchronisation is taken care of for you automatically. And if fact, you cannot change the BodyXPath property by typing in the property grid – you have to open the Body XPath dialog as shown above.

Moral of the story is: beware of circumventing the built-in editors and tools unless you are really, really careful. The tools are usually designed to help keep you out of trouble. 😉

Using BizTalk xpath() Function to Retrieve Optional Nodes

I often suffer a bit of trepidation when trying to retrieve values for optional XML nodes using the built-in xpath() function in an orchestration, as I wonder if it will result in an exception of some kind if the node isn’t found in the message.  This concern is heightened by reading Microsoft’s own documentation of the xpath function on MSDN:

The engine is not schema-aware, so you can only read values from or write values to a node that exists in the containing message (the complete path must exist), or the engine will raise an exception. This is true even if you supply a default value.

The conclusion from this is that if the node doesn’t exist in the message instance, you’ll get a runtime error trying to read from it using xpath(), right?

Wrong! In fact, as Leonid Ganeline proved in an older blog post, you will either get an empty string or “null”, depending on whether you wrap the XPATH expression in a string() function or not:

John Glisson - Geek of the Cloth

Thoughts on integration, technology and what-not...

prashantbiztalkblogs

My BizTalk Experiences

The CRUCIBLE

THINK: It's not illegal....yet.....

paulbouwer.com

life and technology

Abdul Rafay's BizTalk Blog

My experiences with BizTalk related to architecture, development and performance in my enterprise.

Mike Diiorio

Connected Systems and other thoughts

BizTalk musings

Issues, patterns and useful tips for BizTalk development

EAI Guy.net

Enterprise Applicaiton Integration and SOA 2.0

Connected Pawns

Mainly BizTalk & Little Chess

Man Vs. Machine

Why can't we all just get along?

Adventures inside the Message Box

BizTalk, Azure, and other tools in the Microsoft stack - Johann Cooper

Biz(Talk)2

Talk, talk and more talk about BizTalk

Richard Seroter's Architecture Musings

Blog Featuring Code, Thoughts, and Experiences with Software and Services

Sandro Pereira BizTalk Blog

My notes about BizTalk Server 2004, 2006, 2006 R2, 2009, 2010, 2013 and now also Windows Azure BizTalk Services.

BizTalk Events

Calendar of BizTalk events all over the world!

Mind Over Messaging

Musings on BizTalk, Azure, WCF, and Enterprise Integration

The WordPress.com Blog

The latest news on WordPress.com and the WordPress community.