Removing the BOM from Outgoing BizTalk Files

Today we had an issue with sending a file to an external SFTP client – they did not like the Byte Order Mark (BOM) that was automatically pre-pended to the file by the BizTalk file adapter.

A BOM is a short set of invisible characters that is added to the beginning of a UTF-16 or UTF-8 file to indicate the endianess (or byte order) of a text stream. Among other uses, it helps consuming systems determine which encoding (UTF-8 or UTF-16) has been used in creating the stream. The BOM is optional in most cases, and you typically will not see it unless you use a special text editor that can display ISO-8859-1 or CP1252 characters (WinDiff appears to work well for this).

Here is some text that starts with a UTF-8 byte order mark (BOM); notice the three initial odd-looking characters? Represents the sequence 0xEF 0xBB 0xB in hexadecimal, or 239 187 191 in CP-1252 decimal. You won’t see these non-printable characters in most text editors.

Most modern systems can cope with the presence or non-presence of the BOM (especially in UTF-8 where the BOM is really superfluous), but this client clearly had issues with it. So it was off to Dr. Google to find the quickest solution that hopefully involved no custom code (since we are nearly in the Acceptance Testing phase).

Microsoft’s documentation partially came to the rescue, although some claims in this article do not present all the facts (at least not for BizTalk 2010). For example, the statement “If you use a PassThruReceive pipeline or a PassThruTransmit pipeline, a byte order mark is not appended to a message” is questionable, since we were in fact using the PassThruTransmit pipeline and the BOM was definitely present in the output. In addition, the article exclusively suggests creating a custom pipeline as the solution – but this is only necessary if you are not outputting XML.

Our solution? Simply swap the PassThruTransmit pipeline for another OOTB option, the XmlTransmit pipeline. Doing this gives you access to a pipeline PreserveBOM instance property that will strip the BOM from the message for you if you change it from its default setting of “True” to “False”:

BOM_XmlTransmit

That was it! No code changes, no recompilation, not even a restart – just a simple binding update and the troublesome BOM is now history. Note the difference between the before & after output messages compared here in WinDiff:

BOM_WinDiff

BOM’s away!    Winking smile

Duplicate Namespace Issue with Schema Validation in Pipeline Components

Today I was tasked with enabling XML schema validation for an existing WCF receive location in BizTalk. The client was already using the out-of-the-box XmlReceive pipeline, so I figured this would be short work, finished by lunchtime after all the regression tests had been run. Little did I suspect that I would run into a snag that would enlighten me to a new BizTalk “feature”.

Because of a compressed time schedule, we wanted to avoid having to add a new custom pipeline to the solution, even one as simple as containing a standard XmlValidator component. So instead, I opted to use port-based configuration by enabling validation within the XmlDissembler component in the existing XmlReceive pipeline. Of course, this does require specifying the list of specific schemas in the DocumentSpecNames property on the port instance:

SchemaValidation-Settings

The trick here, of course, is to ensure you get these specifications correct, as they need to be fully qualified assembly (FQN) references. Moreover, if you are expecting multiple message types, you need to separate the entries with a pipeline (“|”) delimiter, something not so obvious if you don’t read the Microsoft documentation. Even less obvious is how to deal with multipart schemas that contain multiple root nodes (at least this blog post sheds some light there).

Read more of this post

John Glisson - Geek of the Cloth

Thoughts on integration, technology and what-not...

prashantbiztalkblogs

My BizTalk Experiences

The CRUCIBLE

THINK: It's not illegal....yet.....

paulbouwer.com

life and technology

Abdul Rafay's BizTalk Blog

My experiences with BizTalk related to architecture, development and performance in my enterprise.

Mike Diiorio

Connected Systems and other thoughts

BizTalk musings

Issues, patterns and useful tips for BizTalk development

EAI Guy.net

Enterprise Applicaiton Integration and SOA 2.0

Connected Pawns

Mainly BizTalk & Little Chess

Man Vs. Machine

Why can't we all just get along?

Adventures inside the Message Box

BizTalk, Azure, and other tools in the Microsoft stack - Johann Cooper

Biz(Talk)2

Talk, talk and more talk about BizTalk

Richard Seroter's Architecture Musings

Blog Featuring Code, Thoughts, and Experiences with Software and Services

Sandro Pereira BizTalk Blog

My notes about BizTalk Server 2004, 2006, 2006 R2, 2009, 2010, 2013 and now also Windows Azure BizTalk Services.

BizTalk Events

Calendar of BizTalk events all over the world!

Mind Over Messaging

Musings on BizTalk, Azure, and Enterprise Integration

The WordPress.com Blog

The latest news on WordPress.com and the WordPress community.