EDI Tools for .NET is a software development kit for .NET Framework and .NET Core, which makes it straightforward to parse, generate, validate, acknowledge, split, customize, or in other words, to programmatically manipulate EDI files. It is written in C# and is distributed as a NuGet package.

EDI Tools for .NET Tutorial - Part 2

This is the second part of the EDI Tools for .NET tutorial, it is assumed that you have gone through the first part (EDI Tools for .NET Tutorial - Part 1) and are familiar with the concepts of EDI Template, the C# POCOs, and how to translate or generate EDI files.

The tutorial is divided into several sections:

• The overview will remind you of the concept of EDI Templates and C# POCOs

• How to Validate EDI Objects will teach you to validate EDI files, including HIPAA Snip Levels, EDI Codes, cross-segment dependencies, and to injecting custom code

• Generate EDI Acknowledgments will teach you how to generate acknowledgments such as 997 or 999
• Save EDI file to DB will teach you how to create a database structure for your EDI transaction and save\pull EDI transactions to\from database using Entity Framework DB context
• Convert between EDI and JSON will teach you how to serialize\deserialize EDI to\from JSON using Newtonsoft Json.NET
• Convert between EDI and XML will teach you how to serialize\deserialize EDI to\from XML using XmlSerializer

You don’t have to complete all of the sections at once to get the value out of this tutorial. Try to get as far as you can - even if it's one or two sections.

Setup and Prerequisites

You can see what we'll be building here:

Download C# examples

The narrative will use primarily X12 examples; however, a link to the corresponding EDIFACT/HL7/NCPDP examples will be included for every section. We'll assume that you have some familiarity with EDI and X12, but you should be able to follow along even if you're new to EDI. We'll also assume that you're familiar with .NET and C#. To learn more about EDI go to our What is EDI article.

You'll also need Visual Studio to open the example solutions: Get Visual Studio.

EDI Tools for .NET Main Concepts

• EDI messages are simple C# objects

  EDI Tools for .NET Tutorial - Part 1 introduced the concept of EDI Template, which is nothing more than a C# class, inheriting from our own EdiMessage and comprised of properties annotated with our EDI attributes.

• Parsing EDI file is the process of transposing each message into a C# object

  When an EDI file is translated or parsed, the resulting POCOs are instantiated by the C# EDI library from each corresponding EDI Template. Each POCO represents one transaction (or message) from the EDI file.

• Generating an EDI file is the process of populating C# objects, and then writing them to a stream

  Alternatively, when EDI files are created, POCOs are manually instantiated and populated with data, and then written out to EDI files or streams.

Let's take another look at the colorful diagram from EDI Tools for .NET Tutorial - Part 1, displaying the main operations you can apply to a POCO, and discuss each operation in more detail.

How to Validate EDI Objects

Accuracy is one of the selling points when employing EDI processes within any enterprise. To be able to validate data before it is sent out according to the same criteria as the receiver expects it, well, that saves a lot of roundtrips and delays. Proper validation of EDI documents is essential for businesses and allows them to exchange data rapidly and reliably, thus gaining them a significant competitive advantage.

Validating EDI messages is the same as validating C# models, as in MVC

Touching on the notion of EDI Templates, validation follows the same principle as Microsoft's own data annotations for model validation (MVC) or entity validation (Entity Framework). Just like when the template attributes are applied to a simple C# class to turn it into EDI Template, another set of our own attributes can be applied to the template classes and properties, enabling them for validation.

These validation attributes allow us to reflect the EDI rules from the specifications to the template classes and the validation operation IsValid() in EDI Tools for .NET to take care of enforcing them and providing appropriate error information back to the consumer.

Validation attributes:

• Required
• ListCount
• StringLength
• DataElement

Go to the EDI Validation Attributes article

Conditional validation attributes:

• Conditional
• ConditionalAny
• Exclusion
• Paired
• RequiredAny

Go to the EDI Conditional Validation Attributes article

So, the validation of EDI transactions is a two-step process:

1. Apply validation attributes to classes and properties according to the EDI implementation guideline (standard or partner-specific, or both). Being simple classes, EDI templates can have multiple versions, per partner or EDI version, which will be loaded accordingly by the library.
2. Execute the validation. This is done by calling IsValid() on any POCO deriving from EdiMessage

Code to validate EDI transactions

Create a new Console project following the Create a New Project from Scratch tutorial.

After you finished it, open Program.cs and add the following code:

Stream edi = File.OpenRead(@"C:\\PurchaseOrder.txt");

List<IEdiItem> ediItems;
using (var reader = new X12Reader(edi, "EdiFabric.Templates.X12"))
	ediItems = reader.ReadToEnd().ToList();

var purchaseOrders = ediItems.OfType<TS850>();

foreach (var purchaseOrder in purchaseOrders)
{
	//  Validate
	MessageErrorContext errorContext;
	if (!purchaseOrder.IsValid(out errorContext))
	{
		var errors = errorContext.Flatten();
	}
	else
	{
		//  purchaseOrder is valid, handle it downstream
	}
}

Change the path in File.OpenRead to the path of the sample purchase order file you downloaded earlier.

That's it, now run this in Debug mode and see the magic happen!

Example Code in GitHub for all EDI standards:

• Validate X12 message
• Validate EDIFACT message
• Validate HL7 message
• Validate NCPDP message
• Validate SCRIPT message

Inspect the code

What does the code do? Let's go into the details.

• The lines before the foreach statement simply translate the sample EDI file into a list of C# objects (EDI items).
• Then we use the standard .NET OfType<> method to pull the objects we are interested in, by type
• Finally, the foreach statement iterates through all the purchase orders from the EDI file and each of them is individually validated with the IsValid() method.
• For the invalid messages, a MessageErrorContext object is supplied, which contains the location and reason for the invalid loop, segment, or data element.

That's it really. A simple method to kick off an elaborate validation process under the hood and report back any wrong findings in a structured, EDI-friendly manner.

Validating External EDI Data Element Code Sets

EDI data element code sets are values within the range accepted by the EDI transaction, such as 'delivery type', 'post code', 'currency code'. Some EDI fields contain a code value, and only a certain set of codes are valid under the EDI standard. 

Often, trading partners introduce their own EDI codes that are different than the standard ones for X12 or EDIFACT. EDI Tools for .NET maintain the code lists in the EDI template. These modifications are easily supported in EDI Tools for .NET by either:

• Creating a separate copy of the EDI Template for each partner, and modifying the EDI code sets in the copy
• Using external EDI codes map when validating EDI messages

The standard EDI templates can be copied per trading partner and then each copy can be modified to adhere to the partner specifications. This is useful when partner formats deviate substantially from the standard.

When the changes are only contained to the EDI codes, then an EDI code map can be used, leaving the rest of the template untouched.

EDI Data Element Code Sets Example

Let's assume that Partner A specifies custom EDI Codes for the Transaction Set Purpose Code X12_ID_353. The standard definition for X12, version 4010 is:

[EdiCodes(",00,01,02,03,04,05,06,07,08,10,11,12,13,14,15,16,17," +
"18,19,20,21,22,24,25,26,27,28,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44," +
"45,46,47,48,49,50,51,52,53,54,55,5,6,5C,77,CN,CO,EX,GR,PR,RH,RV,SU,ZZ,")]
public class X12_ID_353
{
}

Another partner, Partner B, however, defines it as a set of only two, not included in the standard, values. In the same EDI template, create a separate class, X12_ID_353_PartnerB, for this particular code set.

[EdiCodes(",PA,PB,")]
public class X12_ID_353_PartnerB
{
}

By default when the EDI Template for the standard 850 is used (the Partner A version), calling IsValid() will check the values from X12_ID_353, but we want it to check against the values from X12_ID_353_PartnerB, when validating messages from Partner B.

This is possible by including an EDI code map as a parameter to IsValid().

In the same Program.cs, change the code to:

Dictionary<Type, Type> codeSetMap = new Dictionary<Type, Type>();
codeSetMap.Add(typeof(X12_ID_353), typeof(X12_ID_353_PartnerB));

purchaseOrder.IsValid(out errorContext, 
     new ValidationSettings { DataElementTypeMap = codeSetMap });

The changes are:

• The first two lines define the EDI Code map, which is a dictionary mapping the source to the destination EDI Code type. This way when the validation logic encounters the source type it will apply the validation according to the destination type.
• Pass that EDI Code map to IsValid() as DataElementTypeMap in ValidationSettings.

Additional information:

• How to validate EDI transactions
• EDI Validation Attributes
• How to configure the EDI validation settings
• How to validate X12 data elements
• How to validate EDIFACT data elements
• How to embed custom validation logic

Generate EDI Acknowledgments

In the previous section, we mentioned the importance of accuracy in the exchange of business documents. Validation is a nifty way for a receiver to discern the quality of the data received, however in the cases it is poor, how would it report it back to the sender? What would be a standard way to let the sender that the data was received and it is of a sufficient (or not) quality?

What are EDI Acknowledgments

EDI Acknowledgments are a standard approach, implemented as part of the EDI communication, to acknowledge receipt and data consistency between trading partners. Broadly speaking, there are two types of acknowledgments - technical, which confirms the receipt of an interchange called TA1 for X12, CONTRL for EDIFACT, and functional, which confirms that the data received was recognized by the receiver and can be parsed\processed, and are called 997 for X12 and CONTRL for EDIFACT. EDIFACT uses the same transaction but different segments are transmitted when used as technical or functional ack.

EDI Tools for .NET generates EDI acknowledgments as C# POCOs, and raises them as events, whilst still reading the EDI file

EDI Tools for .NET employs a parallel approach when dealing with acknowledgments, meaning that whilst an EDI file is still being translated, acknowledgments will be raised via .NET events as functional groups and interchanges are encountered. This allows the consumer to do both operations, translation, and acknowledgment, in parallel and potentially in separate threads, thus providing a non-blocking and fast mechanism to respond immediately and to maintain a single acknowledgment in-memory.

Technical, functional, and implementation EDI acknowledgments

Let's acknowledge our sample EDI file which had the following structure:

A TA1 (X12 technical acknowledgment) will be created for each interchange (ISA\IEA block). A 997 (or 999) will be created for each functional group (GS\GE block). The ISA segment also contains a flag (in purple) telling us whether the sender expects an acknowledgment or not. This flag can either be honored or ignored or overridden depending on the agreement between the partners.

Code to generate X12 TA1 and 997 acknowledgments

Create a new Console project following the Create a New Project from Scratch tutorial.

After you finished it, open Program.cs and add the following code:

var edi = File.OpenRead(@"C:\PurchaseOrder.txt");

var settings = new AckSettings
{
    AckHandler = (s, a) =>
    {
	var tsTa1 = a.Message as TSTA1;
	var ts997 = a.Message as TS997;

	if (tsTa1 != null)
	{
	    // a.Message is TA1 
	}

	if (ts997 != null)
	{
	    //  Inspect the acknowledgment
	    var ackCode = ts997.AK9.FunctionalGroupAcknowledgeCode_01;

	   var ack = AckBuilders.BuildAck(a.InterchangeHeader, 
                                          a.GroupHeader, 
                                          ts997, AckVersion.X12_997);
	}
    },
    MessageHandler = (s, a) =>
    {
	if (!a.ErrorContext.HasErrors)
	{
	}
    },
    AckVersion = AckVersion.X12_997
};

using (var ackMan = new Plugins.Acknowledgments.X12.AckMan(settings))
{
    using (var ediReader = new X12Reader(edi, 
                                    "EdiFabric.Templates.X12"))
    {
	while (ediReader.Read())
		ackMan.Publish(ediReader.Item);
    }
}

Change the path in File.OpenRead to the path of the sample purchase order file you downloaded earlier.

Example Code in GitHub for all EDI standards:

• Generate X12 997, 999 and TA1
• Generate EDIFACT CONTRL

Inspect the code

It begins with creating an instance of AckSettings which holds references to the bodies of the events where the acknowledgments will be raised. This example defines them inline but you can code it in your preferred way. AckSettings defines two event handlers:

1. AckHandler - where all events will be raised, both technical and functional
2. MessageHandler - where each POCO will be reported, e.g. our purchase order in this case

The reason for having a separate handler for the transactions is to be able to process them in it, as opposed to in the EDI reader while loop because:

• The will be validated already
• They will be checked for duplicates
• They will be marked as being in a duplicate functional group or interchange

If you need to ensure any of the 3 points above are valid before processing the transaction downstream, then do so in the MessageHandler .

The AckHandler provides the acks as POCOs (all EDI Templates for the standard acknowledgments can be referenced from EdiFabric.Core), so you can inspect or manipulate them at this stage. You can then generate a final EDI acknowledgment using the EDI Generation routine we learned about in EDI Tools for .NET Tutorial - Part 1 of this tutorial. The original ISA and GS are also included in the event so you can reuse some of the information to create their response counterparts if needed. With that information at hand, the EDI acknowledgment can be turned into a string or written out to a file.

The last step is how to actually inject that AckSettings instance as part of the reader, or any list of EDI items for that matter? You need to create an instance of AckMan taking the AckSettings as a parameter.

using (var ackMan = new Plugins.Acknowledgments.X12.AckMan(settings)) 

Then whilst you read, add the extra step of publishing the currently read item to AckMan

while (ediReader.Read()) 
    ackMan.Publish(ediReader.Item);

You don't need to validate the transactions separately, they are already validated when raised in the MessageHandler.

The ack object in AckHandler is a normal POCO, an instance of an EDI Template.

Additional information:

• How to generate EDI acknowledgments
• EDIFACT CONTRL Error Codes
• X12 997 Error Codes
• X12 999 Error Codes
• X12 TA1 Error Codes

EDI files to Database (DB)

Using Entity Framework Migrations, you can automatically create a database structure for all X12 and EDIFACT EDI templates, sharing segments and complex data elements across all transactions in the same version.

Each EDI Template is prepared for use with Entity Framework:

• Id property is included for every class, to serve as a primary key
• Complex properties are defined as virtual
• Collections of properties are defined as List<>
• A separate DB context is provided for each EDI version

It's amazing how quickly you can create a new database and start saving EDI transactions to it.

Let's demonstrate it:

Create a database for EDI transactions and save an EDI message to it

Create a new Console project following the Create a New Project from Scratch tutorial, and install Entity Framework.

Then add the DbContext.cs file for version 4010 to that project.

The DbContext.cs file contains System.Data.Entity.DbSet for every EDI Template in the selected EDI version, however for our example we only need the purchase order and you'll have to manually remove all other DbSets pointing to items that are not included.

Excellent, so we have our Entity Framework DB context ready, let's use it.

Open Program.cs and add the following code:

Stream edi = File.OpenRead(@"C:\\PurchaseOrder.txt");

List<IEdiItem> ediItems; 
using (var reader = new X12Reader(edi, "EdiFabric..Templates.X12")) 
    ediItems = reader.ReadToEnd().ToList();

var purchaseOrders = ediItems.OfType<TS850>();

using (var db = new X12Context())
{
    db.TS850.AddRange(purchaseOrders);
    db.SaveChanges();
} 

Change the path in File.OpenRead to the path of the sample purchase order file you downloaded earlier.

Example Code in GitHub for all EDI standards:

• Save X12 message to DB
• Save EDIFACT message to DB

Inspect the code

The top part reads our sample EDI file. Once we have all the purchase order POCOs, we create an instance of the DB Context, add the purchase orders to the TS850 entity and finally save it to the database.

The save operation will create the database structure the first time it is executed:

Edit the connection string to point to a specific SQL Server instance.

That's it. Now you can enjoy the full power of Entity Framework to query and maintain the database.

Convert between EDI and JSON

Are any additional operations required to convert between EDI and JSON? Not at all. EDI data, transactions, and control segments are represented as C# POCOs, hence you can use the familiar Newtonsoft Json.NET to serialize to JSON and deserialize from JSON.

Create a new Console project following the Create a New Project from Scratch tutorial

Open Program.cs and add the following code:

List<IEdiItem> ediItems; 
using (var reader = new X12Reader(edi, "EdiFabric.Examples.X12.Templates.V4010")) 
    ediItems = reader.ReadToEnd().ToList();

var purchaseOrders = ediItems.OfType<TS850>();

foreach (var transaction in transactions)
{
    string json = Newtonsoft.Json.JsonConvert.SerializeObject(transaction);
}

Change the path in File.OpenRead to the path of the sample purchase order file you downloaded earlier.

Example Code in GitHub for all EDI standards:

• Convert between X12 and JSON
• Convert between EDIFACT and JSON
• Convert between HL7 and JSON
• Convert between NCPDP and JSON
• Convert between SCRIPT and JSON

Inspect the code

The top part reads our sample EDI file. Then we iterate through each purchase order in the EDI file and convert it to JSON using the functionality provided in Json.NET.

Deserialization is the other way around, we take a JSON that conforms to the EDI Template for 850 and deserialize it to C# POCO, again, using Json.NET.

var purchaseOrder = Newtonsoft.Json.JsonConvert.DeserializeObject<TS850>(json);

Additional information:

• Convert between EDI and JSON

Convert between EDI and XML

Similar to converting between EDI and JSON, we can use the familiar XmlSerializer and DataContractSerializer to convert between EDI and XML.

Create a new Console project following the Create a New Project from Scratch tutorial

Open Program.cs and add the following code:

List<IEdiItem> ediItems; 
using (var reader = new X12Reader(edi, "EdiFabric.Examples.X12.Templates.V4010")) 
    ediItems = reader.ReadToEnd().ToList();

var purchaseOrders = ediItems.OfType<TS850>();

foreach (var transaction in transactions)
{
    var xml = Serialize(transaction);
}
public static XDocument Serialize(EdiMessage instance)
{
    if (instance == null)
        throw new ArgumentNullException("instance");

    var serializer = new XmlSerializer(instance.GetType());
    using (var ms = new MemoryStream())
    {
        serializer.Serialize(ms, instance);
        ms.Position = 0;
        return XDocument.Load(ms, LoadOptions.PreserveWhitespace);
    }
}

Change the path in File.OpenRead to the path of the sample purchase order file you downloaded earlier.

Example Code in GitHub for all EDI standards:

• Convert between X12 and XML
• Convert between EDIFACT and XML
• Convert between HL7 and XML
• Convert between NCPDP and XML
• Convert between SCRIPT and XML

Inspect the code

The top part reads our sample EDI file. Then we iterate through each purchase order in the EDI file and convert it to XML using the functionality provided in XmlSerializer.

Deserialization is the other way around, we take an XML that conforms to the EDI Template for 850 and deserialize it to C# POCO, again, using XmlSerializer.

var purchaseOrder = Deserialize<TS850>(xml);
public static T Deserialize<T>(XElement xml)
{
    var serializer = new XmlSerializer(typeof(T));
    return (T)serializer.Deserialize(xml.CreateReader());
}

Additional information:

• Convert between EDI and XML
• Create XSD from EDI Template

Final Words

Congratulations again! You've created code that validates EDI transactions, acknowledges EDI interchanges and groups, creates DB structure to save and query EDI transactions, and serializes\deserializes to\from JSON & XML.

For a more detailed explanation of each of these topics, check out the rest of the documentation.