What is EDI from a programmer's perspective?
Electronic data interchange (EDI) is the most commonly used B2B and Healthcare technology for computer-to-computer exchange of structured documents between trading partners/health institutions.
In 2019 EDI accounted for 78.4%—$7.00 trillion—of all B2B electronic sales.
The core tenet of EDI is that when two organizations want to communicate electronically, they can exchange EDI messages, transported as files. Each file contains data in a well-known and agreed-upon format, that both organizations understand and adhere to.
Our focus will be on the technical standards that exist to facilitate the exchange of business/healthcare data automatically.
This is what EDI files are:
-
ISA*00* *00* *16*SENDER1 *1B*RECEIVER1 *071216*1406*U*00204*000000263*1*T*>~ GS*IN*SENDER1*RECEIVER1*20071216*1406*000000001*X*004010~ ST*850*0001~ BEG*00*SA*XX-1234**20170301**NA~ PER*BD*ED SMITH*TE*8001234567~ TAX*53247765*SP*CA*********9~ N1*BY*ABC AEROSPACE*9*1234567890101~ N2*AIRCRAFT DIVISION~ N3*2000 JET BLVD~ N4*FIGHTER TOWN*CA*98898~ PO1*1*25*EA*36*PE*MG*XYZ-1234~ MEA*WT*WT*10*OZ~ IT8*******B0~ SCH*25*EA***106*20170615~ CTT*1~ AMT*TT*900~ SE*15*0001~ GE*1*000000001~ IEA*1*000000263~
-
UNB+UNOB:1+SENDER1:14:ZZUK+RECEIVER1:1:ZZUK+071101:1701+131++ORDERS++1++1' UNH+000000101+ORDERS:D:96A:UN' BGM+220+128576+9' DTM+137:20020830:102' PAI+::42' FTX+ZZZ+1+001::91' RFF+CT:652744' DTM+171:20020825:102' NAD+BY+5412345000013::9' RFF+VA:87765432' CTA+OC+:P FORGET' COM+0044715632478:TE' NAD+SU+4012345500004::9' RFF+VA:56225432' CUX+2:GBP:9+3:EUR:4+1.67' DTM+134:2002080120020831:718' TDT+20++30+31' TOD+3++CIF:23:9' LOC+1+BE-BRU' LIN+1++4000862141404:SRS' PIA+1+ABC1234:IN' IMD+C++TU::9' QTY+21:48' MOA+203:699.84' PRI+AAA:14.58:CT:AAE:1:KGM' RFF+PL:AUG93RNG04' DTM+171:20020801:102' PAC+2+:51+CS' PCI+14' LOC+7+3312345502000::9' QTY+11:24' DTM+2:20020915:102' LOC+7+3312345501003::9' QTY+11:24' DTM+2:20020913:102' TAX+7+VAT+++:::17.5+S' UNS+S' CNT+2:1' UNT+38+000000101' UNZ+1+131'
-
FHS|^~\&|LCAMC|LAB|PLUS|PLU|20160927123530|||TEST|F1627112308801 BHS|^~\&|LCAMC|LAB|PLUS|PLU|20160927123530||||B000001 MSH|^~\&|Pharm|GenHosp|IE||2006052911150700||RDS^O13^RDS_O13|MSG00001-|P|2.6 PID|||444-33-3333^^^MPI&GenHosp&L^MR||Everyman^Adam||19600614|M||C|2222 Home St^^Anytown^US^12345||^^^^^555^5552004 ORC|NW|177^SMS|||||1^C^^200611280900^^R^^^^C RXO|Cyclic IV ORC|CH|177A^SMS|||||1^C^^^^^^^^C&177C&SMS&&&*ES+0M|177 RXO| Segment, Requested Give Amount-Minimum: ...|125||ML|... Requested Give Per (Time Unit): ...|H1 RXR|IV RXC|B|D5W|1000|ML RXC|A|KCL|40|MEQ ORC|CH|177B^SMS|||||1^C^^^^^^^^C&177A&SMS&&&ES+0M|177 RXO| Segment, Requested Give Amount-Minimum: ...|100||ML|... Requested Give Per (Time Unit): ...|H1 RXR|IV RXC|B|D5/LR|1000|ML RXC|A|KCL|20|MEQ ORC|CH|177C^SMS|||||1^C^^^^^^^^C&177B&SMS&&&#ES+0M|177 RXO| Segment, Requested Give Amount-Minimum: ...|125||ML|... Requested Give Per (Time Unit): ...|H1 RXR|IV RXC|B|D5/0.45NACL|1000|ML FT1|1|||200605291115-0700|||CO^Co-Pay^HL70017 |00378112001^VerapamilHydrochloride 120 mg TAB^NDC |||1|5&USD^TP FT1|2|||200605291115-0700|||PY^Payment^HL70017 |00378112001^VerapamilHydrochloride 120 mg TAB^NDC |||1|5&USD BTS|1 BHS|^~\&|LCAMC|LAB|PLUS|PLU|20160927123530||||B000001 MSH|^~\&|LCAMC|LAB|PLUS|PLU|20160927123530||ORU^R01|M16271000000000001|T|2.6|||ER|ER PID|1|MG82249Q|MG82249Q||SANTIAGO^SAMUEL||19640507|M|||857 NOBLE AVE A^^BRONX^NY^10473|||||||603248005070 PV1|1|O|||||||||||||||||||||||||||||||||||||RN ORC||603248005070|603248005070||||||20160201 OBR|1||603248005070|005009^CBC WITH DIFFERENTIAL/PLATELET^L|||20160201|20160201||||||||||RN|||||||F OBX|1|NM|6690-2^LOINC^LN^005025^WBC^L||4|X10E3/UL|3.4-10.8||||F|||20160201|RN^^L OBX|2|NM|789-8^LOINC^LN^005033^RBC^L||4.02|X10E6/UL|4.14-5.80|L|||F|||20160201|RN^^L MSH|^~\&|CLINREG|WESTCLIN|HOSPMPI|HOSP|199912121135-0600||QBP^Q21^QBP_Q21|1|D|2.6 QPD|Q21^Get Person Demographics^HL7nnn|111069|112234^^^METRO HOSPITAL|^^^METROHOSPITAL~^^^SOUTH LAB RCP||I BTS|2 FTS|2
-
00TNNQ02074115 FHCSERVSCORP2017086201703271140P12EMEDNYBAT G10466917401004740D0B1YRG7065NNQ10502074115 20151112 AM04C2DK86342SC90C3001AM01C419970213C51CATAHIEMCBHARTAM07EM1D2000005073530E103D716729014912E760000D300D5060D61D80DE20151112DF00DJ1C801EK06KVYZ11NV03AM11D9218GDX0000000{DQ253GDU10177EAM03EZ01DB1023043676AM054C25CDATA16CDATA2HB2HCDATA3DVDATA4HCDATA5DVDATA65CDATA76CDATA8HB2HCDATA9DVDATA10HCDATA11DVDATA129920170860000000003
-
UNA:+./*' UIB+UNOA:0++1234567+++77777777:C:PASSWORDA+7701630:P+19971001:081322' UIH+SCRIPT:010:006:NEWRX+110072+++19971001:081322' PVD+P1+7701630:D3+++++MAIN STREET PHARMACY++6152205656:TE' PVD+PC+6666666:0B+++JONES:MARK++++6152219800:TE' PTT++19541225+SMITH:MARY+F+333445555:SY' DRU+P:CALAN SR 240MG::::240:::::::AA:C42998:AB:C28253+:60:38:AC:C48542+:1 TID -TAKE ONE TABLET TWO TIMES A DAY UNTILGONE+85:19971001:102*ZDS:30:804+0+R:1' UIT+110072+6' UIH+SCRIPT:010:006:NEWRX+110072+++19971001:081322' PVD+P1+7701630:D3+++++MAIN STREET PHARMACY++6152205656:TE' PVD+PC+6666666:0B+++JONES:MARK++++6152219800:TE' PTT++19541225+SMITH:MARY+F+333445555:SY' DRU+P:CALAN SR 240MG::::240:::::::AA:C42998:AB:C28253+:60:38:AC:C48542+:1 TID -TAKE ONE TABLET TWO TIMES A DAY UNTILGONE+85:19971001:102*ZDS:30:804+0+R:1' UIT+110072+6' UIH+SCRIPT:010:006:NEWRX+110072+++19971001:081522' PVD+P1+7701630:D3+++++MAIN STREET PHARMACY++6152205656:TE' PVD+PC+6666666:0B+++JONES:MARK++++6152219800:TE' PTT++19541225+SMITH:MARY+F+333445555:SY' COO+123456:BO+INSURANCE COMPANY NAME++123456789++AA112' DRU+P:CALAN SR 240MG::::240:::::::AA:C42998:AB:C28253+:60:38:AC:C48542+:1 TID -TAKE ONE TABLET TWO TIMES A DAY UNTILGONE+85:19971001:102*ZDS:30:804+0+R:1' UIT+110072+7' UIH+SCRIPT:010:006:NEWRX+110072+++19971001:081522' REQ++1' PVD+P1+7701630:D3+++++MAIN STREET PHARMACY++6152205656:TE' PVD+PC+6666666:0B+++JONES:MARK++++6152219800:TE' PTT++19541225+SMITH:MARY+F+333445555:SY' DRU+P:CALAN SR 240MG::::240:::::::AA:C42998:AB:C28253+:60:38:AC:C48542+:1 TID -TAKE ONE TABLET TWO TIMES A DAY UNTILGONE+85:19971001:102*ZDS:30:804+0+R:1' UIT+110072+7' UIH+SCRIPT:010:006:NEWRX+110072+++19971001:081522' REQ++1' PVD+P1+7701630:D3+++++MAIN STREET PHARMACY++6152205656:TE' PVD+PC+6666666:0B+++JONES:MARK++++6152219800:TE' PTT++19541225+SMITH:MARY+F+333445555:SY' DRU+P:CALAN SR 240MG::::240:::::::AA:C42998:AB:C28253+:60:38:AC:C48542+:1 TID -TAKE ONE TABLET TWO TIMES A DAY UNTILGONE+85:19971001:102*ZDS:30:804+0+R:1' UIT+110072+7' UIH+SCRIPT:010:006:NEWRX+110072+++19971001:081522' PVD+P1+7701630:D3+++++MAIN STREET PHARMACY++6152205656:TE' PVD+PC+6666666:0B+++JONES:MARK++++6152219800:TE' PTT++19541225+SMITH:MARY+F+333445555:SY' DRU+P:PAXIL:::30:::::::AA:C42998:AB:C28253+:30:38:AC:C48542+1 QID-TAKE ONE TABLETTHREE TIMES A DAY+85:19971001:102*ZDS:30:804++R:5+:::22000123456:PD' UIT+110072+6' UIH+SCRIPT:010:006:NEWRX+110072+++19971001:081522' PVD+P1+7701630:D3+++++MAIN STREET PHARMACY++6152205656:TE' PVD+PC+6666666:0B+++JONES:MARK++++6152219800:TE' PTT++19541225+SMITH:MARY+F+333445555:SY' COO+123456:BO+INSURANCE COMPANY NAME++123456789++AA112' DRU+P:CALAN SR 240MG::::240:::::::AA:C42998:AB:C28253+:100:38:AC:C48542+:1 TID -TAKE ONE TABLET TWO TIMES A DAY UNTILGONE+85:19971001:102*ZDS:30:804+0+R:1' UIT+110072+7' UIH+SCRIPT:010:006:NEWRX+110072+++19971001:081522' PVD+P1+7701630:D3+++++MAIN STREET PHARMACY++6152205656:TE' PVD+PC+6666666:0B+++JONES:MARK++++6152219800:TE' PTT++19541225+SMITH:MARY+F+333445555:SY' COO+123456:BO+INSURANCE COMPANY NAME++123456789++AA112' DRU+P:CALAN SR 240MG::::240:::::::AA:C42998:AB:C28253+:60:38:AC:C48542+:1 TID -TAKE ONE TABLET TWO TIMES A DAY UNTILGONE+85:19971001:102*ZDS:30:804+0+R:1' UIT+110072+7' UIH+SCRIPT:010:006:NEWRX+110073+110072++19971001:081522' REQ+C1' PVD+P1+7701630:D3+++++MAIN STREET PHARMACY++6152205656:TE' PVD+PC+6666666:0B+++JONES:MARK++++6152219800:TE' PTT++19541225+SMITH:MARY+F+333445555:SY' COO+123456:BO+INSURANCE COMPANY NAME++123456789++AA112' DRU+P:CALAN SR 240MG::::240:::::::AA:C42998:AB:C28253+:60:38:AC:C48542+:1 TID -TAKE ONE TABLET TWO TIMES A DAY UNTILGONE+85:19971001:102*ZDS:30:804+0+R:1' UIT+110073+8' UIZ++9'
-
5110259012 00323625 0002200023121115111231 5120103 187 121115186 1211091514280009100 CGF-56026482A 13 STL 51301121113432701 0000003460000000019427121115000000000000000 5150100000013021300000000000000001303150000000000 000000 5180107-08140295/04 23-09140029 51801 5190100000010000001000000100000000000000000000200000010000001
EDI Standards
There are multiple EDI standards, each targeting either a business vertical (manufacturing, automotive, retail, healthcare) or a geographic location (North America, Europe, UK, Germany, etc.).
The prevalent EDI standards are all supported by EDI Tools for .NET:
The difference between the EDI standards is substantial in regards to their tags and formal description, however, the underlying principles are very similar.
EDI files are a combination of transactions (or messages, or business data) wrapped up in functional groups, which are themselves wrapped up in interchange envelopes. Three levels of nesting.
With some exemptions, the structure above is valid across all EDI standards, where the names of the interchange, group, and message headers and trailers differ per EDI standard.
Collectively, all header and trailer items are called control segments.
EDI Standard | Interchange Header | Group Header | Message Header | Message Trailer | Group Trailer | Interchange Trailer |
---|---|---|---|---|---|---|
X12 | ISA | GS | ST | SE | GE | IEA |
EDIFACT | UNB | UNG (Optional) | UNH | UNT | UNE (Optional) | UNZ |
HL7 | FHS (Optional) | BHS (Optional) | MSH | BTS (Optional) | FTS (Optional) | |
NCPDP Telco | Transmission Header (Optional) | GS1 (Optional) | Transaction Header | Transmission Trailer (Optional) | ||
NCPDP SCRIPT | UIB | UIH | UIT | UIZ |
EDI Interchanges
EDI interchanges (or envelopes) carry information for the sender and the receiver. When two partners exchange EDI files, an agreement is established between them, and the agreement is set up in their respective software systems.
-
ISA*00* *00* *16*SENDER1 *1B*RECEIVER1 *071216*1406*U*00204*000000263*1*T*>~ ... IEA*1*000000263~
-
UNB+UNOB:1+SENDER1:14:ZZUK+RECEIVER1:1:ZZUK+071101:1701+131++ORDERS++1++1' ... UNZ+1+131'
-
FHS|^~\&|LCAMC|LAB|PLUS|PLU|20160927123530|||TEST|F1627112308801 ... FTS|2
-
UNA:+./*' UIB+UNOA:0++1234567+++77777777:C:PASSWORDA+7701630:P+19971001:081322' ... UIZ++9'
The main points to agree on are:
- Sender and receiver IDs - these are unique identifiers assigned to each partner
- The EDI separators to be used (see below)
- Whether technical or functional EDI acknowledgments are required (X12 and EDIFACT)
- How to handle duplicates, e.g. when control numbers are reset, etc.
- Other arrangments.
EDI sender and receiver IDs serve in the same way as sender's and receiver's addresses are written on postcards or paper envelopes. The paper envelope is a good analogy to depict EDI envelopes.
EDI interchanges also tell the date and time when the document was created and the control number of the envelope.
The combination between the sender code, the receiver code, and the control number is unique and is used to rule out any duplicates.
Envelopes also drive the automatic generation of technical acknowledgments, e.g. if one is required or not, and indicate whether the data contained is for testing or production purposes. The trailer contains a checksum to tally it up with the control number in the header and the count of the included items.
EDI Separators
EDI separators (or EDI delimiters) are used to separate the different EDI items from one another. It is imperative that a valid set of delimiter is used and the various types of delimiters are placed at the correct position within an EDI file.
If not otherwise agreed the default separators per standard are defined as:
EDI Standard | Component | Data Element | Escape | Repetition | Segment | Sub Element |
---|---|---|---|---|---|---|
X12 | > | * | ^ | ~ | ||
EDIFACT | : | + | ? | * | ' | |
HL7 | ^ | | | \ | ~ | \r | & |
NCPDP SCRIPT | : | + | / | * | ' |
The agreed separators are used when EDI files are generated for sending to a destination party. When EDI files are received and interpreted, the separators are included in one or more of the control segments.
X12 separators are in the ISA
Let's begin with X12 (also identical for all HIPAA transactions such as 837P or 835).
All separators are defined in the ISA interchange header. ISA is positional and contains 106 characters. The first 3 are the segment tag 'ISA' and immediately after is the data element separator. The last two characters in the ISA are the component data element separator and the segment separator. Sometimes segments can be postfixed and things such as CR\LF can appear at the end of each segment.
If we have the following sample X12 document:
ISA*00* *00* *16*SENDER1 *1B*RECEIVER1 *071216*1406*U*00204*000000263*1*T*>~ GS*IN*SENDER1*RECEIVER1*20071216*1406*000000001*X*004010~ ST*850*0001~ BEG*00*SA*XX-1234**20170301**NA~ PER*BD*ED SMITH*TE*8001234567~ TAX*53247765*SP*CA*********9~ N1*BY*ABC AEROSPACE*9*1234567890101~ N2*AIRCRAFT DIVISION~ N3*2000 JET BLVD~ N4*FIGHTER TOWN*CA*98898~ PO1*1*25*EA*36*PE*MG*XYZ-1234~ MEA*WT*WT*10*OZ~ IT8*******B0~ SCH*25*EA***106*20170615~ CTT*1~ AMT*TT*900~ SE*15*0001~ GE*1*000000001~ IEA*1*000000263~
The separators are (positions within the ISA segment):
- data element separator at position 4 *
- component data element separator at position 105 >
- segment terminator at position 106 ~
- The repetition separator is at position 83 if the ISA version starting at position 85 is greater than '00401'. In the sample above it is not (the version is ''00204') and therefore 'U' is a standard identifier instead of a repetition separator.
EDIFACT separators are either in the UNA or the default
EDIFACT and EANCOM share the same delimiters and use the default set by default (see the table above).
When an envelope is prefixed with the UNA segment (sitting just before the interchange header UNB), all delimiters are derived from that UNA.
The structure of UNA is positional and contains 9 characters. The first 3 are the segment tag 'UNA' and then:
- component data element separator at position 4 :
- data element separator at position 5 +
- decimal notation at position 6 .
- release indicator at position 7 ?
- reserved (not used) at position 8
- segment terminator at position 9 '
This is an example of an EDIFACT envelope with UNA segment:
UNA :+.? ' UNB+UNOB:1+102096559TEST:16:ZZUK+PARTNERID:01:ZZUK+071101:1701+131++INVOIC++1++1' UNH+509010117+INVOIC:D:00A:UN' BGM+380::*380::+12345678:9+AP' DTM+137:19980610:102' TAX+7+VAT:::+++:::16.00' MOA+124:16.00' MOA+125:100.00' UNT+40+0001' UNZ+1+1'
EDIFACT allows delimiters to be escaped by using a release indicator ('?' by default) before the escaped delimiter. X12 does not allow any symbol to be escaped.
EDI Groups
The next level is the groups (or functional groups). Groups are used to combine transactions of the same type and version. They are logical containers and are mandatory for X12 but optional for EDIFACT. The reason for this is that in X12 the functional groups define the version of all transactions in that group whereas in EDIFACT the version is defined at each transaction separately.
-
GS*IN*SENDER1*RECEIVER1*20071216*1406*000000001*X*004010~ ... GE*1*000000001~
-
UNG+INVOIC+2:1+3:4+971013:1040+5+UN+D:96A:UN+PASSPORT' ... UNE+1+5'
-
BHS|^~\&|LCAMC|LAB|PLUS|PLU|20160927123530||||B000001 ... BTS|2
X12 also acknowledges functional groups, e.g. generates 997 or 999 at the end of each group. EDIFACT on the other hand acknowledges envelopes, e.g. generates CONTRL at the end of each envelope.
Functional groups combine messages of the same type (group of invoices or group of purchase orders but not both) and of the same version. They are optional in EDIFACT but mandatory in X12.
EDI Transactions
The inner-most level of EDI documents contains business transactions. This is where all payloads such as claims or invoices or purchase orders are stacked up.
EDI transactions are defined by the organization governing the respective EDI standard, e.g ASC is for X12, UNECE for EDIFACT, GS1 for EANCOM, etc. They release new versions of the transactions every year.
X12 and EDIFACT are not backward compatible and an invoice 810 in version 003010 is not compatible with an invoice in version 004010. It is primarily because of the EDI codes that can be attached to data elements.
Although there are many versions in circulation the most popular are 004010 for X12 and D96A for EDIFACT. If you are ever going to exchange messages with a trading partner it's highly likely that they will be using one of these two.
An EDI transaction is for example a purchase order, an invoice, or a medical claim. All of these business documents must be formatted according to the agreed standard between any trading/medical partners before they start exchanging EDI files.
EDI Complexity
EDI transactions are described by a set of format documents, or implementation guidelines, mappings, or specs; you name it. This meta-format, describing the format of every EDI document in the way the creator of the document intends it to look like, is at the core of all underlying applications or software systems that will be processing those EDI documents.
There are a few main meta-formats in use today, IBM uses SEF, Microsoft uses XSD, EDI Tools for .NET uses .NET attributes and C# classes, etc., and a multitude of incompatible vendor-specific and proprietary scripts and languages.
The obvious problems here are:
- The meta-formats that exist today are incompatible with each other. This means that an investment in one of them results in a limited choice of EDI software, resources, and vendors.
- None of the existing meta-formats is easily applicable to the Web or APIs. This supports the presumption that EDI is a legacy technology that is ill-suited for the Internet.
EDI Transport
EDI can be viewed as an extra compression or encoding of data. That's pretty much what it is from a programmer's perspective. When a purchase order is pulled out from a database and you need to send it to another application what is the general approach?
I bet that the receiving application exposes a RESTful or SOAP endpoint and you only have to consume it. There will be a mapping code at your end to map the purchase order from the database to the purchase order structure defined at the service endpoint. Why did we need EDI again?
Companies exchanging EDI would rarely expose service endpoints. Instead, they would have a mailbox or some sort of FTP (SFTP, FTPS, etc.) or good old AS2 over HTTP or else. Or simply a file share. The upshot is that they will provide a transport interface that requires EDI documents to be exchanged as files or streams. It's pretty old school.
From the architecture's perspective, this isn't too bad. You will still pull the purchase order out from the database, but then what to map it to? How to generate an EDI file out of it?
The same is valid for the receiver. They'll get an EDI file via FTP or AS2 but how would they read it? What would be the medium?
EDI Templates
Earlier on I alluded to the similarity between EDI and XML. Both can be used to represent hierarchical data structures. Both use meta-formats to define these data structures (XSD, etc.).
Understandably all X12 and EDIFACT transactions are provided as XSDs by their governing organizations. XSD is language-agnostic, and there are good parsers for XML in every programming language. Coupling EDI to XML sounds like a good idea and the effort to get the ball rolling seems justifiable.
If EDI can be defined with XSD and represented as XML, a good bet is that EDI would become obsolete at some point in the future. They will slowly conflate into one by XML absorbing EDI. Or so people thought. Nothing like this has happened to the surprise of many and XML never prevailed. They both continue to co-exist in the same troubled relationship.
The thing is that EDI still needs to be converted to\from XML. It is uncommon for applications to internally represent data using XML. They model their domain data structures using the programming languages they use, be it C++, C#, Java, etc. Therefore XML is predominantly used at the outer borders in data exchange. Internal objects are natively serialized or deserialized to or from XML.
Most of the existing EDI translators rely on this forced marriage between EDI and XML. They convert EDI to XML and vice versa. Then XML can be manipulated using XML DOM and exported to whatever. I must admit that this does appear to be intuitive and I based my first EDI parsers on XML.
What could be better than XML nowadays, perhaps JSON, or YAML? Both assumptions are valid.
EDI Tools for .NET has something better - EDI Templates.
With templates, you can define/represent the structure of EDI transactions in the most developer-friendly way possible - using the native C# programming language classes and attributes. You can then use the built-in functionality in .NET to seamlessly convert between XML, JSON, and the instances of the EDI templates.
Next Steps
Check out the next part of this tutorial, What is EDI Translator, to learn about EDI Templates and How to Parse EDI files.
Comments
0 comments
Please sign in to leave a comment.