Upgrade Broke all of my Outbound EDI Maps by Rearranging RDD Sequence

This is what I did to fix it, in hopes someone with the same problem goes a Googlin’…

We did a minor upgrade from 10.1.600.5 to 10.1.600.20. Our 856 and 810 outbound maps immediately started failing.

I compared the new (broken) output files against ones that worked before the upgrade. The schema didn’t change (as I would expect in a minor version upgrade), but my data was coming out in a different order than it was the day I created the maps:

I figured I could go back and remap all outbound documents for all trading partners (this could take days), or I could try and get Epicor to spit the data out in the “good” order so the map could interpret them with (hopefully) no changes.

There is an option in Report Style under Actions >> Generate EDI Definition that you should run after you set up your RDD’s and have them working. It generates a CSV reference file that effectively “locks” the schema for you. It will protect you against additions/subtractions (like UD Fields) to any of the tables included in your RDD, but apparently it doesn’t influence the order your data is sequenced.

So I went to the Report Data Definition (create a backup of your RDD with the Solution Workbench just to be safe!) and found the Sequence number the tables are listed in. Shot in the dark, I rearranged the Sequence to match my working Datafile:

I reprinted all of my outbound documents and the map then processed them with no issues. I then had to do the same thing for my other outbound maps’ Report Data Definitions.

A good lesson here is to grab a sample set of any working EDI output files that Epicor generates and save them to compare to if you need to later. The mapping software we use (Tie/Kinetix) deletes these files when it processes them. You can write custom workflows to back the files up, but I opted to write my own system to keep tabs on EDI send/ack status and I preserve 30 days worth of these files in case I need them–and last night I was certainly glad I did.

(WHEW, all columns are green again!)

1 Like

Checking for a way to mark resolved…

Thanks Lee, this was really helpful! We always run into 810 issues after an upgrade.

What I discovered is that while the Actions >> Generate EDI Definition doesn’t output the order of your tables correctly, I was able to manually rearrange the CSVs columns in Excel and get the desired result. That saved me the trouble of monkeying with the RDD sequence and is hopefully more durable.

For anyone else facing this issue, the generated CSVs reside at .\EpicorData\Reports\CustomReports\EDI_Definitions

By arranging the CSVs columns horizontally, I was able to affect the vertical arrangement in the output:

image

2 Likes

Do I understand correctly that TIE Kinetix does not use the schema rows, and is therefore sensitive to column order in the output file? I had hoped that dumping our current EDI provider for TIE would remove this headache.

We’d never had an EDI definition for our 810s, so I generated one. The column order came out slightly different from the mostly (but not entirely) stable order that Epicor had been outputting without an EDI definition. Fortunately, I have a backup of every 810 we’ve ever sent. I’ll write a program to generate EDI definitions from old EDI documents.

You can rearrange the columns ( in excel) in the EDI definition if you generated your own definition from the report styles.

Vinay Kamboj

Turns out I didn’t need to write anything. You can create an EDI definition from a known good EDI report with the following steps:

  1. Open the report as a CSV with tilde as the delimiter.
  2. Grab the “Schema_” rows and rotate them.
  3. Replace “Schema_” with the empty string.
  4. Save as a CSV with comma as the delimiter.
  5. Open in a text editor and delete any newlines after the last row. (Not sure if this matters, but Epicor’s EDI definitions do not have a newline at the end of the file.)

The result is byte-for-byte identical to the EDI definition that was in effect when the EDI report was printed.

I am having a problem with the order of the tables. I’ve taken the standard RDD for EDIARFrm, duplicated it, and added the OrderHed table (linked to InvcDtl). I used the sequence number in the RDD to make OrderHed my LAST table. Then, I generated the EDI Definitions file, and I modified that by moving OrderHed to the end.

What I’ve got now, is the “Schema” records come out in the correct order. But in the data section below, the OrderHed is NOT at the end. I could care less that the schema’s in the right order.

Here’s a snapshot of part of the EDI Defintiion file, showing the last few columns. Notice OrderHed is at the end.

Here’s a snapshot of the way the output it coming out:

image

Yes, I know - I can grab OrderHed as a linked table from InvcDtl, but I’m trying to match up with maps that are already in existence. I’ve tried all kinds of things, nothing seems to work. While we could go to workarounds, I’d rather just make the data output match so we don’t have to touch maps.

Thanks!
Kevin Simon

1 Like

You are correct. TIE does not use the schema other than to create their initial map of data. From then on, it’s column order only.

Makes me long for the good-old-days. Bring back the Asteroids game and the handheld Mattel Football game.

I tried writing a library to read and manipulate EDI report files, and I think I understand why EDI VANs don’t use the schema rows. The file format is clever, but doesn’t completely describe the data, leaving a potential for ambiguity if we try to parse the file in a universal way without relying on knowledge of foreign key relationships in the data rows.

We can think of data rows as being divided into two categories: rows that appear exactly once in each document, like Company, and rows that may appear zero or more times in each document, like InvcDtl and InvcMisc. If the data rows were guaranteed to appear in the same order as the schema rows, then detecting the beginning of a new document would be trivial. We would know we’ve encountered a new document whenever the schema index of the current data row type is less than the schema index of the previous row type. I think this is how the file format was intended to work. (Edit: This method also depends on there being at least one row type that appears in every document, which is probably the case.)

But since the order of the data rows is not guaranteed, we have to use another trick that relies on the fact that rows of the same type seem to be grouped within each document. If the row type of the current row is different from the previous row, and is a type that already exists in the current document, then start a new document.

But there is still ambiguity because of the possibility of encountering a row of a type where zero or more may exist, which does not exist in the current document. Because row order is not guaranteed, we cannot tell if this row belongs to the current document or the next document.

Epicor could resolve the ambiguity by guaranteeing that the data rows appear in the same order as the schema, or by adding a document index column. A document index would also resolve the edge case (which probably never happens in practice) where there is only one schema row.

Have you reported the table order issue? Does it have an issue number?

Another fun thing to watch out for: when using an EDI Definition, EDI report styles no longer output the Company row with every document in the report, only the first document. This broke my integrations because my VAN is using the Company row as the delimiter between documents. Epicor says,

This behavior is tied to software change request ERPS-126094 targeted to be included in an upcoming ERP 10.2.400 point update–there are not plans on retroing this change to earlier releases.

Just wait till you upgrade to 10.2.400 and they also change column names on your 810 output - lots of fun.

I created a small library for reading, validating, and fixing Epicor’s EDI files. It includes workarounds for the Company row bug that I mentioned and the row order bug that @SimsTrak reported. I’m using it to pre-process EDI files before they’re uploaded to our VAN. Let me know if you find it useful.

1 Like