[HN Gopher] What makes EDI so hard?
___________________________________________________________________
What makes EDI so hard?
Author : mooreds
Score : 59 points
Date : 2022-01-06 12:35 UTC (10 hours ago)
(HTM) web link (www.stedi.com)
(TXT) w3m dump (www.stedi.com)
| poulsbohemian wrote:
| I cut my teeth on this kind of thing 25 years ago...it's the
| stuff that makes the world go around, but gets shuffled off to
| the back corners of the tech world. Way more interesting and
| nuanced than it gets credit. So many problems that were solved
| decades ago that have been forgotten as the world perpetually
| reinvents the wheel.
| erik_seaberg wrote:
| Maybe we wouldn't reinvent the wheel so often if the old wheels
| PO1**1*2P*0.0001*PE*GE*EDI Core~
|
| didn't require memorizing a huge compression dictionary just to
| work with.
| DennisP wrote:
| You don't have to memorize if you have a computer translate
| for you.
| jcims wrote:
| Why would you do anything else when the world is running at
| 9600 baud.
| jcims wrote:
| Did you ever do any work with Sterling Software? I worked at
| the Commerce division supporting parts of the X.25 and ISDN
| network. Good times.
| mszcz wrote:
| When I worked for a huge electronics distributor in the EU I
| got assigned the EDI hot potato by accident and kinda grew to
| like it. I got to talk to other companies's IT people that
| handled it on the their side instead of business facing sales
| or management people. Started with loving the XML formats and
| hating EDIFACT but that got reversed real fast. One of the
| projects I'm still most proud of from that time is
| consolidating all of our EDI systems into one, coherent (and
| beautifly designed I might add since no one interfered because
| no one cared ;) whole that used EDIFACT as an internal format
| to represent documents. That enabled developing integrations
| quickly by just writing a translator/parser between our format
| and whatever God awful format the other party used.
|
| One thing I do hate with a passion still is the AS2
| transmission later that some vendors used. My god, what a
| shitshow that is. No stable clients, discrepancies between
| implementations and all for what? Essentially transferring
| files. Ugh.
| masa331 wrote:
| I develop an internal system for a company which sells diapers to
| big chain supermarkets in europe. It is connected to them through
| EDI. These are my 2 reasons why EDI is hard and fucked up in my
| opinion:
|
| 1) syntax - we are using XML which has weird element names and
| values and it's mixed camel case with other styles. It's not nice
| or possible to read without a manual/reference
|
| 2) brokers - you connect to others throught "EDI brokers" (dunno
| the right name in english). Basically you don't connect with
| other directly but it always goes through one or more brokers
| which are part of some EDI union or whatnot. So for example order
| from chain to a producer goes like this Supermarket -> Broker ->
| Producer. What this means is that different brokers pass you the
| orders in different way. So i guess it can vary a lot. Our broker
| is maybe the largest in the country and yet the only way how to
| programatically get messages from him is invoking a Java program,
| which downloads the message from his servers and that is possible
| to do exactly once. Then it's up to you to distribute it in your
| company as you wish. God forbid if you corrupt the message or
| something. This is very bad because you have to do a lot more
| work than just do a few REST requests to deal with anything. It
| prolongs basic integration by a lot.
| DennisP wrote:
| Twenty years ago I was working with EDI without XML. The format
| was a series of lines, each with a starting tag followed by a
| delimited list of values.
|
| Each transaction type had a grammar, so each starting tag could
| mean different things depending on its position within the
| grammatical structure of the transaction. We were new to EDI
| and had to roll out a bunch of partners fast, so we bought $60K
| software to help us. Turned out, that software had no
| understanding of the grammar; you had to use a cumbersome GUI
| to essentially write code from scratch to pull out what you
| wanted, with the only control structure available being GOTOs.
| Just pulling out dozen important fields looked like a
| significant engineering effort.
|
| Then our partners said they required us to display to the user
| _everything_ they sent. I ended up ditching the $60K software
| and writing my own tool with a proper parser, so I could input
| the grammar of a new transaction type in an hour and sort
| everything out right. After that it was easy.
|
| We didn't have to deal with brokers at least, we connected to
| everyone directly. I think the company _became_ one of the
| brokers after I left.
| LeChuck wrote:
| I work in container shipping in Europe. Everything is coordinated
| by e-mailing EDI files (baplie, coprar, iftdgn etc.) Back and
| forth. Tens of millions containers per year.
|
| Even though it should be a standard, you can't just e-mail a file
| to someone and except it to work. There's always integration work
| needed on both sides. Even then, errors frequently occur where
| you have to manually adjust something. I work on the operational
| side, but have become quite the edifact expert just to be able to
| do my work.
| onphonenow wrote:
| I did medical billing EDI adjacent stuff with a VMS system (that
| ran decades past point of obsolescence).
|
| One thing - in the past the EDI integrations just had to be
| simple, the feature set was so small - there wasn't room to go
| totally crazy.
|
| Once they mandated full ehealth records and exploded the
| complexity of the medical IT side, clinician productivity (face
| to face with patient) went way down. Integration pain went way
| up. The old tab separated value row turned into GIGANTIC XML
| monstrosities for the smallest things - now thousands of these
| blobs in a document. If you put even 10,000 XML blobs in an
| exchange vs tab separated values - very quickly appreciated the
| old way of doing things. One annoying thing, even the person on
| other end you are dealing with lost all ability to troubleshoot
| issues.
|
| For some reason in some areas - no EDI in the traditional sense
| is being used (ie, govt contracts with idiots who build latest
| buzzword filled solutions - total annoyance to interact with).
| commandlinefan wrote:
| Yeah, worked in medical billing for a while. It's insane. One
| particular example I remember was that there was one code for
| "blood pressure taken while patient lying down" and a
| completely different code for "blood pressure taken while
| patient sitting up" (rather than, say, one code for "blood
| pressure" and another for "patient position").
| joenathanone wrote:
| I'm going to have to implement an EDI medical billing system
| soon (moving from XLS/FTP based billing, say what you will
| about it but relatively easy to understand and automate) and
| after looking over the spec, I am not looking forward to it at
| all. Ended up purchasing a product called Cozyroc which
| integrates into SSIS(SQL Server Integration Services) which can
| do SQL to EDI translation with a map provided, should make
| things easier but still not looking forward to it.
| devadvance wrote:
| Overall solid introduction to the space. EDI was one of the
| things that boggled my mind early in my career.
|
| > What we're seeing here is that while a standard can be
| opinionated about the structure of a document and the naming of
| fields, it cannot be opinionated about the contents of a business
| transaction - the contents of a business transaction are dictated
| by the idiosyncrasies of the business itself.
|
| Maybe it's the engineer in me speaking, but this seems like a
| pretty unfortunate stance to take. Sort of similar to saying, "we
| have the concept of HTTP status codes, but we cannot possibly
| opine on the codes themselves."
|
| A bit hyperbolic for sure, but not having opinions on critical
| multi-party information has led to things like the reluctance of
| financial services to support non-ASCII values for customer
| names.
| jimnotgym wrote:
| This both brings back my past and a future project I have waiting
| for me.
|
| The discipline of dealing with other people's systems is a great
| learning point.
|
| The last person I integrated with was totally bemused by my huge
| list of questions. They always seem to assume that the way their
| system exports orders is the norm!
|
| A:"When do you export orders, as soon as they are placed or on
| some schedule?"
|
| B:"As soon as they are placed".
|
| Some time later in the conversation...
|
| B:"You needn't run it that early, we don't export orders until
| after 7.30am".
|
| You really can't trust anything in EDI until you have seen the
| orders!
| jimnotgym wrote:
| I just remembered a proprietry ERP system I worked with that
| accepted EDI orders placed in a certain directory on their
| Solaris server. Every 30 seconds it would empty that folder and
| begin to process the orders.
|
| The company must have implemented a design internally to spawn
| a new process to deal with each new order. The first I noticed
| was that the log file it spat out was not in order, it had
| messages about different parts of different orders all
| intermingled... then 1 day I put 700 orders in and it launched
| 700 processes and the server ran out of swap space and
| crashed.... Their support team felt I was unreasonable! I had
| to rate limit afterwards with my system drip feeding the
| orders! It is funny to think that it is still in use I
| understand, and busier than ever!
| blipvert wrote:
| I haven't done any work on EDI for many years so it might be a
| bit out of date, but in case it's of any use to anyone I wrote an
| EDI parsing library and processor for many of the EDIFACT message
| specs:
|
| https://github.com/davidcoles/medici
| time0ut wrote:
| I spent a decade working with ANSI X12. Writing software to
| generate it, streaming parsers for mutli-gigabyte batches, tools
| to display it, consume it, validate it, etc. Really interesting
| world to deal with. Every trading partner had their own
| interpretation of certain things. In my experience (healthcare)
| things slowly improved over time. Supposedly, FHIR will will take
| over in that space, but I have my doubts.
| nradov wrote:
| FHIR can now be used instead of X12 for prior authorizations
| but in the US it can't be used for claims yet. Claims will
| continue to rely on X12 for many years.
| allynjalford wrote:
| Healthcare wasn't using HL7? Another rather old, heavy data
| exchange format.
| nradov wrote:
| HL7 is a standards development organization, not a data
| exchange format. They publish several major standards
| including V2 Messaging, CDA, and FHIR.
| time0ut wrote:
| It is used for exchange of clinical data like admission
| notifications and health records. X12 is used for
| administrative data like enrollment and claim submission. I
| have worked with both.
| arpinum wrote:
| FHIR will take over, but I wouldn't call that a good thing. the
| development of FHIR is extremely haphazard, most of it will
| turn into a generic KV store and miss a lot of the great
| systems thinking in EDI. It's as if they lowered the barrier to
| entry at the expense of robust document exchange.
|
| Hopefully Stedi can make adopting EDI easier and increase its
| popularily over the next 50 years and there isn't such a demand
| to create alternative systems.
| tamiral wrote:
| i love how most of the comments on here are talking about EDI
| back when they first started and here I am consolidating 6
| different systems into 1 EDI VAN and its considered new
| capabilities -_-
| iask wrote:
| EDI is not hard!!!! Period! Like some file formats, it's
| unnecessarily complicated. The biggest problem in this industry
| is that there's a monopoly and many companies are being FORCED to
| use these arcane and costly service providers.
|
| I build and integrate EDI solutions for small/mid and retail
| juggernauts.
| neolithic wrote:
| We're a small business looking into EDI with some of our
| suppliers, but are finding the pricing quite painful for our
| level of business when talking to some service providers. Which
| service providers do you see as arcane and costly?
| eatonphil wrote:
| Took me a while searching around in that article to figure out
| what EDI is:
|
| > Since each business has multiple trading partners, and each of
| its trading partner operates on different business systems,
| "point to point" integration of these transactions (that is,
| mapping Walmart's internal database format directly to the
| QuickBooks JSON API) would require the recipient to have detailed
| knowledge of many different transaction formats - for example, a
| company selling to three customers running on NetSuite, SAP, and
| QuickBooks, respectively, would need to understand NetSuite XML,
| SAP IDoc, and QuickBooks JSON. Maintaining familiarity with so
| many systems isn't practical; to avoid this explosion of
| complexity, businesses instead use commonly-accepted intermediary
| formats, which are broadly known as Electronic Data Interchange -
| EDI.
| azalemeth wrote:
| There was I, having just read the headline on the main page and
| not yet the article thinking it stood for "equality and
| diversity initiative"...
| philk10 wrote:
| that's DEI :)
| bregma wrote:
| I think we're accepting of all the variants.
| wincy wrote:
| Even DIE, Diversity, Inclusion, and Equity?
| bee_rider wrote:
| It is also one letter off from either EDA or EMI. Which was
| enough for me to in some superposition of "Electronic design
| automation can be complicated" and "Electromagnetic
| interference is confusing."
| allynjalford wrote:
| It's format.. I once had to convert several EDI specs to XML.. It
| was horrid..
___________________________________________________________________
(page generated 2022-01-06 23:01 UTC)