Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/clarkema/x12pp
A pretty-printer for X12 EDI files
https://github.com/clarkema/x12pp
edi rust x12
Last synced: 28 days ago
JSON representation
A pretty-printer for X12 EDI files
- Host: GitHub
- URL: https://github.com/clarkema/x12pp
- Owner: clarkema
- License: gpl-3.0
- Created: 2019-04-16T22:47:11.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2021-06-21T16:58:13.000Z (over 3 years ago)
- Last Synced: 2024-10-31T11:46:46.678Z (about 1 month ago)
- Topics: edi, rust, x12
- Language: Rust
- Homepage:
- Size: 28.3 KB
- Stars: 11
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- Electronic-Interchange-Github-Resources - clarkema/x12pp - Fast cross-platform X12 pretty-printer in Rust (CLI utilities / Golang)
README
# x12pp
`x12pp` is a CLI pretty-printer for X12 EDI files.
X12 is an arcane format consisting of a fixed-length header followed by a series
of segments, each separated by a segment terminator character.
These segments are generally not separated by newlines, so extracting a range of
lines from a file or taking a peek at the start using the usual Unix toolbox
becomes unnecessarily painful.
Of course, you could split the lines using `sed -e 's/~/~\n/g'` and get on with
your day, but:
1. although the `~` is the traditional and most widely-used segment terminator
it's not required -- each X12 file specifies its own terminators as part of
the header.
2. using `sed` or `perl` would mean I wouldn't have a chance to explore fast
stream processing in Rust.
So here we are.
## Installation
### Homebrew
```
$ brew tap clarkema/nomad
$ brew install x12pp
```
### With cargo
```
$ cargo install x12pp
```
### From source
x12pp is written in Rust, so you'll need an up-to-date Rust installation in
order to build it from source. The result is a statically-compiled binary at
`target/release/x12pp`, which you can copy wherever you need.
```
$ git clone https://github.com/clarkema/x12pp
$ cd x12pp
$ cargo build --release
$ ./target/release/x12pp --version
```
## Usage
```
$ x12pp < FILE > NEWFILE
$ x12pp FILE -o NEWFILE
# Strip newlines out instead with:
$ x12pp --uglify FILE
```
See manpage or `--help` for more.
## Benchmarks
All tests were performed on an Intel Core i9-7940X, using a 1.3G X12 test file
located on a RAM disk. In each case, shell redirection was used to
pipe the file through the test command and into `/dev/null` in order to get
as close as possible to measuring pure processing time. For example:
`$ time sed -e 's/~/~\n/g' < test-file > /dev/null`
| Tool | Command | Terminator detection | Pre-wrapped? | SIGPIPE? | Time |
|-------------|-------------------------------|----------------------|--------------|----------|-------|
| x12pp | `x12pp` | ✓ | ✓ | ✓ | 1.3s |
| GNU sed 4.7 | `sed -e 's/~/~\n/g'` | ✗ | ✗ | ✗ | 7.6s |
| perl 5.28.2 | `perl -pe 's/~[\r\n]*/~\n/g'` | ✗ | ✓ but slower | ✗ | 8.5s |
| edicat | `edicat` | ✓ | ✓ | ✓ | 7m41s |
### Notes
1. 'SIGPIPE' refers to whether a command can return a partial result without
having to process the entire input. One of the motivations for `x12pp` was
to be able to run `x12pp < FILE | head -n 100` without having to plough
through a multi-gigabyte file.
2. Of course you could write a Perl script that _did_ correctly read the
segment terminator before processing the rest of the file.
3. Perl produces the correct output with input data that is already wrapped,
but _much_ slower; around 24 seconds compared to 8.5.
4. See https://github.com/notpeter/edicat for edicat