Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/guiabolso/fixed-length-file-handler
Handlers for Fixed Length files in a beautiful Kotlin DSL
https://github.com/guiabolso/fixed-length-file-handler
dsl file fixed-length fixed-length-format fixed-length-records fixed-width kotlin parser parsing parsing-library
Last synced: 3 months ago
JSON representation
Handlers for Fixed Length files in a beautiful Kotlin DSL
- Host: GitHub
- URL: https://github.com/guiabolso/fixed-length-file-handler
- Owner: GuiaBolso
- License: apache-2.0
- Created: 2019-11-06T15:56:23.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2023-02-02T20:04:19.000Z (almost 2 years ago)
- Last Synced: 2024-09-26T00:02:43.836Z (3 months ago)
- Topics: dsl, file, fixed-length, fixed-length-format, fixed-length-records, fixed-width, kotlin, parser, parsing, parsing-library
- Language: Kotlin
- Homepage:
- Size: 95.7 KB
- Stars: 13
- Watchers: 4
- Forks: 4
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Fixed Length File Handler
[](https://github.com/GuiaBolso/fixed-length-file-handler/actions/workflows/build.yml)
[](https://github.com/GuiaBolso/fixed-length-file-handler/blob/master/LICENSE)
## Introduction
When processing data from some systems (mainly legacy ones), it's usual to have Fixed Length Files, which are files that contain lines which content is split using a specific length for each field of a record.
This kind of files are sometimes tricky to handle as many times there is a spaghetti of string manipulations and padding, and character counting and... Well, many things to take care of.
This library comes to the rescue of programmers dealing with fixed length files. It enables you to simply define how your records are structured and it will handle these records for you in a nice Kotlin DSL for further processing.
## Using with Gradle
Import it into your dependencies:
```kotlin
dependencies {
implementation("br.com.guiabolso:FixedLengthFileHandler:{version}")
}
```
## Basic Usage
The basic usage assumes that you're reading a file with a single type of record.
Given a Fixed-Length File:
#### Definition
| Field | Type | Initial Position | Final Position Exclusive |
| ----- | ---- | ---------------- | ------------------------ |
| UserName | String | 0 | 30 |
| User Document | Int | 30 | 39 |
| User Registry Date | LocalDate | 39 | 49 |
#### File
```
FirstUsername 1234567892019-02-09
SecondAndLongerUsername 9876543212018-03-10
ThirdUsernameWithShorterDoc 0000001232017-04-11
```
We can parse it with the `fixedLengthFileParser` DSL:
```kotlin
data class MyUserRecord(val username: String, val userDoc: Int, val registryDate: LocalDate)
val fileInputStream: InputStream = getFileInputStream()
fixedLengthFileParser(fileInputStream) {
MyUserRecord(
field(0, 30, Padding.PaddingRight(' ')),
field(30, 39, Padding.PaddingLeft('0')),
field(39, 49)
)
}
```
The library is prepared to handle `Left Padding` and `Right Padding`. It's also prepared to handle many of Kotlin/Java types.
## Closing the file stream
**Attention** - You're responsible for closing the stream after processing the sequence, so be sure to close it!
## Default parsing
This library is prepared to handle some of the most usual Kotlin/Java types. More types may be added if they're required. The default types are:
- String
- Int
- UInt
- Double
- Long
- ULong
- Char
- Boolean (Case insensitive)
- LocalDate (Using default DateTimeFormatter)
- LocalTime (Using default DateTimeFormatter)
- LocalDateTime (Using default DateTimeFormatter)
- BigDecimal
- Enum types
## Decimal Parsing
There might be scenarios where the default parsing of a decimal (Double / BigDecimal) isn't enough, and you need to declare a special scale, for example `4299` might represent `42.99` (scale of 2, initially undeclared in the String).
For this particular case, you can use `decimalField` instead of `field`:
```kotlin
decimalField(from = 0, toExclusive = 0, scale = 3, padding = NoPadding)
```
## Custom parsing
There might be times where the default types are not enough, and you need a custom parser for a given record.
For example: You know that a specific number contains a currency, and the last two digits are used for the cents.
This library is prepared to handle cases where you need custom parsing for a String, by modifying the `field` invocation:
```kotlin
// Parsing the field 0000520099 to 5200.99
field(15, 25, Padding.PaddingLeft('0')) { str: String -> StringBuilder(str).insert(str.length - 2, ".").toString().toBigDecimal() }
```
## Advanced Usage
For an unknown reason, many Fixed-Length file providers use the same file for more than one record, denoting a specific bit for record identification, so there's a possibility that this happens:
```
1 FirstUserName 123.12
1 SecondUserName 002.50
2 123456789 2019-02-09UserDocs
2 000812347 2018-03-08AnotherUserDocs
```
In this cases, the software must look at the first `char` to determine the record type. This situation is usually what leads to a spaghetti string manipulation. We can solve it by using this library's "advanced" options:
```kotlin
data class FirstRecordType(username: String, userMoney: BigDecimal)
data class SecondRecordType(userCode: Int, registerDate: LocalDate, docs: String)
fixedLengthFileParser(fileInputStream) {
withRecord({ line -> line[0] == '1' }) {
FirstRecordType(
field(2, 22, Padding.PaddingRight(' ')),
field(22, 28, Padding.PaddingLeft('0'))
)
}
withRecord( { line -> line[0] == '2' }) {
SecondRecordType(
field(2, 15, Padding.PaddingRight(' ')),
field(15, 25),
field(25, 40, Padding.PaddingRight(' '))
)
}
}
```
## Features
- The file is streamed into a sequence of values, and is never loaded in its entirety to the memory. You should expect this to have a good performance over a very big file.
- The Kotlin DSL makes it easier to define the file parsing in a single point, and the sequence processing can be done anywhere
## Changelog
Check the complete changelog [here](./CHANGELOG.md)