https://github.com/ysden123/scala-csvreader
Scala CSV file (stream) reader
https://github.com/ysden123/scala-csvreader
csv file parser reader scala stream
Last synced: about 1 year ago
JSON representation
Scala CSV file (stream) reader
- Host: GitHub
- URL: https://github.com/ysden123/scala-csvreader
- Owner: ysden123
- License: mit
- Created: 2018-10-25T16:37:53.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2021-04-04T15:48:07.000Z (almost 5 years ago)
- Last Synced: 2025-01-27T06:11:52.938Z (about 1 year ago)
- Topics: csv, file, parser, reader, scala, stream
- Language: Scala
- Size: 31.3 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.adoc
- License: LICENSE
Awesome Lists containing this project
README
= scala-csvreader
Scala CSV file (stream) reader
== Build status
Master: image:https://travis-ci.org/ysden123/scala-csvreader.svg?branch=master[Build Status: master,link=https://travis-ci.org/ysden123/scala-csvreader]
Develop: image:https://travis-ci.org/ysden123/scala-csvreader.svg?branch=develop[Build Status: develop,link=https://travis-ci.org/ysden123/scala-csvreader]
== CsvReader
CsvReader is simple CSV reader with pure Scala.
The reader supports:
* arbitrary case class for record data
* file with or without header line
* setting either continue or stop after error during parsing a record
* setting different delimiter character
* error handling
* record handling
== Usage
=== Record Data Class
Record data class is a case class. For example:
[source,scala]
----
case class TestData1(i: Int, s: String, d: Double)
----
Some parameters may be Option, For example
[source,scala]
----
case class TestData2(i: Int, s: Option[String], d: Double) {
def this(params: Seq[String]) {
this(params(0).toInt,
if (params(1).isEmpty)
None
else
Some(params(1)),
params(2).toDouble
)
}
}
----
Note in this case the auxiliary constructor should be defined!
=== How to work with CsvReader
[source,scala]
----
def recordHandler(record:TestData]:Unit={
...
}
def errorHandler(error:String]:Unit={
...
}
CsvReader.reader[TestData](Source.fromFile("data.csv"), recordHandler, errorHandler)
.parse()
----
or
[source,scala]
----
// Empty record handler
def handler(testData: TestData1): Unit = {}
val list = CsvReader.reader[TestData1](Source.fromResource("test-data1.csv"), handler, errorHandler).toList
----
=== Use of modificators:
* withDelimiter
* withContinueOnError
* withHeaderLine
* withCustomTransformer
=== How to work with custom transformer
Custom transformer allows to customize creating an object.
recordHandler method should be called for each created object.
errorHandler method should be called for each error during creating object.
Example:
[source,scala]
----
def transformer(line: Seq[String], recordHandler: TestData1 => Unit, errorHandler: String => Unit): TestData1 = {
try {
val data = TestData1(333, "test", 77.12)
recordHandler(data)
data
} catch {
case e: Exception =>
errorHandler(e.getMessage)
throw e
}
}
----
See CsvReaderTest test("customTransformer") and test("customTransformer with error")
== CsvStreamReader
CsvStreamReader is CSV reader with using the Akka stream.
The reader supports:
* arbitrary case class for record data
* file with or without header line
* setting either continue or stop after error during parsing a record
* setting different delimiter character
* error handling
* record handling
== Usage
=== Record Data Class
Record data class is a case class. For example:
[source,scala]
----
case class TestData1(i: Int, s: String, d: Double)
----
Some parameters may be Option, For example
[source,scala]
----
case class TestData2(i: Int, s: Option[String], d: Double) {
def this(params: Seq[String]) {
this(params(0).toInt,
if (params(1).isEmpty)
None
else
Some(params(1)),
params(2).toDouble
)
}
}
----
Note in this case the auxiliary constructor should be defined!
=== How to work with CsvStreamReader
[source,scala]
----
def recordHandler(record:TestData]:Unit={
...
}
def errorHandler(error:String]:Unit={
...
}
implicit val actorSystem: ActorSystem = ActorSystem("CsvStreamReaderTest")
implicit val materializer: ActorMaterializer = ActorMaterializer()
val reader = CsvStreamReader.reader(Source.fromResource("test-data1.csv"), recordHandler, errorHandler)
Await.ready(reader.run(), Duration.Inf)
source.close()
actorSystem.terminate()
----
=== How to get list of items
[source,scala]
----
def errorHandler(error:String]:Unit={
...
}
implicit val actorSystem: ActorSystem = ActorSystem("CsvStreamReaderTest")
implicit val materializer: ActorMaterializer = ActorMaterializer()
val list = ListBuffer.empty[TestData1]
val reader = CsvStreamReader.reader(Source.fromResource("test-data1.csv"),
item => list += item,
errorHandler)
Await.ready(reader.run(), Duration.Inf)
println(s"list.length=${list.length}")
source.close()
actorSystem.terminate()
----
=== Use of modificators:
* withDelimiter
* withContinueOnError
* withHeaderLine
* withCustomTransformer
=== How to work with custom transformer
See "How to work with custom transformer" for CsvReader (above)