https://github.com/gtiwari333/java-read-big-xml-to-csv
java read big xml file and convert to csv
https://github.com/gtiwari333/java-read-big-xml-to-csv
graalvm graalvm-native-image java xml xml-parser
Last synced: about 1 year ago
JSON representation
java read big xml file and convert to csv
- Host: GitHub
- URL: https://github.com/gtiwari333/java-read-big-xml-to-csv
- Owner: gtiwari333
- Created: 2020-07-09T20:26:07.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2024-05-24T15:14:26.000Z (about 2 years ago)
- Last Synced: 2025-03-29T21:22:30.456Z (over 1 year ago)
- Topics: graalvm, graalvm-native-image, java, xml, xml-parser
- Language: Java
- Homepage: https://ganeshtiwaridotcomdotnp.blogspot.com/2020/07/java-read-huge-xml-file-and-convert-to.html
- Size: 132 KB
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## Sample project for
- https://ganeshtiwaridotcomdotnp.blogspot.com/2011/08/xml-parsing-using-saxparser-with.html
- https://ganeshtiwaridotcomdotnp.blogspot.com/2020/07/java-read-huge-xml-file-and-convert-to.html
- https://ganeshtiwaridotcomdotnp.blogspot.com/2020/07/graalvm-setup-and-generate-native-image.html
## This uses Java SAX parser to parse a big xml file efficiently and convert to csv file
First, import into Intellij as Maven project. Make sure you have maven
## Requirement: make sure you have java installed (run `java -version`)
## Generate Jar file - it will be generated in target folder
`./mvnw clean package`
## Run
`java -jar target/xmltocsv-FINAL.jar PATH/input.xml PATH/output.csv`
## Time/Memory Consumption
RAM Used: less than 190MB
Time Taken:
File: 118MB big2.xml @ 16GB Ram, Core i5, 6MB L3 cache, SSD
- JDK8 - 8-9 sec
- JDK 11 - 6-7 sec
- JDK 14 - 5 sec
# Included xml files - unzip before use
- small.xml - 1.44KB file
- big1.zip - 2MB file
- big2.zip - 118MB file
- big3.7z - 6.58GB file
# Native Image with GraalVM
Refer to this https://ganeshtiwaridotcomdotnp.blogspot.com/2020/07/graalvm-setup-and-generate-native-image.html blog for how to setup GraalVM and native-image utility
Run the following to generate native image. Make sure graalvm sdk is chosen as java sdk.
`sdk use java 21.0.3-graal`
`./mvnw clean package install`
- Run native executable
```
$ ./target/xmltocsv PATH/big2.xml PATH/big2.csv
```