https://github.com/apache/arrow-julia

Official Julia implementation of Apache Arrow
https://github.com/apache/arrow-julia

apache-arrow julia

Last synced: 16 days ago
JSON representation

Official Julia implementation of Apache Arrow

• Host: GitHub
• URL: https://github.com/apache/arrow-julia
• Owner: apache
• License: apache-2.0
• Created: 2020-08-28T12:54:32.000Z (over 4 years ago)
• Default Branch: main
• Last Pushed: 2025-03-20T15:44:56.000Z (about 1 month ago)
• Last Synced: 2025-03-30T20:01:56.677Z (about 1 month ago)
• Topics: apache-arrow, julia
• Language: Julia
• Homepage: https://arrow.apache.org/julia/
• Size: 2.03 MB
• Stars: 289
• Watchers: 30
• Forks: 64
• Open Issues: 109
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


# Arrow

[![docs](https://img.shields.io/badge/docs-latest-blue&logo=julia)](https://arrow.apache.org/julia/)

[![CI](https://github.com/apache/arrow-julia/workflows/CI/badge.svg)](https://github.com/apache/arrow-julia/actions?query=workflow%3ACI)

[![codecov](https://app.codecov.io/gh/apache/arrow-julia/branch/main/graph/badge.svg)](https://app.codecov.io/gh/apache/arrow-julia)

[![deps](https://juliahub.com/docs/Arrow/deps.svg)](https://juliahub.com/ui/Packages/Arrow/QnF3w?t=2)

[![version](https://juliahub.com/docs/Arrow/version.svg)](https://juliahub.com/ui/Packages/Arrow/QnF3w)

[![pkgeval](https://juliahub.com/docs/Arrow/pkgeval.svg)](https://juliahub.com/ui/Packages/Arrow/QnF3w)

This is a pure Julia implementation of the [Apache Arrow](https://arrow.apache.org) data standard.  This package provides Julia `AbstractVector` objects for

referencing data that conforms to the Arrow standard.  This allows users to seamlessly interface Arrow formatted data with a great deal of existing Julia code.

Please see this [document](https://arrow.apache.org/docs/format/Columnar.html#physical-memory-layout) for a description of the Arrow memory layout.

## Installation

The package can be installed by typing in the following in a Julia REPL:

```julia

julia> using Pkg; Pkg.add("Arrow")

```

## Local Development

When developing on Arrow.jl it is recommended that you run the following to ensure that any

changes to ArrowTypes.jl are immediately available to Arrow.jl without requiring a release:

```sh

julia --project -e 'using Pkg; Pkg.develop(path="src/ArrowTypes")'

```

## Format Support

This implementation supports the 1.0 version of the specification, including support for:

  * All primitive data types

  * All nested data types

  * Dictionary encodings and messages

  * Extension types

  * Streaming, file, record batch, and replacement and isdelta dictionary messages

It currently doesn't include support for:

  * Tensors or sparse tensors

  * Flight RPC

  * C data interface

Third-party data formats:

  * CSV, parquet and avro support via the existing [CSV.jl](https://github.com/JuliaData/CSV.jl), [Parquet.jl](https://github.com/JuliaIO/Parquet.jl) and [Avro.jl](https://github.com/JuliaData/Avro.jl) packages

  * Other Tables.jl-compatible packages automatically supported ([DataFrames.jl](https://github.com/JuliaData/DataFrames.jl), [JSONTables.jl](https://github.com/JuliaData/JSONTables.jl), [JuliaDB.jl](https://github.com/JuliaData/JuliaDB.jl), [SQLite.jl](https://github.com/JuliaDatabases/SQLite.jl), [MySQL.jl](https://github.com/JuliaDatabases/MySQL.jl), [JDBC.jl](https://github.com/JuliaDatabases/JDBC.jl), [ODBC.jl](https://github.com/JuliaDatabases/ODBC.jl), [XLSX.jl](https://github.com/felipenoris/XLSX.jl), etc.)

  * No current Julia packages support ORC

See the [full documentation](https://arrow.apache.org/julia/) for details on reading and writing arrow data.