https://github.com/intake/intake-avro

Apache Avro plugin for Intake
https://github.com/intake/intake-avro

Last synced: 2 months ago
JSON representation

Apache Avro plugin for Intake

• Host: GitHub
• URL: https://github.com/intake/intake-avro
• Owner: intake
• License: bsd-2-clause
• Created: 2018-02-07T16:51:50.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2020-02-06T14:58:06.000Z (over 6 years ago)
• Last Synced: 2026-01-24T17:13:29.105Z (4 months ago)
• Language: Python
• Size: 55.7 KB
• Stars: 2
• Watchers: 3
• Forks: 3
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Intake-avro

[![Build Status](https://travis-ci.org/ContinuumIO/intake-avro.svg?branch=master)](https://travis-ci.org/ContinuumIO/intake-avro)

[![Documentation Status](https://readthedocs.org/projects/intake-avro/badge/?version=latest)](http://intake-avro.readthedocs.io/en/latest/?badge=latest)

The [Apache Avro](https://avro.apache.org/) format is a popular method for information

exchange, particularly for message and reord distribution in high-throughput systems.

In [Intake](https://github.com/ContinuumIO/intake/), there are two plugins provided for reading avro data:

- `avro_table` is appropriate for data which have a schema simple enough to be

  represented as columns with simple types, i.e., *flat* schema. The output of

  this plugin are dataframes, and the reading is optimized for speed.

- `avro_sequence` is more generic and can handle all possible avro schema, but

  it produces generic python sequences of dictionaries, and is consequently

  much slower than the table reader.

### Installation

The conda install instructions are:

```

conda install -c conda-forge fastavro uavro

conda install -c intake intake_avro

```

### Examples

See the notebook in the examples/ directory.