https://github.com/wayfair-incubator/avro-to-bigquery

https://github.com/wayfair-incubator/avro-to-bigquery

hacktoberfest

Last synced: about 6 hours ago
JSON representation

• Host: GitHub
• URL: https://github.com/wayfair-incubator/avro-to-bigquery
• Owner: wayfair-incubator
• License: mit
• Created: 2021-10-18T13:18:31.000Z (about 3 years ago)
• Default Branch: main
• Last Pushed: 2024-10-31T05:26:36.000Z (7 days ago)
• Last Synced: 2024-10-31T06:20:54.983Z (7 days ago)
• Topics: hacktoberfest
• Language: Python
• Homepage:
• Size: 40 KB
• Stars: 11
• Watchers: 3
• Forks: 11
• Open Issues: 13
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Security: SECURITY.md

Awesome Lists containing this project

README

        
# Avro to Bigquery

[![CI pipeline status](https://github.com/wayfair-incubator/avro-to-bigquery/workflows/CI/badge.svg?branch=main)](https://github.com/wayfair-incubator/avro-to-bigquery/actions/workflows/main.yml)

[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.0-4baaaa.svg)](https://github.com/wayfair-incubator/avro-to-bigquery/blob/main/CODE_OF_CONDUCT.md)

## About The Project

Avro to bigquery is a simple project to create a bigquery schema from an avro schema.

## Getting Started

### Installation`

   ```sh

   pip install avro-to-bigquery

   ```

## Usage

With avro_schema.avsc:

```json

{

    "type": "record",

    "name": "User",

    "namespace": "example.avro",

    "fields": [

        {"name": "favorite_number", "type": "int", "doc": "Favorite number"}

    ]

}

```

```python

>>> import json

>>> from pathlib import Path

>>> from avro_to_bigquery import convert_schema

>>> schema = json.loads(Path("avro_schema.avsc").read_text())

>>> print(convert_schema(schema))

(SchemaField('favorite_number', 'INTEGER', 'NULLABLE', 'Favorite number', (), ()),)

```

## Roadmap

See the [open issues](https://github.com/wayfair-incubator/avro-to-bigquery/issues) for a list of proposed features (and known issues).

## Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are **greatly appreciated**. For detailed contributing guidelines, please see [https://github.com/wayfair-incubator/avro-to-bigquery/blob/main/CONTRIBUTING.md](CONTRIBUTING.md)

## License

Distributed under the `MIT` License. See `LICENSE` for more information.

## Contact

Project Link: [https://github.com/wayfair-incubator/avro-to-bigquery/](https://github.com/wayfair-incubator/avro-to-bigquery/)