Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/supermodel/supermodel-cli
Boring data modeling
avro avro-schema graphql json json-ld json-schema openapi swagger
Last synced: 02 Jul 2024
![](https://github.com/supermodel.png)
https://github.com/HariSekhon/DevOps-Python-tools
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
avro aws cloudformation devops docker dockerhub elasticsearch gcf gcp hadoop hbase hdfs json linux parquet pyspark python solr spark travis-ci
Last synced: 27 Jun 2024
![](https://github.com/HariSekhon.png)
https://github.com/FasterXML/jackson-dataformats-binary
Uber-project for standard Jackson binary format backends: avro, cbor, ion, protobuf, smile
avro cbor hacktoberfest jackson-backends protobuf smile
Last synced: 20 Jun 2024
![](https://github.com/FasterXML.png)
https://github.com/Chabane/bigdata-playground
A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
angular apache-flink apache-spark avro big-data docker graphql hadoop hbase kafka kops machine-learning mongodb nodejs parquet python scala spark-sql spark-streaming twitter-api
Last synced: 17 Jun 2024
![](https://github.com/Chabane.png)
https://github.com/mongodb/mongo-kafka
MongoDB Kafka Connector
avro bson cdc confluent-hub kafka kafka-connect mongodb sink sink-connector source source-connector
Last synced: 07 Jun 2024
![](https://github.com/mongodb.png)
https://github.com/hpgrahsl/kafka-connect-mongodb
**Unofficial / Community** Kafka Connect MongoDB Sink Connector -> integrated 2019 into the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector
avro azure-cosmosdb bson cdc change-data-capture confluent-hub connector cosmosdb debezium json kafka kafka-connect mongodb sink sink-connector
Last synced: 07 Jun 2024
![](https://github.com/hpgrahsl.png)
https://github.com/salsify/avromatic
Generate Ruby models from Avro schemas
Last synced: 02 Jun 2024
![](https://github.com/salsify.png)
https://github.com/salsify/avro-schema-registry
Implementation of the Confluent Schema Registry API as a Rails application
avro hacktoberfest schema-registry
Last synced: 02 Jun 2024
![](https://github.com/salsify.png)
![](https://github.com/salsify.png)
https://github.com/dasch/avro_turf
A library that makes it easier to use the Avro serialization format from Ruby.
avro avro-data ruby schema schema-registry
Last synced: 02 Jun 2024
![](https://github.com/dasch.png)
https://github.com/confluentinc/examples
Apache Kafka and Confluent Platform examples and demos
avro cdc cloud confluent connect connector debezium demo docker examples jdbc kafka ksql kubernetes microservices monitoring quickstart replicator schema-registry sql
Last synced: 02 Jun 2024
![](https://github.com/confluentinc.png)
https://github.com/supermodel/supermodel
Supermodel browser library and CLI tool for working with JSON Schema data models
avro graphql json json-ld json-schema openapi swagger
Last synced: 02 Jun 2024
![](https://github.com/supermodel.png)
https://github.com/niqdev/kafka-scala-examples
Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala
avro interactive-queries kafka kafka-connect kafka-scala kafka-streams ksql schema-registry
Last synced: 01 Jun 2024
![](https://github.com/niqdev.png)
https://github.com/higherkindness/mu-haskell
Mu (μ) is a purely functional framework for building micro services.
avro defines-schemas graphql grpc hacktoberfest haskell monads mu mu-graphql mu-haskell protocol-buffers rpc type-level type-level-programming
Last synced: 31 May 2024
![](https://github.com/higherkindness.png)
https://github.com/capitalone/DataProfiler
What's in your data? Extract schema, statistics and entities from datasets
avro csv data-analysis data-labels data-science dataprofiling dataset gdpr graph-data machine-learning network-data nlp npi pandas pii privacy python security sensitive-data tabular-data
Last synced: 22 May 2024
![](https://github.com/capitalone.png)
https://github.com/bolcom/avro-schema-viewer
Visualizer for Avro Schemas (.avsc) - Try it yourself at:
angular angular-material avro avro-schema
Last synced: 16 May 2024
![](https://github.com/bolcom.png)
https://github.com/mtth/avsc
Avro for JavaScript :zap:
avro big-data binary-format encoding javascript schema-evolution serialization typescript
Last synced: 16 May 2024
![](https://github.com/mtth.png)
https://github.com/confluentinc/schema-registry
Confluent Schema Registry for Kafka
avro avro-schema confluent json json-schema kafka protobuf protobuf-schema rest-api schema schema-registry schemas
Last synced: 07 May 2024
![](https://github.com/confluentinc.png)
https://github.com/streamthoughts/kafka-connect-file-pulse
🔗 A multipurpose Kafka Connect connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka
amazon-s3 avro azure-storage csv etl file-streaming google-cloud grok-filters kafka kafka-connect kafka-connector kafka-producer xml
Last synced: 07 May 2024
![](https://github.com/streamthoughts.png)
https://github.com/lensesio/schema-registry-ui
Web tool for Avro Schema Registry |
Last synced: 07 May 2024
![](https://github.com/lensesio.png)
https://github.com/Cinchoo/ChoETL
ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
avro cinchoo-etl csharp csv dotnet etl etl-framework flat json keyvalue parquet parquet-files parser reader writer xml yaml
Last synced: 05 May 2024
![](https://github.com/Cinchoo.png)
https://github.com/thekvs/cpp-serializers
Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++
apache-avro avro boost c-plus-plus capn-proto cereal cpp flatbuffers msgpack performance-testing protobuf serialization thrift yas
Last synced: 01 May 2024
![](https://github.com/thekvs.png)
https://github.com/indix/schemer
Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
avro graphql-api json parquet schema-inference schema-registry spark tsv
Last synced: 30 Apr 2024
![](https://github.com/indix.png)
https://github.com/sksamuel/avro4s
Avro schema generation and serialization / deserialization for Scala
avro avro-schema coproduct scala scala-macros schema-generation serialization
Last synced: 30 Apr 2024
![](https://github.com/sksamuel.png)
https://github.com/zarusz/SlimMessageBus
Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.
apache-kafka avro azure azure-event-hubs azure-service-bus bus c-sharp ddd dotnet kafka message-bus messaging mqtt pub-sub rabbitmq redis request-response
Last synced: 25 Apr 2024
![](https://github.com/zarusz.png)
https://github.com/chezou/sparkavro
Load Avro data into Spark with sparklyr
Last synced: 25 Apr 2024
![](https://github.com/chezou.png)
https://github.com/malcolmgreaves/avro-codegen
Scala code generator for Avro schemas.
avro avro-schema codegen data scala serialization
Last synced: 22 Apr 2024
![](https://github.com/malcolmgreaves.png)
https://github.com/dflemstr/rq
Record Query - A tool for doing record analysis and transformation
avro command-line-tool javascript json lodash messagepack protobuf rust toml yaml
Last synced: 20 Apr 2024
![](https://github.com/dflemstr.png)
https://github.com/pennsignals/aptos
:sunny: A tool for validating data using JSON Schema and converting JSON Schema documents into different data-interchange formats
avro avro-schema cli command-line-tool data-interchange json-schema python3 schema-conversion validation
Last synced: 15 Apr 2024
![](https://github.com/pennsignals.png)
https://github.com/Altinity/clickhouse-sink-connector
Replicate data from MySQL, Postgres and MongoDB to ClickHouse
avro cdc clickhouse debezium kafka kafka-connect mongo mysql postgresql replication sqlserver
Last synced: 13 Apr 2024
![](https://github.com/Altinity.png)
https://github.com/ticean/awesome-kafka-in-clojure
Awesome information for using Kafka and Confluent Platform in Clojure.
avro avro-schema awesome awesome-list awesome-lists clojure confluent confluent-platform kafka kafka-connect kafka-streams ksql microservices microservices-architecture
Last synced: 09 Apr 2024
![](https://github.com/ticean.png)
https://github.com/Adelost/javascript-serialization-benchmark
Comparison and benchmark of JavaScript serialization libraries (Protocol Buffer, Avro, BSON, etc.)
avro benchmark binary-serialization comparison javascript performance protobuf protocol-buffers serialization
Last synced: 08 Apr 2024
![](https://github.com/Adelost.png)
https://github.com/piotr-yuxuan/dove
Infer specs from SpecificRecord Java classes of any Avro named type (record, fixed, or enum)
avro avro-format avro-schema clojure clojure-spec clojure-specs spec specific-schemas
Last synced: 08 Apr 2024
![](https://github.com/piotr-yuxuan.png)
https://github.com/ovotech/kafka-avro-confluent
Kafka De/Serializer using avro and Confluent's Schema Registry
avro company-kaluza confluent kafka
Last synced: 08 Apr 2024
![](https://github.com/ovotech.png)
https://github.com/petersonjr/MetadataCrawler
A simple tool to extract metadata from relational databases
avro crawler database-schemas java jdbc metadata rdms relational-databases
Last synced: 01 Apr 2024
![](https://github.com/petersonjr.png)
https://github.com/spotify/ratatool
A tool for data sampling, data generation, and data diffing
avro bigquery parquet protobuf scala scalacheck
Last synced: 31 Mar 2024
![](https://github.com/spotify.png)
https://github.com/divolte/divolte-collector
Divolte Collector
analytics analytics-tracking avro clickstream divolte-collector gcs hdfs java kafka pubsub
Last synced: 31 Mar 2024
![](https://github.com/divolte.png)
https://github.com/asyncapi/avro-schema-parser
An AsyncAPI schema parser for Avro 1.x schemas.
avro avro-schema avro-schema-registry get-global-node-release-workflows get-global-releaserc nodejs parser
Last synced: 30 Mar 2024
![](https://github.com/asyncapi.png)
https://github.com/OneCricketeer/schema-registry-transfer-smt
Kafka Connect Transform to copy Avro schemas between Schema Registries
avro confluent kafka kafka-connect kafka-connect-smt schema-registry
Last synced: 26 Mar 2024
![](https://github.com/OneCricketeer.png)
https://github.com/nodefluent/schema-registry
:orange_book: json & avro http schema registry backed by Kafka
avro json kafka node registry rest schema schema-registry
Last synced: 26 Mar 2024
![](https://github.com/nodefluent.png)
https://github.com/deviceinsight/kafkactl
Command Line Tool for managing Apache Kafka
apache-kafka avro cli fish golang kafka zsh
Last synced: 22 Mar 2024
![](https://github.com/deviceinsight.png)
https://github.com/sevenmind/kaufmann_ex
Kafka backed service library.
avro concurrency elixir kafka microservice schema
Last synced: 22 Mar 2024
![](https://github.com/sevenmind.png)
https://github.com/pmacct/pmacct
pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].
avro bgp bmp geoip2 ipfix json kafka libpcap mysql ndpi netflow nflog pmacct postgresql rabbitmq rpki sflow sql sqlite3
Last synced: 22 Mar 2024
![](https://github.com/pmacct.png)
https://github.com/only-cliches/NoProto
Flexible, Fast & Compact Serialization with RPC
apache-avro avro bson data-buffers databases deserialization flatbuffers flexbuffers json messagepack protocol-buffers rpc schemas serialization zero-copy
Last synced: 17 Mar 2024
![](https://github.com/only-cliches.png)
https://github.com/RandomFractals/vscode-data-preview
Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files
array arrow avro config csv data excel extension json parquet perspective viewer vscode yaml
Last synced: 17 Mar 2024
![](https://github.com/RandomFractals.png)
https://github.com/flix-tech/schema-registry-php-client
A PHP 8+ API client for the Confluent Schema Registry REST API based on Guzzle 7 - http://docs.confluent.io/current/schema-registry/docs/index.html
api-client avro avro-schema confluent event-driven-architecture guzzlehttp php schema-evolution schema-registry
Last synced: 17 Mar 2024
![](https://github.com/flix-tech.png)
https://github.com/anair-it/schema-evolution
Demo - Schema evolution using Apache Avro and Google Protocol buffers
avro java protocol-buffers schema-evolution
Last synced: 17 Mar 2024
![](https://github.com/anair-it.png)
https://github.com/andrewinci/Insulator
A client UI to inspect Kafka topics, consume, produce and much more
apache-kafka avro consumer javafx kafka kafka-consumer kotlin schema topic tornadofx ui
Last synced: 16 Mar 2024
![](https://github.com/andrewinci.png)