{"id":44478826,"url":"https://github.com/keltia/fetiche-rs","last_synced_at":"2026-02-12T23:13:32.016Z","repository":{"id":176496195,"uuid":"552146839","full_name":"keltia/fetiche-rs","owner":"keltia","description":"Rust crates for managing Surveillance data — ingress/egress formats for ECTL","archived":false,"fork":false,"pushed_at":"2026-01-30T15:45:57.000Z","size":16240,"stargazers_count":1,"open_issues_count":14,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-31T08:44:13.224Z","etag":null,"topics":["aeronautical","cli-app","library","rust","rust-library"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/keltia.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-10-15T23:34:23.000Z","updated_at":"2025-10-14T10:41:52.000Z","dependencies_parsed_at":"2024-05-05T23:29:05.909Z","dependency_job_id":"579462c4-16bf-424b-9462-af39e0ccd23e","html_url":"https://github.com/keltia/fetiche-rs","commit_stats":null,"previous_names":["keltia/fetiche-rs"],"tags_count":76,"template":false,"template_full_name":null,"purl":"pkg:github/keltia/fetiche-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keltia%2Ffetiche-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keltia%2Ffetiche-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keltia%2Ffetiche-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keltia%2Ffetiche-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/keltia","download_url":"https://codeload.github.com/keltia/fetiche-rs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keltia%2Ffetiche-rs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29385727,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-12T22:07:52.078Z","status":"ssl_error","status_checked_at":"2026-02-12T22:07:49.026Z","response_time":55,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aeronautical","cli-app","library","rust","rust-library"],"created_at":"2026-02-12T23:13:31.072Z","updated_at":"2026-02-12T23:13:32.011Z","avatar_url":"https://github.com/keltia.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- omit in TOC --\u003e\n\n# fetiche-rs\n\n[![Fetiche Logo](docs/fetiche-rs-icon.jpg)]\n\n\u003e **FETICHE: Framework to import/fetch/transform various aeronautical data**\n\n[![Build status](https://github.com/keltia/fetiche-rs/actions/workflows/rust.yml/badge.svg)](https://github.com/keltia/fetiche-rs/actions/workflows/rust.yml)\n[![Build status (develop)](https://github.com/keltia/fetiche-rs/actions/workflows/develop.yml/badge.svg)](https://github.com/keltia/fetiche-rs/actions/workflows/develop.yml)\n[![dependency status](https://deps.rs/repo/github/keltia/fetiche-rs/status.svg)](https://deps.rs/repo/github/keltia/fetiche-rs)\n[![Docs](https://img.shields.io/docsrs/dmarc-rs)](https://docs.rs/fetiche-rs)\n[![GitHub release](https://img.shields.io/github/release/keltia/dmarc-rs.svg)](https://github.com/keltia/fetiche-rs/releases/)\n[![GitHub issues](https://img.shields.io/github/issues/keltia/fetiche-rs.svg)](https://github.com/keltia/fetiche-rs/issues)\n[![fetiche-rs: 1.56+]][Rust 1.56]\n[![SemVer](https://img.shields.io/badge/semver-2.0.0-blue)](https://semver.org/spec/v2.0.0.html)\n[![License](https://img.shields.io/crates/l/mit)](https://opensource.org/licenses/MIT)\n\nLicensed under the [MIT](LICENSE) license.\n\n1. [About](#about)\n2. [Installation](#installation)\n3. [Usage](#usage)\n4. [Structure](#structure-and-design)\n5. [MSRV](#msrv)\n6. [TODO](#todo)\n7. [Contributing](#contributing)\n\n## About\n\n**Fetiche** is a framework with a set of libraries and utilities dealing with various data formats and import/conversion\nutilities for Aeronautical data about drones and aircraft.\n\nThis is now divided into different crates with libraries (`fetiche-engine`, `fetiche-formats`, `fetiche-sources`) shared\nby the binary crates (`acutectl`, `opensky-history` and now `process-data`).\n\nBinary crates include command-line utilities (`acutectl` and `opensky-history`) to perform import from a file or\nfetch data from different sites. There is now `process-data` which include several tasks aimed at gathering statistics\nand metrics about our drone and flight data.\n\n`acutectl` is the main data fetching utility, relaying the `fetiche-sources` and `fetiche-formats` crates to provide\na single interface to multi sources.\n\n`process-data`  works on the drones and flights data through [Clickhouse] and does various SQL-backed procedures to\ngather\nand calculates metrics including distances (2D and 3D).\n\n`adsb-to-parquet` is a temporary converter between the CSV files we receive the ADS-B data into compressed parquet\nfiles.\nAs my patch to improve [bdt]  has been merged, `bdt` is now used instead.\n\n`opensky-history` is for retrieving historical data from [Opensky]. This access is managed through an SSH-based shell to\nthe Impala database. This is for everything older than 1h of real-time data which does complicate things. This utility\nuse the [pyopensky] Python module (embedded through the `inline-python` crate).\n\n## Installation\n\nIt might be available at some point as crates on [Crates.io]  but for the moment just as a private repository on\n[GitHub]. Installation can be done either through a compiled binary for your platform or by cloning the repo and\ncompiling.\n\n### Cargo Features\n\nThere is one feature enabled by default, called `privacy`. This is for truncating the drone ID to a less-easily\nidentifiable value. See `Cargo.toml` for this.\n\nThis is intentionally *not* a run-time option but a compile-time one.\n\n## Usage\n\nFor the moment, there are 3 binaries called `acutectl` (with `.exe` on Windows), `opensky-history` and `process-data`.\nThe former is used to fetch data into their native format (csv, json). It uses `fetiche-engine` for all the code related\nto accessing, authenticating and fetching data in various ways.\n\nRight now, `acutectl` use blocking HTTP calls and is not using any `async` features.\n\nHowever, while working on streaming support for Opensky, I have been experimenting with [tokio] for async support and\n`acutectl` might eventually become fully-async. It does help for some stuff including signal (read ^C) support.\n\nAll the commands are described in more details in the [acutectl README.md](acutectl/README.md),\n[opensky-history README.md](opensky-history/README.md) and [process-data](process-data/README.md) files.\n\n## `fetiched` (managed in the `fetiched` crate)\n\nOn UNIX systems, there is a new command called `fetiched`. It is a daemon running the latest engine, detaching itself\nfrom the terminal and accepting requests through an [GRPC] interface. The Windows version will have to be run from a\nspecific terminal with the `serve` command.\n\nIn the near future, `fetiched` is evolving into an Actor-based subsystem (using [Actix] ) to manage\norchestration between the internal modules. We do have an engine actor, a configuration actor, etc.\n\nMore details in the specific [Fetiched README.md](fetiched/README.md).\n\n\u003e NOTE: This is WIP\n\n### Data Model\n\nEach source has its own data model which complicates things, apart from [ASTERIX] with Cat129 for drone data, each\ncompany/service provider use their own data model. To ease managing drone data, I started to define my own `DronePoint`\nas a common data model (extracted from the data sent by [ASD] with some fields with different types -- like actual `f32`\ninstead of the string format) and real timestamp. In fact, now that I have fixed `Asd` struct fields handling and types,\nit is not needed.\n\nSee the `fetiche-formats` crate for more details.\n\n## MSRV\n\nThe Minimum Supported Rust Version is *1.56* due to the 2021 Edition.\n\n## Supported platforms\n\n* Unix (tested on FreeBSD, Linux and macOS)\n* Windows\n    * cmd.exe\n    * [Nushell]\n    * Powershell (preferred)\n\n## TODO\n\nHere are some of the things I've been working on. Some of these are registered as issues on [GitHub issues].\n\n- ~~support more parameters (like dates, etc.)~~\n- ~~fetch and analyse from Aeroscope~~\n- ~~fetch and analyse from Asd~~\n- ~~divide into crates for sharing more code.~~\n- ~~use a common data model for drone data~~\n- ~~Support for Opensky (same)~~\n- ~~make `acutectl` use `fetiche-engine` instead of its own `task.rs`~~.\n- ~~add streaming support for sources like opensky~~.\n- ~~rename `drone-utils` into the more proper `fetiche-rs`~~.\n- ~~Data formats conversion framework~~\n- ~~caching tokens (like ASD ones) locally~~\n- ~~merge `import-adsb` and `cat21conv` into `acutectl`~~.\n- ~~Add a `Store` module to handle long-running jobs and their output.~~\n- ~~Retrieve historical data from the [Opensky] site.~~\n- ~~Support for Flightaware AeroAPI and Firehose.~~\n- ~~Apache Parquet as output format.~~\n- ~~Migrate from the embedded [DuckDB] to a proper server-based DB [Clickhouse]~~\n- Add more tests \u0026 benchmarks.\n\nUncertain:\n\n- build `fetiched` as the core daemon and making all other talk to it through gRPC.\n- link to HashiCorp Vault for storing credentials and tokens\n- support for Safesky for ADS-B data\n- Support for Sherlock formats and access methods\n\n## Contributing\n\nPlease see [CONTRIBUTING.md](CONTRIBUTING.md) for some simple rules.\n\nI use Git Flow for this package so please use something similar or the usual GitHub workflow.\n\n1. Fork it [here](https://github.com/keltia/fetiche-rs/fork)\n2. Checkout the develop branch (`git checkout develop`)\n3. Create your feature branch (`git checkout -b my-new-feature`)\n4. Commit your changes (`git commit -am 'Add some feature'`)\n5. Push to the branch (`git push origin my-new-feature`)\n6. Create a new Pull Request\n\n[ASD]: https://eur.airspacedrone.com/\n\n[Actix]: https://actix.rs/\n\n[ASTERIX]: https://www.eurocontrol.int/asterix/\n\n[fetiche-rs: 1.56+]: https://img.shields.io/badge/Rust%20version-1.56%2B-lightgrey\n\n[Mozilla]: https://mozilla.org/\n\n[Opensky]: https://www.opensky-network.org/\n\n[Parquet]: https://parquet.apache.org/\n\n[RUST]: https://www.rust-lang.org/\n\n[Rust 1.56]: https://blog.rust-lang.org/2021/10/21/Rust-1.56.0.html\n\n[Safesky]: https://safesky.app/\n\n[HCL]: https://developer.hashicorp.com/terraform/language\n\n[GitHub issues]: https://github.com/keltia/fetiche-rs/issues\n\n[tokio]: https://crates.io/crates/tokio\n\n[GRPC]: https://en.wikipedia.org/wiki/GRPC\n\n[pyopensky]: https://pypi.org/project/pyopensky/\n\n[Nushell]: https://nushell.sh/\n\n[Actix]: https://actix.rs/\n\n[DuckDB]: https://duckdb.org/\n\n[Clickhouse]: https://clickhouse.com/\n\n[bdt]: https://github.com/datafusion-contrib/bdt\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeltia%2Ffetiche-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkeltia%2Ffetiche-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeltia%2Ffetiche-rs/lists"}