{"id":21812818,"url":"https://github.com/gforien/flightbeat","last_synced_at":"2026-05-01T00:31:54.170Z","repository":{"id":57547929,"uuid":"301351287","full_name":"gforien/flightbeat","owner":"gforien","description":"Projet INSA 4e année : Heatmap des trajets d'avions les plus demandés et les plus coûteux avec la suite ELK et l'API Skyscanner.","archived":false,"fork":false,"pushed_at":"2021-10-25T09:11:21.000Z","size":6086,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-10-04T08:49:30.239Z","etag":null,"topics":["beats","data-vizualisation","elasticsearch","elk-stack","flights-api","kibana","scrapping"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gforien.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-05T09:08:57.000Z","updated_at":"2021-12-15T16:38:02.000Z","dependencies_parsed_at":"2022-09-26T16:31:11.552Z","dependency_job_id":null,"html_url":"https://github.com/gforien/flightbeat","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/gforien/flightbeat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gforien%2Fflightbeat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gforien%2Fflightbeat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gforien%2Fflightbeat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gforien%2Fflightbeat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gforien","download_url":"https://codeload.github.com/gforien/flightbeat/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gforien%2Fflightbeat/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32481553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"ssl_error","status_checked_at":"2026-04-30T13:12:06.837Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beats","data-vizualisation","elasticsearch","elk-stack","flights-api","kibana","scrapping"],"created_at":"2024-11-27T14:20:23.673Z","updated_at":"2026-05-01T00:31:54.138Z","avatar_url":"https://github.com/gforien.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# flightbeat :airplane::zap:\n\n![](https://img.shields.io/badge/5TC-ELK-blueviolet)\n![](https://img.shields.io/badge/license-Apache-grey)\n\n#### Beat written in Go to retrieve flight prices from Skyscanner API and insert them into an elasticsearch cluster\n\n## :construction_worker: Build and run the beat\n```bash\n$ mage build\n$ ./flightbeat -e -d \"*\"\n# input : flightbeat.yml (default)\n# output: console (default)\n```\n\n## :rocket: Deploy\nThe beat has to always be active on 1 node (out of 16).\nA periodic NodeJS job, **leader.js** makes sure that there is always a leader designated to run the beat.\n*(and because a NodeJS job doesn't handle hard reboot, a cron job makes sure every hour that **leader.js** is up and running).*\nIn case of failure of the leader, the nodes perform a leader election to elect a new leader who will run the beat.\n\n\n## :wrench: Generate your Beat from elastic/beats with mage (oct. 2020)\n1. install go\n2. `go get github.com/elastic/beats`\n3. `go get github.com/magefile/mage`\n4. `cd elastic/beats; mage GenerateCustomBeat` (fill everything explicitly, there is no default value)\n5. `cd gforien/custombeat`\n6. disable W10 App execution alias  + copy python.exe to go/bin/python3.exe  + disable W10 real-time protection (re-enable after)\n7. `make setup; make`\n\n\n## :books: Sources\nThis project is sampled from :\n- [Beats developer guide](https://www.elastic.co/guide/en/beats/devguide/current/new-beat.html)\n- [Go by example](https://gobyexample.com/)\n- [HTTP requests in Go](https://medium.com/rungo/making-external-http-requests-in-go-eb4c015f8839)\n- [Biggest airports in the world](https://gettocenter.com/airports/top-100-airports-in-world/1000)\n- [Flight routes database](https://www.kaggle.com/open-flights/flight-route-database)\n##\n#### [Gabriel FORIEN](https://github.com/gforien)\n![](https://upload.wikimedia.org/wikipedia/commons/b/b9/Logo_INSA_Lyon_%282014%29.svg)\n\n\n\n\u003c!-- # {Beat}\n\nWelcome to {Beat}.\n\nEnsure that this folder is at the following location:\n`${GOPATH}/src/github.com/gforien/flightbeat`\n\n## Getting Started with {Beat}\n\n### Requirements\n\n* [Golang](https://golang.org/dl/) 1.7\n\n### Init Project\nTo get running with {Beat} and also install the\ndependencies, run the following command:\n\n```\nmake setup\n```\n\nIt will create a clean git history for each major step. Note that you can always rewrite the history if you wish before pushing your changes.\n\nTo push {Beat} in the git repository, run the following commands:\n\n```\ngit remote set-url origin https://github.com/gforien/flightbeat\ngit push origin master\n```\n\nFor further development, check out the [beat developer guide](https://www.elastic.co/guide/en/beats/libbeat/current/new-beat.html).\n\n### Build\n\nTo build the binary for {Beat} run the command below. This will generate a binary\nin the same directory with the name flightbeat.\n\n```\nmake\n```\n\n\n### Run\n\nTo run {Beat} with debugging output enabled, run:\n\n```\n./flightbeat -c flightbeat.yml -e -d \"*\"\n```\n\n\n### Test\n\nTo test {Beat}, run the following command:\n\n```\nmake testsuite\n```\n\nalternatively:\n```\nmake unit-tests\nmake system-tests\nmake integration-tests\nmake coverage-report\n```\n\nThe test coverage is reported in the folder `./build/coverage/`\n\n### Update\n\nEach beat has a template for the mapping in elasticsearch and a documentation for the fields\nwhich is automatically generated based on `fields.yml` by running the following command.\n\n```\nmake update\n```\n\n\n### Cleanup\n\nTo clean  {Beat} source code, run the following command:\n\n```\nmake fmt\n```\n\nTo clean up the build directory and generated artifacts, run:\n\n```\nmake clean\n```\n\n\n### Clone\n\nTo clone {Beat} from the git repository, run the following commands:\n\n```\nmkdir -p ${GOPATH}/src/github.com/gforien/flightbeat\ngit clone https://github.com/gforien/flightbeat ${GOPATH}/src/github.com/gforien/flightbeat\n```\n\n\nFor further development, check out the [beat developer guide](https://www.elastic.co/guide/en/beats/libbeat/current/new-beat.html).\n\n\n## Packaging\n\nThe beat frameworks provides tools to crosscompile and package your beat for different platforms. This requires [docker](https://www.docker.com/) and vendoring as described above. To build packages of your beat, run the following command:\n\n```\nmake release\n```\n\nThis will fetch and create all images required for the build process. The whole process to finish can take several minutes.\n --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgforien%2Fflightbeat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgforien%2Fflightbeat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgforien%2Fflightbeat/lists"}