{"id":24087970,"url":"https://github.com/tinybirdco/vehicle-tracking-demo","last_synced_at":"2025-09-01T22:37:16.833Z","repository":{"id":191620866,"uuid":"684477872","full_name":"tinybirdco/vehicle-tracking-demo","owner":"tinybirdco","description":null,"archived":false,"fork":false,"pushed_at":"2023-10-03T11:07:42.000Z","size":126,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-27T05:25:03.605Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://vehicle-tracking-demo.vercel.app","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinybirdco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-29T07:57:36.000Z","updated_at":"2023-08-30T11:02:17.000Z","dependencies_parsed_at":"2025-02-27T05:24:39.430Z","dependency_job_id":"6730681d-3691-4f70-8354-6362d4ef4600","html_url":"https://github.com/tinybirdco/vehicle-tracking-demo","commit_stats":null,"previous_names":["tinybirdco/vehicle-tracking-demo"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tinybirdco/vehicle-tracking-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fvehicle-tracking-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fvehicle-tracking-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fvehicle-tracking-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fvehicle-tracking-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinybirdco","download_url":"https://codeload.github.com/tinybirdco/vehicle-tracking-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fvehicle-tracking-demo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273203218,"owners_count":25063275,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-01T02:00:09.058Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-10T03:56:42.841Z","updated_at":"2025-09-01T22:37:16.785Z","avatar_url":"https://github.com/tinybirdco.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vehicle Tracking Demo\n\nThe use case is a fleet manager that wants to check the real-time status of their vehicles, in terms of the latest VIN, odometer reading, and GPS coordinates per vehicle.\n\n## Data Project\n\n```txt\n├── datasources\n│   ├── fixtures\n│   ├── kafka_sample_gps.datasource\n│   ├── kafka_sample_vnet.datasource\n│   ├── sample_gps_mv.datasource\n│   ├── sample_latest_gps_mv.datasource\n│   ├── sample_latest_odo_mv.datasource\n│   ├── sample_latest_vin_mv.datasource\n│   ├── sample_vnet_events_mv.datasource\n│   ├── sample_vnet_mv.datasource\n│   └── vehicle_hierarchy.datasource\n├── endpoints\n│   ├── api_active_vehicles.pipe\n│   ├── api_events_trend.pipe\n│   ├── api_filter_company.pipe\n│   ├── api_filter_serial.pipe\n│   ├── api_latest_gps.pipe\n│   ├── api_odo_gps_per_vehicle.pipe\n│   ├── api_top_odom.pipe\n│   ├── api_vin_odo_per_vehicle.pipe\n│   └── api_vin_odo_per_vehicle_fulldownload.pipe\n├── pipes\n│   ├── sample_gps_materialization.pipe\n│   ├── sample_latest_gps_materialization.pipe\n│   ├── sample_latest_odo.pipe\n│   ├── sample_latest_vin.pipe\n│   ├── sample_vnet_events_mat.pipe\n│   └── sample_vnet_materialization.pipe\n```\n\nIn the `/data-project/datasources` folder, we have two Kafka data sources, one vehicle hierarchy (i.e. dimension table), and several materialized data sources.\n\nIn the `/data-project/pipes` folder, we have several materialized view pipes to extract JSON strings and calculate the latest values.\n\nIn the `/data-project/endpoints` folder, we have the API endpoints that are used in the front end.\n\nNote: before tb pushing, you need to create the kafka connection with `tb connection create kafka --bootstrap-servers $BOOTSTRAP_SERVERS --key $SASL_PLAIN_USERNAME --secret $SASL_PLAIN_PASSWORD --connection-name tb_confluent`\n\nThen, a `tb push --push-deps` will work.\n\n## Data Generator\n\nIn the `/data-generator` folder, there are four Python scripts to generate dummy data:\n\n1. `gen_vehicles.py` generates `vehicle_hierarchy.csv`, which is a CSV file containing a unique identifier for each vehicle and the associated dimensional data (company, serial number, etc.). This file must be generated first because it is referenced by the other scripts.\n\n2. `gen_kafka_vnet.py` produces messages to the Kafka topic `demo_vehicle_vnet`, containing data about VIN, odometer readings, and other data types. This script references `vehicle_hierarchy.csv` to ensure repeatability and a common hierarchy. Each message contains a Kafka header. The script loops through several `data_types` for each vehicle to ensure repeatability (so the front end will show data changing for a specific vehicle).\n\n3. `gen_kafka_gps.py` produces messages to the Kafka topic `demo_vehicle_gps`, containing data about GPS location. This script references `vehicle_hierarchy.csv` to ensure repeatability and a common hierarchy. Each message contains a Kafka header.\n\n4. `gen_csv_vnet.py` is optional - it is used to generate monthly CSV files to backfill data (in case history is important).\n\nTo use the kafka scripts, create a `.env` file in the `/data-generator` folder with the following content:\n\n```.env\nBOOTSTRAP_SERVERS=\u003cYOUR_SERVER\u003e\nSASL_PLAIN_USERNAME=\u003cYOUR_USERNAME\u003e\nSASL_PLAIN_PASSWORD=\u003cYOUR_PASSWORD\u003e\n```\n\nNote: do not forget to add the generated vehicle_hierarchy.csv file into the vehicle_hierarchy Data Source.\n\n```bash\ncd ../data-project\ntb datasource append vehicle_hierarchy ../data-generator/vehicle_hierarchy.csv\n```\n\n## Dashboard\n\nThe dashboard was built using [Tremor](https://www.tremor.so/).\n\nThere are two tabs to visualize data from the two Kafka topics. There are dropdowns to filter on a specific company or serial number. There is also a dropdown used to refresh the dashboard on-demand (default), or every five seconds. The latter option is useful when demonstrating real-time updates to a single vehicle (you can filter on a specific serial number and see the values update every five seconds).\n\nThere is also a text input field to enter a token. By default, the dashboard uses a token that has access to all data. There is another token named `demo dashboard token company` that has a filter on a single company, which you can use in the dashboard to show row-level security.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Fvehicle-tracking-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinybirdco%2Fvehicle-tracking-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Fvehicle-tracking-demo/lists"}