{"id":24087955,"url":"https://github.com/tinybirdco/tinybird-scooters","last_synced_at":"2025-08-07T23:29:03.566Z","repository":{"id":211146816,"uuid":"728323490","full_name":"tinybirdco/tinybird-scooters","owner":"tinybirdco","description":"Build a real-time logistics management application using Confluent and Tinybird","archived":false,"fork":false,"pushed_at":"2024-01-05T20:36:13.000Z","size":296,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-10T03:56:17.156Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinybirdco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-12-06T17:39:16.000Z","updated_at":"2024-01-09T19:50:21.000Z","dependencies_parsed_at":"2023-12-06T19:28:15.038Z","dependency_job_id":"22aec474-6343-4fad-9efe-6e2528690745","html_url":"https://github.com/tinybirdco/tinybird-scooters","commit_stats":null,"previous_names":["tinybirdco/tinybird-scooters"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Ftinybird-scooters","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Ftinybird-scooters/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Ftinybird-scooters/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Ftinybird-scooters/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinybirdco","download_url":"https://codeload.github.com/tinybirdco/tinybird-scooters/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240982517,"owners_count":19888515,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-10T03:56:29.127Z","updated_at":"2025-02-27T05:21:24.669Z","avatar_url":"https://github.com/tinybirdco.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scooter IoT Analytics Demo\n\nThis demo uses streaming IoT time series data from Confluent Cloud on AWS plus scooter dimensional tables in Amazon S3 to create real-time Scooter IoT Analytics for visualization in a frontend application (we used Retool).\n\n## Instructions\n\n### 1. Set up a free Tinybird Workspace\n\nNavigate to [tinybird.co/signup](https://www.tinybird.co/signup) and create a free account. Create a new Workspace (name it whatever you want).\n\n### 2. Clone the repository\n\n```bash\ngit clone https://github.com/tinybirdco/tinybird-scooters.git\ncd tinybird-scooters\n```\n\n### 3. Install the Tinybird CLI and dependencies\n\n```bash\npython -mvenv .e\n. .e/bin/activate\npip install tinybird-cli ndjson confluent_kafka\n```\n\n### 4. Authenticate to Tinybird\n\nCopy your user admin token from [ui.tinybird.co/tokens](https://ui.tinybird.co/tokens). Your user admin token is the token with the format `admin \u003cyour email address\u003e`.\n\nIn the Tinybird CLI, run the following command\n\n```bash\ncd tinybird\nexport TB_TOKEN=\u003cyour user admin token\u003e\ntb auth -i\n```\n\nIf you intend to push this to your own repo, add the `.tinyb` file to your `.gitignore`, as it contains your user admin token.\n\n```bash\necho \".tinyb\" \u003e\u003e .gitignore\n```\n\n### Update `data_gen.py` with your Confluent cluster details\n\nThis demo uses Confluent Cloud to handle data streaming of scooter telemetry events. If you're new to Confluent, you can signup for an account [here](https://www.confluent.io/get-started). We use a Python script to generate fake data and send it to Confluent Cloud. You'll need to update the `data_gen.py` file with your Confluent details:\n\n```python\nCONFLUENT_SERVER = 'your_confluent_bootstrap_server'\nCONFLUENT_KEY = 'your_confuent_access_key'\nCONFLUENT_SECRET = 'your_confluent_access_secret'\n```\n\nOnce you've done that, run the data generator to begin sending data to your Confluent topic:\n\n```bash\npython data_gen.py\n```\n\n### Create a the scooter telemetry events Data Source in Tinybird\n\nYou're going to capture streaming data from Confluent into Tinybird so you can query it and build your APIs.\n\nYou can do this in the Tinybird UI using the [Confluent Connector](https://www.tinybird.co/docs/ingest/confluent.html), or in the CLI using the following command:\n\n```bash\ntb connection create kafka\n# Kafka Bootstrap Server: \u003cyour Confluent server\u003e\n# Key: \u003cyour Confluent access key\u003e\n# Secret: \u003cyour Confluent secret\u003e\n# Connection name: \u003cGive your connection a name, defaults to bootstrap server\u003e\n```\n\nThen, update the `scooter_telem_events.datasource` file as follows:\n\n```\nSCHEMA \u003e\n    `battery_percent` Int16 `json:$.battery_percent`,\n    `fault_severity` String `json:$.fault_severity`,\n    `journey_id` String `json:$.journey_id`,\n    `latitude` Float32 `json:$.latitude`,\n    `longitude` Float32 `json:$.longitude`,\n    `scooter_id` String `json:$.scooter_id`,\n    `status` String `json:$.status`,\n    `timestamp` DateTime `json:$.timestamp`\n\nENGINE \"MergeTree\"\nENGINE_PARTITION_KEY \"toYear(timestamp)\"\nENGINE_SORTING_KEY \"timestamp\"\n\nKAFKA_CONNECTION_NAME \u003cyour connection name\u003e\nKAFKA_TOPIC \u003cyour Confluent topic\u003e\nKAFKA_GROUP_ID \u003cchoose a consumer group ID\u003e\n```\n\n### Create a scooter info Data Source\n\nYou're going to enrich scooter telemetry events with dimensions in S3. To do that, upload the [`scooter_info.csv`](/tinybird/scripts/scooter_info.csv) to an S3 bucket, and use the [S3 Connector](https://www.tinybird.co/docs/ingest/s3.html) to sync that file to Tinybird.\n\nAlternatively, you can simply upload the `scooter_info.csv` file to Tinybird directly.\n\n### Push the remaining resources to the Tinybird server\n\nPush the remaining Pipes and Data Sources to Tinybird with...\n\n```\ntb push\n```\n\n### Build a frontend!\n\nYou'll now have several Tinybird APIs that you can build with. We used Retool to build a simple frontend application. If you'd like to see how we did that, watch the [recording of our live coding session](https://www.youtube.com/watch?v=rf7ZannHDf0)\n\n## Contributing\n\nIf you find any issues or have suggestions for improvements, please submit an issue or a [pull request](https://github.com/tinybirdco/scooter-rental-iot-analytics/pulls?q=is%3Apr+is%3Aopen+sort%3Aupdated-desc).\n\n## License\n\nThis code is available under the MIT license. See the [LICENSE](https://github.com/tinybirdco/scooter-rental-iot-analytics/blob/main/LICENSE.txt) file for more details.\n\n## Need help?\n\n\u0026bull; [Community Slack](https://www.tinybird.co/community) \u0026bull; [Tinybird Docs](https://www.tinybird.co/docs) \u0026bull;\n\n## Authors\n\n- [Cameron Archer](https://github.com/tb-peregrine)\n- [Alasdair Brown](https://github.com/sdairs)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Ftinybird-scooters","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinybirdco%2Ftinybird-scooters","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Ftinybird-scooters/lists"}