{"id":24087927,"url":"https://github.com/tinybirdco/weather-data-api","last_synced_at":"2026-06-11T08:31:44.662Z","repository":{"id":198761529,"uuid":"651646132","full_name":"tinybirdco/weather-data-api","owner":"tinybirdco","description":"A demo to illustrate how to go from an API design to implementing (and hosting) it on Tinybird.","archived":false,"fork":false,"pushed_at":"2024-06-04T02:37:22.000Z","size":185,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-02-27T05:24:34.494Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinybirdco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-09T18:00:11.000Z","updated_at":"2024-06-04T02:37:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"5139f3d7-b167-4ad7-bcdd-0b546e46ed49","html_url":"https://github.com/tinybirdco/weather-data-api","commit_stats":null,"previous_names":["tinybirdco/weather-data-api"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tinybirdco/weather-data-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fweather-data-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fweather-data-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fweather-data-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fweather-data-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinybirdco","download_url":"https://codeload.github.com/tinybirdco/weather-data-api/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Fweather-data-api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34190583,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-10T03:56:11.414Z","updated_at":"2026-06-11T08:31:44.641Z","avatar_url":"https://github.com/tinybirdco.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# weather-data-api\nA demo to illustrate how to go from an API design to implementing (and hosting) it on Tinybird. \n\nWhile this example focuses on publishing weather data, the underlying concepts should apply to any other data domain.\n\nThere are a lot of moving pieces, but here is a general outline of the resources provided:\n* Python script for getting weather data from OpenWeatherMap and posting it to a Tinybird Data Source using the [Events API](https://www.tinybird.co/docs/ingest/events-api.html).\n* How-tos for creating a ``incoming_weather_data`` Data Source and a ``reports`` Pipe. \n* Python script for making requests to the weather API ``reports`` endpoint.  \n\n### That endpoint looks like this: \n`https://api.tinybird.co/v0/pipes/reports.json`\n[Endpoint documentation](https://ui.tinybird.co/23b79dee-af6b-43c5-a5eb-93b3c717e7b8/pipe/t_402200151ae947ea95bd2e5cf8c8dcf0/endpoint)\n\nThese endpoints make the weather data available to clients such as this Grafana dashboard:\n\n![Dashboard](images/weather-dashboard.png)\n\n\n## Getting started \n\nYou can start by making a clone or fork of this repository. \n\nNext, you will need to create Tinybird and OpenWeatherMap accounts, create a Tinybird Data Source and Pipe, then crank up your weather data feed. When all that is in place, you will be all set to launch your own weather API.\n\n### Set up accounts\n\n+ Go to [OpenWeatherMap](https://openweathermap.org/) and create a [free](https://openweathermap.org/price#weather) account for the \"Current Weather and Forecasts\" API.\n  + Generate an API key. An environmental variable will be set to this key value. \n+ Go to [Tinybird](http://tinybird.co) and create an account with the forever-free Build tier.\n  + Reference your Admin Token or create a User Token with read permissions. An environmental variable will be set to this key value.\n \n### Create Data Source and build Pipe\n+ In Tinybird, create ``incoming_weather_data`` Data Source and ``reports`` Pipe.\n  + See below for details on creating these. When it comes to creating the Pipe, the Tinybird command-line interface (CLI) comes highly recommended. With it you can automate the process by pushing a provide Pipe definition file, rather than manually building the Pipe in the user interface (UI) by setting up three Nodes.  \t\n\n### Start up near-real-time weather data feed\n+ Set up Python script to load near-real-time OpenWeatherMap data into Tinybird.\n  + Configure ./config/.env with the OpenWeatherMap and Tinybird tokens.\n```\nTINYBIRD_TOKEN={COPY-IN-YOUR-TINYBIRD-TOKEN}\nOPENWEATHERMAP_TOKEN={COPY-IN-YOUR-OPENTWEATHERMAO-TOKEN}\n```\n  + Run the ./scripts/get_and_send_data.py script. This script is designed to be used with a 'scheduler', so runs once and quits.\n    * Set up scheduler to run script every ten minutes. \n \nWhen you are ready to build, this [checklist](#checklist) may help for rolling out your instance.\n\n## Details\n\n### Weather report data schema\n\nThe incoming data feed emits weather `report` objects with the following structure:\n\n```json\n{\n\t\"timestamp\": \"2023-05-04 18:07:08\",\n\t\"site_name\": \"New York City\",\n\t\"temp_f\": 52.12,\n\t\"precip\": 0.0,\n\t\"humidity\": 79,\n\t\"pressure\": 1017,\n\t\"wind_speed\": 10.36,\n\t\"wind_dir\": 190,\n\t\"clouds\": 100,\n\t\"description\": \"overcast clouds\"\n}\n```\n\n\u003ca id=\"markdown-createds\" name=\"create-ds\"\u003e\u003c/a\u003e\n### Creating Data Source\n\nThis Events API request creates a new `incoming_weather_data` Data Source by referencing a `report` JSON object:\n\n```curl\ncurl \\\n      -X POST 'https://api.tinybird.co/v0/events?name=incoming_weather_data' \\\n      -H \"Authorization: Bearer {TOKEN}\" \\\n      -d $' {\"timestamp\": \"2023-05-01 12:45:53\",\"site_name\": \"New York City\",\"temp_f\": 59.65,\"precip\": 0.0,\"humidity\": 41,\"pressure\": 994,\"wind_speed\": 21.85,\"wind_dir\": 240,\"clouds\": 100,\"description\": \"overcast clouds\"}'\n```\n## API design\n\n### API Endpoints \n* One `/reports.json` endpoint that returns weather `report` JSON objects. Out-of-the-box, CSV, Ndjson, and Parquet formats are also available. \n\n### Query parameters \n\nThe ‘reports’ API Endpoint will support the following query parameters:\n\n* ```start_time``` and ```end_time``` for defining a period of interest. Timestamps are formatted with the “YYYY-MM-DD HH:mm:ss” pattern and are in Coordinated Universal Time (UTC). \n  * If these request parameters are not included, the endpoint will return data from the previous 24 hours. \n  * If end_time is not included in the request, it defaults to the time of the request (i.e. ‘now’). \n  * If only an end_time is included, the start_time will default to 24 hours before the end_time. \n\t\n* ```city``` for selecting a single city of interest. If not included in the request, data from the entire US will be returned. Values for this parameter are case insensitive.\n\n* ```sensor_type``` for selecting a single type of weather data to return. The following values are supported: temp, precip, wind, humidity, pressure, and clouds. If not used, all weather data types are reported. When ‘wind’ is selected, both speed and direction are returned. \n\t\n* ```max_results``` for limiting the amount of reports to return in the response. The default value is 1000. \n\n## Example endpoint requests\nTo help illustrate how the API Endpoint should work, below are some example requests. The root URL for all of these examples is https://api.tinybird.co/v0/pipes/reports.json. In these examples, just the /reports.json portion is referenced along with the query parameters.\n\n* Requesting the 1,000 most recent reports from all cities since yesterday:  \n[/reports.json](https://api.tinybird.co/v0/pipes/reports.json?token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzchttps://api.tinybird.co/v0/pipes/reports.json?max_results=1000\u0026sensor_type=all\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzc)\n* Due to defaults this is equivalent to:  \n[/reports.json?max_results=1000\u0026sensor_type=all]([/reports.json?max_results=1000\u0026sensor_type=all](https://api.tinybird.co/v0/pipes/reports.json?max_results=1000\u0026sensor_type=all\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzchttps://api.tinybird.co/v0/pipes/reports.json?max_results=1000\u0026sensor_type=all\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzc))\n\n* Requesting reports from the first week of June 2023:  \n[/reports.json?start_time=2023-06-01 00:00:00\u0026end_time=2023-06-08 00:00:00](https://api.tinybird.co/v0/pipes/reports.json?start_time=2023-06-01%2000:00:00\u0026end_time=2023-06-08%2000:00:00\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzchttps://api.tinybird.co/v0/pipes/reports.json?max_results=1000\u0026sensor_type=all\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzc)\n\n* Requesting full reports from Minneapolis since yesterday:  \n[/reports.json?city=minneapolis](https://api.tinybird.co/v0/pipes/reports.json?city=minneapolis\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzchttps://api.tinybird.co/v0/pipes/reports.json?max_results=1000\u0026sensor_type=all\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzc)\n\n* Requesting the 1,000 most recent temperatures, over the past day, from all cities:  \n[/reports.json?sensor_type=temp](https://api.tinybird.co/v0/pipes/reports.json?sensor_type=temp\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzchttps://api.tinybird.co/v0/pipes/reports.json?max_results=1000\u0026sensor_type=all\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzc)\n\n* Request the 100 most recent weather reports from across the US:  \n[/reports.json?max_results=100](https://api.tinybird.co/v0/pipes/reports.json?max_results=100\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzchttps://api.tinybird.co/v0/pipes/reports.json?max_results=1000\u0026sensor_type=all\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzc)\n\n* Request temperature data for the city of Houston, and for June 16, 2023, midnight to midnight local time (CDT). \n[/reports.json?city=houston\u0026sensor_type=temp\u0026start_time=2023-06-03 05:00:00\u0026end_time=2023-06-04 05:00:00](https://api.tinybird.co/v0/pipes/reports.json?city=houston\u0026sensor_type=temp\u0026start_time=2023-06-16%2006:00:00\u0026end_time=2023-06-17%2006:00:00\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzchttps://api.tinybird.co/v0/pipes/reports.json?max_results=1000\u0026sensor_type=all\u0026token=p.eyJ1IjogIjIzYjc5ZGVlLWFmNmItNDNjNS1hNWViLTkzYjNjNzE3ZTdiOCIsICJpZCI6ICJmNWZlYjg3ZS0wM2Q0LTRhN2MtODEwNy00ZDEzZThmNjgxNjMifQ.1i32I7ZMUm6pvZ_DEyu-XasBKKx1XYTEHzF8k4eRAzc)\n\n## Nodes \n\nThe ``reports`` Pipe has three Nodes:\n\n+ Node 1: Filtering by the city and time period of interest\n+ Node 2: Selecting the sensor report type\n+ Node 3: Applying the ‘max results’ parameter to limit the number of report objects in the response\n\nThese Nodes are designed to be applied in this order, and the final Node is used to generate API Endpoint responses. \n\n### Node 1 named ``city_and_period_of_interest``\n\nApplying ``city``, ``start_time``, and ``end_time`` query parameters. This is a case where we pull in every field (SELECT \\*) and do not take this opportunity to drop fields. The fields arriving via the Event API have already been curated by a Python script. \n\n```sql\n%\nSELECT *\nFROM incoming_weather_data\nWHERE\n    1=1\n     {% if defined(city) %}\n        AND lowerUTF8(site_name) LIKE lowerUTF8({{ String(city, description=\"Name of US City to get data for. Data is available for the 175 most populated cities in the US. Optional and defaults to all cities.\") }})\n     {% end %}\n     {% if defined(start_time) and defined(end_time) %}\n         AND toDateTime(timestamp) BETWEEN {{ DateTime(start_time, description=\"'YYYY-MM-DD HH:mm:ss'. UTC. Optional and defaults to 24 hours ago. Defines the start of the period of interest. \") }} AND {{ DateTime(end_time, description=\"'YYYY-MM-DD HH:mm:ss'. UTC. Optional and defaults to time of request. Defines the end of the period of interest.\") }}\n     {% end %}\n     {% if not defined(start_time) and not defined(end_time) %}\n        AND toDateTime(timestamp) BETWEEN addDays(now(),-1) AND now()\n     {% end %}\n     {% if defined(start_time) and not defined(end_time) %}\n         AND toDateTime(timestamp) BETWEEN {{ DateTime(start_time) }} AND now()\n     {% end %}\n     {% if not defined(start_time) and defined(end_time) %}\n         AND toDateTime(timestamp) BETWEEN addDays(toDateTime({{DateTime(end_time)}}),-1) AND {{ DateTime(end_time) }}\n     {% end %}\nORDER BY timestamp DESC\n```\n### Node 2 named ``select_sensor_type``\n\nHere we support the ``sensor_type`` query parameter. When used, just that data type is selected, along with the timestamp and site_name attributes. Note that 'wind' is a special case and two data types are returned (speed and direction). \"Special cases\" often result in confusion and 'special' code, so this should be updated to have separate wind_dir and wind_vel as sensor_type otpions. \n\n```sql\n%\nWITH\n{{ String(sensor_type, 'all', description=\"Type of weather data to return. Default is all types. Available types: 'temp', 'precip', 'wind', 'humidity', 'pressure', and 'clouds'. Wind returns both velocity and direction paramters. Units: temperature (F), precip (inches per hour), wind (mph and degrees) humidity (%), pressure (hPa), clouds (% coverage), \")}}\n\nSELECT\n    timestamp,\n    site_name,\n    {% if defined(sensor_type) and sensor_type == 'temp' %} temp_f\n    {% elif defined(sensor_type) and sensor_type == 'precip' %} precip\n    {% elif defined(sensor_type) and sensor_type == 'wind' %} wind_speed, wind_dir\n    {% elif defined(sensor_type) and sensor_type == 'humidity' %} humidity\n    {% elif defined(sensor_type) and sensor_type == 'pressure' %} pressure\n    {% elif defined(sensor_type) and sensor_type == 'clouds' %} clouds\n    {% else %}\n        temp_f, precip, wind_speed, wind_dir, humidity, pressure, clouds, description\n    {% end %} \nFROM city_and_period_of_interest\n```\n\n### Node 3 named ``endpoint``\n\nHere we apply the ``max_results`` parameter. The naming here follows a convention of marking the Node you want to publish as an API with the name ``endpoint``. You can use whatever name you want. \n\n```sql\n%\nSELECT * \nFROM select_sensor_type\nORDER BY timestamp DESC\nLIMIT {{ Int32(max_results, 1000, description=\"The maximum number of reports to return per response. Defaults to 1000.\") }}\n\n```\n\u003ca id=\"markdown-checklist\" name=\"checklist\"\u003e\u003c/a\u003e\n## Checklist for replicating this demo\n\n- [ ] Set up accounts\n  - [ ] OpenWeatherMap\n  - [ ] Tinybird\n\t\n- [ ] Establishing data feed\n  - [ ] Create Tinybird Data Source referencing report JSON object. See [example command](#create-ds)\n  - [ ] Set-up, configure and deploy the “get_and_post_data” Python script. \n\t\t\n- [ ] Build Tinybird Pipe \n  * With CLI\n    - [ ] Push provided Pipe file to your Workspace\n  * With UI\n    - [ ] Create ‘reports’ Pipe\n    - [ ] Create Nodes \n\n- [ ] Publish and test API Endpoint\n\n\n## Next steps? \n\nSome ways to iterate the weather data API Endpoint: \n\n- [X] Add sensor types of ``wind_vel`` and ``wind_dir``.\n- [ ] Enable selecting a comma-delimited list of City names.\n- [ ] Parameterize number of days set start_time with (not a query parameter, but a configurable Pipe setting.\n- [ ] Enable selecting a comma-delimited list of sensor types.\n- [ ] Add geographic metadata:\n  - [ ] Capture OpenWeatherMap geo metadata for current set of cities. \n  - [ ] Enable data retrieval by geographic area, such as US States.\n  - [ ] Design an endpoint to serve up site metadata, including geographic metadata to support filtering by location. \n\n\n\n\n\n \n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Fweather-data-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinybirdco%2Fweather-data-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Fweather-data-api/lists"}