{"id":19476642,"url":"https://github.com/oklabflensburg/open-data-api","last_synced_at":"2025-04-25T14:32:21.571Z","repository":{"id":191630755,"uuid":"685058686","full_name":"oklabflensburg/open-data-api","owner":"oklabflensburg","description":"Open Data Schnittstelle zur freien Nutzung von Rohdaten in einem maschinenlesbarem json Format mit aggregierten Datensätzen.","archived":false,"fork":false,"pushed_at":"2025-04-19T08:47:12.000Z","size":3669,"stargazers_count":6,"open_issues_count":0,"forks_count":7,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-19T15:09:38.482Z","etag":null,"topics":["datascience","fossgis","geography","geojson-data","geospatial","hacktoberfest","json-api","open-source","opendata","opendata-api","opengovernmentdata"],"latest_commit_sha":null,"homepage":"https://api.oklabflensburg.de","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oklabflensburg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-08-30T12:33:36.000Z","updated_at":"2025-04-19T08:47:15.000Z","dependencies_parsed_at":"2024-01-15T04:20:50.233Z","dependency_job_id":"9ca8b4ef-3a62-4705-a98b-262273091ce7","html_url":"https://github.com/oklabflensburg/open-data-api","commit_stats":{"total_commits":60,"total_committers":2,"mean_commits":30.0,"dds":0.01666666666666672,"last_synced_commit":"ecb97c4c84aaf6e2c20266e80f61c7aee509ddb5"},"previous_names":["oklabflensburg/open-data-api"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oklabflensburg%2Fopen-data-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oklabflensburg%2Fopen-data-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oklabflensburg%2Fopen-data-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oklabflensburg%2Fopen-data-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oklabflensburg","download_url":"https://codeload.github.com/oklabflensburg/open-data-api/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250834165,"owners_count":21494921,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datascience","fossgis","geography","geojson-data","geospatial","hacktoberfest","json-api","open-source","opendata","opendata-api","opengovernmentdata"],"created_at":"2024-11-10T19:41:16.188Z","updated_at":"2025-04-25T14:32:21.527Z","avatar_url":"https://github.com/oklabflensburg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Open Data API\n\n\u003e These API endpoints are open to everyone. Please use GitHub issues to report or request anything.\n\n![Screenshot Documentation](https://raw.githubusercontent.com/oklabflensburg/open-data-api/main/screenshot_open_data_api.jpg)\n\n\n## API Host\n\nThe API can be accessed at the following URL:\n\n[https://api.oklabflensburg.de](https://api.oklabflensburg.de)\n\n\n---\n\n\n## Retrieving and Inserting API Data\n\nTo retrieve and insert data into your PostgreSQL database, follow the instructions in the [SETUP.md](SETUP.md) document.\n\n\n## How to Use\n\nFor detailed usage instructions and examples, refer to the [USAGE.md](USAGE.md).\n\n\n---\n\n\n## Prerequisites\n\nBefore running your own instance of the Open Data API, make sure to install the necessary system dependencies on your Ubuntu machine.\n\n\n### Install Dependencies:\n\n```sh\nsudo apt update\nsudo apt install wget\nsudo apt install git git-lfs\nsudo apt install python3 python3-pip python3-venv\nsudo apt install postgresql-16 postgresql-postgis gdal-bin\n```\n\n\n## Install Firewall\n\n```sh\nsudo apt install ufw\nsudo ufw allow 22/tcp\nsudo ufw allow 80/tcp\nsudo ufw allow 443/tcp\nsudo ufw allow 5665/tcp\nsudo ufw enable\nsudo ufw status\n```\n\n---\n\n\n## Creating Dedicated User Accounts\n\nIt is recommended to run the Open Data API as a dedicated user. Below are the commands to create a user named `oklab` and set up the necessary permissions.\n\n```sh\nsudo adduser oklab\nsudo usermod -a -G www-data oklab\nsudo mkdir -p /srv/oklab\nsudo chown -R oklab:oklab /srv/oklab\nsudo chmod 770 -R /srv/oklab\n```\n\n\n---\n\n\n## Prepare Database\n\n1. Modify PostgreSQL Configuration:\n\nOpen and edit the PostgreSQL configuration file `/etc/postgresql/16/main/pg_hba.conf` and add the following lines:\n\n```sh\nlocal   oklab           oklab                                   trust\nhost    oklab           oklab           127.0.0.1/32            trust\n```\n\nAfter editing the configuration, restart PostgreSQL:\n\n```sh\nsudo systemctl restart postgresql.service\nsudo systemctl status postgresql.service\n```\n\n\n2. Create Database and User:\n\nSwitch to the `postgres` user and create the necessary database and user:\n\n```sh\nsudo -i -u postgres\ncreateuser -d oklab\ncreatedb -O oklab oklab\npsql -U oklab\nexit\n```\n\n\n3. Enable PostGIS Extension:\n\nLog into the `oklab` database and enable the necessary extensions:\n\n```sh\npsql -U postgres\n\\c oklab\n\nCREATE EXTENSION IF NOT EXISTS pg_trgm;\nCREATE EXTENSION IF NOT EXISTS postgis;\nCREATE EXTENSION IF NOT EXISTS hstore;\n\nALTER TABLE geography_columns OWNER TO oklab;\nALTER TABLE geometry_columns OWNER TO oklab;\nALTER TABLE spatial_ref_sys OWNER TO oklab;\n\n\\q\n```\n\n\n---\n\n\n## Repository Setup\n\nTo run your own instance of the Open Data API, first clone the repository, initialize a Python virtual environment, and install the required dependencies.\n\n\n### Clone Repository and Install Dependencies:\n\n```sh\ngit clone https://github.com/oklabflensburg/open-data-api.git\ncd open-data-api\npython3 -m venv venv\nsource venv/bin/activate\npip3 install -r requirements.txt\ndeactivate\n```\n\n\n### Configure Environment Variables:\n\nCreate a `.env` file and add the following environment variables with your values:\n\n```sh\nDB_PASS=YOUR_PASSWORD_HERE\nDB_HOST=localhost\nDB_USER=oklab\nDB_NAME=oklab\nDB_PORT=5432\n```\n\n\n---\n\n\n## Import Data\n\nTo use all available Open Data API endpoints, you will need to import data into the database.\n\n\n### Import District Data:\n\n```sh\ncd ..\ngit clone https://github.com/oklabflensburg/open-social-map.git\ncd open-social-map\npsql -U oklab -h localhost -d oklab -p 5432 \u003c data/flensburg_stadtteile.sql\ncp ../open-data-api/.env .\ncd tools\npython3 -m venv venv\nsource venv/bin/activate\npip3 install -r requirements.txt\npython3 insert_districts.py ../static/flensburg_stadtteile.geojson\ndeactivate\npsql -U oklab -h localhost -d oklab -p 5432 \u003c data/flensburg_sozialatlas.sql\npsql -U oklab -h localhost -d oklab -p 5432 \u003c data/flensburg_sozialatlas_metadaten.sql\n```\n\n\n### If You Need to Clean the Database:\n\nIf the data import process fails or you need to reset, use this command to delete all tables (be cautious):\n\n\n```sh\npsql -U oklab -h localhost -d oklab -p 5432 \u003c data/cleanup_database_schema.sql\n```\n\nAfterward, repeat the above data import steps.\n\n\n---\n\n\n## Import Monuments Data\n\n```sh\ncd ..\ngit clone https://github.com/oklabflensburg/open-monuments-map.git\ncd open-monuments-map\ngit lfs pull\npsql -U oklab -h localhost -d oklab -p 5432 \u003c data/denkmalliste_schema.sql\ncp ../open-data-api/.env .\ncd tools\npython3 -m venv venv\nsource venv/bin/activate\npip3 install -r requirements.txt\npython3 insert_boundaries.py ../data/denkmalliste_geometrien.geojson\npython3 insert_monuments.py ../data/stadt-flensburg-denkmalschutz.geojson\ndeactivate\ncd ..\n```\n\n\n---\n\n\n## Open Data API Usage\n\n### Running the API Locally:\n\nOnce the data is imported, you can start the Open Data API locally using the following command:\n\n```sh\ncd ../open-data-api\nsource venv/bin/activate\nuvicorn app.main:app --reload\n```\n\n### Testing the API:\n\nTo test the API, you can run this `curl` command:\n\n```sh\ncurl -X 'GET' 'http://localhost:8000/demographics/v1/details'  -H 'accept: application/json'\n```\n\n\n---\n\n\n## Setting Up the Service\n\nTo set up the Open Data API as a service, create a file `/usr/lib/systemd/system/open-data-api.service` and add the following configuration:\n\n```conf\n[Unit]\nDescription=Instance to serve the open data api\nAfter=network.target\nRequires=postgresql.service\n\n[Service]\nType=simple\nUser=oklab\nGroup=www-data\nDynamicUser=true\nWorkingDirectory=/srv/oklab/open-data-api\nPrivateTmp=true\nEnvironmentFile=/srv/oklab/open-data-api/.env\nExecStart=/srv/oklab/open-data-api/venv/bin/uvicorn \\\n        --proxy-headers \\\n        --forwarded-allow-ips='*' \\\n        --workers=4 \\\n        --port=6720 \\\n        app.main:app\nExecReload=/bin/kill -HUP ${MAINPID}\nRestartSec=1\nRestart=always\n\n[Install]\nWantedBy=multi-user.target\n```\n\n\n### Enabling the Service:\n\nStart and enable the service to run at startup:\n\n```sh\nsudo systemctl start open-data-api.service\nsudo systemctl status open-data-api.service\nsudo systemctl enable open-data-api.service\n```\n\n\n---\n\n\n## Setting Up the Web Server\n\n### Installing Dependencies:\n\nInstall the required packages for the web server:\n\n```sh\nsudo apt install nginx certbot python3-certbot-nginx\n```\n\n\n### Web Server Configuration:\n\nEdit the Nginx configuration to secure the server and set up HTTPS.\nAn example Nginx configuration file can be found here [nginx.conf](config/nginx.conf)\n\n\n### Test Configuration and Install SSL Certificates:\n\n```sh\nsudo nginx -t\nsudo certbot\n```\n\n\n---\n\n\n## Control Web Server\n\nUse the following commands to start, check status, and enable the Nginx service:\n\n```sh\nsudo systemctl start nginx.service\nsudo systemctl status nginx.service\nsudo systemctl enable nginx.service\n```\n\n\n---\n\n\n## How to Contribute\n\nContributions are welcome! Please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) guide for details on how to get involved.\n\n\n---\n\n\n## License\n\nThis repository is licensed under [CC0-1.0](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foklabflensburg%2Fopen-data-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foklabflensburg%2Fopen-data-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foklabflensburg%2Fopen-data-api/lists"}