{"id":17873825,"url":"https://github.com/napo/overturemaps_italy","last_synced_at":"2025-06-14T19:03:23.012Z","repository":{"id":187994518,"uuid":"677934918","full_name":"napo/overturemaps_italy","owner":"napo","description":"an extraction of the first release of data made by OvertureMaps Foundation for each italian region script\u0026data","archived":false,"fork":false,"pushed_at":"2023-08-27T15:11:50.000Z","size":70506,"stargazers_count":14,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-08T14:11:30.909Z","etag":null,"topics":["duckdb","geospatial","geospatial-data","italy","opendata","overturemaps","regions"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"wtfpl","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/napo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-08-13T06:08:35.000Z","updated_at":"2024-07-31T01:20:15.000Z","dependencies_parsed_at":"2023-09-26T10:18:09.184Z","dependency_job_id":null,"html_url":"https://github.com/napo/overturemaps_italy","commit_stats":{"total_commits":26,"total_committers":1,"mean_commits":26.0,"dds":0.0,"last_synced_commit":"9f46a8345f60abfb369174392051d29de37ff30d"},"previous_names":["napo/overturemaps_italy"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/napo%2Foverturemaps_italy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/napo%2Foverturemaps_italy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/napo%2Foverturemaps_italy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/napo%2Foverturemaps_italy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/napo","download_url":"https://codeload.github.com/napo/overturemaps_italy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246911489,"owners_count":20853654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["duckdb","geospatial","geospatial-data","italy","opendata","overturemaps","regions"],"created_at":"2024-10-28T11:05:27.712Z","updated_at":"2025-04-02T23:43:46.427Z","avatar_url":"https://github.com/napo.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# overturemaps italy\nOn this page you will find the links to the data extraction of the first release of OvertureMaps Foundation and all the scripts needed to replicate the operation.\u003cbr/\u003e\nThe data version is [Release Overture 2023-07-26-alpha.0](https://overturemaps.org/download/).\u003cbr/\u003e\nThe license for buildings is ODbL, while for places it is CDLA-Permissive 2.0.\u003cbr/\u003e\nThe data is enriched with ISTAT ids and released in geopackage format.\u003cbr/\u003e\nIn this repository there is also the dataset of the non-generalized Italian municipal borders at 2023 of ISTAT (CC-BY license) in geoparquet format.\n\n### prerequisites\njust duckdb\n## data download\nnote: the coordinate reference system is EPSG:4326\n### places\n1. [Abruzzo](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_abruzzo.gpkg) 16Mb\n2. [Basilicata](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_basilicata.gpkg) 5.9Mb\n3. [Calabria](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_calabria.gpkg) 19.8Mb\n4. [Campania](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_campania.gpkg) 61.3Mb\n5. [Emilia-Romagna](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_emiliaromagna.gpkg) 53.5Mb\n6. [Friuli Venezia-Giulia](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_friuliveneziagiulia.gpkg) 13.8Mb\n7. [Lazio](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_lazio.gpkg) 66.2Mb\n8. [Liguria](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_liguria.gpkg) 20.1Mb\n9. [Lombardia](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_lombardia.gpkg) 96.9Mb\n10. [Marche](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_marche.gpkg) 20.2Mb\n11. [Molise](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_molise.gpkg) 3.2Mb\n12. [Piemonte](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_piemonte.gpkg) 46.2Mb\n13. [Puglia](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_puglia.gpkg) 44.9Mb\n14. [Sardegna](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_sardegna.gpkg) 19.9Mb\n15. [Sicilia](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_sicilia.gpkg) 53.4Mb\n16. [Toscana](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_toscana.gpkg) 51.3Mb\n17. [Trentino Alto Adige](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_trentinoaltoadige.gpkg) 15.7Mb\n18. [Umbria](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_umbria.gpkg) 11.5Mb\n19. [Valle d'Aosta](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_valledaosta.gpkg) 2.3Mb\n20. [Veneto](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/places_veneto.gpkg) 55.8Mb\n\n### buildings\n1. [Abruzzo](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_abruzzo.gpkg) 66.1Mb\n2. [Basilicata](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_basilicata.gpkg) 49.9Mb\n3. [Calabria](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_calabria.gpkg) 96.4Mb\n4. [Campania](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_campania.gpkg) 197.3Mb\n5. [Emilia-Romagna](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_emiliaromagna.gpkg) 509.0Mb\n6. [Friuli Venezia-Giulia](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_friuliveneziagiulia.gpkg) 248.5Mb\n7. [Lazio](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_lazio.gpkg) 290.6Mb\n8. [Liguria](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_liguria.gpkg) 200.3Mb\n9. [Lombardia](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_lombardia.gpkg) 640.9Mb\n10. [Marche](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_marche.gpkg) 94.8Mb\n11. [Molise](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_molise.gpkg) 20.4Mb\n12. [Piemonte](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_piemonte.gpkg) 473.8Mb\n13. [Puglia](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_puglia.gpkg) 662.0Mb\n14. [Sardegna](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_sardegna.gpkg) 235.5Mb\n15. [Sicilia](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_sicilia.gpkg) 347.0Mb\n16. [Toscana](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_toscana.gpkg) 512.0Mb\n17. [Trentino Alto Adige](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_trentinoaltoadige.gpkg) 125.1Mb\n18. [Umbria](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_umbria.gpkg) 46.7Mb\n19. [Valle d'Aosta](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_valledaosta.gpkg) 24.9Mb\n20. [Veneto](https://s3.eu-central-1.amazonaws.com/overturemaps.italy/buildings_veneto.gpkg) 925.7Mb\n\n## process\n### prepare the data\nInstall [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and download the data\u003cbr/\u003e\nIf you want download all\n```\naws s3 cp --region us-west-2 --no-sign-request --recursive s3://overturemaps-us-west-2/release/2023-07-26-alpha.0/ \u003cDESTINATION\u003e\n```\nTo download single themes:\n- admins\u003cbr/\u003e *s3://overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=admins*\n- buildings\u003cbr/\u003e *s3://overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=buildings*\n- places\u003cbr/\u003e*s3://overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=places*\n- transportation\u003cbr/\u003e*s3://overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=transportation*\n\n### process description\nall data is incorporated into a duckdb file on which the following SQL operations are then performed\n- extraction from the downloaded parquet files into a duckdb table taking into account the bounding box of Italy with a buffer of 5km\n- from the created table, a new table is created with a selection of columns so that each contains only one value. E.g. for names you choose the languages of Italian, English, German and French\n- the table is enriched with new columns corresponding to ISTAT codes through a spatial relationship with the geometries of Italian municipalities\n- based on these attributes, data are extracted at the level of each Italian region\n\n### execution\n#### prepare data\nbash script\n```bash\nduckdb omf_italy -c \".read 00_prepare_tables_istat.sql\"\n```\n#### create geopackages of the places for earch region of Italy\nbash script\n```bash\nduckdb omf_italy -c \".read 01_extraction_places_italy.sql\"\n```\n#### create geopackages of the buildings for earch region of Italy\n```bash\nduckdb omf_italy -c \".read 02_extraction_buildings_italy.sql\"\n```\n\n#### assign the SRS to each file\nbash script\n```bash\nfor i in `ls *.gpkg`;\n  do\n    name=`basename $i .gpkg`;\n    tmpname=`echo $name`_tmp.gpkg;\n    echo \"assign WGS84 to $i\";\n    ogr2ogr -a_srs EPSG:4326 -f \"GPKG\" $tmpname $i;\n    mv $tmpname $i;\n    echo \"done!\";\ndone\n```\n\n#### creation of geoparquet files\nbash script\n```bash\nwget https://github.com/planetlabs/gpq/releases/download/v0.11.0/gpq-linux-amd64.tar.gz\ntar xfvz gpq-linux-amd64.tar.gz \nchmod 755 gpq\nregions=\"abruzzo basilicata calabria campania emiliaromagna friuliveneziagiulia lazio liguria lombardia marche molise piemonte puglia sardegna sicilia toscana trentinoaltoadige umbria valledaosta veneto\"\nurl=\"https://s3.eu-central-1.amazonaws.com/overturemaps.italy/\"\nplaceslbl=\"places_\"\ngpkglbl=\".gpkg\"\nparquetlbl=\".parquet\"\nbuildingslbl=\"buildings_\"\nfor r in $regions \ndo\n\td=\"$url$placeslbl$r$gpkglbl\"\n\twget $d\n\tduckdb -c \"load spatial;CREATE TABLE $placeslbl$r as select * from st_read('$placeslbl$r$gpkglbl', layer='$placeslbl$r');ALTER TABLE $placeslbl$r  RENAME geom TO geometry;COPY  (SELECT * FROM $placeslbl$r ) TO 'tmp.parquet' (FORMAT PARQUET, CODEC 'ZSTD');\"\n\trm $placeslbl$r$gpkglbl\n\t./gpq convert tmp.parquet $placeslbl$r$parquetlbl\n\trm tmp.parquet\ndone;\nfor r in $regions \ndo\n\td=\"$url$buildingslbl$r$gpkglbl\"\n\twget $d\n\tduckdb -c \"load spatial;CREATE TABLE $buildingslbl$r as select * from st_read('$buildingslbl$r$gpkglbl', layer='$r');ALTER TABLE $buildingslbl$r  RENAME geom TO geometry;COPY  (SELECT * FROM $buildingslbl$r ) TO 'tmp.parquet' (FORMAT PARQUET, CODEC 'ZSTD');\"\n\trm $buildingslbl$r$gpkglbl\n\t./gpq convert tmp.parquet $buildingslbl$r$parquetlbl\n\trm tmp.parquet\ndone;\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnapo%2Foverturemaps_italy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnapo%2Foverturemaps_italy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnapo%2Foverturemaps_italy/lists"}