{"id":25984988,"url":"https://github.com/jjcfrancisco/popgis","last_synced_at":"2025-11-07T07:01:55.545Z","repository":{"id":246268828,"uuid":"820592085","full_name":"jjcfrancisco/popgis","owner":"jjcfrancisco","description":"A blazing fast way to insert GeoJSON, ShapeFile \u0026 OsmPBF into a PostGIS database.","archived":false,"fork":false,"pushed_at":"2024-09-11T22:07:38.000Z","size":11621,"stargazers_count":9,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-09-13T07:12:01.401Z","etag":null,"topics":["database","gis","postgis","rust"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/popgis","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jjcfrancisco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-26T19:32:09.000Z","updated_at":"2024-09-11T22:07:39.000Z","dependencies_parsed_at":"2024-06-27T00:04:26.200Z","dependency_job_id":"f3635910-dd83-4f0e-a048-6fb596ad73a0","html_url":"https://github.com/jjcfrancisco/popgis","commit_stats":null,"previous_names":["jjcfrancisco/popgis"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jjcfrancisco%2Fpopgis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jjcfrancisco%2Fpopgis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jjcfrancisco%2Fpopgis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jjcfrancisco%2Fpopgis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jjcfrancisco","download_url":"https://codeload.github.com/jjcfrancisco/popgis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242023193,"owners_count":20059297,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","gis","postgis","rust"],"created_at":"2025-03-05T11:52:46.161Z","updated_at":"2025-11-07T07:01:50.499Z","avatar_url":"https://github.com/jjcfrancisco.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PopGIS\nA blazing fast way to insert large GeoJSON, ShapeFile \u0026 OsmPBF into a PostGIS database.\n\n## Why?\nImporting large datasets into a PostGIS database can take a long time and the aim of PopGIS is to optimize the performance of such operations. **PopGIS is 2x faster than ogr2ogr**, particularly with very large input files against remote databases. Although the performance improvement for smaller datasets may be minimal, the efficiency gains for larger datasets are considerable. For more details, go to the [benchmarks](#benchmarks) section.\n\n## Installation\nYou can install PopGIS directly by running the Cargo install command\n```bash\ncargo install popgis\n```\n\n## Usage\nBelow are the available commands and flags for PopGIS: \n\n#### `input`\nspecifies the path to the GeoJSON or ShapeFile you'd like to insert into a PostGIS database.\n\n#### `uri`\nspecifies the URI of the PostGIS database where you'd like to insert the input data.\n\n#### `schema`\nspecifies the schema where the table will be created. **Optional**. *Default is public.*\n\n#### `table`\nspecifies the name of the resulting table.\n\n#### `srid`\nspecifies the SRID of the input data. **Optional**. *Default is 4326.*\n\n#### `mode`\nspecifies the mode of the operation. **Optional**. *Default is overwrite*. Read more [here](#modes).\n\n#### `reproject`\nreprojects the input data to the specified SRID. **Optional**.\n\n#### Examples\n```bash\n## GeoJSON -\u003e PostGIS ##\npopgis --input spain.geojson \\\n       --uri postgresql://my_username:my_password@localhost:5432/my_database \\\n       --schema osm \\\n       --table waters \\\n       --srid 3857\n\n## ShapeFile -\u003e PostGIS ##\npopgis -i water_polygons.shp \\\n       -u  postgresql://my_username:my_password@localhost:5432/my_database \\\n       -s osm \\\n       -t waters\n       -m overwrite\n\n## Reproject a GeoJSON from 4326 to 3857 -\u003e PostGIS ##\npopgis --input spain.geojson \\\n       --uri postgresql://my_username:my_password@localhost:5432/my_database \\\n       --schema osm \\\n       --table waters \\\n       --srid 4326 \\\n       --reproject 3857\n\npopgis --input andalucia-latest.osm.pbf\n       --uri postgresql://my_username:my_password@localhost:5432/my_database \\\n       --schema osm \\\n       --table andalucia\n\n```\n\n#### Modes\nThe **overwrite** mode will delete existing table if name of schema/table is the same and will write into the new table. The **fail** mode, it ensures that if the table already exists in the database, the job will fail to prevent data loss.\n\n## Benchmarks\nAlthough non extensive, the benchmarking shows **PopGIS is twice faster than ogr2ogr**. This is most noticeable with large files.\n\n### ShapeFile\n\n| file size |  `popgis` took | `ogr2ogr` took | environment |\n|-----------|----------------|----------------|-------------|\n| 1.2GB     | **36sec**      | 1min 15sec     | local [PostGIS](https://hub.docker.com/r/kartoza/postgis/)       | \n| 1.2GB     | **36min**      | 1h 14min       | virtual machine ([n2-standard-4](https://cloud.google.com/compute/docs/general-purpose-machines)) [PostGIS](https://hub.docker.com/r/kartoza/postgis/) |\n\n\u003e The file used for this test can be found [here](https://osmdata.openstreetmap.de/data/water-polygons.html).\n\n### GeoJSON\n\n| file size |  `popgis` took | `ogr2ogr` took | environment |\n|-----------|----------------|----------------|-------------|\n| 103.9MB   | **2sec**       | 5sec           | local [PostGIS](https://hub.docker.com/r/kartoza/postgis/)       | \n| 103.9MB   | **2min 14sec** | 5min           | virtual machine ([n2-standard-4](https://cloud.google.com/compute/docs/general-purpose-machines)) [PostGIS](https://hub.docker.com/r/kartoza/postgis/) |\n\n\u003e The file used for this test can be found [here](https://data.cityofnewyork.us/City-Government/NYC-Street-Centerline-CSCL-/exjm-f27b).\n\n### OsmPBF\nComing soon.\n\n## Future implementations\n\n* Allow filtering based on a key value pair.\n* Add GeoParquet support.\n* From PostGIS to GeoJSON/ShapeFile.\n* Reintroduce the append mode (temporarily removed in `v0.4.0` due to inconsistent results).\n* Examples to pipe the standard output of `what-osm-pbf` with `PopGIS` as input.\n\n## Limitations\n\n* PopGIS does not currently support nested GeoJSON properties.\n* When using `osm.pbf`, use the smallest Geofabrik areas to get the best performance - try using it in conjuction with [`what-osm-pbf`](https://crates.io/crates/what-osm-pbf) CLI.\n\n## License\nSee [`LICENSE`](./LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjjcfrancisco%2Fpopgis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjjcfrancisco%2Fpopgis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjjcfrancisco%2Fpopgis/lists"}