{"id":13425104,"url":"https://github.com/omniscale/imposm3","last_synced_at":"2025-03-15T19:32:38.079Z","repository":{"id":10390101,"uuid":"12539646","full_name":"omniscale/imposm3","owner":"omniscale","description":"Imposm imports OpenStreetMap data into PostGIS","archived":false,"fork":false,"pushed_at":"2025-02-24T12:13:44.000Z","size":7369,"stargazers_count":737,"open_issues_count":77,"forks_count":157,"subscribers_count":44,"default_branch":"master","last_synced_at":"2025-02-24T12:40:04.298Z","etag":null,"topics":["database","geo","go","imposm"],"latest_commit_sha":null,"homepage":"http://imposm.org/docs/imposm3/latest/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/omniscale.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-09-02T12:34:08.000Z","updated_at":"2025-02-24T12:13:48.000Z","dependencies_parsed_at":"2023-01-13T16:24:26.198Z","dependency_job_id":"e9d5803b-132a-4420-9c60-76bdf0e97b50","html_url":"https://github.com/omniscale/imposm3","commit_stats":{"total_commits":872,"total_committers":30,"mean_commits":"29.066666666666666","dds":0.06307339449541283,"last_synced_commit":"33e15acf7b3e0df1c4af25c5eda312842ef0cbb4"},"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omniscale%2Fimposm3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omniscale%2Fimposm3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omniscale%2Fimposm3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omniscale%2Fimposm3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/omniscale","download_url":"https://codeload.github.com/omniscale/imposm3/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243781845,"owners_count":20347156,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","geo","go","imposm"],"created_at":"2024-07-31T00:01:04.778Z","updated_at":"2025-03-15T19:32:38.072Z","avatar_url":"https://github.com/omniscale.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"Imposm\n======\n\nImposm is an importer for OpenStreetMap data. It reads PBF files and imports the data into PostgreSQL/PostGIS. It can also automatically update the database with the latest changes from OSM.\n\nIt is designed to create databases that are optimized for rendering (i.e. generating tiles or for WMS services).\n\nImposm \u003e=3 is written in Go and it is a complete rewrite of the previous Python implementation.\nConfigurations/mappings and cache files are not compatible with Imposm 2, but they share a similar architecture.\n\nThe development of Imposm is sponsored by [Omniscale](https://omniscale.com/).\n\n*Imposm is in production use by the authors. It is actively maintained, with a focus on resolving future incompatibilities with dependencies such as PostGIS. However, there is no capacity for end-user support, and no new features will be developed beyond its existing scope.*\n\n\nFeatures\n--------\n\n* High-performance\n* Diff support\n* Custom database schemas\n* Generalized geometries\n\n\n### In detail\n\n\n- High performance:\n  Parallel from the ground up. It distributes parsing and processing to all available CPU cores.\n\n- Custom database schemas:\n  Creates tables for different data types. This allows easier styling and better performance for rendering in WMS or tile services.\n\n- Unify values:\n  For example, the boolean values `1`, `on`, `true` and `yes` all become ``TRUE``.\n\n- Filter by tags and values:\n  Only import data you are going to render/use.\n\n- Efficient nodes cache:\n  It is necessary to store all nodes to build ways and relations. Imposm uses a file-based key-value database to cache this data.\n\n- Generalized tables:\n  Automatically creates tables with lower spatial resolutions, perfect for rendering large road networks in low resolutions.\n\n- Limit to polygons:\n  Limit imported geometries to polygons from GeoJSON, for city/state/country imports.\n\n- Easy deployment:\n  Single binary with only runtime dependencies to common libs (GEOS and LevelDB).\n\n- Automatic OSM updates:\n  Includes background service (`imposm run`) that automatically downloads and imports the latest OSM changes.\n\n- Route relations:\n  Import all relation types including routes.\n\n- Support for table namespace (PostgreSQL schema)\n\n\nPerformance\n-----------\n\n* Imposm makes full use of all available CPU cores\n* Imposm uses bulk inserts into PostgreSQL with `COPY FROM`\n* Imposm uses efficient intermediate caches for reduced IO load during ways and relations building\n\n\nAn import in diff-mode on a Hetzner AX102 server (AMD Ryzen 9 7950X3D, 256GB RAM and NVMe storage) of a 78GB planet PBF (2024-01-29) with generalized tables and spatial indices, etc. takes around 7:30h. This is for an import that is ready for minutely updates. The non-diff mode is even faster.\n\nIt's recommended that the memory size of the server is roughly twice the size of the PBF extract you are importing. For example: You should have 192GB RAM or more for a current (2024) 78GB planet file, 8GB for a 4GB regional extract, etc.\nImports with spinning disks will take significantly longer and are not recommended.\n\n\nInstallation\n------------\n\n### Binary\n\n[Binary releases are available at GitHub.](https://github.com/omniscale/imposm3/releases)\n\nThese builds are for x86 64bit Linux and require *no* further dependencies. Download, untar and start `imposm`.\nBinaries are compatible with Debian 10 and other distributions from 2022 or\nnewer. You can build from source if you need to support older distributions.\n\n### Source\n\nThere are some dependencies:\n\n#### Compiler\n\nYou need [Go](http://golang.org). 1.17 or higher is recommended.\n\n#### C/C++ libraries\n\nOther dependencies are [libleveldb][] and [libgeos][].\nImposm was tested with recent versions of these libraries, but you might succeed with older versions.\nGEOS \u003e=3.2 is recommended, since it became much more robust when handling invalid geometries.\n\n\n[libleveldb]: https://github.com/google/leveldb/\n[libgeos]: https://libgeos.org/\n\n#### Compile\n\nThe quickest way to install Imposm is to call:\n\n    go install github.com/omniscale/imposm3/cmd/imposm@latest\n\nThis will download, compile and install Imposm to `~/go/bin/imposm`. You can change the location by setting the `GOBIN` environment.\n\nThe recommended way for installation is:\n\n    git clone https://github.com/omniscale/imposm3.git\n    cd imposm3\n    make build\n\n`make build` will build Imposm into your local path and it will add version information to your binary.\n\nYou can also directly use go to build or install imposm with `go build ./cmd/imposm`. However, this will not set the version information.\n\nGo compiles to static binaries and so Imposm has no runtime dependencies to Go.\nJust copy the `imposm` binary to your server for deployment. The C/C++ libraries listed above are still required though.\n\nSee also `Dockerfile` for instructions on how to build standalone binary packages for Linux.\n\n#### LevelDB\n\nFor better performance you should use LevelDB \u003e1.21. You can still build with support for 1.21 with ``go build -tags=\"ldbpre121\"`` or ``LEVELDB_PRE_121=1 make build``.\n\n\nUsage\n-----\n\n`imposm` has multiple subcommands. Use `imposm import` for basic imports.\n\nFor a simple import:\n\n    imposm import -connection postgis://user:password@host/database \\\n        -mapping mapping.json -read /path/to/osm.pbf -write\n\nYou need a JSON file with the target database mapping. See `example-mapping.json` to get an idea what is possible with the mapping.\n\nImposm creates all new tables inside the `import` table schema. So you'll have `import.osm_roads` etc. You can change the tables to the `public` schema:\n\n    imposm import -connection postgis://user:passwd@host/database \\\n        -mapping mapping.json -deployproduction\n\n\nYou can write some options into a JSON configuration file:\n\n    {\n        \"cachedir\": \"/var/local/imposm\",\n        \"mapping\": \"mapping.json\",\n        \"connection\": \"postgis://user:password@localhost:port/database\"\n    }\n\nTo use that config:\n\n    imposm import -config config.json [args...]\n\nFor more options see:\n\n    imposm import -help\n\n\nNote: TLS/SSL support is disabled by default due to the lack of renegotiation support in Go's TLS implementation. You can re-enable encryption by setting the `PGSSLMODE` environment variable or the `sslmode` connection option to `require` or `verify-full`, eg: `-connect postgis://host/dbname?sslmode=require`. You will need to disable renegotiation support on your server to prevent connection errors on larger imports. You can do this by setting `ssl_renegotiation_limit` to 0 in your PostgreSQL server configuration.\n\n\nDocumentation\n-------------\n\nThe latest documentation can be found here: \u003chttp://imposm.org/docs/imposm3/latest/\u003e\n\nSupport\n-------\n\nThere is a [mailing list at Google Groups](http://groups.google.com/group/imposm) for all questions. You can subscribe by sending an email to: `imposm+subscribe@googlegroups.com`\n\nFor commercial support [contact Omniscale](http://omniscale.com/contact).\n\nDevelopment\n-----------\n\nThe source code is available at: \u003chttps://github.com/omniscale/imposm3/\u003e\n\nYou can report any issues at: \u003chttps://github.com/omniscale/imposm3/issues\u003e\n\nLicense\n-------\n\nImposm is released as open source under the Apache License 2.0. See LICENSE.\n\nAll dependencies included as source code are released under a BSD-ish license. See LICENSE.dep.\n\nAll dependencies included in binary releases are released under a BSD-ish license except the GEOS package.\nThe GEOS package is released as LGPL3 and is linked dynamically. See LICENSE.bin.\n\n\n### Test ###\n\n#### Unit tests ####\n\nTo run all unit tests:\n\n    make test-unit\n\n\n#### System tests ####\n\nThere are system test that import and update OSM data and verify the database content.\nYou need `osmosis` to create the test PBF files.\nThere is a Makefile that creates all test files if necessary and then runs the test itself.\n\n    make test\n\nCall `make test-system` to skip the unit tests.\n\nWARNING: It uses your local PostgreSQL database (`imposm_test_import`, `imposm_test_production` and `imposm_test_backup` schema). Change the database with the standard `PGDATABASE`, `PGHOST`, etc. environment variables.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomniscale%2Fimposm3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fomniscale%2Fimposm3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomniscale%2Fimposm3/lists"}