https://github.com/jsanz/osm2es

Utility to upload data into Elasticsearch from OSM pbf files
https://github.com/jsanz/osm2es

Last synced: 11 months ago
JSON representation

Utility to upload data into Elasticsearch from OSM pbf files

• Host: GitHub
• URL: https://github.com/jsanz/osm2es
• Owner: jsanz
• Created: 2022-02-09T18:05:45.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2022-03-14T15:20:04.000Z (about 4 years ago)
• Last Synced: 2025-03-01T17:23:45.947Z (over 1 year ago)
• Language: Python
• Size: 23.4 KB
• Stars: 1
• Watchers: 4
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# OSM to Elasticsearch

## Set up

Create a virtual environment and install the libraries with `pip install -r requirements.txt`

## Run

```text

$ python scripts/osm2es.py --help

usage: python3 osm2es.py

Imports OSM data into Elasticsearch

positional arguments:

  input_file            OSM input PBF file

optional arguments:

  -h, --help            show this help message and exit

  --index-name INDEX_NAME

                        Index name

  --es-url ES_URL       Elasticsearch url (default: http://localhost:9200)

  --es-user ES_USER     Elasticsearch user (default: elastic)

  --es-pwd ES_PWD       Elasticsearch password (default: changeme)

  --es-replicas ES_REPLICAS

                        Index replicas (default: 0)

  --workers WORKER_COUNT

                        Number of worker threads to run (default: 1)

  --cache-size DB_CACHE_SIZE

                        Number of documents to accumulate before sending to ES (default: 5000)

  -v                    Enable verbose output.

```

**Notes**:

* The script will overwrite the index passed so be sure you are OK with that

* By default it will use a single worker in parallel with the data read. You may want to try but 6 to 8 workers should work best