{"id":19881414,"url":"https://github.com/giscience/osmatrix-processing","last_synced_at":"2025-07-04T16:34:24.811Z","repository":{"id":149844513,"uuid":"187789768","full_name":"GIScience/osmatrix-processing","owner":"GIScience","description":"No longer maintained: Processing of osmatrix in postgreSQL","archived":false,"fork":false,"pushed_at":"2019-05-24T11:49:48.000Z","size":102,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-01T02:46:45.033Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GIScience.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-21T07:59:46.000Z","updated_at":"2019-05-24T11:49:50.000Z","dependencies_parsed_at":"2023-09-21T01:04:04.872Z","dependency_job_id":null,"html_url":"https://github.com/GIScience/osmatrix-processing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/GIScience/osmatrix-processing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GIScience%2Fosmatrix-processing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GIScience%2Fosmatrix-processing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GIScience%2Fosmatrix-processing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GIScience%2Fosmatrix-processing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GIScience","download_url":"https://codeload.github.com/GIScience/osmatrix-processing/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GIScience%2Fosmatrix-processing/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263577256,"owners_count":23483130,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T17:14:10.662Z","updated_at":"2025-07-04T16:34:24.789Z","avatar_url":"https://github.com/GIScience.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OSMatrix processing \n\n## This Repository is no longer maintained!\n\n## Steps will be done once:\n\n### Scripts to Create OSMatrix Schema \n\n+ Excecute main_schema.sh, mention number of attribute tables you want\n    * Such as: \n        * ./main_schema 25\n\n### Processing Data for OSMatrix\n\n#### 1. Downloading\n\n+ Download the planet file from Planet.osm.org you want to process\n    + wget http://planet.openstreetmap.org/pbf/full-history/history-latest.osm.pbf\n\n#### 2. Extracting Region/Cutting/Bounding Box\n\n+ Download OSMconvert tool for Operating System you want\n    +  Available from http://wiki.openstreetmap.org/wiki/Osmconvert#Download\n+ Extracting region (Europe) using OSMConvert command\n    * osmconvert planet-file -b=rectangle coordinates (left, bottom, right, top) -o=outputfile\n    * osmconvert /data/OSMatrix/2016-09-01_full_history/history-latest.osm.pbf -b=-10.53538,34.66683,38.00957,61.07390 -o=/data/OSMatrix/extracted_regions/europe.pbf\n\n#### 3. Extracting Timestamps after cutting out Region from a OSM history file\n\n+ Download Osmium command line tool and install it from:\n    * http://osmcode.org/osmium-tool/\n+ Command to filter the timestamps\n    + Option 1 (osm2pgsql version 0.9 and above)\n        *Extract timestamps as pbf format, because osm2pgsql version 0.9 and above supports the importing of pbf files and provide all extra attributes\n    + Option 2 (osm2pgsql versions below 0.9)\n        * Importing pbf files does not work with --extra attributes and lacks information such as osm_timestamps etc. \n        * osm.bz2 format does work and provide all extra attributes while importing data, which can be accomplished directly in osmium time-filter command by specifying output file format as osm.bz2 (this way you can avoid converting step for all timestamps)\n            * osmium time-filter input-file timestamp YY-MM-DDTHH:MM:SSZ-o output-file (osm, pbf, osm.bz2)\n            * osmium-tool-1.3.1/build/osmium time-filter extracted_regions/europe.pbf 2006-01-01T00:00:00Z -o europe_ts/2006-01-01.pbf\n\n#### Note:\n##### (Convert files in case if you already have osm or osm.bz2 file formats or want to work with them instead of pbf) \n\n+ All timestamps need to convert from .pbf to .osm or .osm.bz2 format if extracted as pbf files using OSMConvert tool. \n    * osmconvert input-file.pbf -o=output-file.osm\n        * osmconvert /data/OSMatrix/europe_ts/2006-01-01.pbf -o=/data/OSMatrix/europe_ts/2006-01-01.osm\n    or \n        * osmconvert /data/OSMatrix/Europe_ts/2006-01-01.pbf | bzip2 \u003e 2006-01-01.osm.bz2\n\n+ Dumping cells table\n    * cells_osmatrix_europe.sql file is available in the repository (cells_dump directory) which is created by a pg_dump utility\n    * To dump a single table \n        * pg_dump -Fp --data-only -t tablename -d databasename \u003e dumpfile.sql\n    * To import/restore\n        * psql databasename \u003c dumpfile.sql\n        * psql osmatrix3000 \u003c cells_osmatrix_europe.sql\n\n\n## Steps will be repeated for each timestamp:\n\n#### 4. Importing Data to Database\n\n+ Using osm2pgsql tool\n    * osm2pgsql -d databasename -U username -W --hstore-all --style --extra-attributes -C --number-processes input-file\n    * osm2pgsql -d osmatrix-DB -U osmatrix -W --hstore-all --style /usr/share/osm2pgsql/default.style --extra-attributes -C 36000 --number-processes 11 /data/OSMatrix/Europe_ts/2006-01-01.pbf\n\n#### 5. Processing\n\n+ Use an executable jar file build from this project (https://gitlab.com/giscience/osmatrix_processing/) with the neccessary attributes\n    * java -server -Xms -jar [.jar-file] -oh [host-name] -od [database-name] -ou [username] -op [‘password’] -t -c -f -v [version number, the number to put here is the number of available timestamps + 1] -ts \"YYYY-MM-DD HH:MM:SS\"\n    * java -server -Xms32000m -jar osmatrix_processing_2016-09-16.jar -oh server.name.something.de:5432 -od osmatrix-DB -ou osmatrix -op '' -t 10 -c 1000 -f 1000 -v 1 -ts \"2006-01-01 00:00:00\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgiscience%2Fosmatrix-processing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgiscience%2Fosmatrix-processing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgiscience%2Fosmatrix-processing/lists"}