{"id":17136568,"url":"https://github.com/matkoniecz/osm_iterator","last_synced_at":"2025-04-23T04:26:50.285Z","repository":{"id":57449723,"uuid":"138583058","full_name":"matkoniecz/osm_iterator","owner":"matkoniecz","description":"loads .osm file and allows to call function on all OSM objects in dataset","archived":false,"fork":false,"pushed_at":"2025-03-14T17:11:16.000Z","size":26,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-18T17:47:27.022Z","etag":null,"topics":["openstreetmap","osm","pip","python","python-library","python3","python3-library"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matkoniecz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-25T11:01:20.000Z","updated_at":"2025-03-14T17:11:20.000Z","dependencies_parsed_at":"2024-10-14T20:04:46.643Z","dependency_job_id":"72c69f4f-b2ee-4f20-9f56-a349c73ce630","html_url":"https://github.com/matkoniecz/osm_iterator","commit_stats":{"total_commits":49,"total_committers":1,"mean_commits":49.0,"dds":0.0,"last_synced_commit":"7f65f1e96933af1c53ea5c7cd7794d39be1485b9"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matkoniecz%2Fosm_iterator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matkoniecz%2Fosm_iterator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matkoniecz%2Fosm_iterator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matkoniecz%2Fosm_iterator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matkoniecz","download_url":"https://codeload.github.com/matkoniecz/osm_iterator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250369160,"owners_count":21419195,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["openstreetmap","osm","pip","python","python-library","python3","python3-library"],"created_at":"2024-10-14T20:04:43.649Z","updated_at":"2025-04-23T04:26:50.278Z","avatar_url":"https://github.com/matkoniecz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"This code loads .osm file and allows to call function on all OSM objects in dataset.\n\n# Installation\n\n`pip install osm-iterator`\n\nLikely `pip3 install osm-iterator` if `pip` points to Python2 pip.\n\nIt is distributed as an `osm_iterator` PyPI package.\n\n[![PyPI version](https://badge.fury.io/py/osm-iterator.svg)](https://pypi.org/project/osm-iterator/)\n\n# Usage example\n\n## Download data and show it\nThis usage example includes downloading data using `requests` library, that you may need to install (also available via pip).\n```\nfrom osm_iterator import osm_iterator\nimport requests\nimport os.path\n\ndef download_from_overpass(query, output_filepath):\n  print(query)\n  url = \"http://overpass-api.de/api/interpreter\"\n  r = requests.get(url, params={'data': query})\n  result = r.text\n  with open(output_filepath, 'w') as file:\n      file.write(str(result))\n\ndef show_places(element):\n    place_tag = element.get_tag_value(\"place\")\n    name_tag = element.get_tag_value(\"name\")\n    osm_object_url = element.get_link()\n    if place_tag != None:\n        print(name_tag, \"(\", place_tag, \") is \", osm_object_url)\n\nfilepath = \"places_in_Kraków.osm\"\nquery = \"\"\"\n[out:xml][timeout:2500];\narea[name='Kraków']-\u003e.searchArea;\n(\n  node[\"place\"](area.searchArea);\n  way[\"place\"](area.searchArea);\n  relation[\"place\"](area.searchArea);\n);\nout center;\n\"\"\"\n\nif os.path.isfile(filepath) == False:\n    download_from_overpass(query, filepath)\nosm = osm_iterator.Data(filepath)\nosm.iterate_over_data(show_places)\n```\n\n## Load data only\n\n```\nfrom osm_iterator import osm_iterator\n\nglobal osm_object_store\nosm_object_store = []\n\ndef record_objects(element):\n    global osm_object_store\n    print(element.element.tag, element.element.attrib['id'])\n    osm_object_store.append({\"type\": element.get_type(), \"id\": element.get_id()})\n\nfilepath = \"output.osm\"\nosm = osm_iterator.Data(filepath)\nosm.iterate_over_data(record_objects)\nfor entry in osm_object_store:\n    print(entry)\n```\n\n# Running tests\n\n```nosetests3``` or ```python3 -m unittest``` or ```python3 tests.py```\n\n# History\n\nDesign explanation: this code has deeply suboptimal handling of pretty much everything. For start, all data is loaded into memory and then duplicated in-memory dataset is created.\n\nAs result, attempt to process any large datasets will cause issues due to excessive memory consumption.\n\nThis situation is consequence of following facts\n\n* This code was written during my first attempt to process OSM data using Python\n* API allows (at least in theory) to painlessly switch to real iterator that is not loading all data into memory at once\n* So far this was good enough for my purposes so I had no motivation to spend time on improving something that is not a bottleneck\n\nThough, if someone has good ideas for improvements (especially in form of a working code) - comments and pull requests are welcomed.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatkoniecz%2Fosm_iterator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatkoniecz%2Fosm_iterator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatkoniecz%2Fosm_iterator/lists"}