{"id":34957839,"url":"https://github.com/shadz3rg/ru_address","last_synced_at":"2025-12-26T22:28:28.070Z","repository":{"id":49777461,"uuid":"106952345","full_name":"shadz3rg/ru_address","owner":"shadz3rg","description":"Обработка ГАР (ex. ФИАС/КЛАДР), в т.ч. конвертация в SQL/CSV/TSV","archived":false,"fork":false,"pushed_at":"2024-06-02T15:25:27.000Z","size":64,"stargazers_count":51,"open_issues_count":0,"forks_count":17,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-06-02T17:01:58.306Z","etag":null,"topics":["csv","gis","sql","tsv","xml"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shadz3rg.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-14T18:27:35.000Z","updated_at":"2024-06-02T15:22:56.000Z","dependencies_parsed_at":"2024-06-02T17:01:02.097Z","dependency_job_id":"69256b98-820b-4383-8417-cdbda4932b37","html_url":"https://github.com/shadz3rg/ru_address","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/shadz3rg/ru_address","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shadz3rg%2Fru_address","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shadz3rg%2Fru_address/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shadz3rg%2Fru_address/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shadz3rg%2Fru_address/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shadz3rg","download_url":"https://codeload.github.com/shadz3rg/ru_address/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shadz3rg%2Fru_address/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28062816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-26T02:00:06.189Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","gis","sql","tsv","xml"],"created_at":"2025-12-26T22:28:26.493Z","updated_at":"2025-12-26T22:28:28.061Z","avatar_url":"https://github.com/shadz3rg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"ru_address: обработка БД ГАР (ex. ФИАС/КЛАДР)\n=============================================\n\n| Утилита для конвертации схемы, ключей и данных выгрузки ГАР под импорт в БД **MySQL/PostgreSQL/ClickHouse**.\nБлагодаря SAX парсеру потребляет небольшое количество ресурсов (не более 50 мб памяти).\n\nУстановка\n---------\nТребуется Python 3.6+\n::\n\n    $ git clone https://github.com/shadz3rg/ru_address.git \u0026\u0026 cd ru_address \n    $ # deprecated python setup.py install\n    $ ~/.local/bin/pip install .\n\nИспользование\n-------------\n\n| Для начала необходимо скачать актуальную **XSD** схему и **XML** выгрузку с данными ГАР, распаковать на диск.\nВсе нужные файлы публикуются на оф. сайте ФНС России https://fias.nalog.ru/Frontend\n\n| Установка пакета дает доступ к исполняемому файлу ``ru_address`` и его под-командам:\n\nКонвертация схемы:\n^^^^^^^^^^^^^^^^^^\n\nПоддерживаемые форматы схемы и их параметры\n\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\n.. list-table::\n   :header-rows: 1\n\n   * - Формат\n     - ENV параметры\n   * - MySQL (mysql)\n     - | ``RA_INCLUDE_DROP`` - Добавить ``DROP IF EXIST`` (по умолчанию *\"1\"*)\n       | ``RA_TABLE_ENGINE`` - Движок таблицы (по умолчанию *\"MyISAM\"*)\n   * - PostgreSQL (psql)\n     - | ``RA_INCLUDE_DROP`` - Добавить ``DROP IF EXIST`` (по умолчанию *\"1\"*)\n   * - ClickHouse (ch)\n     - | ``RA_INCLUDE_DROP`` - Добавить ``DROP IF EXIST`` (по умолчанию *\"1\"*)\n       | ``RA_TABLE_ENGINE`` - Движок таблицы (по умолчанию *\"MergeTree\"*)\n\nОписание\n\"\"\"\"\"\"\"\"\n::\n\n    $ ru_address schema --help\n    Usage: python -m ru_address.command schema [OPTIONS] SOURCE_PATH OUTPUT_PATH\n\n    Convert XSD content into target platform schema definitions.\n    Get latest schema at https://fias.nalog.ru/docs/gar_schemas.zip\n    Generate file per table if `output_path` argument is existing directory;\n    else dumps all tables into single file.\n\n    Options:\n      --target [mysql|psql|ch]  Target schema format\n      -t, --table TEXT          Limit table list to process\n      --no-keys                 Exclude keys \u0026\u0026 column index\n      --help                    Show this message and exit.\n\nПримеры\n\"\"\"\"\"\"\"\n.. code-block:: shell\n\n  # Простейший вариант\n  $ ru_address schema /путь/к/файлам /путь/для/экспорта\n  # С ограниченным списком таблиц\n  $ ru_address schema /путь/к/файлам /путь/для/экспорта --table=ADDHOUSE_TYPES --table=HOUSE_TYPES\n  # Экспорт всех таблиц в один файл\n  $ ru_address schema /путь/к/файлам /путь/для/экспорта/schema.sql\n\nКонвертация данных:\n^^^^^^^^^^^^^^^^^^^\n\nПоддерживаемые форматы дампа и их параметры\n\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\n.. list-table::\n   :header-rows: 1\n\n   * - Формат\n     - ENV параметры\n   * - MySQL (mysql)\n     - | ``RA_BATCH_SIZE`` - Разделить ``INSERT INTO`` на части по n-записей (по умолчанию *\"500\"*)\n       | ``RA_SQL_ENCODING`` - Кодировка данных (по умолчанию *\"utf8mb4\"*)\n   * - PostgreSQL (psql)\n     - | ``RA_BATCH_SIZE`` - Разделить ``INSERT INTO`` на части по n-записей (по умолчанию *\"500\"*)\n   * - CSV (csv)\n     - —\n   * - TSV (tsv)\n     - —\n\nОписание\n\"\"\"\"\"\"\"\n::\n\n    $ ru_address dump --help\n    Usage: python -m ru_address.command dump [OPTIONS] SOURCE_PATH OUTPUT_PATH [SCHEMA_PATH]\n\n    Convert XML content into target platform dump files.\n    Get latest data at https://fias.nalog.ru/Frontend\n\n    Options:\n      --target [mysql|psql|csv|tsv]   Target dump format\n      -r, --region TEXT               Limit region list to process\n      -t, --table TEXT                Limit table list to process\n      -m, --mode [direct|per_region|per_table|region_tree]\n                                      Dump output mode (only if `output_path` argument is a valid directory)\n      --help                          Show this message and exit.\n\nПримеры\n\"\"\"\"\"\"\"\n.. code-block:: shell\n\n  # Простейший вариант\n  $ ru_address dump /путь/к/файлам /путь/для/сохранения /путь/к/xsd-схеме\n  # С указанием режима вывода:\n  # - direct        Вывод в единый файл, используется по умолчанию если /путь/для/сохранения - файл\n  # - per_region    Вывод данных в единый файл на каждый регион\n  # - per_table     Вывод данных в единый файл на каждую таблицу\n  # - region_tree   Вывод повторяет исходную структуру файлов (по умолчанию)\n  $ ru_address dump /путь/к/файлам /путь/для/сохранения /путь/к/xsd-схеме --mode=per_table\n  # С ограниченным списком таблиц\n  $ ru_address dump /путь/к/файлам /путь/для/сохранения /путь/к/xsd-схеме --table=ADDHOUSE_TYPES --table=HOUSE_TYPES\n  # С ограниченным списком регионов\n  $ ru_address dump /путь/к/файлам /путь/для/сохранения /путь/к/xsd-схеме --region=01 --region=02\n  # Экспорт всех таблиц в один файл\n  $ ru_address dump /путь/к/файлам /путь/для/экспорта/dump.sql /путь/к/xsd-схеме\n\nFAQ\n---------\nКак передать ENV параметры в приложение?\n   .. code-block:: shell\n\n      # Linux (стандартные ENV переменные, в т.ч. на уровне системы)\n      $ RA_BATCH_SIZE=1000 RA_SQL_ENCODING=utf8 ru_address dump ...\n      # Linux / Windows\n      $ ru_address -e RA_BATCH_SIZE 1000 -e RA_SQL_ENCODING utf8 dump ...  \n     \nКак импортировать CSV данные?\n  .. code-block:: shell\n\n      # На примере MariaDB:\n      $ MariaDB [ru_address]\u003e LOAD DATA INFILE '/var/dump/ADDHOUSE_TYPES.csv' INTO TABLE ADDHOUSE_TYPES FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\\r\\n';\n\nКак импортировать TSV данные?\n  .. code-block:: shell\n\n      # На примере MariaDB:\n      $ MariaDB [ru_address]\u003e LOAD DATA INFILE '/var/dump/ADDHOUSE_TYPES.tsv' INTO TABLE ADDHOUSE_TYPES LINES TERMINATED BY '\\r\\n';\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshadz3rg%2Fru_address","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshadz3rg%2Fru_address","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshadz3rg%2Fru_address/lists"}