{"id":36822470,"url":"https://github.com/kornev/kornev-gin-app","last_synced_at":"2026-01-12T14:01:53.223Z","repository":{"id":181138310,"uuid":"451764117","full_name":"kornev/kornev-gin-app","owner":"kornev","description":"A JSON-RPC 2.0 operation based API for Hive Metastore which used PostgreSQL, Hive JDBC and HDFS.","archived":false,"fork":false,"pushed_at":"2022-05-04T05:21:57.000Z","size":71,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-01-26T11:36:32.015Z","etag":null,"topics":["big-data","clojure","data-catalog","data-discovery","hdfs","hive","metadata"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kornev.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-01-25T06:38:05.000Z","updated_at":"2023-01-10T22:20:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"ce680a94-94b7-48d9-b83a-7319f54694ac","html_url":"https://github.com/kornev/kornev-gin-app","commit_stats":null,"previous_names":["kornev/kornev-gin-app"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kornev/kornev-gin-app","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kornev%2Fkornev-gin-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kornev%2Fkornev-gin-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kornev%2Fkornev-gin-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kornev%2Fkornev-gin-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kornev","download_url":"https://codeload.github.com/kornev/kornev-gin-app/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kornev%2Fkornev-gin-app/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28340212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T12:22:26.515Z","status":"ssl_error","status_checked_at":"2026-01-12T12:22:10.856Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","clojure","data-catalog","data-discovery","hdfs","hive","metadata"],"created_at":"2026-01-12T14:01:24.570Z","updated_at":"2026-01-12T14:01:53.129Z","avatar_url":"https://github.com/kornev.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gin\n\n## Using Gin\n\nMethod **table/describe**:\n\n```shell\ncurl -s -X POST 'http://127.0.0.1:8080/rpc' \\\n  -d '{\"id\": 1, \"jsonrpc\": \"2.0\", \"method\": \"table/describe\", \"params\": [\"processing\", \"user_agg\"]}' \\\n  -H 'content-type: application/json'\n```\n\n```json\n{\n  \"id\": 1,\n  \"jsonrpc\": \"2.0\",\n  \"result\": {\n    \"DB_ID\": 16,\n    \"TBL_ID\": 59,\n    \"SD_ID\": 59,\n    \"CD_ID\": 59,\n    \"DB_NAME\": \"processing\",\n    \"TBL_NAME\": \"user_agg\",\n    \"PART_SPEC\": [\n      {\n        \"INTEGER_IDX\": 0,\n        \"PKEY_NAME\": \"dt\",\n        \"PKEY_TYPE\": \"string\"\n      }\n    ],\n    \"DB_LOCATION_URI\": \"hdfs://ensime/warehouse/tablespace/managed/hive/processing.db\",\n    \"LOCATION\": \"hdfs://ensime/warehouse/tablespace/external/hive/processing.db/user_agg\"\n  }\n}\n```\n\nMethod **partition/list**:\n\n```shell\ncurl -s -X POST 'http://127.0.0.1:8080/rpc' \\\n  -d '{\"id\": 3, \"jsonrpc\": \"2.0\", \"method\": \"partition/list\", \"params\": [198, 2]}' \\\n  -H 'content-type: application/json'\n```\n\n```json\n{\n    \"id\": 3,\n    \"jsonrpc\": \"2.0\",\n    \"result\": [\n        {\n            \"TBL_ID\": 198,\n            \"SD_ID\": 13928,\n            \"PART_ID\": 13717,\n            \"CREATE_TIME\": 1608131750,\n            \"CD_ID\": 198,\n            \"PART_NAME\": \"dt=2020-11-22\",\n            \"LOCATION\": \"hdfs://ensime/warehouse/tablespace/external/hive/incoming_data.db/mediahills/dt=2020-11-22\",\n            \"PART_SPEC_VAL\": [\n                {\n                    \"INTEGER_IDX\": 0,\n                    \"PKEY_NAME\": \"dt\",\n                    \"PKEY_TYPE\": \"date\",\n                    \"PART_KEY_VAL\": \"2020-11-22\"\n                }\n            ]\n        },\n        {\n            \"TBL_ID\": 198,\n            \"SD_ID\": 40501,\n            \"PART_ID\": 40258,\n            \"CREATE_TIME\": 0,\n            \"CD_ID\": 198,\n            \"PART_NAME\": \"dt=2021-02-13\",\n            \"LOCATION\": \"hdfs://ensime/warehouse/tablespace/external/hive/incoming_data.db/mediahills/dt=2021-02-13\",\n            \"PART_SPEC_VAL\": [\n                {\n                    \"INTEGER_IDX\": 0,\n                    \"PKEY_NAME\": \"dt\",\n                    \"PKEY_TYPE\": \"date\",\n                    \"PART_KEY_VAL\": \"2021-02-13\"\n                }\n            ]\n        }\n    ]\n}\n```\n\nMethod **partition/parent**:\n\n```shell\ncurl -s -X POST 'http://127.0.0.1:8080/rpc' \\\n  -d '{\"id\": 4, \"jsonrpc\": \"2.0\", \"method\": \"partition/parent\", \"params\": [59, 1626746198]}' \\\n  -H 'content-type: application/json'\n```\n\n```json\n{\n    \"id\": 4,\n    \"jsonrpc\": \"2.0\",\n    \"result\": {\n        \"TBL_ID\": 59,\n        \"SD_ID\": 52731,\n        \"PART_ID\": 52481,\n        \"CREATE_TIME\": 1626659658,\n        \"CD_ID\": 59,\n        \"PART_NAME\": \"dt=2021-07-18\",\n        \"LOCATION\": \"hdfs://ensime/warehouse/tablespace/external/hive/processing.db/user_agg/dt=2021-07-18\",\n        \"PART_SPEC_VAL\": [\n            {\n                \"INTEGER_IDX\": 0,\n                \"PKEY_NAME\": \"dt\",\n                \"PKEY_TYPE\": \"string\",\n                \"PART_KEY_VAL\": \"2021-07-18\"\n            }\n        ]\n    }\n}\n```\n\nMethod **partition/find**:\n\n```shell\ncurl -s -X POST 'http://127.0.0.1:8080/rpc' \\\n  -d '{\"id\": 61966, \"jsonrpc\": \"2.0\", \"method\": \"partition/find\", \"params\": [248, [\"2021\", \"08\", \"09\", \"15\"]]}' \\\n  -H 'content-type: application/json'\n```\n\n```json\n{\n  \"id\": 61966,\n  \"jsonrpc\": \"2.0\",\n  \"result\": {\n    \"TBL_ID\": 248,\n    \"SD_ID\": 54309,\n    \"PART_ID\": 54059,\n    \"CREATE_TIME\": 1628514173,\n    \"CD_ID\": 248,\n    \"PART_NAME\": \"year=2021/month=08/day=09/hour=15\",\n    \"LOCATION\": \"hdfs://ensime/data/logs/siren/year=2021/month=08/day=09/hour=15\",\n    \"PART_SPEC_VAL\": [\n      {\n        \"INTEGER_IDX\": 0,\n        \"PKEY_NAME\": \"year\",\n        \"PKEY_TYPE\": \"string\",\n        \"PART_KEY_VAL\": \"2021\"\n      },\n      {\n        \"INTEGER_IDX\": 1,\n        \"PKEY_NAME\": \"month\",\n        \"PKEY_TYPE\": \"string\",\n        \"PART_KEY_VAL\": \"08\"\n      },\n      {\n        \"INTEGER_IDX\": 2,\n        \"PKEY_NAME\": \"day\",\n        \"PKEY_TYPE\": \"string\",\n        \"PART_KEY_VAL\": \"09\"\n      },\n      {\n        \"INTEGER_IDX\": 3,\n        \"PKEY_NAME\": \"hour\",\n        \"PKEY_TYPE\": \"string\",\n        \"PART_KEY_VAL\": \"15\"\n      }\n    ]\n  }\n}\n```\n\nMethod **partition/attach**:\n\n```clojure\n(echo\n    (rpc :partition/attach [#_(TBL_ID) 261\n                            #_(PART_KEY_VALS) [\"2019-11-22\" \"moscow\" 13]\n                            #_(DATA_LOCATION) \"/user/pmp/input\"]))\n```\n\n```clojure\n{:id 52445,\n :jsonrpc \"2.0\",\n :result {:PART_COLS [\"dt\" \"city\" \"num\"],\n          :PART_KEY_VALS [\"2019-11-22\" \"moscow\" 13],\n          :TBL_NAME \"test01\",\n          :CD_ID 261,\n          :ADD_PARTITION_SQL \"ALTER TABLE processing.test01 ADD IF NOT EXISTS PARTITION(dt=date '2019-11-22', city='moscow', num=13)\",\n          :DATA_LOCATION \"/user/pmp/input\",\n          :DB_NAME \"processing\",\n          :PART_LOCATION \"/user/pmp/hive/dt=2019-11-22/city=moscow/num=13\",\n          :DB_ID 16,\n          :TBL_ID 261,\n          :LOCATION \"hdfs://ensime/user/pmp/hive\",\n          :SD_ID 64654,\n          :PART_TYPES [\"date\" \"string\" \"int\"]}}\n```\n\n```text\n+-----------------------------------+\n|             partition             |\n+-----------------------------------+\n| dt=2019-11-22/city=moscow/num=13  |\n+-----------------------------------+\n1 rows selected (0.278 seconds)\n\n+------------+--------------+-------------+--------------+-------------+\n| test01.id  | test01.name  |  test01.dt  | test01.city  | test01.num  |\n+------------+--------------+-------------+--------------+-------------+\n| 1          | \"ya.ru\"      | 2019-11-22  | moscow       | 13          |\n| 2          | \"yandex.ru\"  | 2019-11-22  | moscow       | 13          |\n+------------+--------------+-------------+--------------+-------------+\n```\n\nMethod **partition/detach**:\n\n```clojure\n(echo\n (rpc :partition/detach [#_(TBL_ID) 261\n                         #_(PART_KEY_VALS) [\"2019-11-22\" \"moscow\" 13]]))\n```\n\n```clojure\n{:id 95792,\n :jsonrpc \"2.0\",\n :result {:TBL_NAME \"test01\",\n          :CD_ID 261,\n          :PART_KEY_VAL_STR \"2019-11-22moscow13\",\n          :CREATE_TIME 1645776871,\n          :PART_SPEC_VAL [{:INTEGER_IDX 0,\n                           :PKEY_NAME \"dt\",\n                           :PKEY_TYPE \"date\",\n                           :PART_KEY_VAL \"2019-11-22\"}\n                          {:INTEGER_IDX 1,\n                           :PKEY_NAME \"city\",\n                           :PKEY_TYPE \"string\",\n                           :PART_KEY_VAL \"moscow\"}\n                          {:INTEGER_IDX 2,\n                           :PKEY_NAME \"num\",\n                           :PKEY_TYPE \"int\",\n                           :PART_KEY_VAL \"13\"}],\n          :DB_NAME \"processing\",\n          :PART_NAME \"dt=2019-11-22/city=moscow/num=13\",\n          :PART_ID 64453,\n          :PART_LOCATION \"/user/pmp/hive/dt=2019-11-22/city=moscow/num=13\",\n          :TBL_ID 261,\n          :DROP_PARTITION_SQL \"ALTER TABLE processing.test01 DROP IF EXISTS PARTITION(dt=date '2019-11-22', city='moscow', num=13)\",\n          :LOCATION \"hdfs://ensime/user/pmp/hive/dt=2019-11-22/city=moscow/num=13\",\n          :SD_ID 64704}}\n```\n\n## Requirements\n\n* Java 8, 11\n* Hadoop authentication method is **SIMPLE**\n* Supported Hive data types in partitions query is **INT**, **DATE**, **STRING**\n\nBuild:\n```sh\nmake jar\n```\n\nEnvironment variables are required:\n```text\nGIN_HOST\nGIN_PORT\nHADOOP_NAMENODE_ACTIVE\nHADOOP_NAMENODE_PASSIVE\nHADOOP_PROXY_USER (example: spark)\nHIVE_METASTORE_HOST\nHIVE_METASTORE_PORT\nHIVE_METASTORE_USER (example: hive)\nHIVE_METASTORE_PASS (example: hive)\nHIVE_HOST\nHIVE_PORT (example: 10000)\nHIVE_USER\nHIVE_PASS\n```\n\nAvailable endpoints:\n```text\n/rpc\n/health\n```\n\n## License\n\nCopyright (C) 2022 Vadim Kornev.  \nDistributed under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkornev%2Fkornev-gin-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkornev%2Fkornev-gin-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkornev%2Fkornev-gin-app/lists"}