{"id":21222230,"url":"https://github.com/lostintime/hittassign-back","last_synced_at":"2025-03-15T01:16:58.274Z","repository":{"id":148217509,"uuid":"135622701","full_name":"lostintime/hittassign-back","owner":"lostintime","description":"Simple web crawling DSL interpreter","archived":false,"fork":false,"pushed_at":"2018-05-31T18:58:32.000Z","size":121,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-21T17:14:14.893Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lostintime.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-31T18:44:56.000Z","updated_at":"2018-05-31T18:58:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"fd0fdbb5-0e55-4f0d-9520-0849e8266629","html_url":"https://github.com/lostintime/hittassign-back","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostintime%2Fhittassign-back","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostintime%2Fhittassign-back/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostintime%2Fhittassign-back/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostintime%2Fhittassign-back/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lostintime","download_url":"https://codeload.github.com/lostintime/hittassign-back/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243668234,"owners_count":20328042,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-20T22:42:17.440Z","updated_at":"2025-03-15T01:16:58.077Z","avatar_url":"https://github.com/lostintime.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"HIT - Simple Crawling DSL\n=========================\n\nSimple web crawling DSL interpreter.\n\n## Grammar\n\nThis DSL supports few statements and uses indentation for blocks.\n\n### Debug\n\nPrints given message, for debugging\n\n```\ndebug Hello\\ World\n```\n\n### Fetch\n\nFetches JSON from given resource and binds value to a named variable\n\n```\nfetch c https://someapi.com/cities.json\n  debug Total:\\ {c.length}\\ cities\n```\n\nWhere:\n\n  * `c` variable name\n  * `https://someapi.com/cities.json` JSON source\n\n### Foreach\n\nIterate over values at given JSON path (must be an Array).\n\n```\nforeach row c.images\n  debug Row.id:\\ {row.id}\n```\n\nWhere:\n\n  * `row` variable name for element value\n  * `c.images` list variable reference (name: `c` and `$.images` json path)\n\n\n### Download\n\nDownloads remote file to local path.\n\n```\ndownload https://somapi.com/cities/{c.id}.json /tmp/cities/{c.id}.json\n```\n\nWhere: \n\n  * `https://somapi.com/cities/{c.id}.json` source file url\n  * `/tmp/cities/{c.id}.json` path to local file\n\n### Concurrently\n\nSet concurrency level for child block statements.\n\n```\nconcurrently 8\n  debug Concurrency\\ level\\ is\\ 8\\ here\n```\n\n## Script Example\n\n```\ndebug Starting...\n\nfetch h https://somedmoeapi.com/some/list.json\n  concurrently 8\n    foreach c h.cities\n      debug id:\\ {c.id}\n      download https://publiccitiesapi.com/{c.id}/image.png /tmp/cities/{c.id}/image.png\n\ndebug Done.\n```\n\n\n## Run with docker\n\n```bash\ncat sample.hit | docker run -i --rm lostintime/hittassign-back:1.0\n```\n\n## License\n\nAll code in this repository is licensed under the Apache License,\nVersion 2.0.  See [LICENCE](./LICENSE).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flostintime%2Fhittassign-back","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flostintime%2Fhittassign-back","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flostintime%2Fhittassign-back/lists"}