{"id":25284263,"url":"https://github.com/elastos/elastos.carrierclassic.crawler","last_synced_at":"2026-07-17T11:32:35.523Z","repository":{"id":75483163,"uuid":"154262812","full_name":"elastos/Elastos.CarrierClassic.Crawler","owner":"elastos","description":null,"archived":false,"fork":false,"pushed_at":"2019-07-22T02:12:06.000Z","size":66,"stargazers_count":1,"open_issues_count":0,"forks_count":3,"subscribers_count":28,"default_branch":"master","last_synced_at":"2025-10-24T16:32:58.281Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elastos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-10-23T04:34:53.000Z","updated_at":"2023-05-04T08:12:01.000Z","dependencies_parsed_at":"2023-10-22T20:00:24.520Z","dependency_job_id":null,"html_url":"https://github.com/elastos/Elastos.CarrierClassic.Crawler","commit_stats":null,"previous_names":["elastos/elastos.carrierclassic.crawler"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/elastos/Elastos.CarrierClassic.Crawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastos%2FElastos.CarrierClassic.Crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastos%2FElastos.CarrierClassic.Crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastos%2FElastos.CarrierClassic.Crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastos%2FElastos.CarrierClassic.Crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elastos","download_url":"https://codeload.github.com/elastos/Elastos.CarrierClassic.Crawler/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastos%2FElastos.CarrierClassic.Crawler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35580669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-17T02:00:06.162Z","response_time":116,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-12T20:36:58.814Z","updated_at":"2026-07-17T11:32:35.488Z","avatar_url":"https://github.com/elastos.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"Carrier Crawler\n===============\n\n[![Build Status](https://travis-ci.com/elastos/Elastos.NET.Carrier.Crawler.svg)](https://travis-ci.com/elastos/Elastos.NET.Carrier.Crawler)\n\n## Summary\n\nelacrawler is a Elastos Carrier network crawler.\n\n## How crawler works\n\nThe carrier crawler crawls the Carrier network with multiple concurrent instances (default 5 crawlers), allowing the connection by sending **get_nodes** requests to (and get responses from) the number of active Carrier nodes on the network.\n\nWhen a cralwer instance comletes it's mission, a log file containing all space separated IP addresses that it found is created in {currentdate} under your config data directory, with the name {timestamp}.lst.\n\n## IP2Location\n\nIP2Location supports to reveal the geological location for crawled carrier nodes. With IP2Location, the geological depoloyment of almost all active nodes of whole carrier network can vividly demenstrated.\n\nPlease refer to [ip2location.com](https://www.ip2location.com/) to download ip2location database before IP2Location enabled.\n\n## Config Profile\n\nThe list of config items and their description is briefly profiled below:\n\n#### 1. internval\nThe allowing least interval value in munites for the next cralwer to begin crawling carrier nodes.\n\n#### 2. timeout\nThe timeout value in seconds for the **get nodes** request to crawler.\n\n#### 3. data_dir\nThis data directory stores carrier node cached data itself and all dumped files of cralwed carrier nodes information.\n\n#### 4. log\\_level and log\\_file\nThe level value and the file path to for logging.\n\n#### 5. ip2location_database\nThe location of ip2location database relative to current config file.\n\n#### 6. bootstraps\nThe bootstrap nodes list.\n\n## Build from source\n\nWith CMake installed, use the following commands to conduct the whole build steps:\n\n```shell\n$ git clone https://github.com/elastos/Elastos.NET.Carrier.Cralwer\n$ mkdir build\n$ cd build\n$ cmake -DCMAKE_INSTALL_PREFIX=dist ..\n$ make install\n```\n\n## Run\n\nUse commands to run crawler after installation:\n```shell\n$ cd dist/bin\n$ ./elacrawler -c ../etc/carrier/crawler.conf\n```\n\n## Contribution\n\nWe welcome contributions to the Elastos Carrier Project.\n\n## Acknowledgments\n\nA sincere thank you to all teams and projects that we rely on directly or indirectly.\n\n## License\n\nGPLv3\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastos%2Felastos.carrierclassic.crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felastos%2Felastos.carrierclassic.crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastos%2Felastos.carrierclassic.crawler/lists"}