{"id":31844716,"url":"https://github.com/exca-dk/node-util","last_synced_at":"2026-05-16T11:04:28.403Z","repository":{"id":153917402,"uuid":"630634793","full_name":"Exca-DK/node-util","owner":"Exca-DK","description":"Usefull utils for analyzing p2p crypto networks.","archived":false,"fork":false,"pushed_at":"2023-04-20T20:51:04.000Z","size":166,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-19T17:19:22.152Z","etag":null,"topics":["crawler","ethereum","mev","p2p","scanner"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Exca-DK.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-04-20T20:06:47.000Z","updated_at":"2023-10-24T11:38:47.000Z","dependencies_parsed_at":"2024-04-02T00:00:34.268Z","dependency_job_id":null,"html_url":"https://github.com/Exca-DK/node-util","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Exca-DK/node-util","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Exca-DK%2Fnode-util","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Exca-DK%2Fnode-util/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Exca-DK%2Fnode-util/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Exca-DK%2Fnode-util/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Exca-DK","download_url":"https://codeload.github.com/Exca-DK/node-util/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Exca-DK%2Fnode-util/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33100321,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T04:41:52.686Z","status":"ssl_error","status_checked_at":"2026-05-16T04:41:52.009Z","response_time":115,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","ethereum","mev","p2p","scanner"],"created_at":"2025-10-12T07:47:31.736Z","updated_at":"2026-05-16T11:04:28.398Z","avatar_url":"https://github.com/Exca-DK.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Node-Util\nThis repo includes usefull services and tools for monitoring, gathering and analyzing p2p network.\n\nMain modules are:\n### Crawler\nCrawls the p2p network and streams data into the provided target. Additionaly it saves entries to DB.\n\n### Indexer\nAwaits for data from event queue and transforms it\n\n### Scanner\nScans server ports\n\n## Getting Started\n\nYou can decide to run each of the modules as either service or command.\n\n### Prerequisites\n\nIt is recommended to have docker installed since most of the examples are based on it.\n\n  ```sh\n  [Docker](https://github.com/docker/docker-ce/releases) \n  [docker-compose](https://github.com/docker/compose/releases). \n  ```\n\n## Usage\n\nEither use single component in which instructions are placed inside the module or use service example provided in /infra/docker-compose/\n\nSteps to run service:\n\n1. Set env variables:\n    - export NODE_UTIL_SCANNER_REDIS_PASSWORD=\"\"\n    - export NODE_UTIL_DB_USERNAME=\"\"\n    - export NODE_UTIL_DB_PASSWORD=\"\"\n    - export NODE_UTIL_BROKER_CRAWLER_PASSWORD=\"\"\n    - export NODE_UTIL_BROKER_SCANNER_PASSWORD=\"\"\n1. cd ./infra/docker-compose/\n2. docker compose up -d\n\nDefault passwords for broker accounts are: admin_secret, crawler_secret, scanner_secret.\nIf you change them, remember to update the hashes in broker definitions. You can easly regenerate the hashes with gen_pass.sh script\n\n## Metrics\n\nAll services output usefull metrics. You can access them at port 6061 of service.\n\nExample at /infra/docker-compose/ includes prometheus scraper. You can access it at port 9090.\n\n## Database\n\nBy default the postgres interface is exposed on localhost. You can access it with default credentials.\n\nYou can extend the database by simply editing sql tables in /service/db/migration and queries at /service/db/query and recompile everything by using  provided make file at /service\n\nSome tools require access to db such as migration. In order to run them you need to set env variable `NODE_UTIL_DB_ENDPOINT`, `NODE_UTIL_DB_USERNAME` and `NODE_UTIL_DB_PASSWORD`\n\n\n\n\n### Contribution\nContribution is welcome. If you'd like to contribute, please fork, change, commit and send a pull request.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexca-dk%2Fnode-util","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fexca-dk%2Fnode-util","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexca-dk%2Fnode-util/lists"}