{"id":13762818,"url":"https://github.com/tomnomnom/fff","last_synced_at":"2025-04-05T18:11:22.116Z","repository":{"id":41270138,"uuid":"267176182","full_name":"tomnomnom/fff","owner":"tomnomnom","description":"The Fairly Fast Fetcher. Requests a bunch of URLs provided on stdin fairly quickly.","archived":false,"fork":false,"pushed_at":"2024-04-10T16:02:48.000Z","size":8,"stargazers_count":400,"open_issues_count":8,"forks_count":64,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-29T17:11:41.998Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tomnomnom.png","metadata":{"files":{"readme":"README.mkd","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-26T23:49:09.000Z","updated_at":"2025-03-29T10:41:27.000Z","dependencies_parsed_at":"2024-06-20T08:24:22.605Z","dependency_job_id":"21c14469-9ef4-478e-8324-b234f7a8a460","html_url":"https://github.com/tomnomnom/fff","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomnomnom%2Ffff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomnomnom%2Ffff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomnomnom%2Ffff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomnomnom%2Ffff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tomnomnom","download_url":"https://codeload.github.com/tomnomnom/fff/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247378149,"owners_count":20929297,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T14:00:58.405Z","updated_at":"2025-04-05T18:11:21.713Z","avatar_url":"https://github.com/tomnomnom.png","language":"Go","funding_links":[],"categories":["Weapons","Go","Go (531)"],"sub_categories":["Tools"],"readme":"# fff\n\nThe Fairly Fast Fetcher. Requests a bunch of URLs provided on stdin fairly quickly.\n\nThe main idea is to launch a new request every `n` milliseconds, without waiting\nfor the last request to finish first. This makes for consistently fast fetching,\nbut can be hard on system resources (e.g. you might run out of file descriptors).\nThe advantage though, is that hitting a bunch of very slow URLs or URLs that\nresult in timeouts doesn't slow the overall progress very much.\n\n## Install\n\n```\n▶ go get -u github.com/tomnomnom/fff\n```\n\n## Usage\n\nBasic usage:\n```\n▶ cat urls.txt | fff\n```\n\nOptions:\n\n```\n▶ fff --help\nRequest URLs provided on stdin fairly frickin' fast\n\nOptions:\n  -b, --body \u003cdata\u003e         Request body\n  -d, --delay \u003cdelay\u003e       Delay between issuing requests (ms)\n  -H, --header \u003cheader\u003e     Add a header to the request (can be specified multiple times)\n  -k, --keep-alive          Use HTTP Keep-Alive\n  -m, --method              HTTP method to use (default: GET, or POST if body is specified)\n  -o, --output \u003cdir\u003e        Directory to save responses in (will be created)\n  -s, --save-status \u003ccode\u003e  Save responses with given status code (can be specified multiple times)\n  -S, --save                Save all responses\n  -x, --proxy \u003cproxyURL\u003e    Use the provided HTTP proxy\n```\n\n## Tuning\nYou might want to increase your open file descriptor limit before doing anything crazy:\n\n```\n▶ ulimit -n 16384\n```\n\n## TODO\n\n* Create an index file in the output directory\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftomnomnom%2Ffff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftomnomnom%2Ffff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftomnomnom%2Ffff/lists"}