{"id":29698645,"url":"https://github.com/nodyhub/fifi","last_synced_at":"2025-09-07T23:33:44.886Z","repository":{"id":57694740,"uuid":"480286429","full_name":"NodyHub/fifi","owner":"NodyHub","description":"fifi sends to a given list of url's HTTP requests, calculates on each response a signature and groups them based on the values.","archived":false,"fork":false,"pushed_at":"2023-08-07T09:11:00.000Z","size":103,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-06-20T05:14:38.497Z","etag":null,"topics":["go","http","recon"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NodyHub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-11T08:12:23.000Z","updated_at":"2023-07-11T14:10:20.000Z","dependencies_parsed_at":"2024-06-20T04:26:31.020Z","dependency_job_id":null,"html_url":"https://github.com/NodyHub/fifi","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"purl":"pkg:github/NodyHub/fifi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NodyHub%2Ffifi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NodyHub%2Ffifi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NodyHub%2Ffifi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NodyHub%2Ffifi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NodyHub","download_url":"https://codeload.github.com/NodyHub/fifi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NodyHub%2Ffifi/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266658778,"owners_count":23963751,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","http","recon"],"created_at":"2025-07-23T10:10:55.102Z","updated_at":"2025-07-23T10:11:00.828Z","avatar_url":"https://github.com/NodyHub.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fifi\n\nfifi sends to a given list of url's HTTP requests, calculates on each response a signature and groups them based on the values.\n\nResponses with the same signature may indicate similar implementation pattern, technologies and homogenious data processing. \n\n## Background\n\nRecently, spring boot had a wide spreaded RCE vulnerability, known as [Spring4Shell](https://portswigger.net/daily-swig/spring4shell-microsoft-cisa-warn-of-limited-in-the-wild-exploitation) ([CVE-2022-22965](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-22965)). Due to the fact that modern web application are implemented based on micro service pattern, various paths of a domain may end up on different applications/containers/CDN. To limit the attack surface system administrator, DevOps Engineers and SRE's are highly interested in limiting the available information about a service in the public.\n\nThis tool provides help to identify differences in the response headers from a given list of urls.\n\n## Installation\nEther download it from [the release page](https://github.com/NodyHub/fifi/releases) or complie it by yourself:\n```\ngo install github.com/NodyHub/fifi@latest\n```\n\n## Usage and example output\n\n```shell\n[~/git/fifi]% fifi -h\nusage: fifi [files]\nfifi sends to a given list of url's HTTP requests, calculates on each response a signature and groups them based on the values.\n\nDefault reads from stdin\n\nOptions:\n--------\n[files] provide the urls in files.\n  -H string\n    \tHost\n  -X string\n    \tMethod (default \"GET\")\n  -a string\n    \tAuthorization\n  -c string\n    \tCookie\n  -diff string\n    \tSignature diff with json file from previous run\n  -json\n    \tOutput json\n  -m int\n    \tMaximum retries for request (default 3)\n  -r\tInclude HTTP response code in signature calculation\n  -s\tInclude 'Server' response header in signature calculation\n  -t int\n    \tThreads (default 1)\n  -u string\n    \tUser-Agent (default GoLang default)\n  -v\tVerbose output\n  -w int\n    \tWait ms between requests\n  -x int\n    \tTimeout seconds (default 1)\n\ngithub.com/NodyHub/fifi@0.3.0\n[~/git/fifi]% cat ~/uber.url.lst | fifi -v -t 4 -s\n2022/05/04 10:58:04 reading from stdin...\n2022/05/04 10:58:04 Collected 11 different urls, starting analysis\n2022/05/04 10:58:04 parsedArgs.ParallelRequests: 4\n2022/05/04 10:58:04 Thread 3 starts\n2022/05/04 10:58:04 Thread 2 starts\n2022/05/04 10:58:04 Thread 1 starts\n2022/05/04 10:58:04 Thread 0 starts\n2022/05/04 10:58:04 1705792451 https://auth.uber.com/login/?next_url=https%3A%2F%2Fm.uber.com%2F\u0026privileged_op_url=https%3A%2F%2Fm.uber.com%2F\n2022/05/04 10:58:05 1705792451 https://auth.uber.com/login\n2022/05/04 10:58:05 1705792451 https://auth.uber.com/login/social/?from=facebook\u0026state=%7B%22query%22%3A%22%3Fnext_url%3Dhttps%253A%252F%252Fm.uber.com%252F%26privileged_op_url%3Dhttps%253A%252F%252Fm.uber.com%252F%26uber_client_name%3Dm2%22%2C%22csrfToken%22%3A%221650443852-01-FNOsAwdU4I8HWkiFZuimbrTHjauX146ik_Hq9h7k1Ew%22%2C%22app%22%3A%22%22%7D\u0026response_type=token\n2022/05/04 10:58:05 1705792451 https://auth.uber.com/login/?breeze_local_zone=dca11\u0026next_url=https%3A%2F%2Fm.uber.com%2F\u0026state=NUUybaiHU9SIaKz56QjyvtJTz5CJC25zhhyocPV9guM%3D\n2022/05/04 10:58:05 1705792451 https://auth.uber.com/login/\n2022/05/04 10:58:05 1705792451 https://auth.uber.com/login/session\n2022/05/04 10:58:05 Thread 2 finished\n2022/05/04 10:58:05 1705792451 https://auth.uber.com/login/?breeze_local_zone=dca1\u0026state=0A-OdN1vuv_FDbpofRZqJg9maKASCY4k0kCRVEiSDGw%3D\u0026uber_client_name=riderSignUp\u0026uclick_id=840a8ddd-ac10-47e6-aec4-e492968acc42\n2022/05/04 10:58:05 Thread 1 finished\n2022/05/04 10:58:05 ERROR (0): Get \"https://auth.uber.com/login/social\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n2022/05/04 10:58:06 ERROR (0): Get \"https://auth.uber.com/login/social/?next_url=https%3A%2F%2Fm.uber.com%2F\u0026privileged_op_url=https%3A%2F%2Fm.uber.com%2F\u0026uber_client_name=m2\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n2022/05/04 10:58:06 ERROR (1): Get \"https://auth.uber.com/login/social\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n2022/05/04 10:58:07 ERROR (1): Get \"https://auth.uber.com/login/social/?next_url=https%3A%2F%2Fm.uber.com%2F\u0026privileged_op_url=https%3A%2F%2Fm.uber.com%2F\u0026uber_client_name=m2\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n2022/05/04 10:58:08 ERROR (2): Get \"https://auth.uber.com/login/social\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n2022/05/04 10:58:09 ERROR (2): Get \"https://auth.uber.com/login/social/?next_url=https%3A%2F%2Fm.uber.com%2F\u0026privileged_op_url=https%3A%2F%2Fm.uber.com%2F\u0026uber_client_name=m2\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\n2022/05/04 10:58:11 ERROR: maxRetry(3) reached, go to next url\n2022/05/04 10:58:11 2898507639 https://auth.uber.com/login/social/\n2022/05/04 10:58:11 1667219945 https://auth.uber.com/\n2022/05/04 10:58:11 Thread 0 finished\n2022/05/04 10:58:12 ERROR: maxRetry(3) reached, go to next url\n2022/05/04 10:58:12 Thread 3 finished\n\nSummary:\n===================================\nHeaders received in every response:\n===================================\n - X-Frame-Options\n - X-Xss-Protection\n - Alt-Svc\n - Date\n - Server\n - Strict-Transport-Security\n - Cache-Control\n - Vary\n - X-Content-Type-Options\n - Content-Type\n - Via\n - X-Envoy-Upstream-Service-Time\n - X-Uber-Edge\n===================================\n\n-----------------------------------\nSignature: 1667219945 ; URLs: 1\nAdditional headers:\n - Server: ufe\n\nUrls:\n[404] https://auth.uber.com/\n-----------------------------------\n\n-----------------------------------\nSignature: 1705792451 ; URLs: 7\nAdditional headers:\n - Content-Security-Policy\n - Etag\n - Server: ufe\n - Set-Cookie\n - Set-Cookie\n - Timing-Allow-Origin\n - X-Content-Security-Policy\n - X-Csrf-Token\n - X-Webkit-Csp\n\nUrls:\n[200] https://auth.uber.com/login\n[200] https://auth.uber.com/login/\n[200] https://auth.uber.com/login/?breeze_local_zone=dca1\u0026state=0A-OdN1vuv_FDbpofRZqJg9maKASCY4k0kCRVEiSDGw%3D\u0026uber_client_name=riderSignUp\u0026uclick_id=840a8ddd-ac10-47e6-aec4-e492968acc42\n[200] https://auth.uber.com/login/?breeze_local_zone=dca11\u0026next_url=https%3A%2F%2Fm.uber.com%2F\u0026state=NUUybaiHU9SIaKz56QjyvtJTz5CJC25zhhyocPV9guM%3D\n[200] https://auth.uber.com/login/?next_url=https%3A%2F%2Fm.uber.com%2F\u0026privileged_op_url=https%3A%2F%2Fm.uber.com%2F\n[200] https://auth.uber.com/login/session\n[200] https://auth.uber.com/login/social/?from=facebook\u0026state=%7B%22query%22%3A%22%3Fnext_url%3Dhttps%253A%252F%252Fm.uber.com%252F%26privileged_op_url%3Dhttps%253A%252F%252Fm.uber.com%252F%26uber_client_name%3Dm2%22%2C%22csrfToken%22%3A%221650443852-01-FNOsAwdU4I8HWkiFZuimbrTHjauX146ik_Hq9h7k1Ew%22%2C%22app%22%3A%22%22%7D\u0026response_type=token\n-----------------------------------\n\n-----------------------------------\nSignature: 2898507639 ; URLs: 1\nAdditional headers:\n - Content-Security-Policy\n - Etag\n - Server: ufe\n - Set-Cookie\n - Set-Cookie\n - X-Content-Security-Policy\n - X-Csrf-Token\n - X-Webkit-Csp\n\nUrls:\n[404] https://auth.uber.com/login/social/\n-----------------------------------\n\n```\n\n# Application behaviour\n\n```mermaid\ngraph TD\n    A[User] --\u003e|all url's| B(fifi)\n    B --\u003e C{For all url's}\n    C --\u003e D[Send HTTP request]\n    D --\u003e E[Calculate signature]\n    E --\u003e F[Store response, based on signature]\n    F --\u003e C\n    C --\u003e G[Generate output]\n\n```\n\n\n# Similar or related projects\n\n* https://github.com/rverton/wonitor\n* https://github.com/dgtlmoon/changedetection.io\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodyhub%2Ffifi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnodyhub%2Ffifi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodyhub%2Ffifi/lists"}