{"id":21621537,"url":"https://github.com/httpreserve/linkstat","last_synced_at":"2026-01-17T02:10:18.666Z","repository":{"id":57598480,"uuid":"176600526","full_name":"httpreserve/linkstat","owner":"httpreserve","description":"CLI implementation of httpreserve that can test links and retrieve internet archive replacements","archived":false,"fork":false,"pushed_at":"2024-11-21T11:18:12.000Z","size":49,"stargazers_count":10,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-11-25T00:02:23.751Z","etag":null,"topics":["archives","cli","code4lib","digipres","digital-preservation","glam","internet-archive","link-checker","wayback-machine","web-archiving"],"latest_commit_sha":null,"homepage":"http://httpreserve.info","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/httpreserve.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-03-19T21:23:53.000Z","updated_at":"2024-11-21T11:18:16.000Z","dependencies_parsed_at":"2024-01-09T19:13:29.987Z","dependency_job_id":null,"html_url":"https://github.com/httpreserve/linkstat","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/httpreserve/linkstat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/httpreserve%2Flinkstat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/httpreserve%2Flinkstat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/httpreserve%2Flinkstat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/httpreserve%2Flinkstat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/httpreserve","download_url":"https://codeload.github.com/httpreserve/linkstat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/httpreserve%2Flinkstat/sbom","scorecard":{"id":471286,"data":{"date":"2025-08-11","repo":{"name":"github.com/httpreserve/linkstat","commit":"cafcb5646a664b786153c683c50cb680f2747550"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.1,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/14 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/linting.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/httpreserve/linkstat/linting.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/linting.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/httpreserve/linkstat/linting.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/tests.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/httpreserve/linkstat/tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/tests.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/httpreserve/linkstat/tests.yml/main?enable=pin","Warn: goCommand not pinned by hash: .github/workflows/linting.yml:33","Warn: goCommand not pinned by hash: .github/workflows/linting.yml:37","Info:   0 out of   4 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 goCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/linting.yml:1","Warn: no topLevel permission defined: .github/workflows/tests.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU General Public License v3.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact 0.0.3 not signed: https://api.github.com/repos/httpreserve/linkstat/releases/104440967","Warn: release artifact 0.0.1 not signed: https://api.github.com/repos/httpreserve/linkstat/releases/16219870","Warn: release artifact 0.0.3 does not have provenance: https://api.github.com/repos/httpreserve/linkstat/releases/104440967","Warn: release artifact 0.0.1 does not have provenance: https://api.github.com/repos/httpreserve/linkstat/releases/16219870"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 5 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-19T13:56:48.823Z","repository_id":57598480,"created_at":"2025-08-19T13:56:48.823Z","updated_at":"2025-08-19T13:56:48.823Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28492057,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T00:50:05.742Z","status":"online","status_checked_at":"2026-01-17T02:00:07.808Z","response_time":85,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archives","cli","code4lib","digipres","digital-preservation","glam","internet-archive","link-checker","wayback-machine","web-archiving"],"created_at":"2024-11-25T00:00:38.326Z","updated_at":"2026-01-17T02:10:18.649Z","avatar_url":"https://github.com/httpreserve.png","language":"Go","funding_links":[],"categories":["Go","Tools \u0026 Software"],"sub_categories":["Utilities"],"readme":"\u003cdiv\u003e\n\u003cp align=\"center\"\u003e\n\u003cimg height=\"300px\" width=\"300px\" id=\"logo\" src=\"https://github.com/httpreserve/httpreserve/raw/main/src/images/httpreserve-logo.png\" alt=\"httpreserve\"/\u003e\n\u003c/p\u003e\n\u003c/div\u003e\n\n# linkstat\n\nCLI implementation of httpreserve that can test links and retrieve Internet\nArchive replacements. The tool can output the result of individual links, or\ntake a CSV list to output collected information in JSON, BoltDB, or CSV format.\n\n## Usage\n```bash\nUsage:  linkstat [Optional -link] [Optional -label]\n                 [Optional -list] [Optional -json]\n                                  [Optional -bolt]\n                                  [Optional -csv]\n                 [Optional -version -v]\n\nOutput: [Json]\nOutput: [CSV]\nOutput: [BoltDB]\nOutput: [Version] 'exponentialDK-httpreserve/0.0.9 ...'\n\nUsage of ./linkstat:\n  -bolt\n    \tOutput to static BoltDB.\n  -csv\n    \tOutput to CSV.\n  -json\n    \tOutput to JSON.\n  -label string\n    \tAnnotate single URL check response with label.\n  -link string\n    \tSeek the status of a single URL: JSON\n  -list string\n    \tProvide a list of URLs to test against in CSV format.\n  -v\tReturn httpreserve version.\n  -version\n    \tReturn httpreserve version.\n```\n\n## Examples\n\n#### Example combining [tikalinkextract][httpreserve-1]\n\nInspired by Harvard Innovation Labs to test the ability of\nhttpreserve-workbench at the time. This CLI version is a simplification of that\nwork but should still produce decent results. HTTPreserve\n[Million Dollar Webpage Project][httpreserve-2]\n\n[httpreserve-1]: https://github.com/httpreserve/tikalinkextract\n[httpreserve-2]: https://github.com/httpreserve/million-dollar-webpage\n\n#### CSV input\n\nAn input CSV `example.csv` might look as follows:\n```csv\n\"BBC News\", \"http://www.bbc.co.uk/news\"\n\"BBC Home\", \"http://www.bbc.co.uk/\"\n\"BBC Radio\", \"http://www.bbc.co.uk/radio\"\n\"Google\", \"http://www.google.com\"\n\"exponentialdecay.co.uk\", \"http://www.exponentialdecay.co.uk\"\n\"Internet Archive\", \"http://www.archive.org\"\n\"perma.cc\", \"http://perma.cc\"\n\"wikipedia.org\", \"http://wikipedia.org\"\n\"The Million Dollar Homepage\", \"http://www.getpixel.net\"\n```\n\nTo output a CSV collecting all of the linkstat results, you can run a command\nas follows:\n```bash\n$ ./linkstat -csv --list example.csv \u003e output.csv\n```\n\nAnd the output looks as follows:\n```\n\"id\",\"filename\",\"link\",\"response code\",\"response text\",\"title\",\"content-type\",\"archived\",\"internet archive response code\",\"internet archive response text\",\"wayback earliest date\",\"internet archive earliest\",\"wayback latest date\",\"internet archive latest\",\"internet archive save link\",\"protocol error\",\"protocol error\",\"analysis version number\",\"analysis version text\",\"stats creation time\"\n\"1651a00b16a12ba06fc6c6b049c7cf7c\",\"BBC News\",\"https://www.bbc.co.uk/news\",\"200\",\"OK\",\"home - bbc news\",\"text/html;charset=utf-8\",\"true\",\"302\",\"Found\",\"09 October 1997\",\"http://web.archive.org/web/19971009011901/http://www.bbc.co.uk/news/\",\"19 March 2019\",\"http://web.archive.org/web/20190319173721/https://www.bbc.co.uk/news\",\"http://web.archive.org/save/https://www.bbc.co.uk/news\",\"\",\"\",\"0.0.9\",\"exponentialDK-httpreserve/0.0.9\",\"1.574649021s\"\n\"57ab6349a47b53b982a939fb1da54fef\",\"BBC Radio\",\"https://www.bbc.co.uk/sounds\",\"200\",\"OK\",\"bbc sounds - music. radio. podcasts\",\"text/html; charset=utf-8\",\"true\",\"302\",\"Found\",\"19 March 2008\",\"http://web.archive.org/web/20080319074038/http://www.bbc.co.uk/sounds\",\"18 March 2019\",\"http://web.archive.org/web/20190318211158/https://www.bbc.co.uk/sounds\",\"http://web.archive.org/save/https://www.bbc.co.uk/sounds\",\"\",\"\",\"0.0.9\",\"exponentialDK-httpreserve/0.0.9\",\"1.660729358s\"\n\"c85da5e372ffe2200e46527b74537ba3\",\"BBC Home\",\"https://www.bbc.co.uk/\",\"200\",\"OK\",\"bbc - home\",\"text/html; charset=utf-8\",\"true\",\"302\",\"Found\",\"21 December 1996\",\"http://web.archive.org/web/19961221203254/http://www0.bbc.co.uk/\",\"19 March 2019\",\"http://web.archive.org/web/20190319141018/https://www.bbc.co.uk/\",\"http://web.archive.org/save/https://www.bbc.co.uk/\",\"\",\"\",\"0.0.9\",\"exponentialDK-httpreserve/0.0.9\",\"1.95442772s\"\n\"b3bd672c1014e07e87ef4a357a161528\",\"exponentialdecay.co.uk\",\"http://www.exponentialdecay.co.uk\",\"206\",\"Partial Content\",\"ross spencer, digital preservation, archives, python developer, golang developer, uk, nz\",\"text/html\",\"true\",\"302\",\"Found\",\"17 September 2008\",\"http://web.archive.org/web/20080917054811/http://www.exponentialdecay.co.uk/\",\"13 November 2018\",\"http://web.archive.org/web/20181113021338/http://exponentialdecay.co.uk/\",\"http://web.archive.org/save/http://www.exponentialdecay.co.uk\",\"\",\"\",\"0.0.9\",\"exponentialDK-httpreserve/0.0.9\",\"425.368183ms\"\n```\n\n#### An individual link\n\nThe command: `./linkstat -link https://github.com/ -label \"GitHub\"` will\noutput:\n```json\n{\n   \"FileName\": \"GitHub\",\n   \"AnalysisVersionNumber\": \"0.0.15\",\n   \"AnalysisVersionText\": \"exponentialDK-httpreserve/0.0.15\",\n   \"SimpleRequestVersion\": \"httpreserve-simplerequest/0.0.4\",\n   \"Link\": \"https://github.com/\",\n   \"Title\": \"github: let’s build from here · github\",\n   \"ContentType\": \"text/html; charset=utf-8\",\n   \"ResponseCode\": 200,\n   \"ResponseText\": \"OK\",\n   \"SourceURL\": \"https://github.com/\",\n   \"ScreenShot\": \"snapshots are not currently enabled\",\n   \"InternetArchiveLinkEarliest\": \"http://web.archive.org/web/20080514210148/http://github.com/\",\n   \"InternetArchiveEarliestDate\": \"2008-05-14 21:01:48 +0000 UTC\",\n   \"InternetArchiveLinkLatest\": \"http://web.archive.org/web/20230829062855/https://github.com/\",\n   \"InternetArchiveLatestDate\": \"2023-08-29 06:28:55 +0000 UTC\",\n   \"InternetArchiveSaveLink\": \"http://web.archive.org/save/https://github.com/\",\n   \"InternetArchiveResponseCode\": 302,\n   \"InternetArchiveResponseText\": \"Found\",\n   \"RobustLinkEarliest\": \"\u003ca href='http://web.archive.org/web/20080514210148/http://github.com/' data-originalurl='https://github.com/' data-versiondate='2008-05-14'\u003eHTTPreserve Robust Link - simply replace this text!!\u003c/a\u003e\",\n   \"RobustLinkLatest\": \"\u003ca href='http://web.archive.org/web/20230829062855/https://github.com/' data-originalurl='https://github.com/' data-versiondate='2023-08-29'\u003eHTTPreserve Robust Link - simply replace this text!!\u003c/a\u003e\",\n   \"PWID\": \"urn:pwid:archive.org:2023-08-29T06:28:55Z:page:https://github.com/\",\n   \"Archived\": true,\n   \"Error\": false,\n   \"ErrorMessage\": \"\",\n   \"StatsCreationTime\": \"7.070152149s\"\n}\n```\n\n## Archiving Weblinks\n\n* [Find and Connect Project:][linkstat-1] Nicola Laurent on the impact of\nbroken links.\n* [Binary Trees? Automatically Identifying the links between born digital records:][linkstat-2]\nI write about hyperlinks as a public record in own right when submitted as part\nof a documentary heritage.\n* [HiberActive Pilot][linkstat-3] A scholarly publishing tool that extracts\nURLs, returns both the original URL and a perma-link.\n* [IIPC Awesome List][linkstat-4] A list of web-archiving links that invites\ncontributions from the community to keep it up-to-date.\n\n[linkstat-1]: http://www.findandconnectwrblog.info/2016/11/broken-links-broken-trust/\n[linkstat-2]: https://www.youtube.com/watch?v=Ked9GRmKlRw\n[linkstat-3]: https://www.era.lib.ed.ac.uk/handle/1842/23366\n[linkstat-4]: https://github.com/iipc/awesome-web-archiving\n\n## License\n\nGNU General Public License Version 3. [Full Text](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhttpreserve%2Flinkstat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhttpreserve%2Flinkstat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhttpreserve%2Flinkstat/lists"}