{"id":13683305,"url":"https://github.com/firecat53/urlscan","last_synced_at":"2025-10-21T04:41:02.929Z","repository":{"id":38355852,"uuid":"2470328","full_name":"firecat53/urlscan","owner":"firecat53","description":"Mutt and terminal url selector (similar to urlview)","archived":false,"fork":false,"pushed_at":"2025-05-18T00:31:25.000Z","size":383,"stargazers_count":236,"open_issues_count":6,"forks_count":37,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-10-04T21:41:43.897Z","etag":null,"topics":["mutt","python","urlscan","urlview"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/firecat53.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2011-09-27T20:00:31.000Z","updated_at":"2025-10-04T13:58:47.000Z","dependencies_parsed_at":"2023-11-13T06:24:37.346Z","dependency_job_id":"970bbb94-f956-4be7-80f5-98d66ce76fac","html_url":"https://github.com/firecat53/urlscan","commit_stats":{"total_commits":180,"total_committers":20,"mean_commits":9.0,"dds":"0.21666666666666667","last_synced_commit":"b9fb2990aa23eada10c646002e65bea300a0299b"},"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"purl":"pkg:github/firecat53/urlscan","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firecat53%2Furlscan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firecat53%2Furlscan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firecat53%2Furlscan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firecat53%2Furlscan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/firecat53","download_url":"https://codeload.github.com/firecat53/urlscan/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firecat53%2Furlscan/sbom","scorecard":{"id":400734,"data":{"date":"2025-08-11","repo":{"name":"github.com/firecat53/urlscan","commit":"44c01517598bc7973f3b1f4fd43e662caa73d545"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.6,"checks":[{"name":"Code-Review","score":2,"reason":"Found 7/29 approved changesets -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: jobLevel 'contents' permission set to 'write': .github/workflows/main.yml:69","Warn: no topLevel permission defined: .github/workflows/main.yml:1"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":3,"reason":"2 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 3","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:35: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:54: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/main.yml:59: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:74: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:78: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/main.yml:83: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:129: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/main.yml:134: update your workflow using https://app.stepsecurity.io/secureworkflow/firecat53/urlscan/main.yml/main?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:28","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:29","Info:   0 out of   7 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   3 third-party GitHubAction dependencies pinned","Info:   0 out of   2 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU General Public License v2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":8,"reason":"2 out of the last 2 releases have a total of 2 signed artifacts.","details":["Info: signed release artifact: urlscan-1.0.6-py2.py3-none-any.whl.sigstore.json: https://github.com/firecat53/urlscan/releases/tag/1.0.6","Info: signed release artifact: urlscan-1.0.4-py2.py3-none-any.whl.sigstore.json: https://github.com/firecat53/urlscan/releases/tag/1.0.4","Warn: release artifact 1.0.6 does not have provenance: https://api.github.com/repos/firecat53/urlscan/releases/187028487","Warn: release artifact 1.0.4 does not have provenance: https://api.github.com/repos/firecat53/urlscan/releases/186031427"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Packaging","score":10,"reason":"packaging workflow detected","details":["Info: Project packages its releases by way of GitHub Actions.: .github/workflows/main.yml:114"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 8 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-18T20:01:35.651Z","repository_id":38355852,"created_at":"2025-08-18T20:01:35.652Z","updated_at":"2025-08-18T20:01:35.652Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280207158,"owners_count":26290613,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mutt","python","urlscan","urlview"],"created_at":"2024-08-02T13:02:06.919Z","updated_at":"2025-10-21T04:41:02.923Z","avatar_url":"https://github.com/firecat53.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Urlscan\n\n[![main](https://github.com/firecat53/urlscan/actions/workflows/main.yml/badge.svg)](https://github.com/firecat53/urlscan/actions/workflows/main.yml)\n\n## Contributors\n\nScott Hansen \\\u003ctech@firecat53.net\\\u003e (Author and Maintainer)\n\nMaxime Chatelle \\\u003cxakz@rxsoft.eu\\\u003e (Debian Maintainer)\n\nDaniel Burrows \\\u003cdburrows@debian.org\\\u003e (Original Author)\n\n## Purpose and Requirements\n\nUrlscan is a small program that is designed to integrate with the \"mutt\"\nmailreader to allow you to easily launch a Web browser for URLs contained in\nemail messages. It is a replacement for the \"urlview\" program.\n\nRequires: Python 3.7+ and the python-urwid library\n\n## Features\n\nUrlscan parses an email message or file and scans it for URLs and email\naddresses. It then displays the URLs and their context within the message, and\nallows you to choose one or more URLs to send to your Web browser.\nAlternatively, it send a list of all URLs to stdout.\n\nRelative to urlview, urlscan has the following additional features:\n\n- Support for emails in quoted-printable and base64 encodings. No more stripping\n  out =40D from URLs by hand!\n\n- The context of each URL is provided along with the URL. For HTML mails, a\n  crude parser is used to render the HTML into text. Context view can be toggled\n  on/off with `c`.\n\n- URLs are shortened by default to fit on one line. Viewing full URL (for one or\n  all) is toggled with `s` or `S`.\n\n- Jump to a URL by typing the number.\n\n- Incremental case-insensitive search with `/`.\n\n- Execute an arbitrary function (for example, copy URL to clipboard) instead of\n  opening URL in a browser.\n\n- Use `l` to cycle through whether URLs are opened using the Python webbrowser\n  module (default), xdg-open (if installed) or opened by a function passed on\n  the command line with `--run` or `--run-safe`.\n\n- Configure colors and keybindings via ~/.config/urlscan/config.json. Generate\n  default config file for editing by running `urlscan -g`. Cycle through\n  available palettes with `p`. Set display width with `--width`.\n\n- Set the urwid color mode with `--colors`. Options include 'true' (2**24\n  colors), '256', '88', '16', '8', or 'mono'. Default is '16'.\n\n- Copy URL to clipboard with `C` or to primary selection with `P`.  Requires\n  xsel or xclip.\n\n- Run a command with the selected URL as the argument or pipe the selected\n  URL to a command.\n\n- Show complete help menu with `F1`. Hide header on startup with `--nohelp`.\n\n- Use a custom regular expression with `-E` for matching urls or any\n  other pattern. In junction with `-r`, this effectively turns urlscan\n  into a general purpose CLI selector-type utility.\n\n- Scan certain email headers for URLs. Currently `Link`, `Archived-At` and\n  `List-*` are scanned when `--headers` is passed.\n\n- Queue multiple URLs for opening and open them all at once with `a` and `o`.\n\n## Installation and setup\n\nTo install urlscan, install from your distribution repositories, from Pypi, or do\na local development install with pip -e:\n\n    pipx install urlscan\n\n    OR\n\n    pip install --user urlscan\n\n    OR\n\n    cd \u003cpath/to/urlscan\u003e \u0026\u0026 pip install --user -e .\n\n**NOTE**\n\n    The minimum required version of urwid is 1.2.1.\n\nOnce urlscan is installed, add the following lines to your .muttrc:\n\n    macro index,pager \\cb \"\u003cpipe-message\u003e urlscan\u003cEnter\u003e\" \"call urlscan to\n    extract URLs out of a message\"\n\n    macro attach,compose \\cb \"\u003cpipe-entry\u003e urlscan\u003cEnter\u003e\" \"call urlscan to\n    extract URLs out of a message\"\n\nOnce this is done, Control-b while reading mail in mutt will automatically\ninvoke urlscan on the message.\n\n\u003e Note for Neomutt users: [As of version\n\u003e `2023-05-17`](https://github.com/neomutt/neomutt/releases/tag/20230517) true\n\u003e color support was implemented. If you are using true color support with Neomutt,\n\u003e or are encountering the error `setupterm: could not find terminfo database`,\n\u003e then you should also add `TERM=xterm-256color` to your macro in `.muttrc`.\n\u003e See more here [#135](https://github.com/firecat53/urlscan/issues/135). For example:\n\u003e `macro index,pager  \\cb \"\u003cpipe-message\u003e  TERM=xterm-256color urlscan\u003cEnter\u003e\" \"call urlscan to\nextract URLs out of a message\"`\n\nTo choose a particular browser, set the environment variable BROWSER. If BROWSER\nis not set, xdg-open will control which browser is used, if it's available.:\n\n    export BROWSER=/usr/bin/epiphany\n\n\n## Command Line usage\n\n    urlscan OPTIONS \u003cfile\u003e\n\n    OPTIONS [-c, --compact]\n            [-d, --dedupe]\n            [-E, --regex \u003cexpression\u003e]\n            [-f, --run-safe \u003cexpression\u003e]\n            [-g, --genconf]\n            [-H, --nohelp]\n            [    --headers]\n            [-n, --no-browser]\n            [-p, --pipe]\n            [-r, --run \u003cexpression\u003e]\n            [-R, --reverse]\n            [-s, --single]\n            [-w, --width]\n            [-W  --whitespace-off]\n            [-C, --colors {true,256,88,16,8,mono}]\n\nUrlscan can extract URLs and email addresses from emails or any text file.\nCalling with no flags will start the curses browser. Calling with '-n' will just\noutput a list of URLs/email addressess to stdout. The '-c' flag removes the\ncontext from around the URLs in the curses browser, and the '-d' flag removes\nduplicate URLs. The '-R' flag reverses the displayed order of URLs and context.\nFiles can also be piped to urlscan using normal shell pipe mechanisms: `cat\n\u003csomething\u003e | urlscan` or `urlscan \u003c \u003csomething\u003e`. The '-W' flag condenses the\ndisplay output by suppressing blank lines and ellipses lines.\n\nInstead of opening a web browser, the selected URL can be passed as the argument\nto a command using `--run-safe \"\u003ccommand\u003e {}\"` or `--run \"\u003ccommand\u003e {}\"`. Note\nthe use of `{}` in the command string to denote the selected URL. Alternatively,\nthe URL can be piped to the command using `--run-safe \u003ccommand\u003e --pipe` (or\n`--run`). Using --run-safe with --pipe is preferred if the command supports it,\nas it is marginally more secure and tolerant of special characters in the URL.\n\n## Theming\n\nRun `urlscan -g` to generate ~/.config/urlscan/config.json with the default\ncolor and black \u0026 white palettes. This can be edited or added to, as desired.\nThe first palette in the list will be the default. Configure the palettes\naccording to the [Urwid display attributes][1].\n\nPartial example from default palette:\n```json\n        \"default\": [\n            [\n                \"header\",   # Urwid widget name\n                \"white\",    # Standard foreground color\n                \"dark blue\",# Standard background color\n                \"standout\", # Monochrome settings\n                \"#ffffff\",  # Extended foreground color (88 through True colors)\n                \"#0000aa\"   # Extended foreground color (88 through True colors)\n            ],\n            [\n                \"footer\",\n                ...\n```\n\nDisplay width can be set with `--width`.\n\n## Keybindings\n\nRun `urlscan -g` to generate ~/.config/urlscan/config.json. All of the keys will\nbe listed. You can either leave in place or delete any that will not be altered.\n\nTo unset a binding, set it equal to \"\".  For example: `\"P\": \"\"`\n\nThe follow actions are supported:\n\n- `add_url` -- add a URL to the queue (default: `a`)\n- `all_escape` -- toggle unescape all URLs (default: `u`)\n- `all_shorten` -- toggle shorten all URLs (default: `S`)\n- `bottom` -- move cursor to last item (default: `G`)\n- `clear_screen` -- redraw screen (default: `Ctrl-l`)\n- `clipboard` -- copy highlighted URL to clipboard using xsel/xclip (default: `C`)\n- `clipboard_pri` -- copy highlighted URL to primary selection using xsel/xclip (default: `P`)\n- `context` -- show/hide context (default: `c`)\n- `del_url` -- delete URL from the queue (default: `d`)\n- `down` -- cursor down (default: `j`)\n- `help_menu` -- show/hide help menu (default: `F1`)\n- `link_handler` -- cycle link handling (webbrowser, xdg-open, --run-safe or --run) (default: `l`)\n- `next` -- jump to next URL (default: `J`)\n- `open_queue` -- open all URLs in queue (default: `o`)\n- `open_queue_win` -- open all URLs in queue in new window (default: `O`)\n- `open_url` -- open selected URL (default: `space` or `enter`)\n- `palette` -- cycle through palettes (default: `p`)\n- `previous` -- jump to previous URL (default: `K`)\n- `quit` -- quit (default: `q` or `Q`)\n- `reverse` -- reverse display order (default: `R`)\n- `shorten` -- toggle shorten highlighted URL (default: `s`)\n- `top` -- move to first list item (default: `g`)\n- `up` -- cursor up (default: `k`)\n\n## Known bugs and limitations\n\n- Running urlscan sometimes \"messes up\" the terminal background. This seems to\n  be an urwid bug, but I haven't tracked down just what's going on.\n\n- Extraction of context from HTML messages leaves something to be desired.\n  Probably the ideal solution would be to extract context on a word basis rather\n  than on a paragraph basis.\n\n- The HTML message handling is a bit kludgy in general.\n\n- multipart/alternative sections are handled by descending into all the\n  sub-parts, rather than just picking one, which may lead to URLs and context\n  appearing twice. (Bypass this by selecting the '--dedupe' option)\n\n## Build/development\n\n- pyproject.toml is configured for [hatch][2] for building and submitting to pypi.\n- flake.nix is available for a development shell or building/testing the package\n  if desired. `nix develop`\n- To update TLD list: `wget https://data.iana.org/TLD/tlds-alpha-by-domain.txt`\n- GitHub Action will upload to TestPyPi on each push to `main`. To create a\n  GitHub and PyPi release, create a new tag (formatting below) and push tags.\n\n        \u003ctag name on first line\u003e\n        \n        * Release note 1\n        * Release note 2\n        * ...\n\n[1]: http://urwid.org/manual/displayattributes.html#display-attributes  \"Urwid display attributes\"\n[2]: https://hatch.pypa.io/latest/  \"Hatch\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirecat53%2Furlscan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffirecat53%2Furlscan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirecat53%2Furlscan/lists"}