{"id":13442742,"url":"https://github.com/vindarel/cl-torrents","last_synced_at":"2026-01-28T04:03:01.363Z","repository":{"id":46661253,"uuid":"105710461","full_name":"vindarel/cl-torrents","owner":"vindarel","description":"Searching torrents on popular trackers - CLI, readline, GUI, web client. Tutorial and binaries (issue tracker on https://gitlab.com/vindarel/cl-torrents/)","archived":false,"fork":false,"pushed_at":"2025-01-14T11:10:07.000Z","size":1129,"stargazers_count":110,"open_issues_count":0,"forks_count":4,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-05T22:29:10.072Z","etag":null,"topics":["1337","1337x","common-lisp","pirate-bay","torrents","tutorial","web-scraping"],"latest_commit_sha":null,"homepage":"https://vindarel.github.io/cl-torrents/tutorial.html","language":"Common Lisp","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vindarel.png","metadata":{"files":{"readme":"README.org","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-03T22:33:05.000Z","updated_at":"2025-02-18T06:33:11.000Z","dependencies_parsed_at":"2024-10-28T04:00:53.528Z","dependency_job_id":"cfb77ed8-64af-471a-a6b5-d175698e682a","html_url":"https://github.com/vindarel/cl-torrents","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"purl":"pkg:github/vindarel/cl-torrents","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vindarel%2Fcl-torrents","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vindarel%2Fcl-torrents/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vindarel%2Fcl-torrents/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vindarel%2Fcl-torrents/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vindarel","download_url":"https://codeload.github.com/vindarel/cl-torrents/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vindarel%2Fcl-torrents/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28837868,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T02:10:51.810Z","status":"ssl_error","status_checked_at":"2026-01-28T02:10:50.806Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["1337","1337x","common-lisp","pirate-bay","torrents","tutorial","web-scraping"],"created_at":"2024-07-31T03:01:49.989Z","updated_at":"2026-01-28T04:03:01.344Z","avatar_url":"https://github.com/vindarel.png","language":"Common Lisp","funding_links":[],"categories":["Common Lisp","Applications"],"sub_categories":["Internet"],"readme":"[[https://gitlab.com/vindarel/cl-torrents][file:https://gitlab.com/vindarel/cl-torrents/badges/master/pipeline.svg]]\n\n  This is a little app to  search for torrents on popular trackers and\n  to open them with a local or a remote client.\n\n  It comes as:\n\n- a *lisp library*,\n- a *command line* interface,\n- a terminal *readline interface*,\n- a simple *GUI*.\n- The *[[https://github.com/vindarel/cl-torrents-web][Web UI]]* works too.\n\n  We currently  scrape [[http://1337x.to][1337x.to]]  (since v0.9), [[https://www.torrentdownloads.me][torrentdownloads.me]]\n  (v0.10) and [[https://torrent-paradise.ml][torrents-paradise]] (v0.11) and present the results sorted by seeders.\n\n  Download for GNU/Linux 64 bits: see [[https://gitlab.com/vindarel/cl-torrents/tags][Releases]] (/23MB, self-contained\n  executable/).\n\nNOTE: torrents-paradise has a rich collection, a simple website as in KISS, it works on IPFS and has\nit offers an API… frankly, I want to use its web service and click on its adds.\n\n#+html: \u003cp align=\"center\"\u003e\u003cimg src=\"assets/img-colored-results.png\" /\u003e\u003c/p\u003e\n\n#+html: \u003cp align='center'\u003e\u003cimg src='assets/img-readline.png' /\u003e\u003c/p\u003e\n\n#+html: \u003cp align=\"center\"\u003e\u003cimg src=\"assets/torrents-tk.png\" /\u003e\u003c/p\u003e\n\nWith the new default Yaru theme (and [[https://lisp-journey.gitlab.io/blog/pretty-gui-in-common-lisp-with-nodgui-tk-themes/][there are more]]):\n\n#+html: \u003cp align=\"center\"\u003e\u003cimg src=\"assets/tk-gui-yaru-theme.png\" /\u003e\u003c/p\u003e\n\n\n** Installation\n\n   The library is on Quicklisp (february 2019) and [[http://ultralisp.org/][Ultralisp]].\n\n: (ql:quickload \"torrents\")\n: (torrents:search-torrents \"tears of steel\")\n\nNote: the master branch needs [[https://github.com/libre-man/cl-transmission/][cl-transmission]] which was added\nto the Quicklisp distribution of december, 2019.\n\n   See the download link of the binary above.  It is a self-contained executable for\n   GNU/Linux  64-bits.    You  do  /not/   need  to  install   a  Lisp\n   implementation. It's a 23MB binary.\n\nTo build the readline app, do:\n\n: make build\n\nTo build the Tk GUI:\n\n: make build-gui\n\nthen run\n\n: ./torrents --help\n: ./torrents-tk \u0026\n\n\n_Troubleshooting_\n\nThe Tk GUI needs [[https://notabug.org/cage/nodgui/][nodgui]] newer than march, 2019.\n\nIf a search doesn't work with a =CRYPTO_num_locks= error message:\n\n#+BEGIN_EXPORT ascii\n\u003e ./torrents matrix\nsearching '(matrix)' on 1337... no results.\nerror: The alien function \"CRYPTO_num_locks\" is undefined.\nsearching '(matrix)' on DOWNLOADSME... no results.\nerror: The alien function \"CRYPTO_num_locks\" is undefined.\n#+END_EXPORT\n\nYou might need to upgrade your OpenSSL version. See [[https://gitlab.com/vindarel/cl-torrents/issues/5][this issue]].\n\n\n** Usage\n\n   We can  use this little app  both in a  Lisp REPL (Slime) and  in a\n   terminal, as a command line tool or from its embedded interactive prompt.\n\n   Results   are  cached   in  =~/.cl-torrents/cache/=.   Delete  this\n   directory if you want new results.\n\n*** Lisp REPL\n\n   Search for torrents:\n\n#+BEGIN_SRC text\n(use-package :torrents)\n(search-torrents \"matrix\")\n  [...]\n  6: The Matrix Revolutions (2003) BRRip [Dual Audio] [Hindi+Eng]\n  5: Matrix (1999)Blu-Ray 720p Dublado PT-BR - mo93438\n  4: The Matrix Trilogy (1999-2003) BDRip 1080p Dual Audio [ Hind\n  3: The Matrix Trilogy (1999-2003) BluRay BDRip 1080p AC3\n  2: The Matrix Trilogy (1999-2003) + Extras 1080p BluRay x264 Du\n  1: The Matrix Reloaded (2003) FullHD, Dual Audio: English + Spa\n  0: Matrix FRENCH DVDRIP 1999 COOL\n#+END_SRC\n\n  To help finding the most  relevant results (the website sometimes is\n  a bit relaxed  with our search terms), each keyword  is printed in a\n  different color:\n\n#+html: \u003cp align=\"center\"\u003e\u003cimg src=\"assets/img-colored-results.png\" /\u003e\u003c/p\u003e\n\n  (you need  to enable ansi  colors in Slime,  see [[https://github.com/enriquefernandez/slime-repl-ansi-color][slime-repl-ansi-color]].\n  Disable this feature with =(setf cl-ansi-text:*enabled* nil)=)\n\n\n  We get a magnet link with:\n\n#+BEGIN_SRC lisp\n(magnet 0)\n;; \"magnet:?xt=urn:btih:40eca43690cf1b99b0a4d485ebf4855d20b0bac5\"\n#+END_SRC\n\n  We can open it with a torrent client (transmission-gtk by default):\n\n#+BEGIN_SRC lisp\n(download 0)\n#+END_SRC\n\n  Download it with a remote client (only transmission-remote so far):\n\n  you need settings, see below. TODO\n#+BEGIN_SRC lisp\n;; in ~/.config/torrents.lisp\n(setf *remote-client-url* \"my.url.without.port\")\n(setf *remote-client-username* \"transmission\")\n(setf *remote-client-password* \"password\")\n;; port is 9091 by default.\n#+END_SRC\n\n#+BEGIN_SRC lisp\n(remote-download 0)\n#+END_SRC\n\n  And voilà :)\n\n  We may add more features, eventually.  Our goal is rather to write a\n  tutorial to show you diverse Common Lisp topics.\n\n**** Parameters\n\n=*cache-p*=: if nil, don't use the cache system.\n\n*** Command line\n\nUse the  options given below,\n\n#+BEGIN_SRC text\n$ torrents -h\nCL-torrents. Usage:\n\nAvailable options:\n  -d, --details            print more details (like the torrent's url)\n  -h, --help               print this help text\n  -i, --interactive        enter an interactive repl\n  -m, --magnet ARG         get the magnet link of the given search result.\n  -n, --nb ARG             set the maximum number of results to print.\n  -o, --open INDEX         download with a torrent client (transmission-gtk by default)\n  -v, --version            print the version\n#+END_SRC\n\nExample: below we  search for \"matrix…\", we display only  1 result and\nwe get the magnet link of the first result.\n\n: $ torrents -n 1 matrix french 1999 cool -m 0\n\n#+html: \u003cp align='center'\u003e\u003cimg src='assets/img-cli.png' /\u003e\u003c/p\u003e\n\n*** Readline interface\n\nStart it with =torrents -i=.\n\nSee the =help= and the help of each command.\n\nThe available commands are (use the auto-completion):\n\n- =search=: search for torrents, print the results sorted by number of seeders.\n- =magnet \u003ci\u003e=, where /i/ is a result index,\n- =open= or =firefox \u003ci\u003e=: open the given torrent page in a web browser\n- =download  \u003ci\u003e=:   open  a  magnet   link  with  a   torrent  client\n  (transmission-gtk by default)\n- =filter  \u003ctext\u003e=:  show  results  that   have  this  text  in  their\n  title. This reduces the tab-completion of ids for all commands.\n- =highlight=  (and  =unhighlight=):  highlight  the  given  words  in\n  yellow, for this session.\n- =url \u003ci\u003e=\n- =nb-results= =n= to set the number of results to print to /n/\n- =details=: toggle the display of details\n- =quit= (or =C-c=)\n\nAnd the available settings to change with =set= are (use =set='s autocompletion):\n\n- =*details*=: if  true, print more  details below each row  (like the\n  url). Defaults to false.\n- =*nb-results*=: how many results to print. Defaults to 20.\n- =*browser*=: the browser name to use to open urls. Defaults to Firefox.\n- =*torrent-client*=: the desktop torrent client name. Defaults to Transmission.\n- =*cache-p*=: if true (the default), use the cached results.\n\nYou can also set them in a configuration file.\n\nNote that we are in  a regular [[https://tiswww.case.edu/php/chet/readline/readline.html#SEC9][readline]] interface, so the  usual keybindings and\noptions are available. For example, Readline obeys the =~/.inputrc=\nstartup file. You can change the way the TAB key does completion:\n\n: TAB: menu-complete\n\nif you add this, the first press on TAB will insert the first\ncompletion candidate (VS listing all the choices, and requiring a\nsecond keypress). For cl-torrents, it is convenient because we can\nenter the first result quickly: I typically do =search foo= then\n=download TAB=.\n\nNote: I found out that =C-x C-r= re-reads the inputrc file, so you can\ntry without quitting cl-torrents.\n\n\n#+html: \u003cp align='center'\u003e\u003cimg src='assets/img-readline.png' /\u003e\u003c/p\u003e\n\n*** Configuration files (in development)\n\n    =cl-torrents= will  read two configuration files.  An ini-style one,\n    and a lispy one.\n\n**** Ini-style config\n\n    First, it will search for a =~.torrents.conf= file\n    from =~/.config/.torrents.conf= and  =~/.torrents.conf=.  The last\n    one takes precedence.\n\n    For example, those are the default values:\n\n#+BEGIN_SRC conf\n[default]\nscrapers = 1337 DOWNLOADSME\nbrowser = firefox\nnb-results = 20\ncache-p = true  # use the cache system.\n#+END_SRC\n\nMoreover, you  can set parameters related to the repl:\n\n#+BEGIN_SRC text\n[default]\n# again, default values:\nconfirm-exit = true  # ask with a yes/no prompt before quiting the app.\nverbose = false\nhistory = true  # use the repl history.\nwrite-history = true  # read the history, but don't write to it.\n#+END_SRC\n\n  You can set  them for all replic-based apps  in =~/.replic.conf=, or\n  override them in =.torrents.conf=.\n\n  /Note: we only use a \"default\" profile for now./\n\n**** Lisp init file\n\nIf the  file =~/.torrents.lisp=  exists, it will  be loaded  after the\n=.conf= one and before the command line arguments.\n\nThe option =--no-userinit= prevents it from loading.\n\nYou can  write whatever you  want there, it is  just a lisp  file that\nwill be  =load='ed at startup.\n\n/Note/:  what we  will eventually  do then  is to  expose cl-torrents'\nmechanism via hooks.\n\n**** Extending the app, creating commands\n\nOne thing you can do is define  additional commands to be found at the\nrepl. We do so by following [[https://github.com/vindarel/replic/][replic]]'s mechanism:\n\n- define functions inside the =torrents.user= package\n- define the completion choices for some functions\n- =export=   the    functions   you   want   to    see   turned   into\n  commands. =cl-torrents=, with the help of the =replic= library, will\n  automatically turn  them into commands available  at the application\n  prompt.\n\nYou  can  copy the  example  below  in =~/.torrents.lisp=  and  re-run\n=torrents -i= to try this out.\n\n#+BEGIN_SRC lisp\n(in-package :torrents.user)\n\n(defparameter *names* '()\n  \"List of names (string) given to `hello`. Will be autocompleted by `goodbye`.\")\n\n(defun hello (name)\n  \"Takes only one argument. Adds the given name to the global\n  `*names*` global variable, used to complete arguments of `goodbye`. \"\n  (format t \"hello ~a~\u0026\" name)\n  (push name *names*))\n\n(defun goodbye (name)\n  \"Says goodbye to name, where `name` should be completed from what was given to `hello`.\"\n  (format t \"goodbye ~a~\u0026\" name))\n\n;; Custom completion for goodbye:\n(replic.completion:add-completion \"goodbye\" (lambda () *names*))\n\n;; and export the two functions to find them as commands.\n(export '(hello\n          goodbye))\n#+END_SRC\n\n** Ideas, todos\n\n- [X] cache (on files) v0.3\n- [X] CI\n- use a local copy of TPB\n- [X] build  with  ECL. Used  [[https://hub.docker.com/r/daewok/lisp-devel/][the  lisp-devel]]  docker\n  image. Result: a 52Mo  executable with a runtime error.\n- [X] open an url\n- download the torrent file\n- [X] ini config file, to parse with [[https://common-lisp.net/project/py-configparser/][py-configparser]]\n- [X] add torrent to a local torrent client\n- [X]  add  the   torrent  to  a  remote   transmission  client,  with\n  [[https://github.com/libre-man/cl-transmission/][cl-transmission]]. v0.12\n- [X] self-contained web app ([[https://lispcookbook.github.io/cl-cookbook/scripting.html][doc]]). POC: [[https://github.com/vindarel/cl-torrents-web][cl-torrents-web]].\n- [X] interactive prompt, with completion of commands.\n\n** Release notes\n\n[[https://gitlab.com/vindarel/cl-torrents/tags][https://gitlab.com/vindarel/cl-torrents/tags]]\n\n*** WIP\n- added:  a simple  Tk GUI  (search,  open in  browser, download  with\n  torrent desktop client).\n\n*** v0.12\n- june, 2021: added [[https://torrent-paradise.ml][torrents-paradise]] API.\n\n*** v0.11\n- november, 2019: added connection to transmission-remote\n- added rarbg scraper (needs new tests)\n- june, 2019: changed the location of the lisp init file from\n  =~/.torrents.lisp= to =~/.config/torrents.lisp=.\n- added:  a =filter=  command,  to only  display  results whose  title\n  contains a given  string. It reduces the TAB-completion  of ids (but\n  doesn't constrain it).\n- added: all functions can now TAB-complete the list of ids.\n- added: scrape and display torrents' size.\n- fixed =-d= cli arg.\n- added:  load  =~/.torrents.lisp=,  create   new  commands  from  the\n  =torrents.user= package.\n- added   the    ability   to   read   an    ini-like   config   file,\n  =~/.torrents.conf=. All exported variables  from the =:torrent= package\n  can be overriden. They are written without earmuffs:\n#+BEGIN_SRC lisp\n           :*nb-results*\n           :*browser*\n           :*torrent-client*\n           :*cache-p*\n#+END_SRC\ngives\n#+BEGIN_SRC text\n# ~/.torrents.conf\n# those are the defaults.\n[default]\nnb-results = 20\nbrowser = firefox\ntorrent-client = firefox\ncache-p = true\n#+END_SRC\n  See more on replic.\n  (all parameters need more tests)\n- added missing =-o= option to download with a torrent client (transmission-gtk).\n- =cl-torrents.conf= configuration file:\n  - choose the list of scrapers.\n- =download= command (open magnet link with transmission by default).\n- use the [[https://github.com/vindarel/replic/][replic]] library to build the repl, the commands and their completions.\n  - -\u003e new builtin =highlight= and =unhighlight= commands,\n  - -\u003e better error handling (don't exit the repl on an error).\n- fix cli usage without a keyword search, print help.\n\n*** v0.10\n\n- added [[https://www.torrentdownloads.me][torrentdownloads.me]] scraper.\n- new parameter, =*cache-p*=: if nil, don't use the cache.\n\n*** v0.9\n\n- added [[http://1337x.to][1337x.to]] scraper.\n- removed piratebay.to, katcr.co, torrent.cd, they're down :/\n\n*** v0.8\n\n- added a readline-based repl  with some commands: =search=, =magnet=,\n  =open=, with their completion.  See =help=.  Built with [[https://github.com/mrkkrp/cl-readline][cl-readline]],\n  but might use [[https://github.com/vindarel/replic][replic]] anytime soon.\n\n*** v0.7\n\n- fix cache not created by binary\n- create cache in =~/.cl-torrents/cache/=\n\n\n* Dev\n\nClone cl-torrents in =~/quicklisp/local-projects=,\n\nnow you can =(ql:quickload :torrents)=.\n\nWe use our [[https://github.com/vindarel/replic/][replic]] library to automatically build the readline repl.\n\nUnit tests:\n\n: make test\n\nEnd-to-end tests (checking that the websites respond and our scrapers still work):\n\n: make test-end2end\n\nTry the Ltk GUI: load =gui-tk.lisp= and run =(main)=.\n\n\n* Resources\n\n  Don't miss these good resources:\n\n- http://lisp-lang.org/\n- [[Https://github.com/CodyReichert/awesome-cl][Awesome-cl]]\n- [[https://lispcookbook.github.io/cl-cookbook/][the Common Lisp Cookbook]]\n- http://quickdocs.org/\n- (my) [[https://lisp-journey.gitlab.io/][lisp-journey]]: introductory posts, lists of resources, selection\n  of screencasts.\n\n** Tutorial\n\n  *update, 2021*: this tutorial is one of the first things I wrote\n  when discovering CL and it probably bitrot a bit. I'd recommend the\n  Cookbook now as I ported most of its useful content there.\n\n   Writing   a   little   web   scraper   like   this   one   is   not\n   difficult. However, I had to spend  some time to find out the right\n   libraries and resources.  It is also  not trivial at first to start\n   a Lisp project.  So the first thing  we did is write a tutorial. It\n   is a mix of hopefully useful stuff:\n\n- web scraping,\n- async web scraping,\n- trying out things at the REPL,\n- where to find documentation,\n- creating and loading a project,\n- basic data structures and gotchas,\n- some useful libraries,\n- unit tests, with mocks, running tests from the shell, continuous integration,\n- parsing command line arguments, creating executables, continuous delivery,\n- basics of error handling,\n- ...\n\n  It will eventually  tackle more topics and  features (more settings,\n  working with a local copy of TPB…) but in the meanwhile, read\n\n  the [[https://vindarel.github.io/cl-torrents/tutorial.html][tutorial]] !\n\n  It was built with https://github.com/fniessen/org-html-themes.\n\n** Bypassing ISP blocking\n\nSome Internet Service Providers block access to torrent sites, as it's\nthe case in France. They currently don't block the HTTP version of\ntorrents-paradise, but that one works on IPFS anyways. You can bypass\nthem by using other DNS servers than your ISP's.\n\nHere's how to do it on Ubuntu: https://doc.ubuntu-fr.org/dns (french)\n\nYou can use [[https://www.opennic.org/][OpenNIC]] addresses or again the ones of the [[https://www.fdn.fr/actions/dns/][French Data Network]] association.\n\n* Licence\n\nMIT.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvindarel%2Fcl-torrents","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvindarel%2Fcl-torrents","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvindarel%2Fcl-torrents/lists"}