{"id":13508420,"url":"https://github.com/fangfufu/httpdirfs","last_synced_at":"2025-04-14T00:59:40.732Z","repository":{"id":42468936,"uuid":"141453630","full_name":"fangfufu/httpdirfs","owner":"fangfufu","description":"A filesystem which allows you to mount HTTP directory listings or a single file, with a permanent cache. Now with Airsonic / Subsonic support! ","archived":false,"fork":false,"pushed_at":"2024-12-06T01:18:39.000Z","size":963,"stargazers_count":805,"open_issues_count":21,"forks_count":60,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-04-14T00:59:28.329Z","etag":null,"topics":["airsonic","filesystem","filesystem-utils","funkwhale","fuse-filesystem","http","http-client","https","libcurl","libfuse","mount","subsonic","subsonic-client"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fangfufu.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-07-18T15:24:36.000Z","updated_at":"2025-04-04T09:29:10.000Z","dependencies_parsed_at":"2024-01-31T08:50:21.957Z","dependency_job_id":"e03f0d3d-16f1-4b68-8c60-95988c1fbc2f","html_url":"https://github.com/fangfufu/httpdirfs","commit_stats":{"total_commits":554,"total_committers":24,"mean_commits":"23.083333333333332","dds":0.2057761732851986,"last_synced_commit":"de9e5942bb7fdffeac081302f4c1668403dd8886"},"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fangfufu%2Fhttpdirfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fangfufu%2Fhttpdirfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fangfufu%2Fhttpdirfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fangfufu%2Fhttpdirfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fangfufu","download_url":"https://codeload.github.com/fangfufu/httpdirfs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248804790,"owners_count":21164132,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airsonic","filesystem","filesystem-utils","funkwhale","fuse-filesystem","http","http-client","https","libcurl","libfuse","mount","subsonic","subsonic-client"],"created_at":"2024-08-01T02:00:52.788Z","updated_at":"2025-04-14T00:59:40.709Z","avatar_url":"https://github.com/fangfufu.png","language":"C","readme":"[![CodeQL](https://github.com/fangfufu/httpdirfs/actions/workflows/codeql.yml/badge.svg)](https://github.com/fangfufu/httpdirfs/actions/workflows/codeql.yml)\n[![CodeFactor](https://www.codefactor.io/repository/github/fangfufu/httpdirfs/badge)](https://www.codefactor.io/repository/github/fangfufu/httpdirfs)\n[![Codacy Badge](https://app.codacy.com/project/badge/Grade/30af0a5b4d6f4a4d83ddb68f5193ad23)](https://app.codacy.com/gh/fangfufu/httpdirfs/dashboard?utm_source=gh\u0026utm_medium=referral\u0026utm_content=\u0026utm_campaign=Badge_grade)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=fangfufu_httpdirfs\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=fangfufu_httpdirfs)\n\n# HTTPDirFS - HTTP Directory Filesystem with a permanent cache, and Airsonic / Subsonic server support\n\nHave you ever wanted to mount those HTTP directory listings as if it was a\npartition? Look no further, this is your solution. HTTPDirFS stands for Hyper\nText Transfer Protocol Directory Filesystem.\n\nThe performance of the program is excellent. HTTP connections are reused through\ncurl-multi interface. The FUSE component runs in the multithreaded mode.\n\nThere is a permanent cache system which can cache all the file segments you have\ndownloaded, so you don't need to these segments again if you access them later.\nThis feature is triggered by the ``--cache`` flag. This is similar to the\n``--vfs-cache-mode full`` feature of\n[rclone mount](https://rclone.org/commands/rclone_mount/#vfs-cache-mode-full)\n\nThere is support for Airsonic / Subsonic server. This allows you to mount a\nremote music collection locally.\n\nIf you only want to access a single file, there is also a simplified\nSingle File Mode. This can be especially useful if the web server does not\npresent a HTTP directory listing.\n\n## Usage\nBasic usage:\n\n    ./httpdirfs -f --cache $URL $MOUNT_POINT\n\nAn example URL would be\n[Debian CD Image Server](https://cdimage.debian.org/debian-cd/). The ``-f`` flag\nkeeps the program in the foreground, which is useful for monitoring which URL\nthe filesystem is visiting.\n\nFor more usage related help, run\n\n    ./httpdirfs --help\n\nor\n\n    man httpdirfs\n\nPlease note that the man page only works if you have installed HTTPDirFS\nproperly.\n\nThe full usage flags is also documented in the [usage](USAGE.md) page.\n\n## Compilation\n\nFor important development related documentation, please refer\n[src/README.md](src/README.md).\n\n### Debian 12 \"Bookworm\"\n\nUnder Debian 12 \"Bookworm\" and newer versions, you need the following\ndependencies:\n\n    libgumbo-dev libfuse3-dev libssl-dev libcurl4-openssl-dev uuid-dev help2man\n    libexpat1-dev pkg-config meson\n\nYou can then compile the program similar to how you compile a typical program\nthat uses the Meson build system:\n\n    meson setup builddir\n    cd builddir\n    meson compile\n\nTo install the program, do the following:\n\n    sudo meson install\n\nTo uninstall the program, do the following:\n\n    sudo ninja uninstall\n\nTo clean the build directory, run:\n\n    ninja clean\n\nFor more information, please refer to this\n[tutorial](https://mesonbuild.com/Tutorial.html).\n\n### Other operating systems\n\nI don't have the resources to test out compilation for Linux distributions\nother than Debian. I also do not have the resources to test out compilation for\nFreeBSD or macOS. Thereforce I have removed the instruction on how to compile\nfor these operating systems in the README for now. Please feel free to send me a\npull request to add them back in. It is known that HTTPDirFS \n[does compile](https://github.com/fangfufu/httpdirfs/issues/165) on FreeBSD.\n\n## Installation\n\nPlease note if you install HTTDirFS from a repository, it can be outdated.\n\n### Debian 12 \"Bookworm\"\n\nHTTPDirFS is available as a package in Debian 12 \"Bookworm\", If you are on\nDebian Bookworm, you can simply run the following\ncommand as ``root``:\n\n apt install httpdirfs\n\nFor more information on the status of HTTDirFS in Debian, please refer to\n[Debian package tracker](https://tracker.debian.org/pkg/httpdirfs-fuse)\n\n### Arch Linux\n\nHTTPDirFS is available in the\n[Arch User Repository](https://aur.archlinux.org/packages/httpdirfs).\n\n### NixOS\n\nHTTPDirFS is available as a [package](https://mynixos.com/nixpkgs/package/httpdirfs).\n\n### FreeBSD\n\nHTTPDirFS is available in the\n[FreeBSD Ports Collection](https://www.freshports.org/sysutils/fusefs-httpdirfs/).\n\n## Airsonic / Subsonic server support\n\nThe Airsonic / Subsonic server support is dedicated the my Debian package\nmaintainer Jerome Charaoui.You can mount the music collection on your\nAirsonic / Subsonic server (*sonic), and browse them using your favourite file\nbrowser.\n\nYou simply have to supply both ``--sonic-username`` and ``--sonic-password`` to\ntrigger the *sonic server mode. For example:\n\n    ./httpdirfs -f --cache --sonic-username $USERNAME --sonic-password $PASSWORD $URL $MOUNT_POINT\n\nYou definitely want to enable the cache for this one, otherwise it is painfully\nslow.\n\nThere are two ways of mounting your *sonic server\n- the index mode\n- and the ID3 mode.\n\nIn the index mode, the filesystem is presented based on the listing on the\n``Index`` link in your *sonic's home page.\n\nIn ID3 mode, the filesystem is presented using the following hierarchy:\n 0. Root\n 1. Alphabetical indices of the artists' names\n 2. The arists' names\n 3. All of the albums by a single artist\n 4. All the songs in an album.\n\nBy default, *sonic server is mounted in the index mode. If you want to mount in\nID3 mode, please use the ``--sonic-id3`` flag.\n\nPlease note that the cache feature is unaffected by how you mount your *sonic\nserver. If you mounted your server in index mode, the cache is still valid in\nID3 mode, and vice versa.\n\nHTTPDirFS is also known to work with the following applications, which implement\nsome or all of Subsonic API:\n\n- [Funkwhale](https://funkwhale.audio/) (requires ``--sonic-id3`` and\n``--no-range-check``, more information in\n[issue #45](https://github.com/fangfufu/httpdirfs/issues/45))\n- [LMS](https://github.com/epoupon/lms) (requires ``--sonic-insecure`` and\n``--no-range-check``, more information in\n[issue #46](https://github.com/fangfufu/httpdirfs/issues/46). To mount the\n[demo instance](https://lms-demo.poupon.dev/), you might also need\n``--insecure-tls``)\n- [Navidrome](https://github.com/navidrome/navidrome), more information in\n[issue #51](https://github.com/fangfufu/httpdirfs/issues/51).\n\n## Single file mode\n\nIf you just want to access a single file, you can specify\n``--single-file-mode``. This effectively creates a virtual directory that\ncontains one single file. This operating mode is similar to the unmaintained\n[httpfs](http://httpfs.sourceforge.net/).\n\ne.g.\n\n    ./httpdirfs -f --cache --single-file-mode https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-11.0.0-amd64-netinst.iso mnt\n\nThis can be useful if the web server does not present a HTTP directory listing.\nThis feature was implemented due to Github\n[issue #86](https://github.com/fangfufu/httpdirfs/issues/86)\n\n## Permanent cache system\n\nYou can cache the files you have accessed permanently on your hard drive by\nusing the ``--cache`` flag. The file it caches persist across sessions, but\ncan clear the cache using ``--cache-clear``\n\n\u003e [!WARNING]\n\u003e If ``--cache-location \u003cdir\u003e`` appears before ``--cache-clear``, the entire\n\u003e directory  ``\u003cdir\u003e`` will be deleted instead. Take caution when specifying\n\u003e non-empty directories to be used as cache.\n\nBy default, the cache files are stored under ``${XDG_CACHE_HOME}/httpdirfs``,\n``${HOME}/.cache/httpdirfs``, or the current working directory ``./.cache``,\nwhichever is found first. By default, ``${XDG_CACHE_HOME}/httpdirfs`` is\nnormally ``${HOME}/.cache/httpdirfs``.\n\nEach HTTP directory gets its\nown cache folder, they are named using the escaped URL of the HTTP directory.\n\nOnce a segment of the file has been downloaded once, it won't be downloaded\nagain.\n\nPlease note that due to the way the permanent cache system is implemented. The\nmaximum download speed is around 15MiB/s, as measured using my localhost as the\nweb server. However after you have accessed a file once, accessing it again will\nbe the same speed as accessing your hard drive.\n\nIf you have any patches to make the initial download go faster, please submit a\npull request.\n\nThe permanent cache system relies on sparse allocation. Please make sure your\nfilesystem supports it. Otherwise your hard drive / SSD will get heavy I/O from\ncache file creation. For a list of filesystem that supports sparse allocation,\nplease refer to\n[Wikipedia](https://en.wikipedia.org/wiki/Comparison_of_file_systems#Allocation_and_layout_policies).\n\n## Configuration file support\n\nThis program has basic support for using a configuration file. By default, the\nconfiguration file which the program reads is\n``${XDG_CONFIG_HOME}/httpdirfs/config``, which by\ndefault is at ``${HOME}/.config/httpdirfs/config``. You will have to create the\nsub-directory and the configuration file yourself. In the configuration file,\nplease supply one option per line. For example:\n\n\t--username test\n\t--password test\n\t-f\n\nAlternatively, you can specify your own configuration file by using the\n``--config`` option.\n\n### Log levels\n\nYou can control how much log HTTPDirFS outputs by setting the\n``HTTPDIRFS_LOG_LEVEL`` environmental variable. For details of the different\ntypes of log that are supported, please refer to\n[log.h](https://github.com/fangfufu/httpdirfs/blob/master/src/log.h) and\n[log.c](https://github.com/fangfufu/httpdirfs/blob/master/src/log.c).\n\n## The Technical Details\n\nFor the normal HTTP directories, this program downloads the HTML web pages/files\nusing [libcurl](https://curl.haxx.se/libcurl/), then parses the listing pages\nusing [Gumbo](https://github.com/google/gumbo-parser), and presents them using\n[libfuse](https://github.com/libfuse/libfuse).\n\nFor *sonic servers, rather than using the Gumbo parser, this program parse\n*sonic servers' XML responses using\n[expat](https://github.com/libexpat/libexpat).\n\nThe cache system stores the metadata and the downloaded file into two\nseparate directories. It uses ``uint8_t`` arrays to record which segments of the\nfile had been downloaded.\n\nNote that HTTPDirFS requires the server to support HTTP Range Request, some\nservers support this features, but does not present ``\"Accept-Ranges: bytes`` in\nthe header responses. HTTPDirFS by default checks for this header field. You can\ndisable this check by using the ``--no-range-check`` flag.\n\n## Press Coverage\n\n- Linux Format - Issue [264](https://www.linuxformat.com/archives?issue=264), July 2020\n\n## Contributors\n\nThanks for your contribution to the project!\n\n[![Contributors Avatars](https://contributors-img.web.app/image?repo=fangfufu/httpdirfs)](https://github.com/fangfufu/httpdirfs/graphs/contributors)\n[![Contributors Count](https://img.shields.io/github/contributors-anon/fangfufu/httpdirfs?style=for-the-badge\u0026logo=httpdirfs)](https://github.com/fangfufu/httpdirfs/graphs/contributors)\n\n## Special Acknowledgement\n\n- First of all, I would like to thank\n[Jerome Charaoui](https://github.com/jcharaoui) for being the Debian Maintainer\nfor this piece of software. Thank you so much for packaging it!\n- I would like to thank\n[Cosmin Gorgovan](https://scholar.google.co.uk/citations?user=S7UZ6MAAAAAJ\u0026hl=en)\nfor the technical and moral support. Your wisdom is much appreciated!\n- I would like to thank [Edenist](https://github.com/edenist) for providing FreeBSD\ncompatibility patches.\n- I would like to thank [hiliev](https://github.com/hiliev) for providing macOS\ncompatibility patches.\n- I would like to thank [Jonathan Kamens](https://github.com/jikamens) for providing\na whole bunch of code improvements and the improved build system.\n- I would like to thank [-Archivist](https://www.reddit.com/user/-Archivist/)\nfor not providing FTP or WebDAV access to his server. This piece of software was\nwritten in direct response to his appalling behaviour.\n\n## License\n\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n","funding_links":[],"categories":["C","Direct Downloads","https"],"sub_categories":["Open Directories"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffangfufu%2Fhttpdirfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffangfufu%2Fhttpdirfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffangfufu%2Fhttpdirfs/lists"}