{"id":15023832,"url":"https://github.com/thomasleister/prosody-filer","last_synced_at":"2025-04-12T06:21:55.624Z","repository":{"id":40541253,"uuid":"137931139","full_name":"ThomasLeister/prosody-filer","owner":"ThomasLeister","description":"Golang mod_http_upload_external server for Prosody and Ejabberd","archived":false,"fork":false,"pushed_at":"2024-02-18T12:53:04.000Z","size":111,"stargazers_count":49,"open_issues_count":5,"forks_count":19,"subscribers_count":10,"default_branch":"develop","last_synced_at":"2025-03-26T01:41:29.690Z","etag":null,"topics":["go","golang","messaging","prosody","xmpp"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ThomasLeister.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-19T18:38:21.000Z","updated_at":"2024-12-05T09:23:53.000Z","dependencies_parsed_at":"2024-06-18T22:57:04.910Z","dependency_job_id":null,"html_url":"https://github.com/ThomasLeister/prosody-filer","commit_stats":{"total_commits":67,"total_committers":11,"mean_commits":6.090909090909091,"dds":"0.26865671641791045","last_synced_commit":"c97fa660078926efa897ba6f52071a758e106041"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThomasLeister%2Fprosody-filer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThomasLeister%2Fprosody-filer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThomasLeister%2Fprosody-filer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThomasLeister%2Fprosody-filer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ThomasLeister","download_url":"https://codeload.github.com/ThomasLeister/prosody-filer/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248525783,"owners_count":21118750,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","messaging","prosody","xmpp"],"created_at":"2024-09-24T19:59:29.504Z","updated_at":"2025-04-12T06:21:55.584Z","avatar_url":"https://github.com/ThomasLeister.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Prosody Filer\n\nA simple file server for handling XMPP http_upload requests. This server is meant to be used with the Prosody [mod_http_upload_external](https://modules.prosody.im/mod_http_upload_external.html) module.\n\n**Despite the name, this server is also compatible with Ejabberd and Ejabberd's http_upload module!**\n\n---\n\n## Why should I use this server?\n\nOriginally this software was written to circumvent memory limitations / issues with the Prosody-internal http_upload implementation at the time. These limitations do not exist, anymore. Still this software can be used with Ejabberd and Prosody as an alternative to the internal http_upload servers. \n\n\n## Download\n\nIf you are using regular x86_64 Linux, you can download a finished binary for your system on the [release page](https://github.com/ThomasLeister/prosody-filer/releases). **No need to compile this application yourself**.\n\n\n## Build (optional)\n\nIf you're using something different than a x64 Linux, you need to compile this application yourself.\n\nTo compile the server, you need a full Golang development environment. This can be set up quickly: https://golang.org/doc/install#install\n\nThen checkout this repo:\n\n```sh\ngo install github.com/ThomasLeister/prosody-filer\n```\n\nand switch to the new directory:\n\n```sh\ncd $GOPATH/src/github.com/ThomasLeister/prosody-filer\n```\n\nThe application can now be build:\n\n```sh\n### Build static binary\n./build.sh\n\n### OR regular Go build\ngo build prosody-filer.go\n```\n\n\n## Set up / configuration\n\n\n### Setup Prosody Filer environment\n\nCreate a new user for Prosody Filer to run as:\n\n```sh\nadduser --disabled-login --disabled-password prosody-filer\n```\n\nSwitch to the new user:\n\n```sh\nsu - prosody-filer\n```\n\nCopy\n\n* the binary ```prosody-filer``` and\n* config ```config.example.toml```\n\nto ```/home/prosody-filer/```. Rename the configuration to ```config.toml```.\n\nMake sure the `prosody-filer` binary is executable: \n\n```\nchmod u+x prosody-filer\n```\n\n\n### Configure Prosody\n\nBack in your root shell make sure ```mod_http_upload``` is **dis**abled and ```mod_http_upload_external``` is **en**abled! Then configure the external upload module:\n\n```lua\nComponent \"uploads.myserver.tld\" \"http_upload_external\"\n    http_upload_external_base_url = \"https://uploads.myserver.tld/upload/\"\n    http_upload_external_secret = \"mysecret\"\n    http_upload_external_file_size_limit = 50000000 -- 50 MB\n```\n\nRestart Prosody when you are finished:\n\n    systemctl restart prosody\n\n\n### Alternative: Configure Ejabberd\n\nAlthough this tool is named after Prosody, it can be used with Ejabberd, too! Make sure you have a Ejabberd configuration similar to this:\n\n```yaml\n  mod_http_upload:\n    put_url: \"https://uploads.@HOST@/upload\"\n    external_secret: \"mysecret\"\n    max_size: 52428800\n```\n\n\n### Configure Prosody Filer\n\nProsody Filer configuration is done via the `config.toml` file in TOML syntax. There's not much to be configured. The most important piece is the `secret` setting, which **needs to match the secret defined in your mod_http_upload_external settings!**\n\n```toml\n### IP address and port to listen to, e.g. \"[::]:5050\"\nlistenport      = \"[::1]:5050\"\n\n### Secret (must match the one in prosody.conf.lua!)\nsecret          = \"mysecret\"\n\n### Where to store the uploaded files\nstoreDir        = \"./upload/\"\n\n### Subdirectory for HTTP upload / download requests (usually \"upload/\")\nuploadSubDir    = \"upload/\"\n```\n\n\nIn addition to that, make sure that the nginx user or group can read the files uploaded\nvia prosody-filer if you want to have them served by nginx directly.\n\n\n### Docker usage \n\nTo build container:\n\n```docker build . -t prosody-filer:latest```\n\nTo run container use:\n\n```docker run -it --rm -v $PWD/config.example.toml:/config.toml prosody-filer -config /config.toml```\n\n\n### Systemd service file\n\nCreate a new Systemd service file: ```/etc/systemd/system/prosody-filer.service```\n\n    [Unit]\n    Description=Prosody file upload server\n\n    [Service]\n    Type=simple\n    ExecStart=/home/prosody-filer/prosody-filer\n    Restart=always\n    WorkingDirectory=/home/prosody-filer\n    User=prosody-filer\n    Group=prosody-filer\n    # Group=nginx  # if the files should get served by nginx directly:\n\n    [Install]\n    WantedBy=multi-user.target\n\nReload the service definitions, enable the service and start it:\n\n    systemctl daemon-reload\n    systemctl enable prosody-filer\n    systemctl start prosody-filer\n\nDone! Prosody Filer is now listening on the specified port and waiting for requests.\n\n### FreeBSD service file\n\n- First, install go and build the static binary with `go install github.com/ThomasLeister/prosody-filer`.\nThe binary should be in `~/go/bin/prosody-filer`. \n- Create an user with `pw user add -n prosodyfiler -c 'Prosody Filer' -d /home/prosodyfiler -m -s /usr/sbin/nologin` \nand copy the binary into `/home/prosodyfiler/`.\n- Put this rc script in `/etc/rc.d/prosody-filer`:\n```\n#!/bin/sh\n\n# PROVIDE: prosody_filer\n# REQUIRE: FILESYSTEMS networking\n# KEYWORDS: upload\n\n. /etc/rc.subr\n\nname=\"prosody_filer\"\nprogram_name=\"prosody-filer\"\ntitle=\"Prosody-Filer\"\nrcvar=prosody_filer_enable\nprosody_filer_user=\"prosodyfiler\"\nprosody_filer_group=\"prosodyfiler\"\nprosody_filer_chdir=\"/home/prosodyfiler/\"\n\npidfile=\"/home/prosodyfiler/${program_name}.pid\"\nrequired_files=\"/home/prosodyfiler/config.toml\"\nexec_path=\"/home/prosodyfiler/${program_name}\"\noutput_file=\"/var/log/${program_name}.log\"\n\ncommand=\"/usr/sbin/daemon\"\ncommand_args=\"-r -t ${title} -o ${output_file} -P ${pidfile} -f ${exec_path}\"\n\nload_rc_config $name\n```\n- Execute `sysrc prosody_filer_enable=YES`\n\nYou can now start the service via `service prosody_filer start` and check\nit's log via `/var/log/prosody-filer.log`.\n\n### Configure Nginx\n\nCreate a new config file ```/etc/nginx/sites-available/uploads.myserver.tld```:\n\n```nginx\nserver {\n    listen 80;\n    listen [::]:80;\n    listen 443 ssl;\n    listen [::]:443 ssl;\n\n    server_name uploads.myserver.tld;\n\n    ssl_certificate /etc/letsencrypt/live/uploads.myserver.tld/fullchain.pem;\n    ssl_certificate_key /etc/letsencrypt/live/uploads.myserver.tld/privkey.pem;\n\n    client_max_body_size 50m;\n\n    location /upload/ {\n        if ( $request_method = OPTIONS ) {\n                add_header Access-Control-Allow-Origin '*';\n                add_header Access-Control-Allow-Methods 'PUT, GET, OPTIONS, HEAD';\n                add_header Access-Control-Allow-Headers 'Authorization, Content-Type';\n                add_header Access-Control-Allow-Credentials 'true';\n                add_header Content-Length 0;\n                add_header Content-Type text/plain;\n                return 200;\n        }\n\n        proxy_pass http://[::]:5050/upload/;\n        proxy_request_buffering off;\n    }\n}\n```\n\nEnable the new config:\n\n    ln -s /etc/nginx/sites-available/uploads.myserver.tld /etc/nginx/sites-enabled/\n\nCheck Nginx config:\n\n    nginx -t\n\nReload Nginx:\n\n    systemctl reload nginx\n\n\n#### Alternative configuration for letting Nginx serve the uploaded files\n\n*(not officially supported - user contribution!)*\n\n```nginx\nserver {\n    listen 80;\n    listen [::]:80;\n    listen 443 ssl;\n    listen [::]:443 ssl;\n\n    server_name uploads.myserver.tld;\n\n    ssl_certificate /etc/letsencrypt/live/uploads.myserver.tld/fullchain.pem;\n    ssl_certificate_key /etc/letsencrypt/live/uploads.myserver.tld/privkey.pem;\n\n    location /upload/ {\n        if ( $request_method = OPTIONS ) {\n                add_header Access-Control-Allow-Origin '*';\n                add_header Access-Control-Allow-Methods 'PUT, GET, OPTIONS, HEAD';\n                add_header Access-Control-Allow-Headers 'Authorization, Content-Type';\n                add_header Access-Control-Allow-Credentials 'true';\n                add_header Content-Length 0;\n                add_header Content-Type text/plain;\n                return 200;\n        }\n\n        root /home/prosody-filer;\n        autoindex off;\n        client_max_body_size 51m;\n        client_body_buffer_size 51m;\n        try_files $uri $uri/ @prosodyfiler;\n    }\n    location @prosodyfiler {\n        proxy_pass http://[::1]:5050;\n        proxy_buffering off;\n        proxy_set_header Host $host;\n        proxy_set_header X-Real-IP $remote_addr;\n        proxy_set_header X-Forwarded-Proto $scheme;\n        proxy_set_header X-Forwarded-Host $host:$server_port;\n        proxy_set_header X-Forwarded-Server $host;\n        proxy_set_header X-Forwarded-For $remote_addr;\n    }\n}\n```\n\n## apache2 configuration (alternative to Nginx)\n\n*(This configuration was provided by a user and has never been tested by the author of Prosody Filer. It might be outdated and might not work anymore)*\n\n```\n\u003cVirtualHost *:80\u003e\n    ServerName upload.example.eu\n    RedirectPermanent / https://upload.example.eu/\n\u003c/VirtualHost\u003e\n\n\u003cVirtualHost *:443\u003e\n    ServerName upload.example.eu\n    SSLEngine on\n\n    SSLCertificateFile \"Path to the ca file\"\n    SSLCertificateKeyFile \"Path to the key file\"\n\n    Header always set Public-Key-Pins: ''\n    Header always set Strict-Transport-Security \"max-age=63072000; includeSubdomains; preload\"\n    H2Direct on\n\n    \u003cLocation /upload\u003e\n        Header always set Access-Control-Allow-Origin \"*\"\n        Header always set Access-Control-Allow-Headers \"Content-Type\"\n        Header always set Access-Control-Allow-Methods \"OPTIONS, PUT, GET\"\n\n        RewriteEngine On\n\n        RewriteCond %{REQUEST_METHOD} OPTIONS\n        RewriteRule ^(.*)$ $1 [R=200,L]\n    \u003c/Location\u003e\n\n    SSLProxyEngine on\n\n    ProxyPreserveHost On\n    ProxyRequests Off\n    ProxyPass / http://localhost:5050/\n    ProxyPassReverse / http://localhost:5050/\n\u003c/VirtualHost\u003e\n```\n\n\n## Automatic purge\n\nProsody Filer has no immediate knowlegde over all the stored files and the time they were uploaded, since no database exists for that. Also Prosody is not capable to do auto deletion if *mod_http_upload_external* is used. Therefore the suggested way of purging the uploads directory is to execute a purge command via a cron job:\n\n    @daily    find /home/prosody-filer/upload/ -mindepth 1 -type d -mtime +28 -print0 | xargs -0 -- rm -rf\n\nThis will delete uploads older than 28 days.\n\n\n## Check if it works\n\nGet the log via\n\n    journalctl -f -u prosody-filer\n\nIf your XMPP clients uploads or downloads any file, there should be some log messages on the screen.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomasleister%2Fprosody-filer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthomasleister%2Fprosody-filer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomasleister%2Fprosody-filer/lists"}