{"id":17187987,"url":"https://github.com/icculus/mod_offload","last_synced_at":"2025-03-25T04:27:00.893Z","repository":{"id":106920129,"uuid":"381146105","full_name":"icculus/mod_offload","owner":"icculus","description":"An Apache module to offload bandwidth to caching servers.","archived":false,"fork":false,"pushed_at":"2021-07-06T00:07:40.000Z","size":91,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-30T04:41:32.588Z","etag":null,"topics":["apache-module"],"latest_commit_sha":null,"homepage":"https://icculus.org/mod_offload/","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"zlib","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/icculus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["icculus"],"patreon":"icculus"}},"created_at":"2021-06-28T19:53:24.000Z","updated_at":"2021-08-08T15:17:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"349d201f-be5d-45ac-80cb-3d5391d4b8ad","html_url":"https://github.com/icculus/mod_offload","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icculus%2Fmod_offload","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icculus%2Fmod_offload/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icculus%2Fmod_offload/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/icculus%2Fmod_offload/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/icculus","download_url":"https://codeload.github.com/icculus/mod_offload/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245398099,"owners_count":20608748,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-module"],"created_at":"2024-10-15T01:07:49.233Z","updated_at":"2025-03-25T04:27:00.886Z","avatar_url":"https://github.com/icculus.png","language":"C","readme":"# mod_offload\n\nmod_offload is an Apache module and C program for redistributing server load\nfrom a \"base\" server to one or more \"offload\" servers. This was written\nbecause my site, icculus.org, was constantly slow due to large demand for\nmany large files. Since the webspace was big enough to make a complete copy\nto another server infeasible, and its contents changed frequently enough\nthat I didn't want to limit my users, I came up with this system.\n\nmod_offload is inspired by [The Coral CDN](https://en.wikipedia.org/wiki/Coral_Content_Distribution_Network)\nbut I wanted something more lightweight and less complex. The Coral Cache\nhad to solve problems I could avoid, as can most web operators that aren't\ntrying to mirror the whole internet.\n\nThe gist of this project is that you install the Apache module on the base\nserver, and the C program (as a cgi-bin or standalone daemon), or a legacy PHP\nscript if all else fails, on offload servers where it will handle every\nrequest. The module on the base server will decide if a given web request\nshould be offloaded, and redirects the client to one of the offload servers.\nThe offload server gets a request from the redirected client and sends an\nHTTP HEAD request to the base server, so it can decide if it needs to\n(re)cache the file in question. If so, it pulls a copy from the base server\nvia HTTP and serves it to the client on-the-fly. If not, it feeds the client\nfrom the cached copy. To the users on the base server adding and changing\nfiles and the downloading client, this is all basically transparent, beyond\nthe base server suddenly being less loaded once the caches start owning\ncopies of large and generally-unchanged files.\n\nFor those without a web server at all, the C program can be built as a\nstandalone daemon that provides a very basic HTTP server.\n\nThe C program is designed to take extremely little memory (as a standalone\ndaemon, it takes about 120 kilobytes per request it serves), and can block\n\"download accelerator\" programs that open multiple requests to the server.\n\nOffload servers can block so-called \"download accelerators\"; at most, X\nsimultaneous connections from one IP address may download a given URL.\nAny above that amount are denied. One IP address can download any amount\nof files, and X connections on the \"accelerated\" URL will run at once.\nDownload accelerators, in modern times, usually aren't. Presumably these\nare a relic of sites that throttled bandwidth per-connection...in such a\ncase, they were at best a way to personally game the system, but mostly,\nthey just turn out to be poor citizens of the Internet, as they tend to use\nexponentially more server memory for long periods of time to send the same\nfile, without much increase in performance (actually, they require more\npackets to send the same data). This method will let these people still\nfunction without ruining the experience for everyone else.\n\n","funding_links":["https://github.com/sponsors/icculus","https://patreon.com/icculus"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ficculus%2Fmod_offload","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ficculus%2Fmod_offload","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ficculus%2Fmod_offload/lists"}