{"id":15870552,"url":"https://github.com/ido50/sheepwool","last_synced_at":"2026-01-11T03:03:28.439Z","repository":{"id":54380126,"uuid":"482290296","full_name":"ido50/sheepwool","owner":"ido50","description":"Semi-Static Site Generator and Server","archived":false,"fork":false,"pushed_at":"2024-08-15T10:59:30.000Z","size":812,"stargazers_count":1,"open_issues_count":13,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-01T21:49:05.338Z","etag":null,"topics":["blogging","c","fastcgi","http","lua","openbsd","static-site-generator","website"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"isc","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ido50.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-16T15:27:19.000Z","updated_at":"2025-03-15T00:39:55.000Z","dependencies_parsed_at":"2024-01-24T02:24:34.224Z","dependency_job_id":"66e38380-d4ff-4a8f-a919-ba7169250c5a","html_url":"https://github.com/ido50/sheepwool","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ido50/sheepwool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ido50%2Fsheepwool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ido50%2Fsheepwool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ido50%2Fsheepwool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ido50%2Fsheepwool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ido50","download_url":"https://codeload.github.com/ido50/sheepwool/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ido50%2Fsheepwool/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28274266,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-11T02:08:32.518Z","status":"ssl_error","status_checked_at":"2026-01-11T02:08:32.093Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blogging","c","fastcgi","http","lua","openbsd","static-site-generator","website"],"created_at":"2024-10-06T00:22:03.361Z","updated_at":"2026-01-11T03:03:28.411Z","avatar_url":"https://github.com/ido50.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SheepWool\n\n**Semi-Static Site Generator and Server.**\n\n\u003c!-- vim-markdown-toc GFM --\u003e\n\n- [Synopsis](#synopsis)\n- [Features](#features)\n- [Documentation](#documentation)\n  - [Dependencies](#dependencies)\n  - [Installation](#installation)\n  - [Site Structure](#site-structure)\n  - [Site Generation](#site-generation)\n  - [Site Deployment](#site-deployment)\n- [Status](#status)\n  - [Roadmap](#roadmap)\n- [History](#history)\n- [License](#license)\n\n\u003c!-- vim-markdown-toc --\u003e\n\n## Synopsis\n\n**SheepWool** is a C application for the generation and serving of dynamic websites.\nIt takes a directory tree of static files, templated HTML files, and Lua scripts;\nparses them into a SQLite database, and serves the database over FastCGI.\n\nThis allows website creators to work exactly as with static-site generators\nduring development, but still have a fully dynamic website. The SQLite database\ncan also be used for whatever purpose needed, thus making SheepWool useable\nas a simple web application framework.\n\nSheepWool currently targets OpenBSD and Linux.\n\n## Features\n\n- Entire website content stored in a SQLite database.\n- Templated HTML resources via [etlua](https://github.com/leafo/etlua) (no Markdown support, just pure HTML).\n- Dynamic resources via [Lua](https://lua.org/) 5.3.\n- Automatic compilation of scss files to css.\n- Ability to create redirects and other responses such as 410 Gone.\n- Ability to execute SQL commands from SQL files during website build.\n- Automatic sitemap generation.\n- Sandboxed under OpenBSD using [`pledge(2)`](https://man.openbsd.org/pledge) and [`unveil(2)`](https://man.openbsd.org/unveil) for security.\n\n## Documentation\n\n### Dependencies\n\n- [SQLite](https://sqlite.org/) 3.35.5+\n- [kcgi](https://kristaps.bsd.lv/kcgi/) 0.13.0+\n- [Lua](https://lua.org/) 5.3\n- [libsass](https://sass-lang.com/libsass)\n- libmagic\n- [libcurl](https://curl.se/libcurl/)\n\n### Installation\n\n```\n% ./configure\n% make\n# make install\n```\n\nNote that `make install` currently installs the `sheepwool` binary into the\n/var/www/cgi-bin directory. Example commands in the following sections take into\naccount that this directory is in your `$PATH`.\n\n### Site Structure\n\nWhen generating the website's database, SheepWool recursively traverses the\nsource tree and utilizes the following flow:\n\n- If the root directory contains a file called META, it will be used to create\n  resources with special status codes. The file should contain lines in the\n  following format:\n\n  ```\n  /path/to/resource status [/path/to/new/resource]\n  ```\n\n  Where `status` is `not_found`, `gone` or `permanent`. The first two will cause\n  requests to /path/to/resource to return 404 and 410 statuses, respectively.\n  If the status is `permanent`, a new path for an existing resource should be\n  provided as well, and requests to the old path will result in a 301 redirect\n  to the new path.\n\n- If a directory contains a file called .nowool, the directory is completely\n  skipped (including all its subdirectories).\n\n- If a file is called import.sql (in any directory), it is treated as input for\n  SQLite and simply executed on the database. This is useful for creating new\n  tables, inserting custom data, _et cetra_. It should be noted that every\n  generation of the file re-runs it, so it should handle conflicts gracefully.\n\n- If a file has an .html extension, SheepWool will strip its .html extension\n  for purposes of the file's path in the website (called \"slug\" internally),\n  and read it to look for metadata in HTML comments. Here is an example of an\n  HTML file called hello-world.html in the root directory of the website, and\n  that includes all supported meta attributes:\n\n  ```html\n  \u003c!-- name: Hello World --\u003e\n  \u003c!-- tags: one, two --\u003e\n  \u003c!-- status: pub --\u003e\n  \u003c!-- template: /templates/post --\u003e\n  \u003c!-- ctime: 2022-01-02T16:28:52 --\u003e\n  \u003c!-- mtime: 2022-01-03T12:40:32 --\u003e\n\n  \u003cp\u003eHello World\u003c/p\u003e\n  ```\n\n  This will generate a resource called /hello-world in the website database,\n  whose name/title is \"Hello World\", tagged with the \"one\" and \"two\" tags,\n  whose status is \"pub\" (for \"published\", the default), using the template file\n  /templates/post.html, and with the provided creation and last modification\n  dates from the ctime and mtime attributes, respectively.\n\n  Other supported statuses are \"unpub\" (for unpublished), \"gone\" and \"moved\".\n\n  When serving this file, SheepWool will render the template file /templates/post\n  with the variable \"content\" containing the content of hello-world.html, and\n  more variables described below. Templates themselves can define their own\n  templates, thus creating a chain of templates (there is no limit on the number\n  of templates in the chain). Each time a template is rendered, the entire\n  output is used as the \"content\" variable for the next template in the chain.\n\n  The following variables are available in templates:\n\n  - `status`: the HTTP status code to be returned (usually 200). An integer.\n  - `baseurl`: the base URL of the request (e.g. \"https://my-domain.com\"). A\n    string. Never contains a trailing slash.\n  - `reqpath`: the request's path (e.g. \"/hello-world\"). A string.\n  - `slug`: the matched resource's slug, usually the same as `reqpath`.\n  - `srcpath`: the path in the source directory from which the resource was built.\n  - `name`: the name of the resource (from the \"name\" meta attribute, if exists,\n    otherwise the name of the file, i.e. \"hello-world.html\").\n  - `content`: the content of the resource.\n  - `ctime`: the resource's creation time, a string in the format \"%Y-%m-%dT%H:%M:%S\".\n  - `mtime`: the resource's modification time, same format as `ctime`.\n  - `tags`: a list of zero or more tags.\n\n  See the [etlua documentation](https://github.com/leafo/etlua) for more information on how to write templates.\n\n- If a file has a .lua extension, it is assumed to be a dynamic resource.\n  SheepWool will strip is .lua extension, and insert it into the database as-is.\n  The file can have any format, but MUST return a table with a key called\n  \"render\", which contains a function of the following format:\n\n  ```lua\n  function render(sheepwool, db, req)\n  ```\n\n  In this function, `sheepwool` is a library-table containing utility functions\n  useable by the function; `db` is a pointer to the SQLite database connection;\n  `req` is a table containing metadata about the request.\n\n  The `render` function should return two values: the MIME type of the resource,\n  and the contents. This means Lua resources can be used to generate any type\n  of content, not just HTML.\n\n  The following functions are included in the `sheepwool` library:\n\n  - `query`: executes a SQL query against the database. Always returns an array,\n    even if only one row is returned. Accepts the following parameters: `db`\n    (the database connection pointer), `sql` (the SQL query to execute), and\n    zero or more binding parameters of any type.\n\n  - `execute`: executes a command in a shell with optional standard input and\n    returns its output. Accepts the following parameters: `cmd` (the name or path\n    of the command to execute); `args` (an array of arguments to the command);\n    and optionally `stdin` (data to pipe into the command's standard input).\n    Returns two values: the command's standard output, and the size of the\n    output.\n\n  - `render`: renders an HTML template from the database and returns the output.\n    Accepts the following parameters: `db` (the database connection pointer),\n    `tmpl_name` (the name of the template to render, e.g. \"/templates/post\"),\n    and `context` (a table of metadata to make available as variables inside the\n    template).\n\n  - `post`: makes an HTTP POST request to a URL and returns the response body.\n    Accepts the following parameters: `url` (the URL of the request), `headers`\n    (a table of request headers) and `body` (the body of the request).\n\n  - `base64_encode`: encodes a binary value in base 64 encoding. Accepts the\n    value to encode, and returns the encoded value.\n\n  The following data is included in the `req` table:\n\n  - `scheme`: the request scheme, e.g. \"https\" or \"http\".\n  - `host`: the request host, e.g. \"my-domain.com\".\n  - `method`: the request method, e.g. \"GET\".\n  - `root`: the root path (i.e. SCRIPT_NAME), usually empty.\n  - `path`: the request path, e.g. \"/hello-world\".\n  - `remote`: the client address.\n  - `status`: the (currently estimated) response status, integer.\n  - `params`: a table of request parameters from query string or body.\n    File uploads are also supported. The table will include a table for each\n    uploaded file, with the keys `content`, which includes the contents of the\n    file in base64 encoding; `mime`, the MIME type of the file; and `size`, the\n    size of the file (before base64 encoding).\n  - `headers`: a table of request headers.\n\n  Here's an example of a Lua resource that loads data from the database,\n  renders it into an HTML template, and returns the output.\n\n  ```lua\n  function render(sheepwool, db, req)\n      -- Load five latest posts from the blog\n      local latest_posts = sheepwool.query(db, [[\n          SELECT r.slug, r.name, r.ctime, r.mtime, group_concat(t.tag) AS tags\n          FROM resources r\n          LEFT JOIN tags t ON t.slug = r.slug\n          WHERE r.slug LIKE '/blog/%' AND r.status = 0\n          GROUP BY r.slug\n          ORDER BY r.ctime DESC\n          LIMIT 5\n      ]])\n\n      -- Render into a template\n      return \"text/html\", sheepwool.render(db, \"/templates/index\", {\n          [\"name\"] = \"Some Guy's Personal Blog\",\n          [\"posts\"] = latest_posts,\n      })\n  end\n\n  return {\n      [\"render\"] = render,\n  }\n  ```\n\n- If a file has a .scss extension, it is compiled using libsass and stored as a\n  css file. For example, a file called /styles/main.scss will generate a resource\n  called /styles/main.css.\n\n- If a file in the root directory is called error.html or error.lua, it will be\n  used for rendering error responses. This happens whenever a resource cannot be\n  served for any reason: the requested resource does not exist, or the rendering\n  failed. The same rules as for HTML and Lua files apply. You will probably\n  want to look at the response status in this file to generate different error\n  responses based on the situation.\n\n- Otherwise, the file is treated as a static file. SheepWool will attempt to\n  find out its MIME type using libmagic, and store it in the database as-is.\n\n- When all files have been processed, SheepWool generates a sitemap resource\n  under the /sitemap.xml path.\n\n### Site Generation\n\nTo build or synchronize the website's database file from a source directory,\nsimply execute:\n\n```sh\nsheepwool build /path/to/database /path/to/source\n```\n\nNote that the `build` command will override existing data in the \"resources\",\n\"tags\" and \"vars\" tables, but leave any other tables as-is, so the command\nshould be safe to run on production databases.\n\n### Site Deployment\n\nOn OpenBSD, it is recommended to use [`kfcgi`](https://kristaps.bsd.lv/kcgi/kfcgi.8.html) to deploy a SheepWool website.\nThe `kfcgi` binary is part of the kcgi dependency.\n\nTo serve a website previously built via SheepWool, execute:\n\n```sh\nkfcgi -- sheepwool serve /path/to/database\n```\n\nYou will probably wish to do this in an rc script.\n\nI haven't been able to successfuly use `kfcgi` on Linux, and instead use\n[`spawn-fcgi`](https://github.com/lighttpd/spawn-fcgi), like so:\n\n```sh\nspawn-fcgi -- sheepwool serve /path/to/database\n```\n\n## Status\n\nI am currently using SheepWool to build and serve several websites, including\nmy [personal website](https://ido50.net/), but I cannot guarantee that it is properly secure and\nfree of critical bugs. While I have many years of development experience with\nmany programming languages, this is my first serious C project, so I wouldn't\nbe surprised if there are memory/security-related issues.\n\n### Roadmap\n\nSee [list of enhancements](https://github.com/ido50/sheepwool/issues?q=is%3Aopen+is%3Aissue+label%3Aenhancement) in the issues page.\n\n## History\n\nI've been using the name SheepWool for my content management systems for over\ntwenty years now. Originally, it was a Perl/CGI application, then a Perl/FastCGI\napplication, then a Perl/PSGI application, then a Go application, then a\nPython/ASGI application, and finally a C/FastCGI application. I have written and\nre-written it all these times to fit my needs. There really isn't any resemblance\nbetween this iteration of SheepWool and earlier ones, but it made sense to me\nto continue using this name. Since there were at least four earlier iterations,\nthis one is starting at version 5.0.0, though this is entirely arbitrary.\n\n## License\n\nAll sources use the ISC (like OpenBSD) license. See the [LICENSE.md](LICENSE.md) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fido50%2Fsheepwool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fido50%2Fsheepwool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fido50%2Fsheepwool/lists"}