Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/debian/debiman
debiman generates a static manpage HTML repository out of a Debian archive
https://github.com/debian/debiman
debian manpages
Last synced: 7 days ago
JSON representation
debiman generates a static manpage HTML repository out of a Debian archive
- Host: GitHub
- URL: https://github.com/debian/debiman
- Owner: Debian
- License: apache-2.0
- Created: 2017-01-06T14:58:51.000Z (almost 8 years ago)
- Default Branch: main
- Last Pushed: 2024-04-19T17:17:55.000Z (9 months ago)
- Last Synced: 2024-12-17T04:06:35.618Z (14 days ago)
- Topics: debian, manpages
- Language: Go
- Homepage:
- Size: 2.29 MB
- Stars: 197
- Watchers: 22
- Forks: 48
- Open Issues: 55
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# debiman
[](https://github.com/Debian/debiman/actions/workflows/main.yml)
[](https://goreportcard.com/report/github.com/Debian/debiman)
## Goals
debiman makes (Debian) manpages accessible in a web browser. Its goals are, in order:
1. **completeness**: all manpages in Debian should be available.
2. **visually appealing** and **convenient**: reading manpages should be fun, convenience features (e.g. permalinks, URL redirects, easy navigation) should be available
3. **speed**: manpages should be quick to load, new manpages should be quickly ingested, the program should run quickly for pleasant development
Currently, there is one known bug with regards to completeness ([#12](https://github.com/Debian/debiman/issues/12)).
With regards to speed, debiman can process all manpages of Debian unstable in **less than 10 minutes** on a modern machine. Incremental updates complete in **less than 15 seconds**. For more details, see [PERFORMANCE.md](https://github.com/Debian/debiman/blob/master/PERFORMANCE.md).
## Prerequisites
* mandoc
* a local or remote Debian mirror or an apt-cacher-ng running on localhost:3142
* a number of Go packages (which `go get` will automatically get for you, see below)
* pault.ag/go/debian
* pault.ag/go/archive
* github.com/golang/protobuf/proto
* golang.org/x/crypto/openpgp
* golang.org/x/net/html
* golang.org/x/sync/errgroup
* golang.org/x/text
## Architecture overview
debiman works in 4 stages:
1. All Debian packages of all architectures of the specified suites are discovered. The following optimizations are used to reduce the number of packages, and hence the input size/required bandwidth:
1. packages which do not own any files in /usr/share/man (as per the Contents- archive files) are skipped.
2. each package is downloaded only for 1 of its architectures, as manpages are architecture-independent.
2. Man pages and auxiliary files (e.g. content fragment files which are included by a number of manpages) are extracted from the identified Debian packages.
3. All man pages are rendered into an HTML representation using mandoc(1).
4. An index file for debiman-auxserver (which serves redirects) is written.
Each stage runs concurrently (e.g. Contents and Packages files are
inspected concurrently), but only one stage runs at a time,
e.g. extraction needs to complete before rendering can start.
## Development quick start
### Set up Go
Install the latest supported version of Go from https://go.dev/dl. If you prefer
to install Go from Debian, ensure you get the same version — if you use Debian
stable, you likely need to install from backports.
### Install debiman
To download, compile and install debiman to `~/go/bin`, run:
```
go install github.com/Debian/debiman/cmd/...@main
```
### Run debiman
To synchronize Debian testing to ~/man and render a handful of packages, run:
```
~/go/bin/debiman -serving_dir=~/man -only_render_pkgs=qelectrotech,i3-wm,cron
```
### Test the output
To serve manpages from ~/man on localhost:8089, run:
```
~/go/bin/debiman-minisrv -serving_dir=~/man
```
Note that for a production setup, you should not use debiman-minisrv. Instead,
refer to the web server example configuration files in example/.
### Recompile debiman
To update your debiman installation after making changes to the HTML
templates or code in your `debiman` git working directory, run:
```
go generate github.com/Debian/debiman/...
go install github.com/Debian/debiman/cmd/...
```
## Synchronizing
For https://manpages.debian.org, we run:
```
flock /srv/manpages.debian.org/debiman/exclusive.lock \
nice -n 5 \
ionice -n 7 \
debiman \
-sync_codenames=oldstable,oldstable-backports,stable,stable-backports \
-sync_suites=testing,unstable,experimental \
-serving_dir=/srv/manpages.debian.org/www \
-local_mirror=/srv/mirrors/debian
```
…resulting in the directories wheezy/, wheezy-backports/, jessie/, jessie-backports/, testing/, unstable/ and experimental/ (respectively).
Note that you will *NOT* need to change this command line when a new version of Debian is released.
When interrupted, you can just run debiman again with the same options. It will resume where it left off.
If for some reason you notice corruption or other mistakes in some manpages, just delete the directory in which they are placed, then re-run debiman to download and re-process these pages from scratch.
It is safe to run debiman while you are serving from `-serving_dir`. debiman will swap files atomically using [rename(2)](https://manpages.debian.org/rename(2)).
## Customization
You can copy the `assets/` directory, modify its contents and start
debiman with `-inject_assets` pointed to your directory. Any files whose
name does not end in .tmpl are treated as static files and will be
placed in -serving_dir (compressed and uncompressed).
There are a few requirements for the templates, so that debiman can
re-use rendered manpages (for symlinked manpages):