https://github.com/codeenigma/wikis2pages
Hugo based static HTML generator that takes Markdown wikis as source
https://github.com/codeenigma/wikis2pages
Last synced: 4 months ago
JSON representation
Hugo based static HTML generator that takes Markdown wikis as source
- Host: GitHub
- URL: https://github.com/codeenigma/wikis2pages
- Owner: codeenigma
- Created: 2020-07-29T16:31:29.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2025-01-21T16:42:09.000Z (10 months ago)
- Last Synced: 2025-03-07T03:07:28.473Z (8 months ago)
- Language: SCSS
- Size: 61.5 KB
- Stars: 0
- Watchers: 8
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# wikis2pages
This is a static generator using Hugo to transform GitHub/GitLab's wiki pages (or any similar markdown) into HTML to create documentation websites.
## Dependencies
You will need to install [`ce-dev`](https://github.com/codeenigma/ce-dev) (which relies on Docker and Docker Compose).
Each repository you want to build a docs website for will also need to have a `.wiki2pages.yml` file in the root, which contains the variables Ansible needs to generate Hugo's config file (which you can see at `ce-dev/ansible/config.toml.j2`). There is an example wiki here you can copy:
* https://github.com/codeenigma/wikis2pages-demo
Here is the annotated `ce-provision` YAML file for reference:
```yaml
ce-provision-2.x: # path to project within the content and public directories
src: https://github.com/codeenigma/ce-provision.git # repository containing the Markdown files to convert
src_branch: 2.x # branch of the repo with the Markdown
src_subdir: 'docs' # subdirectory of the repo where the Markdown is kept (if applicable)
# This step requires a 'git push', we script the adding of the token for pushing in our CI (GitHub Actions in this case) directly
dest: https://github.com/codeenigma/ce-provision-docs.git # destination repo for the generated HTML
dest_branch: master # branch of the target repo to push the HTML to
dest_subdir: 2.x # subdirectory of the target repo to save the generated HTML files into (if applicable)
title: ce-provision # the main page title of the HTML pages
type: doc # the site 'type' (currently only supports 'doc' or an empty string, 'doc' enables automatic sidebar building)
base_url: https://codeenigma.github.io/ce-provision-docs/2.x # the eventual URL of the published HTML website, including the 'dest_subdir' if you set one
```
Once you have `ce-dev` installed and your repo has the above file, you're ready to get started.
## Installing and running wiki2pages
Assuming you have installed `ce-dev` *and* you have created a suitable `.wiki2pages.yml` config file in your target repository pointing to a valid set of Markdown pages to convert to HTML, you can fetch `wiki2pages` itself. In a suitable location on your computer:
```sh
git clone git@github.com:codeenigma/wikis2pages.git
cd wikis2pages
/bin/sh init.sh --repo --branch [my branch]
```
For example, to build the documentation for `ce-provision` at version `2.x` you should use this command to initialise the documentation:
```sh
/bin/sh init.sh --repo git@github.com:codeenigma/ce-provision.git --branch 2.x
```
Behind the scenes this creates variables for your wiki in the `ce-dev/ansible` directory, starts up `ce-dev` and runs a `ce-dev provision` command, which executes the Ansible playbook at `ce-dev/ansible/provision.yml` in your `ce-dev` target container. You can now run the included set-up script, generated by the `provision` step, that configures and allows you to manage multiple `wiki2pages` projects at once. It is a simple command:
```sh
/bin/sh set-current.sh
```
This script will pull down or update the local copy of the documentation source markdown files, write the variables for the Ansible playbook at `ce-dev/ansible/deploy.yml` and run a `ce-dev deploy` command, which executes that playbook in the target container for `ce-dev`.
If you know the name of the project you want to set to current, you can pass it as a parameter and avoid the project list like this:
```sh
/bin/sh set-current.sh --project ce-provision-2.x
```
If you want to just run Hugo again you can run `ce-dev deploy` any time.
At this point Hugo's web server will have started on the target container and you should be able to browse your documentation website. See the easy browsing section below for the usage of `ce-dev browse`.
### Where is my HTML?
There will be a `public` directory that gets generated in the repository root and this is the root of the Hugo web server. To understand where your code will be published you need to look at your `.wiki2pages.yml` file in the target repo. For example, the name of the project in [the file for `ce-deploy` is `ce-deploy-1.x`](https://github.com/codeenigma/ce-deploy/blob/1.x/.wikis2pages.yml#L1) so that will be the directory name in `public`, because this is the value that will get copied into Hugo's `config.toml` file at runtime.
*However*, this will not necesarily be the URL in Hugo to access the files. That will be set according to the `base_url` variable in the same `.wiki2pages.yml` file, which is `https://codeenigma.github.io/ce-deploy-docs/1.x` in the `ce-deploy` example. The templated `hugo-daemon.sh` script, which you will find at `/opt/hugo-daemon.sh` on your `ce-dev` container, is recreated every time you run `set-current.sh` or the `ce-dev deploy` command, as you can see here:
* https://github.com/codeenigma/wikis2pages/blob/master/ce-dev/ansible/deploy.yml
And [we can see in the template for the Hugo daemon script](https://github.com/codeenigma/wikis2pages/blob/master/ce-dev/ansible/hugo-daemon.sh.j2#L23) that it uses the Jinja2 `urlsplit()` function to build the path Hugo will serve on, based on the contents of the `base_url` Ansible variable for the wiki, in this case `ce-deploy-docs/1.x`.
So to recap, the `ce-deploy` config file in the repo root, `.wiki2pages.yml`, looks like this:
```yaml
ce-deploy-1.x:
# other vars here...
base_url: https://codeenigma.github.io/ce-deploy-docs/1.x
```
Which means the code will be built in the `wiki2pages` repo under the path `public/ce-deploy-1.x/1.x` and will be served by Hugo on the URL http://wikis2pages-hugo:4000/ce-deploy-docs/1.x/
### Configuring ce-dev for easy browsing
It's possible to make it easier to browse your generated docs by adding URLs to your `ce-dev` configuration. By default this is configured for our `ce-deploy` and `ce-provision` projects, as we have not yet devised a means to automate it:
```yaml
urls:
- http://wikis2pages-hugo:4000/ce-deploy-docs/1.x
- http://wikis2pages-hugo:4000/ce-provision-docs/2.x
```
But you can edit the file at [`wiki2pages/ce-dev/ce-dev.compose.yml`](https://github.com/codeenigma/wikis2pages/blob/master/ce-dev/ce-dev.compose.yml) and change the list of URLs there. Once that list is correct, run the following to rebuild `ce-dev`:
```sh
ce-dev destroy
ce-dev init
/bin/sh set-current.sh
ce-dev browse
```
The last command there, `ce-dev browse` should open the URLs defined automatically in your default browser.
## Adding another wiki
If you want to add a second wiki, just run the `init.sh` script again. For example, if I want to add a docs project for our `ce-deploy` product at version `1.x` I can just execute this command in the `wiki2pages` repo root:
```sh
/bin/sh init.sh --repo git@github.com:codeenigma/ce-deploy.git --branch 1.x
```
Once I have run that command, if I run `set-current.sh` again, as above, I will have my `ce-deploy` docs in the list of options.
## Building a site without ce-dev
Under certain circumstances, usually when using CI to automatically build docs, you may not want to use `ce-dev`, for example when using CI to build documentation. In this case the steps are different. You can look at the CI for `ce-provision` as an example:
* https://github.com/codeenigma/ce-provision/blob/2.x/.github/workflows/ce-provision-publish-docs.yml#L45
```yaml
- name: Initialise wiki2pages for ce-provision 2.x
run: |
/usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh init.sh --repo https://github.com/codeenigma/ce-provision.git --branch 2.x --no-ce-dev"
/usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/provision.yml"
/usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /bin/sh set-current.sh --project ce-provision-2.x --no-ce-dev"
/usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && /home/ce-dev/ansible/bin/ansible-playbook -e 'wiki2pages_build_path=/home/ce-dev/build/wiki2pages launch_hugo_server=false' -i /home/ce-dev/ansible/bin/hosts /home/ce-dev/build/wiki2pages/ce-dev/ansible/deploy.yml"
- name: Run Hugo
run: |
/usr/bin/su - ce-dev -c "cd /home/ce-dev/build/wiki2pages && hugo"
ls -la /home/ce-dev/build/wiki2pages/public/ce-provision-2.x/
```
As you can see, we run `init.sh` normally at first, but with an extra `--no-ce-dev` flag to prevent it trying to start up `ce-dev` for us in an environment where it doesn't exist. We then run the equivalent of `ce-dev provision` manually with Ansible, before running the `set-current.sh` script which gets generated by the `provision.yml` playbook. Finally we run theb `deploy.yml` playbook manually with Ansible and with an important extra variable, `launch_hugo_server=false`. We *do not* start Hugo for browsing, because we only want to generate the HTML. This is done in the `Run Hugo` step where we simply change to the directory where Hugo's `config.toml` file was created and the other set-up was carried out and run the `hugo` command. The variables in `config.toml` should provide everything Hugo needs to build the site without any other input.
## Known issues and limitations
GitLab wikis lets you reference pages in relative link by name instead of file path, and interpolates them at rendering time.
E.g.
```
[We are Code Enigma](We are Code Enigma)
```
This is a valid link in GitLab's wiki syntax and will "magically" be transformed to `/we-are-code-enigma` in the href (and `we-are-code-enigma.md` in the background).
However, this is not possible with Hugo, and all links need to be standardised:
```
[We are Code Enigma](we-are-code-enigma)
```
This ensures they will work in both places.