Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/All-About-Berlin/ursus
Static site generator for All About Berlin
https://github.com/All-About-Berlin/ursus
Last synced: 2 days ago
JSON representation
Static site generator for All About Berlin
- Host: GitHub
- URL: https://github.com/All-About-Berlin/ursus
- Owner: All-About-Berlin
- Created: 2023-01-09T12:11:28.000Z (almost 2 years ago)
- Default Branch: master
- Last Pushed: 2024-10-29T22:19:02.000Z (10 days ago)
- Last Synced: 2024-10-29T22:52:43.358Z (10 days ago)
- Language: Python
- Size: 201 KB
- Stars: 60
- Watchers: 3
- Forks: 8
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
Awesome Lists containing this project
- my-awesome-github-stars - All-About-Berlin/ursus - Static site generator for All About Berlin (Python)
README
# Ursus
Ursus is the static site generator used by [All About Berlin](https://allaboutberlin.com) and my [personal website](https://nicolasbouliane.com). It turns Markdown files and [Jinja](https://jinja.palletsprojects.com/) templates into a static website.
It also renders images in different sizes, renders SCSS, minifies JS and generates Lunr.js search indexes.
This project is in active use and development.
### What's different
- You can use Jinja `{% include %}` tags and `{{ variables }}` inside your Markdown content. This lets you insert constants and embed components inside your content. For example, I use it to insert a table of contents, calculators and constants in my content.
- It transforms images and supports `imgsrc` and `srcset` out of the box, so responsive images are easy to implement. It can also create PDF thumbnails, and can be extended to transform other files.
- You can build linters for your content. The default linters check if internal links and related entries exist. It's easy to write your own linters.
- It's *very* extensible. You can add your own context processors, renderers and linters. You are not stuck with Markdown and Jinja. You can also create your own Markdown extensions.
- It's relatively fast. [All About Berlin](https://allaboutberlin.com) and its hundreds of pages builds in 5 seconds on an M2 Macbook Air. Live reloads take around 400ms. It was originally built to run smoothly on a much older laptop.
## Setup
### Installation
Install Ursus with pip:
```bash
pip install ursus-ssg
```
### Getting started
Call `ursus` to generate a static website. Call `ursus --help` to see the command line options it supports.
By default, Ursus looks for 3 directories, relative to the current directory:
- It looks for content in `./content`
- It looks for page templates in `./templates`
- It generates a static website in `./output`
For example, create a markdown file and save it as `./content/posts/first-post.md`.
```markdown
---
title: Hello world!
description: This is an example page
date_created: 2022-10-10
---
## Hello beautiful world
*This* is a template. Pretty cool eh?
```
Then, create a page template and save it as `./templates/posts/entry.html.jinja`.
```
{{ entry.title }}
{{ entry.body }}
Created on {{ entry.date_created }}
```
Your project should now look like this:
```
my-website/ <- You are here
├─ content/
│ └─ posts/
│ └─ first-post.md
└─ templates/
└─ posts/
└─ entry.html.jinja
```
Call `ursus` to generate a statuc website. It will create `./output/posts/first-post.html`.
### Configuring Ursus
To configure Ursus, create a configuration file.
```python
# Example Ursus config file
# Find all configuration options in `ursus/config.py`.
from ursus.config import config
config.content_path = Path(__file__).parent / 'blog'
config.templates_path = Path(__file__).parent / 'templates'
config.output_path = Path(__file__).parent.parent / 'dist'
config.site_url = 'https://allaboutberlin.com'
config.minify_js = True
config.minify_css = True
```
If you call your configuration file `ursus_config.py`, Ursus loads it automatically.
```
my-website/
├─ ursus_config.py
├─ content/
└─ templates/
```
You can also load a configuration file with the `-w` argument.
```bash
ursus -c /path/to/config.py
```
### Watching for changes
Ursus can rebuild your website when the content or templates change.
```bash
# Rebuild when content or templates change
ursus -w
ursus --watch
```
It can only rebuild the pages that changed. This is much faster, but it does not work perfectly.
```bash
# Only rebuild the pages that changed
ursus -wf
ursus --watch --fast
```
### Serving the website
Ursus can serve the website it generates. This is useful for testing.
```bash
# Serve the static website on port 80
ursus -s
ursus --serve 80
```
This is not meant for production. Use nginx, Caddy or some other static file server for that.
## AI-translated multilingual site
Ursus *can* use OpenAI to translate your content to multiple languages. This feature is disabled by default. This tool is designed to only translate the parts of your content that change. It does not retranslate entire markdown files, but only the sections that changed.
### 1. Configure translations
Add these to your configuration:
```python
# ursus_config.py
# Where .po, .pot and .mo translation files for your templates are stored. Optional.
config.translations_path = Path('...')
# OpenAI API key used to get translations from ChatGPT
config.openai_api_key = '...'
# The language of the original content
config.default_language = 'en'
# The desired translation languages
config.translation_languages = ['de', 'fr',]
# The Markdown fields you wish to translate. Other fields do not get translated.
config.metadata_fields_to_translate = ('title', 'short_title', 'description', )
```
### 2. Enable translations in your content
To translate Markdown files, add `translation_*` attributes to the metadata. The value is the desired URL of the translated page, relative to `config.site_url`.
```markdown
---
title: How to change your address in Germany
short_title: How to change your address
description: ...
...
translation_de: de/ratgeber/adresswechsel.md
translation_fr: fr/guides/changement-dadresse.md
---
When you [move into a new apartment](/guides/moving-in), you must change your address. It's not automatic. This guide shows you how to do it.
```
In this example, the file `guides/address-change.md` would be turned into 3 entries in `context['entries']`:
- `context['entries']['en/guides/address-change.md']`
- `context['entries']['de/ratgeber/adresswechsel.md']`
- `context['entries']['fr/guides/changement-dadresse.md']`
Each entry has a `translations` attribute that points to other entries. You can use it in your templates to link between entries.
```python
context['entries']['en/guides/address-change.md'] == {
'en': 'en/guides/address-change.md',
'de': 'de/ratgeber/adresswechsel.md',
'fr': 'fr/guides/changement-dadresse.md',
}
```
### 3. Render the entries
Your context now contains the original entry and its translations. If a template can render them, they will be rendered.
In the example above, you would need 3 templates:
- `/en/guides/address-change.html.jinja`
- `/de/ratgeber/adresswechsel.html.jinja`
- `/fr/guides/changement-dadresse.html.jinja`
You can use Jinja includes to use the same template in all 3 places:
```jinja
{# This is the content of de/ratgeber/entry.html.jinja #}
{% include "guides/entry.html.jinja" %}
```
### 4. Translate the templates
Jinja templates support localisation. Use `{% trans %}your string here{% endtrans %}` or `{{ _('your string here') }}`. Call `ursus translate` to generate gettext translation files, then call it again to compile your translation files.
## How Ursus works
1. **Context processors** generate the context used to render templates. The context is just a big dictionary that represent your site's entire content. Usually, each content file is turned into an entry.
2. **Renderers** use the context and the templates to render the parts of the final website: pages, thumbnails, static assets, etc.
### Content
**Content** is what fills your website: text, images, videos, PDFs. Content is usually *rendered* to create a working website. Some content (like Markdown files) is rendered with Templates, and other (like images) is converted to a different file format.
Ursus looks for content in `./content`, unless you change `config.content_path`.
### Entries
A single piece of content is called an **Entry**. This can be a single image, a single markdown file, etc.
Each Entry has a **URI**. This is the Entry's unique identifier. The URI is the Entry's path relative to the content directory. For example, the URI of `./content/posts/first-post.md` is `posts/first-post.md`.
### Context
The **Context** contains the information needed to render your website. It's just a big dictionary, and you can put anything in it.
`context['entries']` contains is a dictionary of all your entries. The key is the Entry URI.
**Context processors** each add specific data to the context. For example, `MarkdownProcessor` adds your `.md` content to `context.entries`.
```python
# Example context
{
'entries': {
'posts/first-post.md': {
'title': 'Hello world!',
'description': 'This is an example page',
'date_created': datetime(2022, 10, 10),
'body': '
Hello beautiful world
...',
},
'posts/second-post.md': {
# ...
},
},
# Context processors can add more things to the context
'blog_title': 'Example blog',
'site_url': 'https://example.com/blog',
}
```
### Templates
**Templates** are used to render your Content. They are the theme of your website. Jinja templates, Javascript, CSS and theme images belong in the templates directory.
Ursus looks for templates in `./templates`, unless you change `config.templates_path`.
### Renderers
**Renderers** use the Context and the Templates to generate parts of your static website. For example, `JinjaRenderer` renders Jinja templates, `ImageTransformRenderer` converts and resizes your images, and `StaticAssetRenderer` copies your static assets.
### Output
This is the final static website generated by Ursus. Ursus generates a static website in `./output`, unless you change `config.output_path`.
The content of the output directory is ready to be served by any static file server.
## How context processors work
Context processors transform the context, which is a dict with information about each of your Entries.
Context processors ignore file and directory names that start with `.` or `_`. For example, `./content/_drafts/hello.md` and `./content/posts/_post-draft.md` are ignored.
### MarkdownProcessor
The `MarkdownProcessor` creates context for all `.md` files in `content_path`. The markdown content is in the `body` attribute.
```python
{
'entries': {
'posts/first-post.md': {
'title': 'Hello world!',
'description': 'This is an example page',
'date_created': datetime(2022, 10, 10),
'body': '
Hello beautiful world
...',
},
# ...
},
}
```
It makes a few changes to the default markdown output:
- Put the front matter in the context
- `related_*` keys are replaced by a list of related entry dicts
- `date_` keys are converted to `datetime` objects
- Other attributes are added to the entry object.
- Use responsive images based on `config.image_transforms` settings.
- `![]()
` are converted to `` or `` tags when appropriate.
- Images are lazy-loaded with the `loading=lazy` attribute.
- Jinja tags (`{{ ... }}` and `{% ... %}`) are rendered as-is. You can use `{% include %}` and `{{ variables }}` in your content.
### GetEntriesProcessor
The `GetEntriesProcessor` adds a `get_entries` method to the context. It's used to get a list of entries of a certain type, and sort it.
```jinja
{% set posts = get_entries('posts', filter_by=filter_function, sort_by='date_created', reverse=True) %}
{% for post in posts %}
...
```
### GitDateProcessor
Adds the `date_updated` attribute to all Entries. It uses the file's last commit date.
```python
{
'entries': {
'posts/first-post.md': {
'date_updated': datetime(2022, 10, 10),
# ...
},
# ...
},
}
```
### ImageProcessor
Adds images and PDFs Entries to the context. Dimensions and image transforms are added to each Entry. Use in combination with `config.image_transforms`.
```python
{
'entries': {
'images/hello.jpg': {
'width': 320,
'height': 240,
'image_transforms': [
{
'is_default': True,
'input_mimetype': 'image/jpeg',
'output_mimetype': 'image/webp',
# ...
},
# ...
]
},
# ...
},
}
```
## How renderers work
Renderers use context and templates to generate parts of the static website.
A **Generator** takes your Content and your Templates and produces an Output. It's a recipe to turn your content into a final result. The default **StaticSiteGenerator** generates a static website. You can write your own Generator to output an eBook, a PDF, or anything else.
### ImageTransformRenderer
Renders images in your content directory.
- Images are converted and resized according to `config.image_transforms`.
- Files that can't be transformed (PDF to PDF) are copied as-is to the output directory.
- Images that can't be resized (SVG to anything) are copied as-is to the output directory.
- Image EXIF data is removed.
This renderer does nothing unless `config.image_transforms` is set:
```python
from ursus.config import config
config.image_transforms = {
# ./content/images/test.jpg
# ---> ./output/images/test.jpg
# ./content/images/test.pdf
# ---> ./output/images/test.pdf
'': {
'include': ('images/*', 'documents/*'),
'output_types': ('original'),
},
# ./content/images/test.jpg
# ---> ./output/images/content2x/test.jpg
# ---> ./output/images/content2x/test.webp
'content2x': {
'include': ('images/*', 'illustrations/*'),
'exclude': ('*.pdf', '*.svg'),
'max_size': (800, 1200),
'output_types': ('webp', 'original'),
},
# ./content/documents/test.pdf
# ---> ./output/documents/pdfPreviews/test.png
# ---> ./output/documents/pdfPreviews/test.webp
'pdfPreviews': {
'include': 'documents/*',
'max_size': (300, 500),
'output_types': ('webp', 'png'),
},
}
```
### JinjaRenderer
Renders `*.jinja` files in the templates directory.
The output file has the same name and relative path as the template, but the `.jinja` extension is removed.
```
my-website/
├─ templates/
│ ├─ contact.html.jinja
│ ├─ sitemap.xml.jinja
│ └─ posts/
│ └─ index.html.jinja
└─ output/
├─ contact.html
├─ sitemap.xml
└─ posts/
└─ index.html
```
Files named `entry.*.jinja` will render every entry with the same relative path.
```
my-website/
├─ content/
│ └─ posts/
│ ├─ first-post.md
│ ├─ second-post.md
│ └─ _draft.md
├─ templates/
│ └─ posts/
│ └─ entry.html.jinja
└─ output/
└─ posts/
├─ first-post.html
└─ second-post.html
```
Files or directory names that start with `.` or `_` are not rendered.
```
my-website/
├─ content/
│ └─ posts/
│ ├─ hello-world.md
│ ├─ .hidden.md
│ └─ _drafts
│ └─ not-rendered.md
├─ templates/
│ └─ posts/
│ └─ entry.html.jinja
└─ output/
└─ posts/
└─ hello-world.html
```
### StaticAssetRenderer
Copies all files under `./templates` except `.jinja` files to the same subdirectory in `./output`. Files starting with `.` are ignored. Files and directories starting with `_` are ignored.
```
my-website/
├─ templates/
│ ├─ _ignored.jpg
│ ├─ styles.css
│ ├─ images/
│ │ └─ hello.png
│ └─ js/
│ └─ test.js
└─ output/
├─ styles.css
├─ images/
│ └─ hello.png
└─ js/
└─ test.js
```
It uses hard links instead of copying files, so it does not use extra disk space.
## How generators work
Generators bring it all together. A generator takes all of your files, and generates some final product. There is only `StaticSiteGenerator`, which generates a static website. Custom generators could generate a book or a slideshow from the same content and templates.
## How linters work
Ursus supports linter. They verify the content when `ursus lint` is called. You can find examples in `ursus/linters`.