Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/sourcegraph/docsite

The documentation site used by Sourcegraph
https://github.com/sourcegraph/docsite

lsif-enabled

Last synced: about 2 months ago
JSON representation

The documentation site used by Sourcegraph

Awesome Lists containing this project

README 

        
# docsite



A documentation site generator that fits [Sourcegraph](https://sourcegraph.com)'s needs:



- Markdown source files that are browseable on the file system and readable as plain text (without custom directives or complex front matter or configuration)

- Served by an HTTP server, not generated as static HTML files, to eliminate the need for external static site host configuration (which we found to be error-prone)

- Provides built-in site search for all documentation versions



## Usage



```shell

go get github.com/sourcegraph/docsite/cmd/docsite

docsite -h

```



- `docsite check`: check for common problems (such as broken links)

- `docsite serve`: serve the site over HTTP



To use docsite for docs.sourcegraph.com, see "[Documentation site](https://docs.sourcegraph.com/dev/documentation/site)" in the Sourcegraph documentation.



## Checks



The `docsite check` command runs various checks on your documentation site to find problems:



- invalid links

- broken links

- disconnected pages (with no inlinks from other pages)



If any problems are found, it exits with a non-zero status code.



To ignore the disconnected page check for a page, add YAML `ignoreDisconnectedPageCheck: true` to the top matter in the beginning of the `.md` file. For example:



```

---

ignoreDisconnectedPageCheck: true

---



# My page title

```



## Site data



The site data describes the location of its templates, assets, and content. It is a JSON object with the following properties.



- `content`: a VFS URL for the Markdown content files.

- `contentExcludePattern`: a regular expression specifying Markdown content files to exclude.

- `baseURLPath`: the URL path where the site is available (such as `/` or `/help/`).

- `rootURL`: (optional) the root URL (scheme + host). Only used for rare cases where this is absolutely necessary, such as SEO tags fox example.

- `templates`: a VFS URL for the [Go-style HTML templates](https://golang.org/pkg/html/template/) used to render site pages.

- `assets`: a VFS URL for the static assets referred to in the HTML templates (such as CSS stylesheets).

- `assetsBaseURLPath`: the URL path where the assets are available (such as `/assets/`).

- `redirects`: an object mapping URL paths (such as `/my/old/page`) to redirect destination URLs (such as `/my/new/page`).

- `check` (optional): an object containing a single property `ignoreURLPattern`, which is a [RE2 regexp](https://golang.org/pkg/regexp/syntax/) of URLs to ignore when checking for broken URLs with `docsite check`.

- `search` (optional): an object containing a single proprety `skipIndexURLPattern`, which is a [RE2 regexp](https://golang.org/pkg/regexp/syntax/) pattern that if matching any content file URL will remove that file from the search index.

- `forceServedDownloadedContent` (optional) (dev):  While developing locally, you might want to see how docsite performs when it downloads the doc content remotely. With this set to true, docsite will download the content instead of serving from the filesystem



The possible values for VFS URLs are:



- A **relative path to a local directory** (such as `../myrepo/doc`). The path is interpreted relative to the `docsite.json` file (if it exists) or the current working directory (if site data is specified in `DOCSITE_CONFIG`).

- An **absolute URL to a Zip archive** (with `http` or `https` scheme). The URL can contain a fragment (such as `#mydir/`) to refer to a specific directory in the archive.



  If the URL fragment contains a path component `*` (such as `#*/templates/`), it matches the first top-level directory in the Zip file. (This is useful when using GitHub Zip archive URLs, such as `https://codeload.github.com/alice/myrepo/zip/myrev#*/templates/`. GitHub produces Zip archives with a top-level directory `$REPO-$REV`, such as `myrepo-myrev`, and using `#*/templates/` makes it easy to descend into that top-level directory without needing to duplicate the `myrev` in the URL fragment.)



  If the URL contains the literal string `$VERSION`, it is replaced by the user's requested version from the URL (e.g., the URL path `/@foo/bar` means the version is `foo`). ⚠️ If you are using GitHub `codeload.github.com` archive URLs, be sure your URL contains `refs/heads/$VERSION` (as in `https://codeload.github.com/owner/repo/zip/refs/heads/$VERSION`), not just `$VERSION`. This prevents someone from forking your repository, pushing a commit to their fork with unauthorized content, and then crafting a URL on your documentation site that would cause users to view that unauthorized content (which may contain malicious scripts or misleading information).



### Templates



The templates use [Go-style HTML templates](https://golang.org/pkg/html/template/).



- Document pages are rendered using a template named `document.html`.

- Search result pages are rendered using a template named `search.html`.

- The file `root.html`, if it exists, is loaded when rendering any template. You can define common templates in this file.



See the following examples:



- [about.sourcegraph.com/handbook templates](https://github.com/sourcegraph/about/tree/master/_resources/templates)

- [docs.sourcegraph.com templates](https://github.com/sourcegraph/sourcegraph-public-snapshot/tree/main/doc/_resources/templates)



### Redirects



In addition to the `redirects` property in site data, you can also specify redirects in a text file named `redirects` at the top level of the `assets` VFS. The format is as follows:



```text

FROM-PATH TO-URL STATUS-CODE

```



For example:



```text

# Comments are allowed

/my/old/page /my/new/page 308

/another/page https://example.com/page 308

```



### Specifying site data



The `docsite` tool requires site data to be available in any of the following ways:



- A `docsite.json` file (or other file specified in the `-config` flag's search paths), as in the following example:

  ```json

  {

    "content": "../sourcegraph/doc",

    "baseURLPath": "/",

    "templates": "templates",

    "assets": "assets",

    "assetsBaseURLPath": "/assets/",

    "check": {

      "ignoreURLPattern": "(^https?://)|(^#)|(^mailto:support@sourcegraph\\.com$)|(^chrome://)"

    }

  }

  ```

- In the `DOCSITE_CONFIG` env var, using Zip archive URLs for `templates`, `assets`, and `content`, as in the following example:

  ```

  DOCSITE_CONFIG='{"templates":"https://codeload.github.com/sourcegraph/sourcegraph-public-snapshot/zip/refs/heads/main#*/doc/_resources/templates/","assets":"https://codeload.github.com/sourcegraph/sourcegraph/zip/refs/heads/main#*/doc/_resources/assets/","content":"https://codeload.github.com/sourcegraph/sourcegraph/zip/refs/heads/$VERSION#*/doc/","baseURLPath":"/","assetsBaseURLPath":"/assets/","defaultContentBranch":"main"}' docsite serve

  ```



## Development



## Running locally



To run docsite locally and serve on port `:5080`, run:



```shell

go run ./cmd/docsite/... -config docsite.json serve

```



### Force serving downloaded content



For certain use cases you want to have docsite download the docs content as it does with production configuration. To force this behaviour locally you can set `"forceServedDownloadedContent": true` in you `docsite.json` configuration



### Release a new version



1. Build the Docker image for `linux/amd64`:



   ```sh

   docker build -t sourcegraph/docsite .



   # Use buildx if you're on M1

   docker buildx build --platform linux/amd64 -t sourcegraph/docsite .

   ```



1. Tag and push the image to Docker Hub and GCR:

   ```sh

   export VERSION= # e.g. v1.9.1

   docker tag sourcegraph/docsite sourcegraph/docsite:$VERSION

   docker push sourcegraph/docsite

   docker push sourcegraph/docsite:$VERSION

   ```

1. For internal Sourcegraph usage:

   1. Bump the deployed version by updating the SHA-256 image digest in [all files that define `sourcegraph/docsite:latest@sha256`](https://sourcegraph.sourcegraph.com/search?q=context:global+repo:%5Egithub.com/sourcegraph/*+%28NOT+repo:sourcegraph/kube-backup%29+index.docker.io/sourcegraph/docsite:v.*%40sha256:.*&patternType=regexp&sm=1&groupBy=path).

   1. Once the pull request is merged, wait for the [Buildkite build to pass](https://buildkite.com/sourcegraph/deploy-sourcegraph-cloud/builds?branch=release).

1. For development, bump the version number in [files that define `DOCSITE_VERSION`](https://sourcegraph.com/search?q=context:global+repo:%5Egithub.com/sourcegraph/*+%28NOT+repo:sourcegraph/kube-backup%29+DOCSITE_VERSION:&patternType=literal).