{"id":13598397,"url":"https://github.com/cu/silicon","last_synced_at":"2025-04-10T09:31:07.549Z","repository":{"id":65284351,"uuid":"457823533","full_name":"cu/silicon","owner":"cu","description":"Silicon Notes, a web-based personal knowledge base with few frills","archived":false,"fork":false,"pushed_at":"2024-12-26T02:43:22.000Z","size":261,"stargazers_count":223,"open_issues_count":5,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-12-26T03:23:31.134Z","etag":null,"topics":["flask","knowledge-base","python","self-hosted","wiki"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-10T14:55:11.000Z","updated_at":"2024-12-26T02:32:11.000Z","dependencies_parsed_at":"2023-11-11T15:28:23.663Z","dependency_job_id":"f318084a-fba8-483b-bf3d-6356b739d169","html_url":"https://github.com/cu/silicon","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cu%2Fsilicon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cu%2Fsilicon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cu%2Fsilicon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cu%2Fsilicon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cu","download_url":"https://codeload.github.com/cu/silicon/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248191676,"owners_count":21062551,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flask","knowledge-base","python","self-hosted","wiki"],"created_at":"2024-08-01T17:00:52.256Z","updated_at":"2025-04-10T09:31:07.535Z","avatar_url":"https://github.com/cu.png","language":"Python","readme":"[![Integration tests](https://github.com/cu/silicon/actions/workflows/tests.yaml/badge.svg)](https://github.com/cu/silicon/actions/workflows/tests.yaml)\n\n# What's all this, then?!\n\nSilicon Notes: A somewhat lightweight, low-friction personal knowledge base.\n\n![silicon logo](https://gist.githubusercontent.com/cu/addb3a5f6ba1de11b8fb5eedd212d82a/raw/b56dfbf4cd3ac77487e61cfbdeba7519d8b4f487/favicon-view-192.png)\n\nFeatures:\n\n* Plaintext editing in Markdown, rendering in HTML\n* Language syntax highlighting in rendered HTML\n* Bi-directional page relationships\n* Powerful full-text and page title search\n* Page history\n* A table of contents in the left sidebar, where it belongs\n* A quite-usable mobile layout\n* Built-in documentation\n* No-frills UI\n* No big frameworks, just a few smallish dependencies\n\nFor the rationale on why this was created and paper-thin justifications on\ncertain design decisions, see [this blog article](https://blog.bityard.net/articles/2022/December/the-design-of-silicon-notes-with-cartoons).\n\n\u003ca href=\"https://gist.githubusercontent.com/cu/addb3a5f6ba1de11b8fb5eedd212d82a/raw/58e80468acbdd832d012fd776afac6d58357cbb3/view_full.png\"\u003e\n  \u003cimg src=\"https://gist.githubusercontent.com/cu/addb3a5f6ba1de11b8fb5eedd212d82a/raw/58e80468acbdd832d012fd776afac6d58357cbb3/view_full.png\" width=\"400\" height=\"275\"\u003e\n\u003c/a\u003e\n\n\u003ca href=\"https://gist.githubusercontent.com/cu/addb3a5f6ba1de11b8fb5eedd212d82a/raw/bd5c9022462fe6aa6fd239b07a56777275bccf85/view_full_codemirror.png\"\u003e\n  \u003cimg src=\"https://gist.githubusercontent.com/cu/addb3a5f6ba1de11b8fb5eedd212d82a/raw/bd5c9022462fe6aa6fd239b07a56777275bccf85/view_full_codemirror.png\" width=\"400\" height=\"275\"\u003e\n\u003c/a\u003e\n\n\u003ca href=\"https://gist.githubusercontent.com/cu/addb3a5f6ba1de11b8fb5eedd212d82a/raw/58e80468acbdd832d012fd776afac6d58357cbb3/view_mobile.png\"\u003e\n  \u003cimg src=\"https://gist.githubusercontent.com/cu/addb3a5f6ba1de11b8fb5eedd212d82a/raw/58e80468acbdd832d012fd776afac6d58357cbb3/view_mobile.png\" width=\"248\" height=\"275\"\u003e\n\u003c/a\u003e\n\n# Tech Stack\n\nProjects we rely on and appreciate!\n\n* [Python](https://www.python.org/), of course.\n* [uv](https://github.com/astral-sh/uv) for project management.\n* [Flask](https://flask.palletsprojects.com/), the micro-framework.\n* [Mistune](https://github.com/lepture/mistune) to render Markdown into HTML.\n* [Pygments](https://pygments.org/) for syntax highlighting of code blocks.\n* [python-slugify](https://github.com/un33k/python-slugify) creates URL-friendly\n  \"slugs\" from strings.\n* [python-dotenv](https://github.com/theskumar/python-dotenv) for configuration\n  management.\n* [Gunicorn](https://gunicorn.org/) for deployment.\n* [Pytest](https://pytest.org/) and\n  [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/) for functional testing.\n* [CodeMirror](https://codemirror.net/) (optional) for editor syntax\n  highlighting\n\n# Quickstart\n\n## Running Locally\n\nSee the Development section below for steps on running the app locally.\n\nFor production, this project is configured and deployed much like any other\nFlask project.  For details, see the [Flask configuration handling Docs].\n\n[Flask configuration handling docs]: https://flask.palletsprojects.com/en/stable/config/\n\n## Docker or Podman\n\nIf containers are more your speed, the following commands should get you going.\nYou obviously need to have [Docker](https://www.docker.com) installed. (If you\nhave [Podman](https://podman.io), simply substitute `docker` for `podman`.)\n\n```sh\ndocker run \\\n  -ti \\\n  --rm \\\n  -p 127.0.0.1:5000:5000 \\\n  -v silicon_instance:/home/silicon/instance \\\n  docker.io/bityard/silicon\n```\n\nAnd then open http://localhost:5000/ with your local web browser.\n\nYou could also start it with `docker-compose` (or `docker compose`):\n\n```sh\ncd deploy/docker-local\ndocker-compose up\n```\n\nIf you want to build the image, a `Dockerfile` and a `docker-compose.yaml` file\nare provided, so you can build the container with:\n\n```sh\ndocker build -t docker.io/bityard/silicon .\n```\n\nOr with buildah:\n\n```sh\nbuildah build --format docker -t docker.io/bityard/silicon .\n```\n\nSilicon will listen on port 5000 (plaintext HTTP) and stores all application\ndata in `/home/silicon/instance`.\n\n# Development\n\n## Prerequisites\n\nThis repo tries to be somewhat flexible about its tools and workflow. However,\nalong the happy path you will find some combination of Python 3.9 (or better),\n`uv`, Docker/Podman, and `npm`.\n\nInstall [uv](https://github.com/astral-sh/uv) if necessary. If you are not a fan\nof curlpipes, there are many other ways to install it:\n\n* `pip install --user uv`\n* `pipx install uv`\n* download binaries from the [latest release](https://github.com/astral-sh/uv)\n\n## Setup\n\nSome settings can either be set as environment variables or written to a\nfile named `.env` in the project root. For development, this will suffice:\n\n```sh\nWERKZEUG_DEBUG_PIN=off\n```\n\nYou can set any environment variables mentioned in the Flask or Werkzeug\ndocs, but these are some you might care to know about:\n\n* `FLASK_RUN_HOST`: defaults to `127.0.0.1`\n* `FLASK_RUN_PORT`: defaults to `5000`\n* `INSTANCE_PATH`: where the silicon data (in particular the database) is stored\n* `WERKZEUG_DEBUG_PIN`: the PIN to enable the Werkzeug debug console. Set to\n  \"off\" to disable it if you are sure the app is only listening on localhost.\n* `SECRET_KEY`: A string used in session cookies. For development purposes, this\n  can be anything, but for production it should be a 16-byte (or larger) string\n  of random characters. Setting this is optional as the app will create one\n  (and write it to a file in `INSTANCE_PATH`) if one doesn't exist.\n* `SILICON_EDITOR`: When set to `textarea`, this disables the CodeMirror text\n  editor when editing pages and uses a standard textarea element instead.\n\nTo initialize the database after the configuration settings have been set,\nrun the following command. It will create an `instance` directory in the root\nof the project and initialize the SQLite database from `schema.sql`.\n\n```sh\nuv run flask --app silicon init-db\n```\n\n## Running Silicon\n\nRun the project via the `flask` development server:\n\n```sh\nuv run flask --app silicon run --debug\n```\n\nUnless you changed the defaults, you should be able to access the UI on\nhttp://localhost:5000/\n\n## Running tests and flake8\n\nTo run the tests:\n\n```sh\nuv run pytest\n```\n\nIf you have a tmpfs filesystem, you can set the `TMP` environment variable to\nhave test databases created there (which is faster and results in less\nwear-and-tear on your disk):\n\n```sh\nTMP=/dev/shm uv run pytest\n```\n\nTo make sure all code is formatted to flake8 standards, run `flake8`:\n\n```sh\nuv run flake8 --exclude .venv\n```\n\n# Production Deployment\n\nSilicon Notes is a fairly simple web application which contains no built-in\nauthentication or authorization mechanisms whatsoever. If deploying the\napplication on its own, you should only deploy this to a trusted private network\nsuch as a local LAN segregated from the public Internet by a firewall or VPN.\n**If deploying on a public server, you are responsible for ensuring all access\nto it is secure.**\n\nThe `deploy` direcctory contains various sample deployments that may be helpful\nas starting points for a production deployment.\n\nNormally, it is easiest to host applications like this on their own domain or\nsubdomain, such as https://silicon.example.com/. If you would rather host it\nunder a prefix instead (as in https://example.com/silicon), see [this\nissue](https://github.com/cu/silicon/issues/3) for hints on how to do that.\n\n# Configuring the CodeMirror Editor\n\nSupport for [CodeMirror](https://codemirror.net) as a text editor is included by\ndefault. It does add a lot of \"heft\" to the UI, mostly around having to make a\nseparate network request for each language and addon specified. To use it, you\nalso have to install third-party Javascript/CSS static packages by running ONE\nof the following commands:\n\n```sh\n# If you have `npm` installed locally\n(cd silicon/static \u0026\u0026 npm ci)\n```\n\nOr:\n\n```sh\n# if you have `docker` installed\ndocker run -ti --rm -v $PWD/silicon/static:/app -w /app node:alpine npm ci\n```\n\nCurrently only a handful of languages are enabled for syntax highlighting, if\nyou want to edit the list to suit your needs, you can edit\n`silicon/static/js/edit.js`. You can find a list of supported lanauges\n[here](https://codemirror.net/mode/).\n\nTo disable CodeMirror and use a regular textarea instead, add the following to\nyour `.env` file or environment:\n\n```sh\nSILICON_EDITOR=textarea\n```\n\n# Data Export and Import\n\n## SQL\n\nIn the event that a database migration is needed, follow these steps:\n\n1. Stop the Silicon instance.\n2. Pull down the latest version of this repository.\n3. Run `scripts/dump.sh \u003e silicon_data.sql`.\n\nTo import the data:\n\n4. Move or rename the old `instance/silicon.sqlite`, if it exists.\n5. Run `uv run flask --app silicon init-db`.\n6. Run `sqlite3 instance/silicon.sqlite \u003c silicon_data.sql`.\n7. Start the Silicon instance.\n\nOnce you are satisfied that there are no issues, you can archive (or delete)\nthe old `silicon.sqlite` file and `silicon_data.sql`.\n\n## JSON\n\nIf you want to dump your data as JSON (perhaps to import into another system\nor hack on with your own tools), these scripts are not well tested but might do\nthe job.\n\nExport:\n\n```sh\n#!/usr/bin/env sh\n\nDB=instance/silicon.sqlite\n\nfor table in pages relationships; do\n    sqlite3 $DB -cmd '.mode json' \"select * from $table;\" \u003e $table.json\ndone\n```\n\nImport:\n\n```sh\n#!/usr/bin/env sh\n\nsqlite3 instance/silicon.sqlite \u003c\u003c EOF\nINSERT INTO pages (revision, title, body)\nSELECT\n    json_extract(value, '$.revision'),\n    json_extract(value, '$.title'),\n    json_extract(value, '$.body')\nFROM json_each(readfile('pages.json'));\n\nINSERT INTO relationships\nSELECT\n    json_extract(value, '$.title_a'),\n    json_extract(value, '$.title_a')\nFROM json_each(readfile('relationships.json'));\nEOF\n```\n\n# Suggested Contributions\n\n## Clean Up CSS\n\nThe current style sheets were more or less arrived at by trial and error. Any\nhelp in organizing the rules in a more coherent yet extensible way would be\nmuch appreciated.\n\n## Dark Theme\n\nIt would be nice if the CSS adjusted itself to a dark theme based on the\npreference set by the user in the browser. This should be pretty easy since\nalmost all of the colors are in one file.\n\n## Clean up Javascript\n\nTo put it mildly, my JS skills are not the best. I would very much appreciate\nany suggestions on improvements to the small amount of code there is, or\nwhether there is a better way to organize it. I won't bring in any kind of\nJavascript \"build\" tool as a project dependency, though.\n\n## Diffs Between Revisions\n\nThis is pretty standard on wiki-like apps, but it's not a critical feature\nfor me so I haven't yet mustered up the fortitude to implement it. (It would\nalso likely involve adding a diff library as a dependency.)\n\n## Draft Feature / Autosave / Leap-frog Detection\n\nTo prevent the loss of unsaved changes while editing, we use the browser's\n\"are you sure?\" nag if there has been a change to the editing area since the\npage was loaded. However, there are still (at least) two opportunies to lose\nwork:\n\n1. The browser crashes.\n2. Two simulatenous edits of a page in separate tabs or windows.\n\nThe first is rare and the second is not as serious since both revisions are\nsaved. But it is currently up to the user to recognize what happened and\nremedy the situation by hand.\n\nThese are technically three separate features but I believe they would be\nquite closely coupled if implemented together.\n\n## Refine Tests\n\nThe `tests` directory contains functional tests that were deemed the most\nimportant. But they could be better organized and optimized/flexible. Code\ncoverage is not likely very high. Some tests are lacking or missing because I\nwas not able to work out the right way to test certain things.\n\n## Add Anchors to Headings\n\nAnchors on headers are a very common feature on various CMSes. They let you\nlink directly to headings, e.g.:\n\nhttp://example.com/view/page_title#some-section\n\n## Implement a (Better) Task List Plugin\n\nMistune (the Markdown-\u003eHTML renderer) ships with a [task_lists plugin]. It is\nfunctional, but it renders task list items inside an ordinary `\u003cul\u003e` as just\nanother kind of list item. This means the task list items get prefixed with\n_both_ a bullet point and a checkbox. IMO, this is fairly ugly and the \"right\"\nway to display a task list item is the have the checkbox replace the bullet\npoint.\n\n[task_lists plugin]: https://mistune.readthedocs.io/en/latest/plugins.html#task-lists\n\nTo do this right, I think task lists should be their own separate kind of list\nrather than just another item type in regular lists. It should be possible to\naccomplish this with a Mistune plugin.\n\n# Release Process\n\n1. Set the `version` string in `pyproject.toml` to the version to be released.\n1. Run `uv lock`.\n1. Commit `pyproject.toml` and `uv.lock`.\n1. Push a tag containing just the version number. The tag has to be pushed before a release can be made. GitHub will use the tag name as the release name, so a tag prefix as in `release-1.2.3` will not be kosher.\n   ```\n   git log -1\n   git tag \u003cversion\u003e \u003ccommit\u003e\n   git push origin \u003cversion\u003e\n   ```\n1. This will kick off an action to build a container for each arch and tag/push:\n   * `latest`\n   * major.minor.patch\n1. Create a release in GitHub containing URLs to docker images.\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcu%2Fsilicon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcu%2Fsilicon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcu%2Fsilicon/lists"}