Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/AndydeCleyre/colorcodebot
A simple Telegram bot for syntax highlighting
https://github.com/AndydeCleyre/colorcodebot
Last synced: 3 months ago
JSON representation
A simple Telegram bot for syntax highlighting
- Host: GitHub
- URL: https://github.com/AndydeCleyre/colorcodebot
- Owner: AndydeCleyre
- License: wtfpl
- Created: 2018-04-18T19:16:10.000Z (over 6 years ago)
- Default Branch: develop
- Last Pushed: 2024-01-15T18:25:44.000Z (10 months ago)
- Last Synced: 2024-06-28T09:35:27.481Z (5 months ago)
- Language: Python
- Size: 501 KB
- Stars: 44
- Watchers: 5
- Forks: 6
- Open Issues: 9
-
Metadata Files:
- Readme: README.rst
- License: LICENSE
Awesome Lists containing this project
README
==============
Color Code Bot
==============
Share code snippets as beautiful syntax-highlighted images and HTML on Telegram
===============================================================================
.. list-table::
:widths: auto
:align: center
* - |telegram|
- |quay|
- |actions-ctnr|
- |actions-reqs|
- |actions-fmt|
It's a small bit of Python glue between great projects, including:
- highlight_ (lua, renders HTML)
- pyTelegramBotAPI_
- Silicon_ (rust, renders image)
- guesslang_ (uses TensorFlow; saves you the step of specifying the snippet's language)
- Iosevka_ (the most wonderful monospaced font)
- Fallback fonts:
- Symbols Nerd Font
- NanumGothicCoding
- OpenMoji
The background image is from [Sharon McCutcheon](https://unsplash.com/@sharonmccutcheon).
Usage
-----
Send `@colorcodebot`_ the code you want highlighted,
as a forwarded or original direct message.
Or, add it to your group
and it will send an image of any ``monospace content`` sent in the chat.
.. image:: https://user-images.githubusercontent.com/1787385/174667742-32e414b4-e4f4-41f8-ae38-d6d64c1075f2.png
:alt: Screenshot of the bot in action
:align: right
As a convenience, you can get to a direct chat with it from any other chat,
by typing ``@colorcodebot`` and tapping the button that pops up.
A button returning you (with a shiny new image)
to your original chat will be presented after you send the code.
Development & Deployment
------------------------
The bot should run anywhere with Python, fontconfig, highlight_, Silicon_, and the ability to install TensorFlow.
Or anywhere that can run a container image.
Depending on your hardware, you may see faster syntax guessing (from guesslang_)
by installing ``cuda`` and ``cudnn`` packages.
This is *not* done for the currently hosted container images,
which is the result of ``./mk/ctnr.sh -d prod --push`` run by a GitHub Action.
Outside of the core Python app,
sops_ is used for secrets,
buildah_ for container building,
GitHub Actions for automated container image builds and other CI tasks,
and `wheezy.template`_ and yamlpath_ are extremely handy for
defining+rendering service definitions and other dev/ops maneuvers.
Most of the ``mk/`` and ``start/`` scripts are POSIX,
but ``mk/svcs.zsh`` requires Zsh,
and ``mk/ctnr.sh`` calls ``mk/svcs.zsh``.
Please do `send a message`_ or open an issue with any questions.
Organization
~~~~~~~~~~~~
An abbreviated file tree overview:
.. code:: shell
colorcodebot/
├──app/ # core app that gets deployed
│ ├──requirements.in # loosely versioned reqs for the bot
│ └──sops/ # encrypted deployment-specific data
├──vars..yml # unencrypted deployment-specific data
├──start/ # scripts that help start the bot
├──mk/ # scripts that make things
├──templates/ # used by mk/ scripts to generate files
└──ops-requirements.in # loosely versioned reqs for mk/ and start/ scripts
The following are generated by ``mk/`` scripts:
.. code:: shell
colorcodebot/
├──app/
│ ├──requirements.txt # mk/reqs.sh - lockfile for the bot
│ ├──svcs/ # mk/svcs.zsh - supervised process definitions for s6 [untracked]
│ └──theme_previews.yml # mk/file_ids.sh - {theme_name: image_id} [untracked]
└──ops-requirements.txt # mk/reqs.sh - lockfile for mk/ and start/ scripts
When building a container image with ``mk/ctnr.sh``,
``app`` becomes ``/home/colorcodebot``.
If you want to use the container images already built from this repo,
you'll probably write or mount over:
- ``/home/colorcodebot/theme_previews.yml``
- ``/home/colorcodebot/svcs``
- ``/home/colorcodebot/sops``
- ``/home/colorcodebot/.sops.yaml``
Getting Started
~~~~~~~~~~~~~~~
To run ``colorcodebot.py``, the environment variable ``TG_API_KEY`` must be set,
with a token from `@botfather`_.
.. code:: console
$ python3 -m venv app/venv
$ . ./app/venv/bin/activate
$ python -m pip install -r app/requirements.txt
$ TG_API_KEY='...' ./app/colorcodebot.py
After chatting with the bot, check the logs for your ``chat_id``.
Pass this as an additional environment variable ``ADMIN_CHAT_ID`` to get:
- an updated SQLite db file sent to that chat whenever a user sets a preferred theme
Deployments, Secrets, and Scripts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Encrypted Variables
^^^^^^^^^^^^^^^^^^^
Configure Sops
""""""""""""""
Create one or more age_ keys to use with sops_:
.. code:: console
$ mkdir -p ~/.config/sops/age
$ printf '%s\n' '' '# --- colorcodebot ---' >>~/.config/sops/age/keys.txt
$ age-keygen >>~/.config/sops/age/keys.txt
Public key: age1r50agxl277e24h4ammj0kvpqh224ut8ds67qc2d537dq0uy74shq98dh97
And use that public key in ``.sops.yaml`` to match your desired deployments.
Write colorcodebot Variables
""""""""""""""""""""""""""""
Overwrite ``app/sops/colorcodebot..yml`` with
.. code:: yaml
TG_API_KEY:
(and optionally ``ADMIN_CHAT_ID``) and encrypt it with
.. code:: console
$ sops -e -i app/sops/colorcodebot..yaml
.. You can set ``host`` and ``port`` in ``app/sops/papertrail..yml``
.. the same way, if using that service.
Load colorcodebot Variables
"""""""""""""""""""""""""""
.. code:: console
$ ./start/local.sh -h
Start the bot locally, without process supervision or other svcs
Args: [-d =dev]
You can use ``start/local.sh`` to:
- ensure Python lockfile is updated
- ensure a virtual environment exists
- ensure the venv has all Python dependencies installed
- ensure the venv is activated if one is not already
- update or create ``app/theme_previews.yml`` if file IDs are present in ``vars..yml``
- load decrypted values from ``app/sops/colorcodebot..yml`` into environment variables
- launch the bot (unsupervised, no other services)
You can do just those last two (as seen in the script) with
.. code:: console
$ sops exec-env "app/sops/colorcodebot.${deployment}.yml" app/colorcodebot.py
Unencrypted Variables
^^^^^^^^^^^^^^^^^^^^^
A deployment's unencrypted variables are defined by ``vars..yml``.
There are two top-level keys:
``theme_previews``
mapping of theme names to Telegram file IDs; see `Generating Theme Previews`_
used by: ``mk/file_ids.sh``, ``mk/ctnr.sh``
``svcs``
list of mappings that each define a long-running supervised service
(the bot and optionally a log sender for Papertrail_)
used by: ``mk/svcs.sh``, ``mk/ctnr.sh``
The deployments ``dev`` and ``prod`` are both intended to run inside a container,
built by ``mk/ctnr.sh``.
Note the difference between the ``svc`` definitions
of ``vars.dev.yml`` and ``vars.prod.yml``:
.. code:: diff
--- vars.dev.yml 2021-06-28 11:13:46.347838948 -0400
+++ vars.prod.yml 2021-07-12 14:22:07.638842356 -0400
@@ -4,7 +4,7 @@
exec: >-
sops exec-env
- sops/colorcodebot.dev.yml
+ sops/colorcodebot.prod.yml
"s6-setuidgid colorcodebot ./venv/bin/python
./colorcodebot.py"
@@ -16,7 +16,7 @@
exec: >-
sops exec-file --filename log_files.yml
- ../log_files.dev.yml
+ ../log_files.prod.yml
"remote_syslog -D -c {}"
@@ -24,7 +24,7 @@
sops_templates:
- src: papertrail.log_files.yml.wz
- dest: log_files.dev.yml
+ dest: log_files.prod.yml
- differences:
+ which encrypted variables get set in the environment of the bot process
+ which encrypted config file is created for and read by the remote logger
Now let's compare ``vars.dev.yml`` to ``vars.local.yml``:
.. code:: diff
--- vars.dev.yml 2021-06-28 11:13:46.347838948 -0400
+++ vars.local.yml 2021-07-12 13:57:00.414719676 -0400
@@ -6,14 +6,15 @@
- "s6-setuidgid colorcodebot ./venv/bin/python
+ "./venv/bin/python
./colorcodebot.py"
folder:
run: ../../
log: ../../../logs/colorcodebot
+ cgroups: /sys/fs/cgroup/user.slice/user-1000.slice/[email protected]/app.slice/svcs
- name: papertrail
- enabled: true
+ enabled: false
@@ -22,6 +23,7 @@
folder:
run: log
log: ../../../logs/papertrail
+ cgroups: /sys/fs/cgroup/user.slice/user-1000.slice/[email protected]/app.slice/svcs
- similarities:
+ which encrypted configs are used
- differences:
+ ``local``: no user changing (no ``s6-setuidgid``)
+ ``local``: overrides the default cgroup path used by services with a systemd-flavored one
+ ``local``: disables optional Papertrail remote logging service
Modify one of these to your liking, or copy to ``vars..yml`` with your own deployment name, e.g.:
.. code:: console
$ cp vars.local.yml "vars.$(hostname).yml"
Generating Theme Previews
~~~~~~~~~~~~~~~~~~~~~~~~~
---
OUTDATED!
TODO: Fill in the missing steps in this section
now that the ``/previews`` command has been removed
---
highlight_ has *many* themes, so we picked a subset.
For the user to choose a theme, we need to generate preview images,
and save their file IDs.
Start by creating ``app/theme_previews.yml`` either manually or with ``./mk/file_ids.sh``
.. code:: console
$ ./mk/file_ids.sh -h
Generate theme_previews.yml, with data from vars..yml
Args: [-d =dev] [=app/theme_previews.yml]
For now the value of each entry can be garbage,
what's important is that the keys are the names of the themes you wish to offer.
Send the ``/previews`` command to the bot, and the file IDs you need
will show up in the log as preview images are generated and sent your way.
Enter those into ``vars..yml``,
then generate ``app/theme_previews.yml`` for local deployment with ``mk/file_ids.sh``,
which is automatically called by ``start/local.sh`` and ``mk/ctnr.sh``.
.. _@botfather: https://t.me/botfather
.. _a demo video: https://user-images.githubusercontent.com/1787385/123204250-ae9a0380-d485-11eb-981d-3302220aad58.mp4
.. _age: https://github.com/FiloSottile/age
.. _buildah: https://github.com/containers/buildah
.. _@colorcodebot: https://t.me/colorcodebot
.. _guesslang: https://github.com/yoeo/guesslang
.. _highlight: http://www.andre-simon.de/doku/highlight/highlight.html
.. _Iosevka: https://github.com/be5invis/Iosevka
.. _Papertrail: https://www.papertrail.com
.. _pyTelegramBotAPI: https://github.com/eternnoir/pyTelegramBotAPI
.. _send a message: https://t.me/andykluger
.. _Silicon: https://github.com/Aloxaf/silicon
.. _sops: https://github.com/mozilla/sops
.. _wheezy.template: https://github.com/akornatskyy/wheezy.template
.. _yamlpath: https://github.com/wwkimball/yamlpath
.. |actions-ctnr| image:: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/ci.yml/badge.svg?branch=develop
:alt: Automated Container Build Status
:target: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/ci.yml
.. |actions-fmt| image:: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/fmt.yml/badge.svg?branch=develop
:alt: Format and Lint Status
:target: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/fmt.yml
.. |actions-reqs| image:: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/reqs.yml/badge.svg?branch=develop
:alt: Automated Python Requirements Bump Status
:target: https://github.com/AndydeCleyre/colorcodebot/actions/workflows/reqs.yml
.. |quay| image:: https://img.shields.io/badge/Quay.io-andykluger%2Fcolorcodebot--prod--archlinux-blue?logo=redhat
:alt: Container Image Repository
:target: https://quay.io/repository/andykluger/colorcodebot-prod-archlinux?tab=tags
.. |telegram| image:: https://img.shields.io/badge/Telegram-%40colorcodebot-blue?logo=telegram
:alt: Telegram user @colorcodebot
:target: https://t.me/colorcodebot