{"id":17481775,"url":"https://github.com/jlevy/kmd","last_synced_at":"2025-08-25T16:20:31.843Z","repository":{"id":257992923,"uuid":"795668948","full_name":"jlevy/kmd","owner":"jlevy","description":"An AI-native command line for modern workflows","archived":false,"fork":false,"pushed_at":"2024-10-29T08:35:53.000Z","size":3614,"stargazers_count":7,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-29T09:53:08.632Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jlevy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-03T19:11:59.000Z","updated_at":"2024-10-29T08:35:57.000Z","dependencies_parsed_at":"2024-11-05T10:51:39.417Z","dependency_job_id":null,"html_url":"https://github.com/jlevy/kmd","commit_stats":null,"previous_names":["jlevy/kmd"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlevy%2Fkmd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlevy%2Fkmd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlevy%2Fkmd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlevy%2Fkmd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jlevy","download_url":"https://codeload.github.com/jlevy/kmd/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233723960,"owners_count":18720110,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-18T22:35:48.022Z","updated_at":"2025-01-13T09:50:41.573Z","avatar_url":"https://github.com/jlevy.png","language":"Python","readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cp style=\"max-width: 400px;\"\u003e\n\n\u003cbr/\u003e\n\n\u003cb\u003e⎪K⎪M⎪D⎪\u003c/b\u003e\n\n\u003cb\u003e\u003ci\u003eThe Knowledge Command Line\u003c/i\u003e\u003c/b\u003e\n\n\u003cb\u003eAn AI-native command line for modern workflows.\u003c/b\u003e\n\n⛭\n\n“*Simple should be simple.\nComplex should be possible.*” —Alan Kay\n\n\u003cbr/\u003e\n\n\u003c/p\u003e\n\n\u003c/div\u003e\n\n*⚠️ Pre-release and experimental!\n[Follow or DM me](https://x.com/ojoshe) for future updates or if you have ideas,\nfeedback, or use cases for Kmd.*\n\n## What is Kmd?\n\nKmd (“Knowledge comManD line”) is a power tool for practical knowledge tasks.\nIt’s an early prototype for the exploration of what’s possible with the myriad of AI\ntools we now have.\n\nKmd makes it easier to use APIs and tools such as **OpenAI GPT-4o and o1**, **Anthropic\nClaude 3.5**, **Groq Llama 3.1** (and any others via **LiteLLM**), **Deepgram**,\n**Firecrawl**, **Exa**, **LlamaIndex**, **ChromaDB**, and other Python tools.\n\nUse commands to transcribe videos, summarize and organize transcripts and notes, write\nblog posts, extract or visualize concepts, check citations, convert notes to PDFs or\nbeautifully formatted HTML, or perform numerous other content-related tasks possible by\norchestrating AI tools in the right ways.\n\n## Is Kmd Mature?\n\nNo. Not at all.\n:) It's the result of a few weeks of coding and experimentation, and it's\nvery much in progress.\nPlease help me make it better by sharing your ideas and feedback!\nIt's easiest to DM me at [twitter.com/ojoshe](https://x.com/ojoshe).\nMy contact info is at [github.com/jlevy](https://github.com/jlevy).\n\nSome of this may be a little crazy or ambitious.\nSee more motivation in the philosophy section below.\n\n## What is Included?\n\n- A bash-like, Python-compatible shell based on xonsh, with pretty syntax coloring of\n  commands and outputs\n\n- Tab auto-completion and help on almost everything\n\n- A [generalized frontmatter format](https://github.com/jlevy/frontmatter-format), that\n  for YAML metadata on Markdown, HTML, Python, and other text files\n\n- A [data model](https://github.com/jlevy/kmd/tree/main/kmd/model) that includes items\n  such as documents, resources, concepts, etc., all stored as files within a workspace\n  of files, and with consistent metadata in YAML on text files\n\n- A few dozen built-in commands for listing, showing, and paging through files, etc.\n  (see `help` for full docs)\n\n- An extensible set of actions for all kinds of tasks like editing or summarizing text\n  or transcribing videos (see `help`)\n\n- A way of tracking the provenance of each file (what actions created each item) so you\n  can tell when to skip running a command (like a Makefile)\n\n- A selection system for maintaining context between commands so you can pass outputs of\n  one action into the inputs of another command (this is a bit like pipes but more\n  flexible for sequences of tasks, possibly with many intermediate inputs and outputs)\n\n- A set of preconditions, like whether a document is Markdown or HTML, if it's a\n  transcript with timestamps, and so on, so you and Kmd know what actions might apply to\n  any selection\n\n- A media cache, which is a mechanism for downloading and caching, downsampling, and\n  transcribing video, audio, using Whisper or Deepgram\n\n- A content cache, for downloading and caching web pages or other files\n\n- An LLM-based assistant that wraps the docs and the Kmd source code into a tool that\n  assists you in using or extending Kmd (this part is quite fun!)\n\n- A\n  [Markdown auto-formatter](https://github.com/jlevy/kmd/blob/main/kmd/text_formatting/markdown_normalization.py),\n  so text documents are saved in a normalized form that can be diffed consistently\n\n- If your terminal supports it, some major enhancements to the terminal experience:\n\n  - Sixel graphics support (see images right in the terminal)\n\n  - A local server for serving information on files as web pages that can be accessed as\n    OSC 8 links\n\n  - Sadly, we may have mind-boggling AI tools, but Terminals are still incredibly\n    archaic and don't support these features well (more on this below) but I have a new\n    terminal, Kyrm, that shows these as tooltips and makes every command clickable\n    (please contact me if you'd like an early developer preview, as I'd love feedback)\n\n- A bunch of other small utilities for making all this easier, including:\n\n  - Parsing and representing text docs as sentences, paragraphs, or chunks of text\n\n  - Diffing words and tokens and filtering diffs to control what changes LLMs make to\n    text\n\n  - Tools for detecting file types and automatic, readable file naming conventions\n\n  - Media handling of videos and audio, including downloading and transcribing videos\n\n### Credits\n\nAll of this is only possible by relying on a wide variety of powerful libraries,\nespecially [LiteLLM](https://github.com/BerriAI/litellm),\n[yt-dlp](https://github.com/yt-dlp/yt-dlp),\n[Pydantic](https://github.com/pydantic/pydantic),\n[Rich](https://github.com/Textualize/rich),\n[Ripgrep](https://github.com/BurntSushi/ripgrep), [Bat](https://github.com/sharkdp/bat),\n[jusText](https://github.com/miso-belica/jusText),\n[WeasyPrint](https://github.com/Kozea/WeasyPrint),\n[Marko](https://github.com/frostming/marko), and\n[Xonsh](https://github.com/xonsh/xonsh).\n\n## Installation\n\n### Running the Kmd Shell\n\nThe best way to use Kmd is as its own shell, which is a shell environment based on\n[xonsh](https://xon.sh/). If you've used a bash or Python shell before, xonsh is very\nintuitive.\nIf you don't want to use xonsh, you can still use it from other shells or as a\nPython library.\n\nWithin the Kmd shell, you get a full environment with all actions and commands.\nYou also get intelligent auto-complete and a built-in assistant to help you perform\ntasks.\n\n### Python and Tool Dependencies\n\nThese are needed to run:\n\n- Python 3.11+\n\n- Poetry\n\n- `ffmpeg` (for video conversions), `ripgrep` (for search), `bat` (for prettier file\n  display), `imagemagick` (for image display in modern terminals), `libmagic` (for file\n  type detection)\n\nCheat sheets to get these set up, if you're not already:\n\nFor macOS, I recommend using brew:\n\n```shell\n# Install pyenv, pipx, and other tools:\nbrew update\nbrew install pyenv pipx ffmpeg ripgrep bat eza imagemagick libmagic\n```\n\nFor Ubuntu:\n\n```shell\n# Install pyenv and other tools:\ncurl https://pyenv.run | bash\napt install pipx ffmpeg ripgrep bat eza imagemagick libmagic\n```\n\nNow install a recent Python and Poetry:\n\n```shell\npyenv install 3.12.8  # Or any version 3.11+ should work.\npipx install poetry\npoetry self add \"poetry-dynamic-versioning[plugin]\"  # Helps build versioning.\n```\n\nFor Windows or other platforms, see the pyenv and poetry instructions.\n\n### Building\n\n1. [Fork](https://github.com/jlevy/kmd/fork) this repo (having your own fork will make\n   it easier to contribute actions, add models, etc.).\n\n2. [Check out](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository)\n   the code.\n\n3. Install the package dependencies:\n\n   ```shell\n   poetry install\n   ```\n\n### API Key Setup\n\nYou will need API keys for all services you wish to use.\nConfiguring OpenAI, Anthropic, Groq (for Llama 3), Deepgram (for transcriptions),\nFirecrawl (for web crawling and scraping), and Exa (for web search) are recommended.\n\nThese keys should go in the `.env` file in your current directory.\n\n```shell\n# Set up API secrets:\ncp .env.template .env \n# Now edit the .env file to add all desired API keys\n```\n\n### Running\n\nTo run:\n\n```shell\npoetry run kmd\n```\n\nUse the `self_check` command to confirm tools like `bat` and `ffmpeg` are found and\nconfirm API keys are set up.\n\nOptionally, to install Kmd globally in the current user's Python virtual environment so\nyou can conveniently use `kmd` anywhere, make sure you have a usable Python 3.12+\nenvironment active (such as using `pyenv`), then:\n\n```shell\n./install_local.sh\n```\n\nIf you encounter installation issues, you can also try `./install_local.sh\n--force-reinstall`.\n\nThis does a pip install of the wheel so you can run it as `kmd`.\n\n### Other Ways to Run Kmd\n\nIf desired, you can also run Kmd directly from your regular shell, by giving a Kmd shell\ncommand.\n\n```\n# Transcribe a video and summarize it:\nmkdir myworkspace.kb\ncd myworkspace.kb\nkmd transcribe 'https://www.youtube.com/watch?v=XRQnWomofIY'\n```\n\n## Getting Started\n\n### Use Tab Completion and Help!\n\nTab completion is your friend!\nJust press tab to get lists of commands and guidance on help from the LLM-based\nassistant.\n\nYou can also ask any question directly in the shell.\n\nType `help` for the full documentation.\n\n### An Example: Transcribing Videos\n\nThe simplest way to illustrate how to use Kmd is by example.\nYou can go through the commands below a few at a time, trying each one.\n\nFor each command below you can use tab completion (which shows information about each\ncommand or option) or run with `--help` to get more details.\n\n```shell\n# Check the help page for a full overview:\nhelp\n\n# Confirm kmd is set up correctly with right tools:\ncheck_tools\n\n# The assistant is built into the shell, so you can just ask questions on the\n# command line. Note you can just press Space twice and it will insert the question\n# mark for you:\n? how do I get started with a new workspace\n\n# Set up a workspace to test things out (we'll use fitness as an example):\nworkspace fitness\n\n# A short transcription (use this one or pick any video on YouTube):\ntranscribe https://www.youtube.com/watch?v=KLSRg2s3SSY\n\n# Note there is a selection indicated.\n# We can then look at the selected item easily, because commands often\n# will just work on the selection automatically:\nshow\n\n# Now let's manipulate that transcription. Note we are using the outputs\n# of each previous command, which are auto-selected as input to each\n# subsequent command. You can always run `show` to see the last result.\n\n# Remove the speaker id \u003cspan\u003e tags from the transcript.\nstrip_html\nshow\n\n# Break the text into paragraphs. Note this is smart enough to \"filter\"\n# the diff so even if the LLM modifies the text, we only let it insert\n# newlines.\nbreak_into_paragraphs\nshow\n\n# Look at the paragraphs and (by following the `derived_from` relation\n# this doc up to find the original source) then infer the timestamps\n# and backfill them, inserting timestamped link to the YouTube video\n# at the end of each paragraph.\nbackfill_timestamps\nshow\n\n# How about we add some headings?\ninsert_section_headings\n\n# How about we compare what we just did with what there was there\n# previously? \ndiff\n\n# If you're wondering how that works, it is an example of a command\n# that looks at the selection history.\nselect --history\n\n# And add some summary bullets and a description:\nadd_summary_bullets\nadd_description\n\n# Note we are just using Markdown still but inserting \u003cdiv\u003e tags to\n# add needed structure.\nshow\n\n# Render it as a PDF:\ncreate_pdf\n\n# See the PDF.\nshow\n\n# Cool. But it would be nice to have some frame captures from the video.\n? are there any actions to get screen captures from the video\n\n# Oh yep, there is!\n# But we're going to want to run it on the previous doc, not the PDF.\n# Let's see what the files are so far.\nfiles\n\n# Note we could select the file like this before we run the next command\n# with `select \u003csome-file\u003e.doc.md`. But actually we can see the history\n# of items we've selected:\nselect --history\n\n# And just back up to the previous one.\nselect --previous\n\n# Look at it again. Yep, there should be timestamps in the text.\nshow\n\n# As a side note, not all actions work on all items. So we also have\n# a way to check preconditions to see what attributes a given item has.\n# Note that for this doc `has_timestamps` is true.\npreconditions\n\n# And there is a way to see what commands are compatible with the current\n# selection based on these preconditions.\nsuggest_actions\n\n# Okay let's try it. (If you're using a shell that supports Kmd well,\n# you can just click the command name!)\ninsert_frame_captures\n\n# Note the screen capture images go to the assets folder as assets.\nfiles\n\n# Let's look at that as a web page.\nshow_as_webpage\n\n# Note that works because unlike regular `show`, that command\n# runs actions to convert a pretty HTML format.\nshow_as_webpage --help\n\n# And you can actually how this works by looking at its source:\naction_source_code show_as_webpage\n\n# What if something isn't working right?\n# Sometimes we may want to browse more detailed system logs:\nlogs\n\n# Note transcription works with multiple speakers, thanks to Deepgram\n# diarization. \ntranscribe https://www.youtube.com/watch?v=_8djNYprRDI\nshow\n\n# We can create more advanced commands that combine sequences of actions.\n# This command does everything we just did above: transcribe, format,\n# include timestamps for each paragraph, etc.\ntranscribe_format --help\ntranscribe_format https://www.youtube.com/watch?v=_8djNYprRDI\n\n# Getting a little fancier, this one adds little paragraph annotations and\n# a nicer summary at the top:\ntranscribe_annotate_summarize https://www.youtube.com/watch?v=_8djNYprRDI\n\n# A few more possibilities...\n\n# Note it's fine to rerun commands on the same arguments and whenever\n# possible intermediate results are cached. The philosophy is actions\n# should be cached and idempotent when possible (a bit like a makefile).\n\n# Let's now look at the concepts discussed in that video (adjust the filename\n# if needed):\ntranscribe_format https://www.youtube.com/watch?v=_8djNYprRDI\nfind_concepts\n\n# This is the list of concepts:\nshow\n\n# But we can actually save them as items:\nsave_concepts\n\n# We now have about 40 concepts. But maybe some are near duplicates (like\n# \"high intensity interval training\" vs \"high intensity intervals\").\n# Let's embed them and find near duplicates:\nfind_near_duplicates\n\n# In my case I see one near duplicate, which I'll archive:\narchive\n\n# And for fun now let's visualize them in 3d (proof of concept, this could\n# get a lot better):\ngraph_view --concepts_only\n\n# We can also list all videos on a channel, saving links to each one as\n# a resource .yml file:\nlist_channel https://www.youtube.com/@Kboges\n\n# Look at what we have and transcribe a couple more:\nfiles resources\ntranscribe resources/quality_first.resource.yml resources/why_we_train.resource.yml\n\n# Another interesting note: you can process a really long document.\n# This one is a 3-hour interview. Kmd uses sliding windows that process a\n# group of paragraphs at a time, then stitches the results back together:\ntranscribe_format https://www.youtube.com/watch?v=juD99_sPWGU\n\nshow_as_webpage\n```\n\n### Creating a New Workspace\n\nAlthough you don't always need one, a *workspace* is very helpful for any real work in\nKmd. It's just a directory of files, plus a `.kmd/` directory with various logs and\nmetadata.\n\nNote the `.kmd/cache` directory contains all the downloaded videos and media you\ndownload, so it can get large.\nYou can delete these files if they take up too much space.\n\nTypically, we name them like `health.kb` or `personal.kb`, because that makes it clear\nthey are in a certain format and may have other data.\n\nPick a workspace that encompasses a project or topic, and it lets you keep things\norganized.\n\nType `workspace` any time to see the current workspace.\n\nBy default, when you are not using the shell inside a workspace directory, or when you\nrun Kmd the first time, it uses the default *sandbox workspace*.\n\nOnce you create a workspace, you can `cd` into that workspace and that will become the\ncurrent workspace. (If you're familiar with how the `git` command-line works in\nconjunction with the `.git/` directory, this behavior is very similar.)\n\nTo start a new workspace, run a command like\n\n```\nworkspace health\n```\n\nThis will create a workspace directory called `health.kb` in the current directory.\nYou can run `cd health.kb` or `workspace health` to switch to that directory and begin\nworking.\n\n### Essential Kmd Commands\n\nKmd has quite a few basic commands that are easier to use than usual shell commands.\nYou can always run `help` for a full list, or run any command with the `--help` option\nto see more about the command.\n\nA few of the most important commands for managing files and work are these:\n\n- `check_tools` to confirm your Kmd setup has necessary tools (like bat and ffmpeg).\n\n- `files` lists files in one or more paths, with sorting, filtering, and grouping.\n\n- `workspace` to show or select or create a new workspace.\n  Initially you work in the \"sandbox\" workspace but for more real work you'll want to\n  create a workspace, which is a directory to hold the files you are working with.\n\n- `select` shows or sets selections, which are the set of files the next command will\n  run on, within the current workspace.\n\n- `edit` runs the currently configured editor (based on the `EDITOR` environment\n  variable) on any file, or the current selection.\n\n- `show` lets you show the first file in the current selection or any file you wish.\n  It auto-detects whether to show the file in the console, the browser, or using a\n  native app (like Excel for a .xls file).\n\n- `param` lets you set certain common parameters, such as what LLM to use (if you wish\n  to use non-default model or language).\n\n- `logs` to see full logs (typically more detailed than what you see in the console).\n\n- `history` to see recent commands you've run.\n\n- `import_item` to add a resource such as a URL or a file to your local workspace.\n\nThe set of actions that do specific useful things is much longer, but a few to be aware\nof include:\n\n- `chat` chat with any configured LLM, and save the chat as a chat document.\n\n- `web_search_topic` searches the web using Exa.\n\n- `crawl_webpage` fetches a webpage and scrapes the content as text, using Firecrawl.\n\n- `download_media` downloads video or audio media from any of several services like\n  YouTube or Apple Podcasts, using yt-dlp.\n\n- `transcribe` transcribes video or audio as text document, using Deepgram.\n\n- `proofread` proofreads a document, editing it for typos and errors only.\n\n- `describe_briefly` describes the contents of a document in about a paragraph.\n\n- `summarize_as_bullets` summarizes a text document as a bulleted item.\n\n- `break_into_paragraphs` breaks a long block of text into paragraphs.\n\n- `insert_section_headings` inserts section headings into a document, assuming it is a\n  document (like a transcript after you've run `break_into_paragraphs`) that has\n  paragraphs but no section headers.\n\n- `show_as_webpage` formats Markdown or HTML documents as a nice web page and opens your\n  browser to view it.\n\n- `create_pdf` formats Markdown or HTML documents as a PDF.\n\n## Tips for Use with Other Tools\n\nWhile not required, these tools can make using Kmd easier or more fun.\n\n### Choosing a Terminal\n\nYou can use any favorite terminal to run Kmd.\n\nHowever, you can get a much better terminal experience if you use one with more advanced\nadditional features, such as [OSC 8 link](https://github.com/Alhadis/OSC8-Adoption)\nsupport and [Sixel](https://www.arewesixelyet.com/) graphics.\n\nI tried half a dozen different popular terminals on Mac\n([Terminal](https://support.apple.com/guide/terminal/welcome/mac),\n[Warp](https://www.warp.dev/), [iTerm2](https://iterm2.com/),\n[Kitty](https://sw.kovidgoyal.net/kitty/), [WezTerm](https://wezfurlong.org/wezterm/),\n[Hyper](https://hyper.is/)). Unfortunately, none offer really good support right out of\nthe box, but I encourage you to try\n\n✨**Would you be willing to help test something new?** If you've made it this far and\nare still reading, I have a request.\nSo alongside Kmd, I've begun to build a new terminal app, **Kyrm**, that has the\nfeatures we would want in a modern command line, such as clickable links and commands,\ntooltips, and image support.\nKmd also takes advantage of this support by embedding OSC 8 links.\nIt is *so* much nicer to use.\nI'd like feedback so please [message me](https://twitter.com/ojoshe) if you'd like to\ntry it out an early dev version!\n\n### Choosing an Editor\n\nMost any editor will work.\nKmd respects the `EDITOR` environment variable if you use the `edit` command.\n\n### Using on macOS\n\nKmd calls `open` to open some files, so in general, it's convenient to make sure your\npreferred editor is set up for `.yml` and `.md` files.\n\nFor convenience, a reminder on how to do this:\n\n- In Finder, pick a `.md` or `.yml` file and hit Cmd-I (or right-click and select Get\n  Info).\n\n- Select the editor, such as Cursor or VSCode or Obsidian, and click the \"Change All…\"\n  button to have it apply to all files with that extension.\n\n- Repeat with each file type.\n\n### Using with Cursor and VSCode\n\n[Cursor](https://www.cursor.com/) and [VSCode](https://code.visualstudio.com/) work fine\nout of the box to edit workspace files in Markdown, HTML, and YAML in Kmd workspaces.\n\n### Using with Zed\n\n[Zed](https://zed.dev/) is another, newer editor that works great out of the box.\n\n### Using with Obsidian\n\nKmd uses Markdown files with YAML frontmatter, which is fully compatible with\n[Obsidian](https://obsidian.md/). Some notes:\n\n- In Obsidian's preferences, under Editor, turn on \"Strict line breaks\".\n\n- This makes the line breaks in Kmd's normalized Markdown output work well in Obsidian.\n\n- Some Kmd files also contain HTML in Markdown.\n  This works fine, but note that only the current line's HTML is shown in Obsidian.\n\n- Install the [Front Matter Title\n  plugin](https://github.com/snezhig/obsidian-front-matter-title):\n\n  - Go to settings, enable community plugins, search for \"Front Matter Title\" and\n    install.\n\n  - Under \"Installed Plugins,\" adjust the settings to enable \"Replace shown title in\n    file explorer,\" \"Replace shown title in graph,\" etc.\n\n  - You probably want to keep the \"Replace titles in header of leaves\" off so you can\n    still see original filenames if needed.\n\n  - Now titles are easy to read for all Kmd notes.\n\n### More Command-Line Tools\n\nThese aren't directly related to Kmd but are very useful to know about if you wish to\nhave modern text UIs for your data files.\nThese can work well with files created by Kmd.\n\n- [**Ranger**](https://github.com/ranger/ranger) is a powerful terminal-based file\n  manager that works well with Kmd-generated files.\n\n- [**Visidata**](https://github.com/saulpw/visidata) is a flexible spreadsheet-like\n  multitool for tabular data, handy if you are wanting to manipulate tabular data with\n  Kmd actions.\n\n## Development\n\nDeveloper setup:\n\n```shell\n# Developers should install poetry plugins to help with dev builds and updates:\npoetry self update\npoetry self add \"poetry-dynamic-versioning[plugin]\"\n\n# Run pytests:\npoetry run test\n# Or within the poetry shell:\npytest   # all tests\npytest -s kmd/text_docs/text_doc.py  # one test, with outputs\n\n# Build wheel:\npoetry build\n\n# Before committing, be sure to check formatting/linting issues:\npoetry run lint\n\n# Upgrade packages:\npoetry up\n\n# Update this README:\nsource devtools/generate_readme.xsh\n```\n\nA few debugging tips when finding issues:\n\n```shell\n# To see tracebacks if xonsh does not show them:\n$XONSH_SHOW_TRACEBACK=1\n\n# To dump Python stack traces of all threads (from another terminal):\npkill -USR1 kmd\n```\n\n## The Philosophy of Kmd\n\nHere is a bit more motivation for experimenting with Kmd, why I think it's potentially\nso useful, and some design principles.\n(You may skip ahead to the next section if you just want a more concrete overview!)\n\n### Why Apps Can't Solve All Your Problems\n\nAI has radically changed the way we use software.\nWith LLMs and other generative AI models, we've seen big improvements in two areas:\n\n1. Powerful general-purpose new AI tools (ChatGPT, Perplexity, etc.)\n\n2. AI-powered features within specific SaaS tools that are built for the problem you\n   want to solve, like Notion, Figma, Descript, etc.\n\nWhile we have these powerful cloud apps, we all know numerous situations where our\nproblems aren't easily solved or automated with the usual tools like the ChatGPT\ninterface, Notion, Google Docs, Slack, Excel, and Zapier.\n\nIf you want to use any of the newest AI models and APIs for something not supported by\nan existing tool, you generally have to design and build it yourself—in Python and/or a\nfull-stack web app.\n\nWe are told how AI is replacing developers—and it's true tools like GitHub Copilot and\nCursor can help you write code much faster.\nBut building apps that are good enough people will pay them is hard.\nAnd we can't all drop what we're doing and build new apps.\n\nIt has long been a rule that once products become widely successful, the curse of\n[Conway's Law](https://en.wikipedia.org/wiki/Conway%27s_law) and the complexity of\nfull-stack apps means many companies won't add many of the specific features you want,\nor at best are likely to do it slowly.\n\nIn short, in spite of AI tools accelerating software, certain things don't change: we\nare waiting for developers, product managers, designers, and entrepreneurs to design and\nship solutions for us.\n\n### Why Do We Need an AI-Native Command Line?\n\nSo what does all this have to do with the command line?\n\nWell, the classic Unix-style command line has been the Swiss Army knife for savvy\ndevelopers for decades.\n(The bash shell, still used widely, was released 35 years ago!)\n\nLike many developers, I love the terminal (I even wrote a popular\n[guide on it](https://github.com/jlevy/the-art-of-command-line), with millions of\nreaders).\n\nA fraction of developers do a lot in a terminal because it is the most efficient way to\nsolve many problems.\nBut among most normal people, the command line has pretty bad reputation.\nThis is a fair criticism.\nCommand-line shells generally still suffer from three big issues:\n\n- Old and arcane commands, full of obscure behaviors that relatively few people remember\n\n- A text-based interface many find confusing or ugly\n\n- No easy, “native” support for modern tools, apps, and APIs (especially LLMs—and using\n  `curl` to call OpenAI APIs doesn't count!)\n\nEven worse, command lines haven't gotten much better.\nFew companies make money shipping new command-line tooling.\n(In the last few years this has slowly starting to change with tools like nushell, fish,\nand Warp.)\n\nNonetheless, for all its faults, there is a uniquely powerful thing about the command\nline: With a command line, you can do complex things that were never planned by an app\ndeveloper, a designer, or an enterpreneur building a product.\n\n*You* know your problems better than anyone else.\nAny tool that lets you solve complex problems yourself, without waiting for engineers\nand designers, can radically improve your productivity.\n\nI think it's a good time to revisit this idea.\n\nIn a post-LLM world, it should be possible to do more things without so much time and\neffort spent (even with the help of LLMs) on coding and UI/UX design.\n\nIf we have an idea for a script or a feature or a workflow, we should not have to spend\nweeks or months to iterate on web or mobile app design and full-stack engineering just\nto see how well it works.\n\n### The Goals of Kmd\n\nKmd is an experimental attempt at building the tool I've wanted for a long time, using a\ncommand line as a starting point, and with an initial focus on content-related tasks.\n\nThat brings us to the goals behind building a new, AI-native shell.\n\n- **Make simple tasks simple:** Doing a simple thing (like transcribing a video or\n  proofreading a document) should be as easy as running a single command (not clicking\n  through a dozen menus).\n  We should be able to tell someone how to do something simply by telling them the\n  command, instead of sharing a complex prompt or a tutorial video on how to use several\n  apps.\n\n- **Make complex tasks possible:** Highly complex tasks and workflows should be easy to\n  assemble (and rerun if they need to be automated) by adding new primitive actions and\n  combining primitive actions into more complex workflows.\n  You shouldn't need to be a programmer to use any task—but any task should be\n  extensible with arbitrary code (written by you and an LLM) when needed.\n\n- **Augment human skills and judgement:** Many AI agent efforts aim for pure automation.\n  But even with powerful LLMs and tools, full automation is rare.\n  Invariably, the best results come from human review wherever it's needed—experimenting\n  with different models and prompts, looking at what works, focusing expert human\n  attention in the right places.\n  The most flexible tools augment, not replace, your ability to review and manipulate\n  information. It should help both very technical users, like developers, as well as less\n  technical but sophisticated users who aren't traditional programmers.\n\n- **Accelerate discovery of the workflows that work best:** We have so many powerful\n  APIs, models, libraries, and tools now—but the real bottleneck is in discovering and\n  then orchestrating the right workflows with the right inputs, models, prompts, and\n  human assistance. Anyone should be able to discover new steps and workflows without\n  waiting on engineers or designers.\n\n- **Understand and build on itself:** A truly AI-native programming environment should\n  improve itself! Kmd can read its own code and docs, assist you with its own commands,\n  and write new Kmd actions.\n  Better languages and scripting tools can in fact make LLMs smarter, because it allows\n  them to solve problems in ways that are simpler and less error prone.\n\n### Design Principles\n\nThis boils down to a few specific design choices:\n\n1. Solve problems with simple commands that can be recombined in complex ways (like the\n   old Unix model)\n\n2. Support any APIs or tools, including local and cloud-based LLMs and other generative\n   AI tools (basically anything available in Python)\n\n3. Allow incremental and interactive work via a shell that maintains context (keep files\n   organized in a simple workspace folder, and include content, metadata, current\n   settings and selections, caches, history, and logs all in one place)\n\n4. Support automation and scripting when desired (an interactive history should be\n   automatable)\n\n5. Use local files whenever possible (not tied to a particular SaaS provider)\n\n6. Use simple and transparent file formats (especially text files like Markdown, YAML,\n   and HTML, with intuitive filenames names, so you can edit content with any editors or\n   external tools; avoid opaque formats, sprawling JSON, or data stored in the cloud and\n   accessible only from an app)\n\n7. Keep content human reviewable, diff-able, and editable at any stage of a workflow\n   (don't just assume automation will work; expect it not to and plan for workflows to\n   understand failures, fix them, and resume; use formats that make diffs as easy and\n   clear as possible)\n\n8. Maintain metadata on files, so you always know where each piece of content comes from\n   (and keep this close to the content, as YAML frontmatter)\n\n9. Make operations idempotent (resuming a task or workflow or restarting after a failure\n   should be as simple as running again)\n\n10. Cache slow or costly operations (track dependencies and content hashes and know when\n    things need to rerun, like a makefile)\n\n11. Docs, code, and examples should be self-explanatory (so an LLM assistant should be\n    able to help you use and enhance the tool itself)\n\n12. Make it easy and LLMs to add and dynamically use new commands (an AI-native\n    programming environment should enhance itself!)\n\nKmd may evolve into more than a command line.\nIt's more like a first step toward an item-based information operating system—an\nalternate, more flexible UX and information architecture for knowledge workflows.\nIt could be the tool you need when you don't know what tool you need.\n\n\u003cbr/\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n⛭\n\n\u003cp style=\"max-width: 400px;\"\u003e\n\n“*Civilization advances by extending the number of important operations which we can\nperform without thinking about them.*” —Alfred North Whitehead\n\n\u003c/p\u003e\n\n\u003c/div\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlevy%2Fkmd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjlevy%2Fkmd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlevy%2Fkmd/lists"}