{"id":13574209,"url":"https://github.com/boyter/cs","last_synced_at":"2025-04-05T01:09:05.528Z","repository":{"id":40372663,"uuid":"198756440","full_name":"boyter/cs","owner":"boyter","description":"command line codespelunker or code search","archived":false,"fork":false,"pushed_at":"2024-02-01T00:43:51.000Z","size":14921,"stargazers_count":521,"open_issues_count":5,"forks_count":15,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-10-14T19:10:23.904Z","etag":null,"topics":["cli","code","command-line-tool","search","tui"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/boyter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-25T04:20:23.000Z","updated_at":"2024-09-23T08:43:53.000Z","dependencies_parsed_at":"2024-06-12T07:33:09.574Z","dependency_job_id":null,"html_url":"https://github.com/boyter/cs","commit_stats":null,"previous_names":["boyter/sc"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boyter%2Fcs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boyter%2Fcs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boyter%2Fcs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boyter%2Fcs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/boyter","download_url":"https://codeload.github.com/boyter/cs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247271532,"owners_count":20911587,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","code","command-line-tool","search","tui"],"created_at":"2024-08-01T15:00:48.140Z","updated_at":"2025-04-05T01:09:05.510Z","avatar_url":"https://github.com/boyter.png","language":"Go","readme":"codespelunker (cs)\n----------------------\n\nA command line search tool. Allows you to search over code or text files in the current directory either on\nthe console, via a TUI or HTTP server, using some boolean queries or regular expressions.\n\nConsider it a similar approach to using ripgrep, silver searcher or grep coupled with fzf but in a single tool.\n\nDual-licensed under MIT or the UNLICENSE.\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/boyter/cs)](https://goreportcard.com/report/github.com/boyter/cs)\n[![Coverage Status](https://coveralls.io/repos/github/boyter/cs/badge.svg?branch=master)](https://coveralls.io/github/boyter/cs?branch=master)\n[![Cs Count Badge](https://sloc.xyz/github/boyter/cs/)](https://github.com/boyter/cs/)\n\n[![asciicast](https://asciinema.org/a/589640.svg)](https://asciinema.org/a/589640)\n\n### Support\n\nUsing `cs` commercially? If you want priority support for `cs` you can purchase a years worth https://boyter.gumroad.com/l/vvmyi which entitles you to priority direct email support from the developer.\n\n### Pitch\n\nWhy use cs?\n\n - Reasonably fast\n - Rank results on the fly helping you find things\n - Searches across multiple lines\n - Has a nice TUI interface.\n - Cross-platform (probably needs the new Windows terminal though)\n\nThe reason `cs` exists at all is because I was running into limitations using `rg TERM | fzf` and decided to solve my own\nproblem. \n\n### Install\n\nIf you want to create a package to install things please do. Let me know and ill ensure I add it here.\n\n#### Go Get\n\nIf you have Go \u003e= 1.20 installed\n\n`go install github.com/boyter/cs@v1.3.0`\n\n#### Nixos\n\n`nix-shell -p codespelunker`\n\nhttps://github.com/NixOS/nixpkgs/pull/236073\n\n#### Manual\n\nBinaries for Windows, GNU/Linux and macOS are available from the [releases](https://github.com/boyter/cs/releases) page.\n\n### FAQ\n\n#### Is this as fast as...\n\nNo.\n\n#### You didn't let me finish, I was going to ask if it's as fast as...\n\nThe answer is probably no. It's not directly comparable. No other tool I know of works like this outside of full\nindexing tools such as hound, searchcode, sourcegraph etc... None work on the fly like this does.\n\nWhile `cs` does have some overlap with tools like ripgrep, grep, ack or the silver searcher the reality is it does not\nwork the same way, so any comparison is pointless. It is slower than most of them, but its also doing something different.\n\nYou can replicate some of what it does by piping their output into fzf though if you feel like a flawed comparison.\n\nOn my local machine which at time of writing is a Macbook Air M1 it can search a recent checkout of the linux source\ncode in ~2.5 seconds. While absolute performance is not a design goal, I also don't want this to be a slow tool. As such\nif any obvious performance gains are on the table I will take them.\n\n#### Does it work on normal documents?\n\nSo long as they are text. I wrote it to search code, but it works just as well on full text documents. The snippet\nextraction for example was tested on Pride and Prejudice. If you had a heap of PDF's you could shell script some\nuse of pdftotext and get something searchable.\n\nNote it was designed for code and as such has full .ignore and .gitignore support.\n\n#### Where is the index?\n\nThere is none. Everything is brute force calculated on the fly. For TUI mode there are some shortcuts taken with\ncaching of results to speed things up.\n\n#### How does the ranking work then?\n\nStandard BM25 or TF/IDF or the modified TF/IDF in Lucene https://opensourceconnections.com/blog/2015/10/16/bm25-the-next-generation-of-lucene-relevation/ \nwhich dampens the impact of term frequency.\n\nTechnically speaking it's not accurate because it calculates the weights based on what it matched on and not everything,\nbut it works well enough in practice and is calculated on the fly. Try it out and report if something is not working as\nyou expect?\n\n#### How do you get the snippets?\n\nIt's not fun... https://github.com/boyter/cs/blob/master/snippet.go Have a look at the code. \n\nIt works by passing the document content to extract the snippet from and all the match locations for each term. \nIt then looks through each location for each word, and checks on either side looking for terms close to it. \nIt then ranks on the term frequency for the term we are checking around and rewards rarer terms. \nIt also rewards more matches, closer matches, exact case matches and matches that are whole words.\n\nFor more info read the \"Snippet Extraction AKA I am PHP developer\" section of this blog post https://boyter.org/posts/abusing-aws-to-make-a-search-engine/\n\n#### What does HTTP mode look like?\n\nIt's a little brutalist.\n\n\u003cimg alt=\"scc\" src=https://github.com/boyter/cs/raw/master/cs_http.png\u003e\n\nYou can change its look and feel using `--template-display` and `--template-search`. See https://github.com/boyter/cs/tree/master/asset/templates\nfor example templates you can use to modify things.\n\n```shell\ncs -d --template-display ./asset/templates/display.tmpl --template-search ./asset/templates/search.tmpl\n```\n\n### Usage\n\nCommand line usage of `cs` is designed to be as simple as possible.\nFull details can be found in `cs --help` or `cs -h`. Note that the below reflects the state of master not a release, as such\nfeatures listed below may be missing from your installation.\n\n```\n$ cs -h\ncode spelunker (cs) code search.\nVersion 1.3.0\nBen Boyter \u003cben@boyter.org\u003e\n\ncs recursively searches the current directory using some boolean logic\noptionally combined with regular expressions.\n\nWorks via command line where passed in arguments are the search terms\nor in a TUI mode with no arguments. Can also run in HTTP mode with\nthe -d or --http-server flag.\n\nSearches by default use AND boolean syntax for all terms\n - exact match using quotes \"find this\"\n - fuzzy match within 1 or 2 distance fuzzy~1 fuzzy~2\n - negate using NOT such as pride NOT prejudice\n - regex with toothpick syntax /pr[e-i]de/\n\nSearches can fuzzy match which files are searched by adding\nthe following syntax\n\n - test file:test\n - stuff filename:.go\n\nFiles that are searched will be limited to those that fuzzy\nmatch test for the first example and .go for the second.\nExample search that uses all current functionality\n - darcy NOT collins wickham~1 \"ten thousand a year\" /pr[e-i]de/ file:test\n\nThe default input field in tui mode supports some nano commands\n- CTRL+a move to the beginning of the input\n- CTRL+e move to the end of the input\n- CTRL+k to clear from the cursor location forward\n\nUsage:\n  cs [flags]\n\nFlags:\n      --address string            address and port to listen to in HTTP mode (default \":8080\")\n      --binary                    set to disable binary file detection and search binary files\n  -c, --case-sensitive            make the search case sensitive\n      --dir string                directory to search, if not set defaults to current working directory\n      --exclude-dir strings       directories to exclude (default [.git,.hg,.svn])\n  -x, --exclude-pattern strings   file and directory locations matching case sensitive patterns will be ignored [comma separated list: e.g. vendor,_test.go]\n  -r, --find-root                 attempts to find the root of this repository by traversing in reverse looking for .git or .hg\n  -f, --format string             set output format [text, json, vimgrep] (default \"text\")\n  -h, --help                      help for cs\n      --hidden                    include hidden files\n  -d, --http-server               start http server for search\n  -i, --include-ext strings       limit to file extensions (N.B. case sensitive) [comma separated list: e.g. go,java,js,C,cpp]\n      --max-read-size-bytes int   number of bytes to read into a file with the remaining content ignored (default 1000000)\n      --min                       include minified files\n      --min-line-length int       number of bytes per average line for file to be considered minified (default 255)\n      --no-gitignore              disables .gitignore file logic\n      --no-ignore                 disables .ignore file logic\n  -o, --output string             output filename (default stdout)\n      --ranker string             set ranking algorithm [simple, tfidf, tfidf2, bm25] (default \"bm25\")\n  -s, --snippet-count int         number of snippets to display (default 1)\n  -n, --snippet-length int        size of the snippet to display (default 300)\n      --template-display string   path to display template for custom styling\n      --template-search string    path to search template for custom styling\n  -v, --version                   version for cs\n```\n\nSearches work on single or multiple words with a logical AND applied between them. You can negate with NOT before a term.\nYou can do exact match with quotes, and do regular expressions using toothpicks.\n\nExample search that uses all current functionality\n\n```shell\ncs t NOT something test~1 \"ten thousand a year\" \"/pr[e-i]de/\" file:test\n```\n\nYou can use it in a similar manner to `fzf` in TUI mode if you like, since `cs` will return the matching document path\nif you hit the enter key.\n\n```shell\ncat `cs`\n```\n","funding_links":[],"categories":["Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboyter%2Fcs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fboyter%2Fcs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboyter%2Fcs/lists"}