{"id":13513576,"url":"https://github.com/livegrep/livegrep","last_synced_at":"2025-05-14T04:07:33.574Z","repository":{"id":6868206,"uuid":"8117164","full_name":"livegrep/livegrep","owner":"livegrep","description":"Interactively grep source code. Source for http://livegrep.com/","archived":false,"fork":false,"pushed_at":"2024-12-28T02:38:39.000Z","size":8857,"stargazers_count":2085,"open_issues_count":60,"forks_count":186,"subscribers_count":33,"default_branch":"main","last_synced_at":"2025-04-10T22:35:03.493Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/livegrep.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-02-10T00:19:18.000Z","updated_at":"2025-04-10T02:54:56.000Z","dependencies_parsed_at":"2023-11-16T04:30:11.267Z","dependency_job_id":"2bcfb177-f09d-4388-be1e-54cdcf2e1f40","html_url":"https://github.com/livegrep/livegrep","commit_stats":{"total_commits":1380,"total_committers":62,"mean_commits":"22.258064516129032","dds":0.1942028985507246,"last_synced_commit":"853379ed3fe25c56c9402fdf1deb5f7208cab767"},"previous_names":["nelhage/livegrep"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livegrep%2Flivegrep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livegrep%2Flivegrep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livegrep%2Flivegrep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/livegrep%2Flivegrep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/livegrep","download_url":"https://codeload.github.com/livegrep/livegrep/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254069212,"owners_count":22009508,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T05:00:31.939Z","updated_at":"2025-05-14T04:07:33.539Z","avatar_url":"https://github.com/livegrep.png","language":"C++","readme":"Livegrep [![Build Status](https://github.com/livegrep/livegrep/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/livegrep/livegrep/actions/workflows/ci.yaml)\n========\n\nLivegrep is a tool, partially inspired by Google Code Search, for\ninteractive regex search of ~gigabyte-scale source repositories. You\ncan see a running instance at\n[http://livegrep.com/](http://livegrep.com).\n\nBuilding\n--------\n\nlivegrep builds using [bazel][bazel]. You will need to \n[install][bazel-install] with a version matching that in `.bazelversion`.\nRunning bazel via [bazelisk][bazelisk] will download the right version\nautomatically.\n\nlivegrep vendors and/or fetches all of its dependencies using `bazel`,\nand so should only require a relatively recent C++ compiler to build.\n\nOnce you have those dependencies, you can build using\n\n    bazel build //...\n\nNote that the initial build will download around 100M of\ndependencies. These will be cached once downloaded.\n\n[bazel]: http://www.bazel.io/\n[bazel-install]: http://www.bazel.io/docs/install.html\n[bazelisk]: https://bazel.build/install/bazelisk\n\nInvoking\n--------\n\nTo run `livegrep`, you need to invoke both the `codesearch` backend\nindex/search process, and the `livegrep` web interface.\n\nTo run the sample web interface over livegrep itself, once you have\nbuilt both `codesearch` and `livegrep`:\n\nIn one terminal, start the `codesearch` server like so:\n\n    bazel-bin/src/tools/codesearch -grpc localhost:9999 doc/examples/livegrep/index.json\n\nIn another, run livegrep:\n\n    bazel-bin/cmd/livegrep/livegrep_/livegrep\n\nIn a browser, now visit\n[http://localhost:8910/](http://localhost:8910/), and you should see a\nworking livegrep.\n\n## Using Index Files\n\nThe `codesearch` binary is responsible for reading source code,\nmaintaining an index, and handling searches. `livegrep` is stateless\nand relies only on the connection to `codesearch` over a TCP\nconnection.\n\nBy default, `codesearch` will build an in-memory index over the\nrepositories specified in its configuration file. You can, however,\nalso instruct it to save the index to a file on disk. This has the dual\nadvantages of allowing indexes that are too large to fit in RAM, and\nof allowing an index file to be reused. You instruct `codesearch` to\ngenerate an index file via the `-dump_index` flag and to not launch\na search server via the `-index_only` flag:\n\n    bazel-bin/src/tools/codesearch -index_only -dump_index livegrep.idx doc/examples/livegrep/index.json\n\nOnce `codeseach` has built the index, this index file can be used for\nfuture runs. Index files are standalone, and you no longer need access\nto the source code repositories, or even a configuration file, once an\nindex has been built. You can just launch a search server like so:\n\n    bazel-bin/src/tools/codesearch -load_index livegrep.idx -grpc localhost:9999\n\nThe schema for the `codesearch` configuration file defined using\nprotobuf in [src/proto/config.proto](src/proto/config.proto).\n\n## `livegrep`\n\nThe `livegrep` frontend accepts an optional position argument\nindicating a JSON configuration file; See\n[doc/examples/livegrep/server.json][server.json] for an example, and\n[server/config/config.go][config.go] for documentation of available\noptions.\n\nBy default, `livegrep` will connect to a single local codesearch\ninstance on port `9999`, and listen for HTTP connections on port\n`8910`.\n\n[server.json]: https://github.com/livegrep/livegrep/blob/main/doc/examples/livegrep/server.json\n[config.go]: https://github.com/livegrep/livegrep/blob/main/server/config/config.go\n\n## github integration\n\n`livegrep` includes a helper driver, `livegrep-github-reindex`, which\ncan automatically update and index selected github repositories. To\ndownload and index all of my repositories (except for forks), storing\nthe repos in `repos/` and writing `nelhage.idx`, you might run:\n\n    bazel-bin/cmd/livegrep-github-reindex/livegrep-github-reindex -user=nelhage -forks=false -name=github.com/nelhage -out nelhage.idx\n\nYou can now use `nelhage.idx` as an argument to `codesearch\n-load_index`.\n\n## Local repository browser\n`livegrep` provides the ability to view source files directly in `livegrep`, as\nan alternative to linking files to external viewers. This was initially implemented\nby @jboning [here](https://github.com/livegrep/livegrep/pull/70). There are\na few ways to enable this. The most important steps are to\n1. Generate a config file that `livegrep` can use to figure out where your\n   source files are (locally).\n2. Pass this config file as an argument to the frontend (`-index-config`)\n\n### Generating index manually\n\nSee [doc/examples/livegrep/server.json](doc/examples/livegrep/server.json) for an\nexample config file, and [server/config/config.go](server/config/config.go) for documentation on available options. To enable the file viewer, you must include an [`IndexConfig`](server/config/config.go#L61) block inside of the config file. An example `IndexConfig` block can be seen at [doc/examples/livegrep/index.json](doc/examples/livegrep/index.json).\n\n*Tip: For each repository included in your `IndexConfig`, make sure to include `metadata.url_pattern` if you would like the file viewer to be able to link out to the external host. You'll see a warning in your browser console if you don't do this.*\n\n### Generating index with `livegrep-github-reindex`\nIf you are already using the `livegrep-github-reindex` tool, an IndexConfig index file is generated for you, by default named \"livegrep.json\".\n\nRun the indexer\n```\nbazel-bin/cmd/livegrep-github-reindex/livegrep-github-reindex_/livegrep-github-reindex -user=xvandish -forks=false -name=github.com/xvandish -out xvandish.idx ```\n```\n\nThe indexer will have done these main things:\n1. Clone (or update) all repositories for `user=xvandish` to/in `repos/xvandish`\n2. Create an IndexConfig file - `repos/livegrep.json`\n3. Create a code index, this is whats used to search - `./xvandish.idx`\n\nHere's an abbreviated version of what your directory might look like after running the indexer.\n```\nlivegrep\n│   xvandish.idx\n└───repos\n│   │   livegrep.json\n│   └───xvandish\n│       └───repo1\n│       └───repo2\n│       └───repo3\n```\n\n### Using your generated index\nNow that you generated an index file, it's time to run livegrep with it.\n\nRun the backend:\n```\nbazel-bin/src/tools/codesearch -load_index xvandish.idx -grpc localhost:9999\n```\n\nRun the frontend in another shell instance with the path to the index file located at `repos/livegrep.json`.\n```\nbazel-bin/cmd/livegrep/livegrep_/livegrep -index-config ./repos/livegrep.json\n```\nIn a browser, now visit `http://localhost:8910` and you should see a working\nlivegrep. Search for something, and once you get a result, click on the file\nname or a line number. You should now be taken to the file browser!\n\nDocker images\n-------------\n\nLivegrep's CI builds Docker images [into the livegrep\norganization][docker] docker repository on every merge to `main`. They\nshould be generally usable. For instance, to build+run a livegrep\nindex of this repository, you could run:\n\n```\ndocker run -v $(pwd):/data ghcr.io/livegrep/livegrep/indexer /livegrep/bin/livegrep-github-reindex -repo livegrep/livegrep -http -dir /data\ndocker network create livegrep\ndocker run -d --rm -v $(pwd):/data --network livegrep --name livegrep-backend ghcr.io/livegrep/livegrep/base /livegrep/bin/codesearch -load_index /data/livegrep.idx -grpc 0.0.0.0:9999\ndocker run -d --rm --network livegrep --publish 8910:8910 ghcr.io/livegrep/livegrep/base /livegrep/bin/livegrep -docroot /livegrep/web -listen=0.0.0.0:8910 --connect livegrep-backend:9999\n```\n\nAnd then access http://localhost:8910/\n\nYou can also find the [docker-compose config powering\nlivegrep.com][docker-compose] in the `livegrep/livegrep.com`\nrepository.\n\n[docker]: https://github.com/orgs/livegrep/packages\n[docker-compose]: https://github.com/livegrep/livegrep.com/tree/main/compose\n\nResource Usage\n--------------\n\nlivegrep builds an index file of your source code, and then works\nentirely out of that index, with no further access to the original git\nrepositories.\n\nThe index file will vary somewhat in size, but will usually be 3-5x\nthe size of the indexed text. `livegrep` memory-maps the index file\ninto RAM, so it can work out of index files larger than (available)\nRAM, but will perform better if the file can be loaded entirely into\nmemory. Barring that, keeping the disk on fast SSDs is recommended for\noptimal performance.\n\nRegex Support\n-------------\n\nLivegrep uses Google's [re2](https://github.com/google/re2) regular\nexpression engine, and inherits its [supported\nsyntax](https://github.com/google/re2/wiki/Syntax).\n\nRE2 is mostly PCRE-compatible, but with some [mostly-deliberate\nexceptions](https://swtch.com/~rsc/regexp/regexp3.html#caveats)\n\n\nLICENSE\n-------\n\nLivegrep is open source. See [COPYING](COPYING) for more information.\n","funding_links":[],"categories":["C++","Starlark","Projects built with Bazel"],"sub_categories":["non-Google projects"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flivegrep%2Flivegrep","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flivegrep%2Flivegrep","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flivegrep%2Flivegrep/lists"}