{"id":13674683,"url":"https://github.com/bblfsh/bblfshd","last_synced_at":"2025-04-06T04:09:39.510Z","repository":{"id":17624780,"uuid":"82369481","full_name":"bblfsh/bblfshd","owner":"bblfsh","description":"A self-hosted server for source code parsing","archived":false,"fork":false,"pushed_at":"2023-10-30T15:37:05.000Z","size":17137,"stargazers_count":362,"open_issues_count":30,"forks_count":54,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-03-30T03:06:25.235Z","etag":null,"topics":["ast","babelfish","code-analysis","parser","server","uast"],"latest_commit_sha":null,"homepage":"https://doc.bblf.sh","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bblfsh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-02-18T07:24:41.000Z","updated_at":"2025-02-15T00:19:47.000Z","dependencies_parsed_at":"2024-01-14T17:05:08.210Z","dependency_job_id":"e754f7b3-6287-41bb-b35c-3845f9fde416","html_url":"https://github.com/bblfsh/bblfshd","commit_stats":null,"previous_names":["bblfsh/server","the-babelfish/server"],"tags_count":60,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bblfsh%2Fbblfshd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bblfsh%2Fbblfshd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bblfsh%2Fbblfshd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bblfsh%2Fbblfshd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bblfsh","download_url":"https://codeload.github.com/bblfsh/bblfshd/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247430869,"owners_count":20937874,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast","babelfish","code-analysis","parser","server","uast"],"created_at":"2024-08-02T11:00:57.066Z","updated_at":"2025-04-06T04:09:39.378Z","avatar_url":"https://github.com/bblfsh.png","language":"Go","readme":"# bblfshd [![Build Status](https://travis-ci.org/bblfsh/bblfshd.svg?branch=master)](https://travis-ci.org/bblfsh/bblfshd) [![codecov](https://codecov.io/gh/bblfsh/bblfshd/branch/master/graph/badge.svg)](https://codecov.io/gh/bblfsh/bblfshd) [![license](https://img.shields.io/badge/license-GPL--3.0-blue.svg)](https://github.com/bblfsh/bblfshd/blob/master/LICENSE) [![GitHub release](https://img.shields.io/github/release/bblfsh/bblfshd.svg)](https://github.com/bblfsh/bblfshd/releases)\n\nThis repository contains bblfsh daemon (*bblfshd*), which includes the\nruntime that runs the driver in *containers* and the bblfshctl, a cli tool used\nto control the installed drivers and query the status of the daemon.\n\nDrivers are implemented as Docker images, each having their own repository in the \n[`bblfsh` organization](https://github.com/search?q=topic%3Adriver+org%3Abblfsh\u0026type=Repositories)\non GitHub. For more information, see [bblfsh SDK documentation](https://doc.bblf.sh/writing-a-driver/babelfish-sdk.html).\n\n## Getting Started\n\nSee the [Getting Started](https://doc.bblf.sh/using-babelfish/getting-started.html) guide.\n\n### Quick start\n\nThis project is now part of [source{d} Engine](https://sourced.tech/engine),\nwhich provides the simplest way to get started with a single command.\nVisit [sourced.tech/engine](https://sourced.tech/engine) for more information.\n\n#### Rootless mode\n\nThe recommended way to run *bblfshd* by itself is using Docker:\n\n```sh\ndocker run -d --name bblfshd \\\n  -p 9432:9432 \\\n  -v /var/lib/bblfshd:/var/lib/bblfshd \\\n  -v /proc:/newproc \\\n  --security-opt seccomp=./bblfshd-seccomp.json \\\n  bblfsh/bblfshd\n```\n\nOn macOS, use this command instead to use a Docker volume:\n\n```sh\ndocker run -d --name bblfshd \\\n  -p 9432:9432 \\\n  -v bblfsh-storage:/var/lib/bblfshd bblfsh/bblfshd \\\n  -v /proc:/newproc \\\n  --security-opt seccomp=./bblfshd-seccomp.json \\\n  bblfsh/bblfshd\n```\n\n\nTo understand the flags `-v /proc:/newproc` and `--security-opt seccomp=./bblfshd-seccomp.json`, \nwhere [`bblfshd-seccomp.json`](./bblfshd-seccomp.json) is a file present in this repo, and check \nfurther requirements, please refer to [rootless.md](./rootless.md). `bblfshd` is based on \n[container technology](https://github.com/opencontainers/runc/tree/master/libcontainer)\nand interacts with the kernel at a low level. It exposes a gRPC server at the port `9432` by default \nwhich is used by the [clients](https://github.com/search?q=topic%3Aclient+org%3Abblfsh\u0026type=Repositories)\nto interact with the server. Also, we mount the path `/var/lib/bblfshd/` where\nall the driver images and container instances will be stored.\n\n#### Privileged mode\n\nWe advise against it, but if you prefer to run `bblfshd` in `privileged` mode to skip configuration steps of \n[rootless.md](rootless.md), you could do, in Linux:\n\n```sh\ndocker run -d --name bblfshd --privileged -p 9432:9432 -v /var/lib/bblfshd:/var/lib/bblfshd bblfsh/bblfshd\n```\n\nor macOs:\n\n```sh\ndocker run -d --name bblfshd --privileged -p 9432:9432 -v bblfsh-storage:/var/lib/bblfshd bblfsh/bblfshd\n```\n\n#### Install drivers\n\nNow you need to install the driver images into the daemon, you can install\nthe official images just running the command:\n\n```sh\ndocker exec -it bblfshd bblfshctl driver install --all\n```\n\nYou can check the installed versions by executing:\n```\ndocker exec -it bblfshd bblfshctl driver list\n```\n\n```\n+----------+-------------------------------+---------+--------+---------+-----+-------------+\n| LANGUAGE |             IMAGE             | VERSION | STATUS | CREATED | GO  |   NATIVE    |\n+----------+-------------------------------+---------+--------+---------+-----+-------------+\n| python   | //bblfsh/python-driver:latest | v1.1.5  | beta   | 4 days  | 1.8 | 3.6.2       |\n| java     | //bblfsh/java-driver:latest   | v1.1.0  | alpha  | 6 days  | 1.8 | 8.131.11-r2 |\n+----------+-------------------------------+---------+--------+---------+-----+-------------+\n```\n\nTo test the driver you can execute a parse request to the server with the `bblfshctl parse` command,\nand an example contained in the Docker image:\n\n```sh\ndocker exec -it bblfshd bblfshctl parse /opt/bblfsh/etc/examples/python.py\n```\n\n## SELinux\n\nIf your system has SELinux enabled (which is the default in Fedora, Red Hat, CentOS\nand many others) you'll need to compile and load a policy module before running the\nbblfshd Docker image or running driver containers will fail with a `permission\ndenied` message in the logs. \n\nTo do this, run these commands from the project root:\n\n```bash\ncd selinux/\nsh compile.sh\nsemodule -i bblfshd.pp\n```\n\nIf you were already running an instance of *bblfshd*, you will need to delete the\ncontainer (`docker rm -f bblfshd`) and run it again (`docker run...`).\n\nOnce the module has been loaded with `semodule` the change should persist even\nif you reboot. If you want to permanently remove this module run `semodule -d bblfshd`.\n\nAlternatively, you could set SELinux to permissive module with:\n\n```\necho 1 \u003e /sys/fs/selinux/enforce\n```\n\n(doing this on production systems which usually have SELinux enabled by default\nshould be strongly discouraged).\n\n## Development\n\nIf you wish to work on *bblfshd* , you'll first need [Go](http://www.golang.org)\ninstalled on your machine (version 1.11+ is *required*) and [Docker](https://docs.docker.com/engine/installation/).\nDocker is used to build and run tests in an isolated environment.\n\nFor local development of *bblfshd*, first make sure Go is properly installed and\nthat a [GOPATH](http://golang.org/doc/code.html#GOPATH) has been set. You will\n also need to add `$GOPATH/bin` to your `$PATH`.\n\nNext, using [Git](https://git-scm.com/), clone this repository into\n`$GOPATH/src/github.com/bblfsh/bblfshd`. All the necessary dependencies are\nautomatically installed, so you just need to type `make`. This will compile the\ncode and then run the tests. If this exits with exit status 0, then everything\nis working!\n\n\n### Dependencies\n\nEnsure you have [`ostree`](https://github.com/ostreedev/ostree) and development libraries for `ostree` installed.\n\nYou can install from your distribution pack manager as follow, or built [from source](https://github.com/ostreedev/ostree) (more on that [here](https://ostree.readthedocs.io/en/latest/#building)).\n\nDebian, Ubuntu, and related distributions:\n```\n$ apt-get install libostree-dev\n```\n\nFedora, CentOS, RHEL, and related distributions:\n```bash\n$ yum install -y ostree-devel\n```\n\nArch and related distributions:\n\n```bash\n$ pacman -S ostree\n```\n\n### Building From Source\n\nBuild with:\n\n```\n$ make build\n```\n\n### Running Tests\n\nRun tests with:\n\n```\n$ make test\n```\n\n### Environment variables\n\n- `BBLFSHD_MAX_DRIVER_INSTANCES` - maximal number of driver instances for each language.\n  Default to a number of CPUs.\n\n- `BBLFSHD_MIN_DRIVER_INSTANCES` - minimal number of driver instances that will be run\n  for each language. Default to 1.\n\n### Enable tracing\n\nBblfshd supports [OpenTracing](https://opentracing.io/) that can be used to profile request on a high level or trace\nindividual requests to bblfshd and/or language drivers.\n\nTo enable it, you can use [Jaeger](https://www.jaegertracing.io/docs/1.8/getting-started/).\nThe easiest way is to start all-in-one Jaeger image:\n```\ndocker run -d --name jaeger \\\n  -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \\\n  -p 5775:5775/udp \\\n  -p 6831:6831/udp \\\n  -p 6832:6832/udp \\\n  -p 5778:5778 \\\n  -p 16686:16686 \\\n  -p 14268:14268 \\\n  -p 9411:9411 \\\n  jaegertracing/all-in-one:1.8\n```\n\nFor Docker installation of bblfshd add the following flags:\n```\n--link jaeger:jaeger -e JAEGER_AGENT_HOST=jaeger -e JAEGER_AGENT_PORT=6831 -e JAEGER_SAMPLER_TYPE=const -e JAEGER_SAMPLER_PARAM=1\n```\n\nFor bblfshd running locally, set following environment variables:\n```\nJAEGER_AGENT_HOST=localhost JAEGER_AGENT_PORT=6831 JAEGER_SAMPLER_TYPE=const JAEGER_SAMPLER_PARAM=1\n```\n\nRun few requests, and check traces at http://localhost:16686.\n\nFor enabling tracing in production, consult [Jaeger documentation](https://www.jaegertracing.io/docs/1.8).\n\n## License\n\nGPLv3, see [LICENSE](LICENSE)\n\n","funding_links":[],"categories":["Software","Go"],"sub_categories":["Software Development - IDE \u0026 Tools"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbblfsh%2Fbblfshd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbblfsh%2Fbblfshd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbblfsh%2Fbblfshd/lists"}