{"id":17476070,"url":"https://github.com/janderland/fql","last_synced_at":"2026-01-14T17:03:32.251Z","repository":{"id":37986213,"uuid":"309250154","full_name":"janderland/fql","owner":"janderland","description":"Foundation DB Query Language","archived":false,"fork":false,"pushed_at":"2026-01-02T06:05:59.000Z","size":2370,"stargazers_count":149,"open_issues_count":15,"forks_count":4,"subscribers_count":7,"default_branch":"main","last_synced_at":"2026-01-08T01:22:12.135Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/janderland.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-11-02T03:42:40.000Z","updated_at":"2026-01-04T00:57:47.000Z","dependencies_parsed_at":"2024-06-19T13:35:01.441Z","dependency_job_id":"e60a3b63-a620-4f9e-98bf-d4f5e51c5a66","html_url":"https://github.com/janderland/fql","commit_stats":null,"previous_names":["janderland/fql","janderland/fdbq"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/janderland/fql","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janderland%2Ffql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janderland%2Ffql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janderland%2Ffql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janderland%2Ffql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/janderland","download_url":"https://codeload.github.com/janderland/fql/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janderland%2Ffql/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28427183,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T16:38:47.836Z","status":"ssl_error","status_checked_at":"2026-01-14T16:34:59.695Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-18T19:00:42.893Z","updated_at":"2026-01-14T17:03:32.229Z","avatar_url":"https://github.com/janderland.png","language":"Go","funding_links":[],"categories":["Layers"],"sub_categories":[],"readme":"# FQL\n\n![demo gif](vhs/demo.gif)\n\nFQL provides a query language and an alternative client API for Foundation DB.\nSome things this project aims to achieve are:\n\n- [x] Provide a query language for FDB.\n- [x] Provide a textual description of key-value schemas.\n- [x] Provide a Go API which is structurally equivalent to the query language.\n- [ ] Simplify the ergonomics of the FoundationDB API.\n    - [ ] Gracefully handle multi-transaction range-reads.\n    - [ ] Gracefully handle transient errors.\n- [ ] Provide an environment for exploring FDB data.\n- [ ] Import/Export subsets of FDB data.\n\n## Building \u0026 Running\n\n### Without Docker\n\nWith the Foundation DB client library (\u003e= v6.2.0) and Go (\u003e= v1.20) installed,\nyou can simply run `go build` in the root of this repo. This will create an\n`fql` binary in the root of the repo.\n\n### Docker Environment\n\nBuilding, linting, and testing can all be performed in a Docker environment.\nThis allows any host to perform these operations with only Docker as a\ndependency. The [build.sh](build.sh) script can be used to perform these\noperations. This is the same script used by the CI/CD workflow of this repo.\n\nTo build, lint, \u0026 test the current state of the codebase, run `./build.sh \n--verify`. To learn more about the build script, run `./build.sh --help`.\n\n### Docker Image\n\nFQL is available as a Docker image for executing queries. The first argument\npassed to the container is the contents of the cluster file. The remaining\narguments are passed to the FQL binary.\n\n```bash\n# 'my_cluster:baoeA32@172.20.3.33:4500' is used as the contents\n# for the cluster file. '-log' and '/my/dir(\u003c\u003e)=42' are passed\n# as args to the FQL binary.\ndocker run docker.io/janderland/fql 'my_cluster:baoeA32@172.20.3.33:4500' -log '/my/dir(\u003c\u003e)=42'\n```\n\nWithin the cluster file contents (first argument), any instances of a \nhostname wrapped in curly braces (e.g. '{my_hostname}') are replaced by the \nequivalent IP address. FDB doesn't support connecting to a cluster via \nhostnames, so this functional provides a workaround. This can simplify \nconnecting to a Docker instance of FDB.\n\n```bash\ndocker network create my_net\ndocker run --network my_net --name fdb -d foundationdb/foundationdb\n\n# The substring '{fdb}' in the first argument will be replaced with\n# the IP address of the FDB container started above before the cluster\n# file is written to disk.\ndocker run --network my_net docker.io/janderland/fql 'docker:docker@{fdb}:4500' -log '/my/dir(\u003c\u003e)=42'\n```\n\n## Query Language\n\nHere is the [syntax definition](syntax.ebnf) for the query language. Currently,\nFQL is focused on reading \u0026 writing key-values created using the directory and\ntuple layers. Reading or writing keys of arbitrary byte strings is not \nsupported.\n\nFQL queries are a textual representation of a specific key-value or a schema\ndescribing the structure of many key-values. These queries have the ability to\nwrite a key-value, read one or more key-values, and list directories.\n\n### Components \u0026 Structure\n\nThis section will explain the components and structure of an FQL query. The\nsemantic meaning of these queries will be explained below in the [Kinds of \nQueries](#kinds-of-queries) section.\n\n#### Primitives\n\nFQL utilizes textual representations of the element types supported by the\ntuple layer. These types are known as primitives. Besides as tuple elements,\nprimitives can also be used as the value portion of a key-value.\n\n| Type     | Example                                |\n|:---------|:---------------------------------------|\n| `nil`    | `nil`                                  |\n| `int`    | `-14`                                  |\n| `uint`   | `7`                                    |\n| `bool`   | `true`                                 |\n| `float`  | `33.4`                                 |\n| `string` | `\"string\"`                             |\n| `bytes`  | `0xa2bff2438312aac032`                 |\n| `uuid`   | `5a5ebefd-2193-47e2-8def-f464fc698e31` |\n\nWhen primitives are used as tuple elements, they are encoded using the tuple \nlayer. When they are used as the value portion of a key-value, they are \nencoded by FQL as outlined below.\n\n| Type     | Encoding                          |\n|:---------|:----------------------------------|\n| `nil`    | `nil`                             |\n| `int`    | 64-bit, endianness configurable   |\n| `uint`   | 64-bit, endianness configurable   |\n| `bool`   | single bit, `0` means false       |\n| `float`  | IEEE 754, endianness configurable |\n| `string` | ASCII byte string                 |\n| `bytes`  | As provided                       |\n| `uuid`   | 16-byte string                    |\n\nIdeally, the encoding of these primitives would align with common community \npractices to maximize usefulness. Let me know if you believe it doesn't.\n\nEven though a big int encoding is supported by the tuple layer, FQL does \nnot currently support using big ints.\n\n#### Directories\n\nA directory is specified as a sequence of strings, each prefixed by a forward\nslash:\n\n```fql\n/my/dir/path_way\n```\n\nThe strings of the directory do not need quotes if they only contain\nalphanumericals, underscores, dashes, or periods. To use other symbols, the\nstrings must be quoted:\n\n```\n/my/\"dir@--\\o/\"/path_way\n```\n\nThe quote character may be backslash escaped:\n\n```\n/my/\"\\\"dir\\\"\"/path_way\n```\n\n#### Tuples\n\nA tuple is specified as a sequence of elements, separated by commas, wrapped in\na pair of curly braces. The elements may be a tuple or any of the primitive\ntypes.\n\n```fql\n(\"one\", 2, 0x03, ( \"subtuple\" ), 5825d3f8-de5b-40c6-ac32-47ea8b98f7b4)\n```\n\nThe last element of a tuple may be the `...` token.\n\n```fql\n(0xFF, \"thing\", ...)\n```\n\nAny combination of spaces, tabs, and newlines is allowed after the opening  \nbrace and commas.\n\n```fql\n(\n  1,\n  2,\n  3,\n)\n```\n\n#### Key-Values\n\nA key-value is specified as a directory, tuple, equal symbol, and value appended\ntogether:\n\n```fql\n/my/dir(\"this\", 0)=0xabcf03\n```\n\nThe value following the equal symbol may be any of the primitives or a tuple:\n\n```fql\n/my/dir(22.3, -8)=(\"another\", \"tuple\")\n```\n\nThe value can also be the `clear` token.\n\n```fql\n/some/where(\"home\", \"town\", 88.3)=clear\n```\n\n#### Variables\n\nA variable may be used in place of a directory element, tuple element, or value.\n\n```fql\n/my/dir/\u003c\u003e(\"first\", \u003c\u003e, \"third\")=\u003c\u003e\n```\n\nIf the variable is a tuple element or value, it may contain a list of primitive\ntypes separated by pipes, except for the `nil` type. The variable may also\ncontain the `any` type which is equivalent to specifying every type. Specifying\nno types is also equivalent to specifying the `any` type.\n\n```fql\n/my/dir(\"that\", \u003cint|float|bytes\u003e)=\u003cany\u003e\n```\n\n### Kinds of Queries\n\nThis section showcases the various kinds of FQL queries, their semantic\nmeaning, and the equivalent FDB API calls implemented in Go.\n\n#### Set\n\nSet queries write a single key-value. The query must not contain the `clear`\nor `...` tokens, nor a variable.\n\n```fql\n/my/dir(\"hello\", \"world\")=42\n```\n\n```go\ndb.Transact(func(tr fdb.Transaction) (interface{}, error) {\n  dir, err := directory.CreateOrOpen(tr, []string{\"my\", \"dir\"}, nil)\n  if err != nil {\n    return nil, err\n  }\n\n  val := make([]byte, 8)\n  binary.LittleEndian.PutUint64(val, 42)\n  tr.Set(dir.Pack(tuple.Tuple{\"hello\", \"world\"}), val)\n  return nil, nil\n})\n```\n\n#### Clear\n\nClear queries delete a single key-value. The query must contain the `clear`\ntoken as it's value and must not contain the `...` token or variables.\n\n```fql\n/my/dir(\"hello\", \"world\")=clear\n```\n\n```go\ndb.Transact(func(tr fdb.Transaction) (interface{}, error) {\n  dir, err := directory.Open(tr, []string{\"my\", \"dir\"}, nil)\n  if err != nil {\n    if errors.Is(err, directory.ErrDirNotExists) {\n      return nil, nil\n    }\n    return nil, err\n  }\n\n  tr.Clear(dir.Pack(tuple.Tuple{\"hello\", \"world\"}))\n  return nil, nil\n})\n```\n\n#### Read Single Key\n\nRead-single queries read a single key-value. These queries must not have the\n`...` token or a variable in their key. The value must be a variable.  \nDeserialization of the value is attempted for each type in the order specified\nby the variable. The first successful deserialization is used as the output. If\nthe value cannot be deserialized as any of the types specified then the\nkey-value is not returned or an error is returned, depending on configuration.\n\n```fql\n/my/dir(99.8, 7dfb10d1-2493-4fb5-928e-889fdc6a7136)=\u003cint|string\u003e\n```\n\n```go\ndb.Transact(func(tr fdb.Transaction) (interface{}, error) {\n  dir, err := directory.Open(tr, []string{\"my\", \"dir\"}, nil)\n  if err != nil {\n    if errors.Is(err, directory.ErrDirNotExists) {\n      return nil, nil\n    }\n    return nil, err\n  }\n\n  val := tr.MustGet(dir.Pack(tuple.Tuple{99.8,\n    tuple.UUID{0x7d, 0xfb, 0x10, 0xd1, 0x24, 0x93, 0x4f, 0xb5, 0x92, 0x8e, 0x88, 0x9f, 0xdc, 0x6a, 0x71, 0x36}))\n  \n     \n  if len(val) == 8 {\n      return binary.LittleEndian.Uint64(val), nil\n  }\n  return string(val), nil\n})\n```\n\nAs a shorthand, these query may be specified without the `=` token or value. \nThis implies an empty variable as the value. In the code block below, the \nthree queries are equivalent.\n\n```fql\n/my/dir(99.8, 7dfb10d1-2493-4fb5-928e-889fdc6a7136)\n/my/dir(99.8, 7dfb10d1-2493-4fb5-928e-889fdc6a7136)=\u003c\u003e\n/my/dir(99.8, 7dfb10d1-2493-4fb5-928e-889fdc6a7136)=\u003cany\u003e\n```\n\n#### Read Range of Keys\n\nRead-many queries read a range of values based on a key prefix. These \nqueries have a `...` token or a variable in their key. If a key-value is \nencountered which does not match the schema defined by the query then the\nkey-value is not returned or an error is returned, depending on configuration.\nThese queries are implemented using FDB's range-read mechanism with \nadditional filtering performed on the client. Care must be taken with these \nqueries as they may result in large amounts of data being sent to the \nclient and most of the data being filtered out.\n\n```fql\n/people(3392, \u003cstring|int\u003e, \u003c\u003e)=(\u003cuint\u003e, ...)\n```\n\n```go\ndb.ReadTransact(func(tr fdb.ReadTransaction) (interface{}, error) {\n  dir, err := directory.Open(tr, []string{\"people\"}, nil)\n  if err != nil {\n    if errors.Is(err, directory.ErrDirNotExists) {\n      return nil, nil\n    }\n    return nil, err\n  }\n\n  rng, err := fdb.PrefixRange(dir.Pack(tuple.Tuple{3392}))\n  if err != nil {\n    return nil, err\n  }\n\n  var results []fdb.KeyValue\n  iter := tr.GetRange(rng, fdb.RangeOptions{}).Iterator()\n  for iter.Advance() {\n    kv := iter.MustGet()\n\n    tup, err := dir.Unpack(kv.Key)\n    if err != nil {\n      return nil, err\n    }\n\n    if len(tup) != 3 {\n      return nil, fmt.Errorf(\"invalid kv: %v\", kv)\n    }\n\n    switch tup[0].(type) {\n    default:\n      return nil, fmt.Errorf(\"invalid kv: %v\", kv)\n    case string | int64:\n    }\n\n    val, err := tuple.Unpack(kv.Value)\n    if err != nil {\n      return nil, fmt.Errorf(\"invalid kv: %v\", kv)\n    }\n    if len(val) == 0 {\n      return nil, fmt.Errorf(\"invalid kv: %v\", kv)\n    }\n    if _, isInt := val[0].(uint64); !isInt {\n      return nil, fmt.Errorf(\"invalid kv: %v\", kv)\n    }\n\n    results = append(results, kv)\n  }\n  return results, nil\n})\n```\n\n#### List Directory Paths\n\nIf only a directory is provided as a query, then the directory layer is queried.\nEmpty variables may be included as placeholders for any directory name.\n\n```fql\n/root/\u003c\u003e/items/\u003c\u003e\n```\n\n```go\ndb.ReadTransact(func(tr fdb.ReadTransaction) (interface{}, error) {\n  root, err := directory.Open(tr, []string{\"root\"}, nil)\n  if err != nil {\n    if errors.Is(err, directory.ErrDirNotExists) {\n      return nil, nil\n    }\n    return nil, err\n  }\n\n  oneDeep, err := root.List(tr, nil)\n  if err != nil {\n    return nil, err\n  }\n\n  var results [][]string\n  for _, dir1 := range oneDeep {\n    twoDeep, err := root.List(tr, []string{dir1, \"items\"})\n    if err != nil {\n      return nil, err\n    }\n\n    for _, dir2 := range twoDeep {\n      results = append(results, []string{\"root\", dir1, dir2})\n    }\n  }\n  return results, nil\n})\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjanderland%2Ffql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjanderland%2Ffql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjanderland%2Ffql/lists"}