{"id":31952891,"url":"https://github.com/batchlabs/charlatan","last_synced_at":"2025-10-14T13:25:38.747Z","repository":{"id":35566338,"uuid":"39838354","full_name":"BatchLabs/charlatan","owner":"BatchLabs","description":"SQL-like Query Language","archived":false,"fork":false,"pushed_at":"2016-12-28T17:54:14.000Z","size":156,"stargazers_count":55,"open_issues_count":0,"forks_count":4,"subscribers_count":17,"default_branch":"master","last_synced_at":"2024-06-20T13:41:07.620Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://godoc.org/github.com/BatchLabs/charlatan","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BatchLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-07-28T14:21:43.000Z","updated_at":"2021-04-22T06:10:33.000Z","dependencies_parsed_at":"2022-09-19T16:00:40.298Z","dependency_job_id":null,"html_url":"https://github.com/BatchLabs/charlatan","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/BatchLabs/charlatan","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BatchLabs%2Fcharlatan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BatchLabs%2Fcharlatan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BatchLabs%2Fcharlatan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BatchLabs%2Fcharlatan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BatchLabs","download_url":"https://codeload.github.com/BatchLabs/charlatan/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BatchLabs%2Fcharlatan/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279018708,"owners_count":26086606,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-14T13:25:01.239Z","updated_at":"2025-10-14T13:25:38.742Z","avatar_url":"https://github.com/BatchLabs.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Charlatan\n\n[![Build Status](https://travis-ci.org/BatchLabs/charlatan.svg?branch=master)](https://travis-ci.org/BatchLabs/charlatan)\n\n**Charlatan** is a query engine for lists or streams of records in different\nformats. It natively supports CSV and JSON formats but can easily be extended\nto others.\n\nIt supports an SQL-like query language that is defined below. Queries are\napplied to records to extract values depending on zero or more criteria.\n\n## Query Syntax\n\n```\nSELECT \u003cfields\u003e FROM \u003csource\u003e [ WHERE \u003cvalue\u003e ] [ STARTING AT \u003cindex\u003e ] [ LIMIT [\u003coffset\u003e,] \u003ccount\u003e ]\n```\n\n- `\u003cfields\u003e` is a list of comma-separated field names. Each field name must\n  exist in the source. When reading CSV files, the field names are the column\n  names, while when reading JSON they represent keys.\n- `\u003csource\u003e` is the filename from which the data is read. The API is agnostique\n  on this and one can implement support for any source type.\n- `\u003cvalue\u003e` is a SQL-like value, which can be either a constant (e.g.\n  `WHERE 1`), a field (e.g. `WHERE archived`) or any operation using comparison\n  operators (`=`, `!=`, `\u003c`, `\u003c=`, `\u003e`, `\u003e=`, `AND`, `OR`) and optionally\n  parentheses (e.g. `WHERE (foo \u003e 2) AND (bar = \"yo\")`). The parser allows to\n  use `\u0026\u0026` instead of `AND` and `||` instead of `OR`. It also support inclusive\n  range tests, like `WHERE age BETWEEN 20 AND 30`.\n- `LIMIT N` can be used to keep only the first N matched records. It also\n  support the MySQL way to specify offsets: `LIMIT M, N` can be used to get the\n  first N matched records after the M-th.\n- `STARTING AT \u003cindex\u003e` can be used to skip the first N records. It’s\n  equivalent to the `\u003coffset\u003e` field of the `LIMIT` clause, and if both clauses\n  are used in a query, the last one will be used.\n\nConstant values include strings, integers, floats, booleans and the `null`\nvalue.\n\n### Examples\n\n```sql\nSELECT CountryName FROM sample/csv/population.csv WHERE Year = 2010 AND Value \u003e 50000000 AND Value \u003c 70000000\nSELECT name, age FROM sample/json/people.jsons WHERE stats.walking \u003e 30 AND stats.biking \u003c 300\nSELECT name, age FROM sample/json/people.jsons WHERE stats.walking BETWEEN 20 AND 100 LIMIT 10, 5\n```\n\n### Type Coercion Rules\n\n* int: same value if the constant is an integer. Truncated value if it’s a\n  float. `1` if it’s a `true` boolean. `0` for everything else.\n* float: same value if the constant is an integer or a float. `1.0` if it’s a\n  `true` boolean. `0.0` for everything else.\n* boolean: `true` if it’s a string (even if it’s empty), a `true` boolean, a\n  non-zero integer or float. `false` for everything else.\n* string: the string representation of the constant. `null` becomes `\"null\"`\n\nThese rules mean that e.g. `WHERE 0` is equivalent to `WHERE false` and\n`WHERE \"\"` is equivalent to `WHERE true`.\n\n## API\n\nThe library is responsible for parsing the query and executing against records.\nEverything else is up to you, including how fields are retrieved from records.\n\nNote: code examples below don’t include error handling for clarity purposes.\n\n```go\n// parse the query\nquery, _ := charlatan.QueryFromString(\"SELECT foo FROM myfile.json WHERE foo \u003e 2\")\n\n// open the source file\nreader, _ := os.Open(query.From())\n\ndefer reader.Close()\n\n// skip lines if the query contains \"STARTING AT \u003cn\u003e\"\nskip := query.StartingAt()\n\ndecoder := json.NewDecoder(reader)\n\nfor {\n    // here we use STARTING AT to skip all lines, not only the ones that match\n    // the query. This is not the usual behavior, but we can do whatever we\n    // want here.\n    skip--\n    if skip \u003e= 0 {\n        continue\n    }\n\n    // get a new JSON record\n    r, err := record.NewJSONRecordFromDecoder(decoder)\n\n    if err == io.EOF {\n        break\n    }\n\n    // evaluate the query against the record to test if it matches\n    if match, _ := query.Evaluate(r); !match {\n        continue\n    }\n\n    // extract the values and print them\n    values, _ := query.FieldsValues(r)\n    fmt.Printf(\"%v\\n\", values)\n}\n```\n\nTwo record types are included: `JSONRecord` and `CSVRecord`. Implementing a\nrecord only requires one method: `Find(*Field) (*Const, error)`, which takes a\nfield and return its value.\n\nAs an example, let’s implement a `LineRecord` that’ll be used to get specific\ncharacters on each line of a file, `c0` being the first character:\n\n```go\ntype LineRecord struct { Line string }\n\nfunc (r *LineRecord) Find(f *charlatan.Field) (*charlatan.Const, error) {\n\n    // this is the field value we must return\n    name := f.Name()\n\n    // we reject fields that doesn't start with 'c'\n    if len(name) \u003c 2 || name[0] != 'c' {\n        return nil, fmt.Errorf(\"Unknown field '%s'\", name)\n    }\n\n    // we extract the character index from the field name.\n    index, err := strconv.ParseInt(name[1:], 10, 64)\n    if err != nil {\n        return nil, err\n    }\n\n    // let's not be too strict and accept out-of-range indexes\n    if index \u003c 0 || index \u003e= int64(len(r.Line)) {\n        return charlatan.StringConst(\"\"), nil\n    }\n\n    return charlatan.StringConst(fmt.Sprintf(\"%c\", r.Line[index])), nil\n}\n```\n\nOne can now loop over a file’s content, construct `LineRecord`s from its lines\nand evaluate queries against them:\n\n```go\nquery, _ := charlatan.QueryFromString(\"SELECT c1 FROM myfile WHERE c0 = 'a'\")\n\nf, _ := os.Open(query.From())\ndefer f.Close()\n\ns := bufio.NewScanner(f)\nfor s.Scan() {\n    r := \u0026LineRecord{Line: s.Text()}\n\n    if m, _ := query.Evaluate(r); !m {\n        continue\n    }\n\n    values, _ := query.FieldsValues(r)\n    fmt.Printf(\"%v\\n\", values)\n}\n```\n\n### Examples\n\nTwo examples are included in the repository under `sample/csv/` and\n`sample/json/`.\n\n### Authors\n\n- [Nicolas DOUILLET](https://github.com/minimarcel)\n- [Vincent RISCHMANN](https://github.com/vrischmann)\n- [Baptiste FONTAINE](https://github.com/bfontaine)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbatchlabs%2Fcharlatan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbatchlabs%2Fcharlatan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbatchlabs%2Fcharlatan/lists"}