{"id":13399331,"url":"https://github.com/facebook/duckling","last_synced_at":"2025-05-14T22:05:45.055Z","repository":{"id":38715843,"uuid":"83621289","full_name":"facebook/duckling","owner":"facebook","description":"Language, engine, and tooling for expressing, testing, and evaluating composable language rules on input strings.","archived":false,"fork":false,"pushed_at":"2025-02-22T01:17:27.000Z","size":7572,"stargazers_count":4147,"open_issues_count":137,"forks_count":733,"subscribers_count":79,"default_branch":"main","last_synced_at":"2025-05-07T21:16:46.994Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facebook.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-02T01:45:50.000Z","updated_at":"2025-05-07T10:08:21.000Z","dependencies_parsed_at":"2024-11-25T16:07:27.180Z","dependency_job_id":"1f24b7ab-596e-4a50-ac68-78bb20037c42","html_url":"https://github.com/facebook/duckling","commit_stats":{"total_commits":753,"total_committers":265,"mean_commits":2.841509433962264,"dds":0.7543160690571049,"last_synced_commit":"7520daaeba28691cda8e1b5c3d946028a28fb64b"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebook%2Fduckling","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebook%2Fduckling/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebook%2Fduckling/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebook%2Fduckling/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facebook","download_url":"https://codeload.github.com/facebook/duckling/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254235687,"owners_count":22036962,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T19:00:36.411Z","updated_at":"2025-05-14T22:05:45.007Z","avatar_url":"https://github.com/facebook.png","language":"Haskell","funding_links":[],"categories":["Haskell","Haskell (3)","Text content NER/Topic/Subject/Context","Language processing tools and libraries"],"sub_categories":["Other models"],"readme":"![Duckling Logo](https://github.com/facebook/duckling/raw/main/logo.png)\n\n# Duckling [![Support Ukraine](https://img.shields.io/badge/Support-Ukraine-FFD500?style=flat\u0026labelColor=005BBB)](https://opensource.fb.com/support-ukraine) [![Build Status](https://travis-ci.org/facebook/duckling.svg?branch=master)](https://travis-ci.org/facebook/duckling)\n\nDuckling is a Haskell library that parses text into structured data.\n\n```bash\n\"the first Tuesday of October\"\n=\u003e {\"value\":\"2017-10-03T00:00:00.000-07:00\",\"grain\":\"day\"}\n```\n\n## Requirements\n\nA Haskell environment is required. We recommend using\n[stack](https://docs.haskellstack.org/en/stable/).\n\nOn Linux and MacOS you'll need to install PCRE development headers. On Linux,\nuse your package manager to install them. On MacOS, the easiest way to install\nthem is with [Homebrew](https://brew.sh/):\n\n```bash\nbrew install pcre\n```\n\nIf that doesn't help, try running `brew doctor` and fix the issues it finds.\n\n## Quickstart\n\nTo compile and run the binary:\n\n```bash\nstack build\nstack exec duckling-example-exe\n```\n\nThe first time you run it, it will download all required packages.\n\nThis runs a basic HTTP server. Example request:\n\n```bash\ncurl -XPOST http://0.0.0.0:8000/parse --data 'locale=en_GB\u0026text=tomorrow at eight'\n```\n\nIn the example application, all dimensions are enabled by default. Provide the\nparameter `dims` to specify which ones you want. Examples:\n\n```bash\nIdentify credit card numbers only:\n$ curl -XPOST http://0.0.0.0:8000/parse --data 'locale=en_US\u0026text=\"4111-1111-1111-1111\"\u0026dims=\"[\"credit-card-number\"]\"'\nIf you want multiple dimensions, comma-separate them in the array:\n$ curl -XPOST http://0.0.0.0:8000/parse --data 'locale=en_US\u0026text=\"3 cups of sugar\"\u0026dims=\"[\"quantity\",\"numeral\"]\"'\n```\n\nSee `exe/ExampleMain.hs` for an example on how to integrate Duckling in your\nproject. If your backend doesn't run Haskell or if you don't want to spin your\nown Duckling server, you can directly use [wit.ai](https://wit.ai)'s built-in\nentities.\n\n## Supported dimensions\n\nDuckling supports many languages, but most don't support all dimensions yet\n(**we need your help!**). Please look into\n[this directory](https://github.com/facebook/duckling/blob/master/Duckling/Dimensions)\nfor language-specific support.\n\n| Dimension          | Example input                     | Example value output                                                                                                                                         |\n| ------------------ | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| `AmountOfMoney`    | \"42€\"                             | `{\"value\":42,\"type\":\"value\",\"unit\":\"EUR\"}`                                                                                                                   |\n| `CreditCardNumber` | \"4111-1111-1111-1111\"             | `{\"value\":\"4111111111111111\",\"issuer\":\"visa\"}`                                                                                                               |\n| `Distance`         | \"6 miles\"                         | `{\"value\":6,\"type\":\"value\",\"unit\":\"mile\"}`                                                                                                                   |\n| `Duration`         | \"3 mins\"                          | `{\"value\":3,\"minute\":3,\"unit\":\"minute\",\"normalized\":{\"value\":180,\"unit\":\"second\"}}`                                                                          |\n| `Email`            | \"duckling-team@fb.com\"            | `{\"value\":\"duckling-team@fb.com\"}`                                                                                                                           |\n| `Numeral`          | \"eighty eight\"                    | `{\"value\":88,\"type\":\"value\"}`                                                                                                                                |\n| `Ordinal`          | \"33rd\"                            | `{\"value\":33,\"type\":\"value\"}`                                                                                                                                |\n| `PhoneNumber`      | \"+1 (650) 123-4567\"               | `{\"value\":\"(+1) 6501234567\"}`                                                                                                                                |\n| `Quantity`         | \"3 cups of sugar\"                 | `{\"value\":3,\"type\":\"value\",\"product\":\"sugar\",\"unit\":\"cup\"}`                                                                                                  |\n| `Temperature`      | \"80F\"                             | `{\"value\":80,\"type\":\"value\",\"unit\":\"fahrenheit\"}`                                                                                                            |\n| `Time`             | \"today at 9am\"                    | `{\"values\":[{\"value\":\"2016-12-14T09:00:00.000-08:00\",\"grain\":\"hour\",\"type\":\"value\"}],\"value\":\"2016-12-14T09:00:00.000-08:00\",\"grain\":\"hour\",\"type\":\"value\"}` |\n| `Url`              | \"https://api.wit.ai/message?q=hi\" | `{\"value\":\"https://api.wit.ai/message?q=hi\",\"domain\":\"api.wit.ai\"}`                                                                                          |\n| `Volume`           | \"4 gallons\"                       | `{\"value\":4,\"type\":\"value\",\"unit\":\"gallon\"}`                                                                                                                 |\n\n[Custom dimensions](https://github.com/facebook/duckling/blob/master/exe/CustomDimensionExample.hs)\nare also supported.\n\n## Extending Duckling\n\nTo regenerate the classifiers and run the test suite:\n\n```bash\nstack build :duckling-regen-exe \u0026\u0026 stack exec duckling-regen-exe \u0026\u0026 stack test\n```\n\nIt's important to regenerate the classifiers after updating the code and before\nrunning the test suite.\n\nTo extend Duckling's support for a dimension in a given language, typically 4\nfiles need to be updated:\n\n- `Duckling/\u003cDimension\u003e/\u003cLang\u003e/Rules.hs`\n\n- `Duckling/\u003cDimension\u003e/\u003cLang\u003e/Corpus.hs`\n\n- `Duckling/Dimensions/\u003cLang\u003e.hs` (if not already present in\n  `Duckling/Dimensions/Common.hs`)\n\n- `Duckling/Rules/\u003cLang\u003e.hs`\n\nTo add a new language:\n\n- Make sure that the language code used follows the\n  [ISO-639-1 standard](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes).\n- The first dimension to implement is `Numeral`.\n- Follow\n  [this example](https://github.com/facebook/duckling/commit/24d3f199768be970149412c95b1c1bf5d76f8240).\n\nTo add a new locale:\n\n- There should be a need for diverging rules between the locale and the\n  language.\n- Make sure that the locale code is a valid\n  [ISO3166 alpha2 country code](https://www.iso.org/obp/ui/#search/code/).\n- Follow\n  [this example](https://github.com/facebook/duckling/commit/1ab5f447d2635fe6d48887a501d333a52adff5b9).\n\nRules have a name, a pattern and a production. Patterns are used to perform\ncharacter-level matching (regexes on input) and concept-level matching\n(predicates on tokens). Productions are arbitrary functions that take a list of\ntokens and return a new token.\n\nThe corpus (resp. negative corpus) is a list of examples that should (resp.\nshouldn't) parse. The reference time for the corpus is Tuesday Feb 12, 2013 at\n4:30am.\n\n`Duckling.Debug` provides a few debugging tools:\n\n```bash\n$ stack repl --no-load\n\u003e :l Duckling.Debug\n\u003e debug (makeLocale EN $ Just US) \"in two minutes\" [Seal Time]\nin|within|after \u003cduration\u003e (in two minutes)\n-- regex (in)\n-- \u003cinteger\u003e \u003cunit-of-duration\u003e (two minutes)\n-- -- integer (0..19) (two)\n-- -- -- regex (two)\n-- -- minute (grain) (minutes)\n-- -- -- regex (minutes)\n[Entity {dim = \"time\", body = \"in two minutes\", value = RVal Time (TimeValue (SimpleValue (InstantValue {vValue = 2013-02-12 04:32:00 -0200, vGrain = Second})) [SimpleValue (InstantValue {vValue = 2013-02-12 04:32:00 -0200, vGrain = Second})] Nothing), start = 0, end = 14}]\n```\n\n## License\n\nDuckling is [BSD-licensed](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebook%2Fduckling","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacebook%2Fduckling","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebook%2Fduckling/lists"}