{"id":13746831,"url":"https://github.com/opencog/link-grammar","last_synced_at":"2025-04-04T12:05:24.867Z","repository":{"id":23333417,"uuid":"26693758","full_name":"opencog/link-grammar","owner":"opencog","description":"The CMU Link Grammar natural language parser","archived":false,"fork":false,"pushed_at":"2024-05-31T23:03:09.000Z","size":40030,"stargazers_count":391,"open_issues_count":82,"forks_count":118,"subscribers_count":39,"default_branch":"master","last_synced_at":"2025-03-28T11:06:01.777Z","etag":null,"topics":["english","grammar","link-grammar","natural-language","natural-language-processing","nlp","parser","russian"],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-2.1","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opencog.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-11-15T21:00:05.000Z","updated_at":"2025-03-14T17:19:08.000Z","dependencies_parsed_at":"2023-02-14T22:31:41.780Z","dependency_job_id":"c7cd1113-09a2-48bc-892e-79ea93547b4a","html_url":"https://github.com/opencog/link-grammar","commit_stats":null,"previous_names":[],"tags_count":129,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Flink-grammar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Flink-grammar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Flink-grammar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Flink-grammar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opencog","download_url":"https://codeload.github.com/opencog/link-grammar/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247174387,"owners_count":20896075,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["english","grammar","link-grammar","natural-language","natural-language-processing","nlp","parser","russian"],"created_at":"2024-08-03T06:01:02.513Z","updated_at":"2025-04-04T12:05:24.848Z","avatar_url":"https://github.com/opencog.png","language":"C","funding_links":[],"categories":["C","Others"],"sub_categories":[],"readme":"Link Grammar Parser\n===================\n***Version 5.12.5***\n\n![Main](https://github.com/opencog/link-grammar/actions/workflows/main.yml/badge.svg)\n![node.js](https://github.com/opencog/link-grammar/actions/workflows/bindings-js.yml/badge.svg)\n\nThe Link Grammar Parser exhibits the linguistic (natural language)\nstructure of English, Thai, Russian, Arabic, Persian and limited subsets\nof a half-dozen other languages. This structure is a graph of typed\nlinks (edges) between the words in a sentence. One may obtain the\nmore conventional HPSG (constituent) and dependency style parses\nfrom Link Grammar by applying a collection of rules to convert to\nthese different formats. This is possible because Link Grammar goes\na bit \"deeper\" into the \"syntactico-semantic\" structure of a sentence:\nit provides considerably more fine-grained and detailed information\nthan what is commonly available in conventional parsers.\n\nThe theory of Link Grammar parsing was originally developed in 1991\nby Davy Temperley, John Lafferty and Daniel Sleator, at the time\nprofessors of linguistics and computer science at the Carnegie Mellon\nUniversity.  The three initial publications on this theory provide the\nbest introduction and overview; since then, there have been hundreds\nof publications further exploring, examining and extending the ideas.\n\nAlthough based on the\n[original Carnegie-Mellon code base](https://www.link.cs.cmu.edu/link/),\nthe current Link Grammar package has dramatically evolved and is\nprofoundly different from earlier versions.  There have been innumerable\nbug fixes; performance has improved by several orders of magnitude.\nThe package is fully multi-threaded, fully UTF-8 enabled, and has been\nscrubbed for security, enabling cloud deployment.  Parse coverage of\nEnglish has been dramatically improved; other languages have been added\n(most notably, Thai and Russian). There is a raft of new features,\nincluding support for morphology, dialects, and a fine-grained weight\n(cost) system, allowing vector-embedding-like behaviour. There is a\nnew, sophisticated tokenizer tailored for morphology: it can offer\nalternative splittings for morphologically ambiguous words.\nDictionaries can be updated at run-time, enabling systems that perform\ncontinuous learning of grammar to also parse at the same time. That is,\ndictionary updates and parsing are mutually thread-safe. Classes of\nwords can be recognized with regexes. Random planar graph parsing is\nfully supported; this allows uniform sampling of the space of planar\ngraphs.  A detailed report of what has changed can be found in the\n[ChangeLog](ChangeLog).\n\nThis code is released under the LGPL license, making it freely\navailable for both private and commercial use, with few restrictions.\nThe terms of the license are given in the LICENSE file included with\nthis software.\n\nPlease see the\n[main web page](https://opencog.github.io/link-grammar-website/)\nfor more information.  This version is a continuation of the\n[original CMU parser](http://www.link.cs.cmu.edu/link).\n\n### New!\nAs of version 5.9.0, the system include an experimental system for\ngenerating sentences. These are specified using a \"fill in the blanks\"\nAPI, where words are substituted into wild-card locations, whenever the\nresult is a grammatically valid sentence. Additional details are in\nthe man page: `man link-generator` (in the `man` subdirectory).\n\nThis generator is used in the\n[OpenCog Language Learning](https://github.com/opencog/learn)\nproject, which aims to automatically learn Link Grammars from corpora,\nusing brand-new and innovative information theoretic techniques,\nsomewhat similar to those found in artificial neural nets (deep\nlearning), but using explicitly symbolic representations.\n\nQuick Overview\n---------------\nThe parser includes API's in various different programming languages,\nas well as a handy command-line tool for playing with it.  Here's some\ntypical output:\n```\nlinkparser\u003e This is a test!\n\tLinkage 1, cost vector = (UNUSED=0 DIS= 0.00 LEN=6)\n\n    +-------------Xp------------+\n    +-----\u003eWV-----\u003e+---Ost--+   |\n    +---Wd---+-Ss*b+  +Ds**c+   |\n    |        |     |  |     |   |\nLEFT-WALL this.p is.v a  test.n !\n\n(S (NP this.p) (VP is.v (NP a test.n)) !)\n\n            LEFT-WALL    0.000  Wd+ hWV+ Xp+\n               this.p    0.000  Wd- Ss*b+\n                 is.v    0.000  Ss- dWV- O*t+\n                    a    0.000  Ds**c+\n               test.n    0.000  Ds**c- Os-\n                    !    0.000  Xp- RW+\n           RIGHT-WALL    0.000  RW-\n\n```\nThis rather busy display illustrates many interesting things. For\nexample, the `Ss*b` link connects the verb and the subject, and\nindicates that the subject is singular.  Likewise, the `Ost` link\nconnects the verb and the object, and also indicates that the object\nis singular. The `WV` (verb-wall) link points at the head-verb of\nthe sentence, while the `Wd` link points at the head-noun. The `Xp`\nlink connects to the trailing punctuation. The `Ds**c` link connects\nthe noun to the determiner: it again confirms that the noun is singular,\nand also that the noun starts with a consonant. (The `PH` link, not\nrequired here, is used to force phonetic agreement, distinguishing\n'a' from 'an').  These link types are documented in the\n[English Link Documentation](https://opencog.github.io/link-grammar-website/dict/index.html).\n\nThe bottom of the display is a listing of the \"disjuncts\" used for\neach word. The disjuncts are simply a list of the connectors that\nwere employed to form the links. They are particularly interesting\nbecause they serve as an extremely fine-grained form of a \"part of\nspeech\".  Thus, for example: the disjunct `S- O+` indicates a\ntransitive verb: its a verb that takes both a subject and an object.\nThe additional markup above indicates that 'is' is not only being used\nas a transitive verb, but it also indicates finer details: a transitive\nverb that took a singular subject, and was used (is usable as) the head\nverb of a sentence.  The floating-point value is the \"cost\" of the\ndisjunct; it very roughly captures the idea of the log-probability\nof this particular grammatical usage.  Much as parts-of-speech correlate\nwith word-meanings, so also fine-grains parts-of-speech correlate with\nmuch finer distinctions and gradations of meaning.\n\nThe link-grammar parser also supports morphological analysis. Here is\nan example in Russian:\n```\nlinkparser\u003e это теста\n\tLinkage 1, cost vector = (UNUSED=0 DIS= 0.00 LEN=4)\n\n             +-----MVAip-----+\n    +---Wd---+       +-LLCAG-+\n    |        |       |       |\nLEFT-WALL это.msi тест.= =а.ndnpi\n```\n\nThe `LL` link connects the stem 'тест' to the suffix 'а'. The `MVA`\nlink connects only to the suffix, because, in Russian, it is the\nsuffixes that carry all of the syntactic structure, and not the stems.\nThe Russian lexis is\n[documented here](https://opencog.github.io/link-grammar-website/russian/doc/).\n\nThe Thai dictionary is now fully developed, effectively covering the\nentire language.  An example in Thai:\n```\nlinkparser\u003e นายกรัฐมนตรี ขึ้น กล่าว สุนทรพจน์\n\tLinkage 1, cost vector = (UNUSED=0 DIS= 2.00 LEN=2)\n\n    +---------LWs--------+\n    |           +\u003c---S\u003c--+--VS-+--\u003eO--\u003e+\n    |           |        |     |       |\nLEFT-WALL นายกรัฐมนตรี.n ขึ้น.v กล่าว.v สุนทรพจน์.n\n```\n\nThe `VS` link connects two verbs 'ขึ้น' and 'กล่าว' in a serial verb\nconstruction. A summary of link types is\n[documented here](data/th/README.md). Full documentation of Thai Link\nGrammar can be [found here](data/th/LINKDOC.md).\n\nThai Link Grammar also accepts POS-tagged and named-entity-tagged\ninputs. Each word can be annotated with the Link POS tag. For example:\n\n```\nlinkparser\u003e เมื่อวานนี้.n มี.ve คน.n มา.x ติดต่อ.v คุณ.pr ครับ.pt\nFound 1 linkage (1 had no P.P. violations)\n\tUnique linkage, cost vector = (UNUSED=0 DIS= 0.00 LEN=12)\n\n                          +---------------------PT--------------------+\n    +---------LWs---------+----------\u003eVE----------\u003e+                  |\n    |           +\u003c---S\u003c---+--\u003eO--\u003e+       +\u003c--AXw\u003c-+---\u003eO---\u003e+        |\n    |           |         |       |       |        |         |        |\nLEFT-WALL เมื่อวานนี้.n[!] มี.ve[!] คน.n[!] มา.x[!] ติดต่อ.v[!] คุณ.pr[!] ครับ.pt[!]\n```\n\nFull documentation for the Thai dictionary can be\n[found here](data/th/INPUT_FORMATS.md).\n\nThe Thai dictionary accepts LST20 tagsets for POS and named entities,\nto bridge the gap between fundamental NLP tools and the Link Parser.\nFor example:\n\n```\nlinkparser\u003e linkparser\u003e วันที่_25_ธันวาคม@DTM ของ@PS ทุก@AJ ปี@NN เป็น@VV วัน@NN คริสต์มาส@NN\nFound 348 linkages (348 had no P.P. violations)\n\tLinkage 1, cost vector = (UNUSED=0 DIS= 1.00 LEN=10)\n\n    +--------------------------------LWs--------------------------------+\n    |               +\u003c------------------------S\u003c------------------------+\n    |               |                +----------\u003ePO---------\u003e+          |\n    |               +-----\u003eAJpr-----\u003e+            +\u003c---AJj\u003c--+          +----\u003eO----\u003e+------NZ-----+\n    |               |                |            |          |          |           |             |\nLEFT-WALL วันที่_25_ธันวาคม@DTM[!] ของ@PS[!].pnn ทุก@AJ[!].jl ปี@NN[!].n เป็น@VV[!].v วัน@NN[!].na คริสต์มาส@NN[!].n\n```\n\nNote that each word above is annotated with LST20 POS tags and NE tags.\nFull documentation for both the Link POS tags and the LST20 tagsets can\nbe [found here](data/th/TAGSETS.md). More information about LST20, e.g.\nannotation guideline and data statistics, can be\n[found here](https://arxiv.org/abs/2008.05055).\n\nThe `any` language supports uniformly-sampled random planar graphs:\n```\nlinkparser\u003e asdf qwer tyuiop fghj bbb\nFound 1162 linkages (1162 had no P.P. violations)\n\n             +-------ANY------+-------ANY------+\n    +---ANY--+--ANY--+        +---ANY--+--ANY--+\n    |        |       |        |        |       |\nLEFT-WALL asdf[!] qwer[!] tyuiop[!] fghj[!] bbb[!]\n```\nThe `ady` language does likewise, performing random morphological\nsplittings:\n```\nlinkparser\u003e asdf qwerty fghjbbb\nFound 1512 linkages (1512 had no P.P. violations)\n\n                                  +------------------ANY-----------------+\n    +-----ANY----+-------ANY------+                  +---------LL--------+\n    |            |                |                  |                   |\nLEFT-WALL asdf[!ANY-WORD] qwerty[!ANY-WORD] fgh[!SIMPLE-STEM].= =jbbb[!SIMPLE-SUFF]\n```\n\n\nTheory and Documentation\n------------------------\nAn extended overview and summary can be found in the\n[Link Grammar Wikipedia page](https://en.wikipedia.org/wiki/Link_grammar),\nwhich touches on most of the import, primary aspects of the theory.\nHowever, it is no substitute for the original papers published on the\ntopic:\n* Daniel D. K. Sleator, Davy Temperley,\n  [\"Parsing English with a Link Grammar\"](http://www.cs.cmu.edu/afs/cs.cmu.edu/project/link/pub/www/papers/ps/tr91-196.pdf)\n  October 1991 *CMU-CS-91-196*.\n* Daniel D. Sleator, Davy Temperley,\n  [\"Parsing English with a Link Grammar\"](http://www.cs.cmu.edu/afs/cs.cmu.edu/project/link/pub/www/papers/ps/LG-IWPT93.pdf),\n  *Third International Workshop on Parsing Technologies* (1993).\n* Dennis Grinberg, John Lafferty, Daniel Sleator,\n  [\"A Robust Parsing Algorithm for Link Grammars\"](http://www.cs.cmu.edu/afs/cs.cmu.edu/project/link/pub/www/papers/ps/tr95-125.pdf),\n  August 1995 *CMU-CS-95-125*.\n* John Lafferty, Daniel Sleator, Davy Temperley,\n  [\"Grammatical Trigrams: A Probabilistic Model of Link Grammar\"](http://www.cs.cmu.edu/afs/cs.cmu.edu/project/link/pub/www/papers/ps/gram3gram.pdf),\n  1992 *AAAI Symposium on Probabilistic Approaches to Natural Language*.\n\nThere are many more papers and references listed on the\n[primary Link Grammar website](https://opencog.github.io/link-grammar-website/)\n\nSee also the\n[C/C++ API documentation](https://opencog.github.io/link-grammar-website/api/index.html).\nBindings for other programming languages, including python3, java\nand node.js, can be found in the [bindings directory](bindings).\n(There are two sets of javascript bindings: one set for the library API,\nand another set for the command-line parser.)\n\nContents\n--------\n\n| Content       | Description |\n| ------------- |-------------|\n| LICENSE     | The license describing terms of use |\n| ChangeLog | A compendium of recent changes. |\n| configure | The GNU configuration script |\n| autogen.sh | Developer's configure maintenance tool |\n| link-grammar/*.c | The program.  (Written in ANSI-C) |\n| ---- | ---- |\n| bindings/autoit/  | Optional AutoIt language bindings. |\n| bindings/java/ | Optional Java language bindings. |\n| bindings/js/ | Optional JavaScript language bindings. |\n| bindings/lisp/ | Optional Common Lisp language bindings. |\n| bindings/node.js/ | Optional node.js language bindings. |\n| bindings/ocaml/ | Optional OCaML language bindings. |\n| bindings/python/  | Optional Python3 language bindings. |\n| bindings/python-examples/ | Link-grammar test suite and Python language binding usage example. |\n| bindings/swig/ | SWIG interface file, for other FFI interfaces. |\n| bindings/vala/  | Optional Vala language bindings. |\n| ---- | ---- |\n| data/en/ | English language dictionaries. |\n| data/en/4.0.dict | The file containing the dictionary definitions. |\n| data/en/4.0.knowledge | The post-processing knowledge file. |\n| data/en/4.0.constituents | The constituent knowledge file. |\n| data/en/4.0.affix | The affix (prefix/suffix) file. |\n| data/en/4.0.regex | Regular expression-based morphology guesser. |\n| data/en/tiny.dict | A small example dictionary. |\n| data/en/words/ | A directory full of word lists. |\n| data/en/corpus*.batch | Example corpora used for testing. |\n| ---- | ---- |\n| data/ru/ | A full-fledged Russian dictionary |\n| data/th/ | A full-fledged Thai dictionary (100,000+ words) |\n| data/ar/ | A fairly complete Arabic dictionary |\n| data/fa/ | A Persian (Farsi) dictionary |\n| data/de/ | A small prototype German dictionary |\n| data/lt/ | A small prototype Lithuanian dictionary |\n| data/id/ | A small prototype Indonesian dictionary |\n| data/vn/ | A small prototype Vietnamese dictionary |\n| data/he/ | An experimental Hebrew dictionary |\n| data/kz/ | An experimental Kazakh dictionary |\n| data/tr/ | An experimental Turkish dictionary |\n| ---- | ---- |\n| morphology/ar/ | An Arabic morphology analyzer |\n| morphology/fa/ | An Persian morphology analyzer |\n| ---- | ---- |\n| debug/ | Information about debugging the library |\n| msvc/ | Microsoft Visual-C project files |\n| mingw/ | Information on using MinGW under MSYS or Cygwin |\n\nUNPACKING and signature verification\n------------------------------------\nThe system is distributed using the conventional `tar.gz` format;\nit can be extracted using the `tar -zxf link-grammar.tar.gz` command\nat the command line.\n\nA tarball of the latest version can be downloaded from:\u003cbr\u003e\n[https://www.gnucash.org/link-grammar/downloads/](https://www.gnucash.org/link-grammar/downloads/)\n\nThe files have been digitally signed to make sure that there was no\ncorruption of the dataset during download, and to help ensure that\nno malicious changes were made to the code internals by third\nparties. The signatures can be checked with the gpg command:\n\n`gpg --verify link-grammar-5.12.5.tar.gz.asc`\n\nwhich should generate output identical to (except for the date):\n```\ngpg: Signature made Thu 26 Apr 2012 12:45:31 PM CDT using RSA key ID E0C0651C\ngpg: Good signature from \"Linas Vepstas (Hexagon Architecture Patches) \u003clinas@codeaurora.org\u003e\"\ngpg:                 aka \"Linas Vepstas (LKML) \u003clinasvepstas@gmail.com\u003e\"\n```\nAlternately, the md5 check-sums can be verified. These do not provide\ncryptographic security, but they can detect simple corruption. To\nverify the check-sums, issue `md5sum -c MD5SUM` at the command line.\n\nTags in `git` can be verified by performing the following:\n```\ngpg --recv-keys --keyserver keyserver.ubuntu.com EB6AA534E0C0651C\ngit tag -v link-grammar-5.10.5\n```\n\n\nCREATING the system\n-------------------\nTo compile the link-grammar shared library and demonstration program,\nat the command line, type:\n```\n./configure\nmake\nmake check\n```\n\nTo install, change user to \"root\" and say\n```\nmake install\nldconfig\n```\n\nThis will install the `liblink-grammar.so` library into `/usr/local/lib`,\nthe header files in `/usr/local/include/link-grammar`, and the\ndictionaries into `/usr/local/share/link-grammar`.  Running `ldconfig`\nwill rebuild the shared library cache.  To verify that the install was\nsuccessful, run (as a non-root user)\n```\nmake installcheck\n```\n\n### Optional system libraries\nThe link-grammar library has optional features that are enabled automatically\nif `configure` detects certain libraries. These libraries are optional on most\nof the systems and if the feature they add is desired, corresponding libraries\nneed to be installed before running `configure`.\n\nThe library package names may vary on various systems (consult Google if\nneeded...).  For example, the names may include `-devel` instead of `-dev`, or\nbe without it altogether. The library names may be without the prefix `lib`.\n\n* `libsqlite3-dev` (for SQLite-backed dictionary)\u003cbr\u003e\n* `libz1g-dev` or `libz-devel` (currently needed for the bundled `minisat2`)\u003cbr\u003e\n* `libedit-dev` (see [Editline](#Editline))\u003cbr\u003e\n* `libhunspell-dev` or `libaspell-dev` (and the corresponding English dictionary).\u003cbr\u003e\n* `libtre-dev` or `libpcre2-dev` (much faster than the libc REGEX\nimplementation, and needed for correctness on FreeBSD and Cygwin).\u003cbr\u003e\nUsing `libpcre2-dev` is strongly recommended. It must be used on certain\nsystems (as specified in their BUILDING sections).\n\n### Editline\nIf `libedit-dev` is installed, then the arrow keys can be used to edit\nthe input to the link-parser tool; the up and down arrow keys will\nrecall previous entries.  You want this; it makes testing and\nediting much easier.\n\n\n### Node.js Bindings\nTwo versions of node.js bindings are included. One version wraps the\nlibrary; the other uses emscripten to wrap the command-line tool. The\nlibrary bindings are in `bindings/node.js` while the emscripten wrapper\nis in `bindings/js`.\n\nThese are built using `npm`. First, you must build the core C library.\nThen do the following:\n```\n   cd bindings/node.js\n   npm install\n   npm run make\n```\nThis will create the library bindings and also run a small unit test\n(which should pass). An example can be found in\n`bindings/node.js/examples/simple.js`.\n\nFor the command-line wrapper, do the following:\n```\n   cd bindings/js\n   ./install_emsdk.sh\n   ./build_packages.sh\n```\n\n### Python3 Bindings\nThe Python3 bindings are built by default, providing that\nthe corresponding Python development packages are installed.\n(Python2 bindings are no longer supported.)\n\nThese packages are:\n- Linux:\n   * Systems using 'rpm' packages: `python3-devel`\n   * Systems using 'deb' packages: `python3-dev`\n- Windows:\n   * Install Python3 from https://www.python.org/downloads/windows/ .\n     You also have to install SWIG from http://www.swig.org/download.html .\n- macOS:\n   * Install python3 using [HomeBrew](http://brew.sh/).\n\nNOTE: Before issuing `configure` (see below) you have to validate that\nthe required python versions can be invoked using your `PATH`.\n\nThe use of the Python bindings is *OPTIONAL*; you do not need these if you\ndo not plan to use link-grammar with Python.  If you like to disable\nPython bindings, use:\n\n```\n./configure --disable-python-bindings\n```\n\nThe `linkgrammar.py` module provides a high-level interface in Python.\nThe `example.py` and `sentence-check.py` scripts provide a demo,\nand `tests.py` runs unit tests.\n\n- macOS:\n   * Due to file permissions settings, macOS users may need to install\n     python bindings into custom directory locations. This can be\n     done by saying\n     `make install pythondir=/where/to/install`\n\n### Java Bindings\nBy default, the `Makefile`s attempt to build the Java bindings.\nThe use of the Java bindings is *OPTIONAL*; you do not need these if\nyou do not plan to use link-grammar with Java.  You can skip building\nthe Java bindings by disabling as follows:\n```\n./configure --disable-java-bindings\n```\n\nIf `jni.h` isn't found, or if `ant` isn't found,\nthen the java bindings will not be built.\n\nNotes about finding `jni.h`:\u003cbr\u003e\nSome common java JVM distributions (most notably, the ones from Sun)\nplace this file in unusual locations, where it cannot be\nautomatically found.  To remedy this, make sure that environment variable\n`JAVA_HOME` is set correctly. The configure script looks for `jni.h` in\n`$JAVA_HOME/Headers` and in `$JAVA_HOME/include`; it also examines\ncorresponding locations for `$JDK_HOME`.  If `jni.h `still cannot be\nfound, specify the location with the `CPPFLAGS` variable: so, for example,\n```\nexport CPPFLAGS=\"-I/opt/jdk1.5/include/:/opt/jdk1.5/include/linux\"\n```\nor\n```\nexport CPPFLAGS=\"-I/c/java/jdk1.6.0/include/ -I/c/java/jdk1.6.0/include/win32/\"\n```\nPlease note that the use of `/opt` is non-standard, and most system\ntools will fail to find packages installed there.\n\nInstall location\n----------------\nThe `/usr/local` install target can be over-ridden using the\nstandard GNU `configure --prefix` option; so, for example:\n```\n./configure --prefix=/opt/link-grammar\n```\n\nBy using `pkg-config` (see below), non-standard install locations\ncan be automatically detected.\n\nCustom builds\n-------------\nAdditional config options are printed by\n```\n./configure --help\n```\n\nThe system has been tested and works well on 32 and 64-bit Linux\nsystems, FreeBSD, macOS, as well as on Microsoft Windows systems.\nSpecific OS-dependent notes follow.\n\n### BUILDING from the [GitHub repository](https://github.com/opencog/link-grammar)\n\nEnd users should download the tarball (see\n[UNPACKING and signature verification](#unpacking-and-signature-verification)).\n\nThe current GitHub version is intended for developers (including anyone who\nis willing to provide a fix, a new feature or an improvement). The tip of\nthe master branch is often unstable, and can sometimes have bad code in it\nas it is under development. It also needs installing of development tools\nthat are not installed by default. Due to these reason the use of the GitHub\nversion is discouraged for regular end users.\n\n#### Installing from GitHub\nClone it:\n`git clone https://github.com/opencog/link-grammar.git`\u003cbr\u003e\nOr download it as a ZIP:\u003cbr\u003e\n`https://github.com/opencog/link-grammar/archive/master.zip`\n\n#### Prerequisite tools\nTools that may need installation before you can build link-grammar:\n\n`make` (the `gmake` variant may be needed)\u003cbr\u003e\n`m4`\u003cbr\u003e\n`gcc` or `clang`\u003cbr\u003e\n`autoconf`\u003cbr\u003e\n`libtool`\u003cbr\u003e\n`autoconf-archive`\u003cbr\u003e\n`pkg-config` (may be named `pkgconf` or `pkgconfig`)\u003cbr\u003e\n`pip3` (for the Python bindings)\u003cbr\u003e\n\nOptional:\u003cbr\u003e\n`swig` (for language bindings)\u003cbr\u003e\n`flex`\u003cbr\u003e\nApache Ant (for Java bindings)\u003cbr\u003e\n`graphviz` (if you wish to use the word-graph display feature)\n\nThe GitHub version doesn't include a `configure` script.\nTo generate it, use:\n```\nautogen.sh\n```\n\nIf you get errors, make sure you have installed the above-listed\ndevelopment packages, and that your system installation is up to date.\nEspecially, missing `autoconf` or `autoconf-archive` may\ncause strange and misleading errors.\n\nFor more info about how to proceed, continue at the section\n[CREATING the system](#creating-the-system) and the relevant sections after it.\n\n#### Additional notes for developers\n\nTo configure **debug** mode, use:\n```\nconfigure --enable-debug\n```\nIt adds some verification debug code and functions that can\npretty-print several data structures.\n\nA feature that may be useful for debugging is the word-graph\ndisplay.  It is enabled by default. For more details on this feature, see\n[Word-graph display](link-grammar/tokenize/README.md#word-graph-display).\n\n\n### BUILDING on FreeBSD\n\nThe current configuration has an apparent standard C++ library mixing problem\nwhen `gcc` is used (a fix is welcome). However, the common practice on FreeBSD\nis to compile with `clang`, and it doesn't have this problem. In addition,\nthe add-on packages are installed under `/usr/local`.\n\nSo here is how `configure` should be invoked:\n```\nenv LDFLAGS=-L/usr/local/lib CPPFLAGS=-I/usr/local/include \\\nCC=clang CXX=clang++ configure\n```\n\nNote that `pcre2` is a required package as the existing `libc`\nregex implementation doesn't have the needed level of regex support.\n\nSome packages have different names than the ones mentioned in the previous\nsections:\n\n`minisat` (minisat2)\n`pkgconf` (pkg-config)\n\n### BUILDING on macOS\nPlain-vanilla Link Grammar should compile and run on Apple macOS\njust fine, as described above.  At this time, there are no reported\nissues.\n\nIf you do NOT need the java bindings, you should almost surely\nconfigure with:\n```\n./configure --disable-java-bindings\n```\n\nIf you do want Java bindings, be sure to set the JDK_HOME environment\nvariable to wherever `\u003cHeaders/jni.h\u003e` is.   Set the JAVA_HOME variable\nto the location of the java compiler.  Make sure you have ant\ninstalled.\n\nIf you would like to build from GitHub\n(see [BUILDING from the GitHub repository](#building-from-the-github-repository))\nyou can install the tools that are listed there using\n[HomeBrew](http://brew.sh/).\n\n### BUILDING on Windows\nThere are three different ways in which link-grammar can be compiled\non Windows.  One way is to use Cygwin, which provides a Linux\ncompatibility layer for Windows. Another way is use the\nMSVC system. A third way is to use the MinGW system, which uses the\nGnu toolset to compile windows programs. The source code supports\nWindows systems from Vista on.\n\nThe Cygwin way currently produces the best result, as it supports line editing\nwith command completion and history and also supports word-graph displaying on\nX-windows. (MinGW currently doesn't  have `libedit`, and the MSVC port\ncurrently doesn't support command completion and history, and also spelling.\n\n### BUILDING on Windows (Cygwin)\nThe easiest way to have link-grammar working on MS Windows is to\nuse Cygwin, a Linux-like environment for Windows making it possible\nto port software running on POSIX systems to Windows.  Download and\ninstall [Cygwin](http://www.cygwin.com/).\n\nNote that the installation of the `pcre2` package is required because the libc\nREGEX implementation is not capable enough.\n\nFor more details See [mingw/README-Cygwin.md](mingw/README-Cygwin.md).\n\n### BUILDING on Windows (MinGW)\nAnother way to build link-grammar is to use MinGW, which uses the GNU\ntoolset to compile POSIX-compliant programs for Windows. Using MinGW/MSYS2 is\nprobably the easiest way to obtain workable Java bindings for Windows.\nDownload and install MinGW/MSYS2 from [msys2.org](msys2.org).\n\nNote that the installation of the `pcre2` package is required because the libc\nREGEX implementation is not capable enough.\n\nFor more details see [mingw/README-MinGW64.md](mingw/README-MinGW64.md).\n\n### BUILDING and RUNNING on Windows (MSVC)\nMicrosoft Visual C/C++ project files can be found in the `msvc` directory.\nFor directions see the [README.md](msvc/README.md) file there.\n\nRUNNING the program\n-------------------\nTo run the program issue the command (supposing it is in your PATH):\n```\nlink-parser [arguments]\n```\n\nThis starts the program.  The program has many user-settable variables\nand options. These can be displayed by entering `!var` at the link-parser\nprompt.  Entering `!help` will display some additional commands.\n\nThe dictionaries are arranged in directories whose name is the 2-letter\nlanguage code. The link-parser program searches for such a language\ndirectory in that order, directly or under a directory names `data`:\n\n1. Under your current directory.\n2. Unless compiled with MSVC or run under the Windows console:\n   At the installed location (typically in `/usr/local/share/link-grammar`).\n3. If compiled on Windows: In the directory of the link-parser\n   executable (may be in a different location than the link-parser\n   command, which may be a script).\n\nIf link-parser cannot find the desired dictionary, use verbosity\nlevel 4 to debug the problem; for example:\n```\nlink-parser ru -verbosity=4\n```\n\nOther locations can be specified on the command line; for example:\n```\nlink-parser ../path/to-my/modified/data/en\n```\n\nWhen accessing dictionaries in non-standard locations, the standard\nfile-names are still assumed (*i.e.* `4.0.dict`, `4.0.affix`, *etc.*).\n\nThe Russian dictionaries are in `data/ru`. Thus, the Russian parser\ncan be started as:\n```\nlink-parser ru\n```\n\nIf you don't supply an argument to link-parser, it searches for a\nlanguage according to your current locale setup. If it cannot find such\na language directory, it defaults to \"en\".\n\nIf you see errors similar to this:\n```\nWarning: The word \"encyclop\" found near line 252 of en/4.0.dict\nmatches the following words:\nencyclop\nThis word will be ignored.\n```\n\nthen your UTF-8 locales are either not installed or not configured.\nThe shell command `locale -a` should list `en_US.utf8` as a locale.\nIf not, then you need to `dpkg-reconfigure locales` and/or run\n`update-locale` or possibly `apt-get install locales`, or\ncombinations or variants of these, depending on your operating\nsystem.\n\n\nTESTING the system\n------------------\nThere are several ways to test the resulting build.  If the Python\nbindings are built, then a test program can be found in the file\n`./bindings/python-examples/tests.py` -- When run, it should pass.\nFor more details see [README.md](bindings/python-examples/README.md)\nin the `bindings/python-examples` directory.\n\nThere are also multiple batches of test/example sentences in the\nlanguage data directories, generally having the names `corpus-*.batch`\nThe parser program can be run in batch mode, for testing the system\non a large number of sentences.  The following command runs the\nparser on a file called `corpus-basic.batch`;\n```\nlink-parser \u003c corpus-basic.batch\n```\n\nThe line `!batch` near the top of corpus-basic.batch turns on batch\nmode.  In this mode, sentences labeled with an initial `*` should be\nrejected and those not starting with a `*` should be accepted.  This\nbatch file does report some errors, as do the files `corpus-biolg.batch`\nand `corpus-fixes.batch`.  Work is ongoing to fix these.\n\nThe `corpus-fixes.batch` file contains many thousands of sentences\nthat have been fixed since the original 4.1 release of link-grammar.\nThe `corpus-biolg.batch` contains biology/medical-text sentences from\nthe BioLG project. The `corpus-voa.batch` contains samples from Voice\nof America; the `corpus-failures.batch` contains a large number of\nfailures.\n\nThe following numbers are subject to change, but, at this time, the\nnumber of errors one can expect to observe in each of these files\nare roughly as follows:\n```\nen/corpus-basic.batch:      88 errors\nen/corpus-fixes.batch:     371 errors\nlt/corpus-basic.batch:      15 errors\nru/corpus-basic.batch:      47 errors\n```\nThe bindings/python directory contains a unit test for the Python\nbindings. It also performs several basic checks that stress the\nlink-grammar libraries.\n\n\nUSING the system\n-----------------\nThere is an API (application program interface) to the parser.  This\nmakes it easy to incorporate it into your own applications.  The API\nis documented on the web site.\n\n\n### USING CMake\nThe `FindLinkGrammar.cmake` file can be used to test for and set up\ncompilation in CMake-based build environments.\n\n\n### USING pkg-config\nTo make compiling and linking easier, the current release uses\nthe pkg-config system. To determine the location of the link-grammar\nheader files, say `pkg-config --cflags link-grammar`  To obtain\nthe location of the libraries, say `pkg-config --libs link-grammar`\nThus, for example, a typical makefile might include the targets:\n```\n.c.o:\n   cc -O2 -g -Wall -c $\u003c `pkg-config --cflags link-grammar`\n\n$(EXE): $(OBJS)\n   cc -g -o $@ $^ `pkg-config --libs link-grammar`\n```\n\n### Using JAVA\nThis release provides java files that offer three ways of accessing\nthe parser.  The simplest way is to use the org.linkgrammar.LinkGrammar\nclass; this provides a very simple Java API to the parser.\n\nThe second possibility is to use the LGService class.  This implements\na TCP/IP network server, providing parse results as JSON messages.\nAny JSON-capable client can connect to this server and obtain parsed\ntext.\n\nThe third possibility is to use the org.linkgrammar.LGRemoteClient\nclass, and in particular, the parse() method.  This class is a network\nclient that connects to the JSON server, and converts the response\nback to results accessible via the ParseResult API.\n\nThe above-described code will be built if Apache `ant` is installed.\n\n\n### Using the JSON Network Server\nThe network server can be started by saying:\n```\njava -classpath linkgrammar.jar org.linkgrammar.LGService 9000\n```\n\nThe above starts the server on port 9000. It the port is omitted,\nhelp text is printed.  This server can be contacted directly via\nTCP/IP; for example:\n```\ntelnet localhost 9000\n```\n\n(Alternately, use netcat instead of telnet). After connecting, type\nin:\n```\ntext:  this is an example sentence to parse\n```\n\nThe returned bytes will be a JSON message providing the parses of\nthe sentence.  By default, the ASCII-art parse of the text is not\ntransmitted. This can be obtained by sending messages of the form:\n```\nstoreDiagramString:true, text: this is a test.\n```\n\n### Spell Guessing\nThe parser will run a spell-checker at an early stage, if it\nencounters a word that it does not know, and cannot guess, based on\nmorphology.  The configure script looks for the aspell or hunspell\nspell-checkers; if the aspell devel environment is found, then\naspell is used, else hunspell is used.\n\nSpell guessing may be disabled at runtime, in the link-parser client\nwith the `!spell=0` flag.  Enter `!help` for more details.\n\nCaution: aspell version 0.60.8 and possibly others have a memory leak.\nThe use of spell-guessing in production servers is strongly discouraged.\nKeeping spell-guessing disabled (`=0`) in `Parse_Options` is safe.\n\n\n### Multi-threading\nIt is safe to use link-grammar in multiple threads. Threads may share\nthe same dictionary.  Parse options can be set on a per-thread basis,\nwith the exception of verbosity, which is a global, shared by all\nthreads.  It is the only global.\n\nLinguistic Commentary\n=====================\n\nPhonetics\n---------\nA/An phonetic determiners before consonants/vowels are handled by a\nnew PH link type, linking the determiner to the word immediately\nfollowing it.  Status: Introduced in version 5.1.0 (August 2014).\nMostly done, although many special-case nouns are unfinished.\n\n\nDirectional Links\n-----------------\nDirectional links are needed for some languages, such as Lithuanian,\nTurkish and other free word-order languages. The goal is to have\na link clearly indicate which word is the head word, and which is\nthe dependent. This is achieved by prefixing connectors with\na single *lower case* letter: h,d, indicating 'head' and 'dependent'.\nThe linkage rules are such that h matches either nothing or d, and\nd matches h or nothing. This is a new feature in version 5.1.0\n(August 2014). The website provides additional documentation.\n\nAlthough the English-language link-grammar links are un-oriented,\nit seems that a de facto direction can be given to them that is\ncompletely consistent with standard conceptions of a dependency\ngrammar.\n\nThe dependency arrows have the following properties:\n\n * Anti-reflexive (a word cannot depend on itself; it cannot point\n   at itself.)\n\n * Anti-symmetric (if Word1 depends on Word2, then Word2 cannot\n   depend on Word1) (so, e.g. determiners depend on nouns, but\n   never vice-versa)\n\n * The arrows are neither transitive, nor anti-transitive: a single\n   word may be ruled by several heads.  For example:\n```text\n    +------\u003eWV-------\u003e+\n    +--\u003eWd--\u003e+\u003c--Ss\u003c--+\n    |        |        |\nLEFT-WALL   she    thinks.v\n```\nThat is, there is a path to the subject, \"she\", directly from the\nleft wall, via the Wd link, as well as indirectly, from the wall\nto the root verb, and thence to the subject.  Similar loops form\nwith the B and R links.  Such loops are useful for constraining\nthe possible number of parses: the constraint occurs in\nconjunction with the \"no links cross\" meta-rule.\n\n * The graphs are planar; that is, no two edges may cross. See,\n   however, the \"link-crossing\" discussion below.\n\n\nThere are several related mathematical notions, but none quite\ncapture directional LG:\n\n * Directional LG graphs resemble DAGS, except that LG allows only\n   one wall (one \"top\" element).\n\n * Directional LG graphs resemble strict partial orders, except that\n   the LG arrows are usually not transitive.\n\n * Directional LG graphs resemble\n   [catena](http://en.wikipedia.org/wiki/Catena_(linguistics))\n   except that catena are strictly anti-transitive -- the path to\n   any word is unique, in a catena.\n\n\nLink Crossing\n-------------\nThe foundational LG papers mandate the planarity of the parse graphs.\nThis is based on a very old observation that dependencies almost never\ncross in natural languages: humans simply do not speak in sentences\nwhere links cross.  Imposing planarity constraints then provides a\nstrong engineering and algorithmic constraint on the resulting parses:\nthe total number of parses to be considered is sharply reduced, and\nthus the overall speed of parsing can be greatly increased.\n\nHowever, there are occasional, relatively rare exceptions to this\nplanarity rule; such exceptions are observed in almost all languages.\nA number of these exceptions are given for English, below.\n\nThus, it seems important to relax the planarity constraint, and find\nsomething else that is almost as strict, but still allows infrequent\nexceptions.  It would appear that the concept of \"landmark transitivity\"\nas defined by Richard Hudson in his theory of \"Word Grammar\", and then\nadvocated by Ben Goertzel, just might be such a mechanism.\n\nftp://ftp.phon.ucl.ac.uk/pub/Word-Grammar/ell2-wg.pdf\u003cbr\u003e\nhttp://www.phon.ucl.ac.uk/home/dick/enc/syntax.htm\u003cbr\u003e\nhttp://goertzel.org/ProwlGrammar.pdf\n\nPlanarity: Theory vs. Practice\n------------------------------\nIn practice, the planarity constraint allows very efficient algorithms\nto be used in the implementation of the parser. Thus, from the point of\nview of the implementation, we want to keep planarity. Fortunately,\nthere is a convenient and unambiguous way to have our cake and eat it,\ntoo. A non-planar diagram can be drawn on a sheet of paper using\nstandard electrical-engineering notation: a funny symbol, wherever\nwires cross. This notation is very easily adapted to LG connectors;\nbelow is an actual working example, already implemented in the current\nLG English dictionary. *All* link crossings can be implemented in this\nway!  So we do not have to actually abandon the current parsing\nalgorithms to get non-planar diagrams. We don't even have to modify them!\nHurrah!\n\nHere is a working example: \"I want to look at and listen to everything.\"\nThis wants two `J` links pointing to 'everything'.  The desired diagram\nwould need to look like this:\n\n```text\n    +----\u003eWV----\u003e+\n    |            +--------IV----------\u003e+\n    |            |           +\u003c-VJlpi--+\n    |            |           |    +---xxx------------Js-------\u003e+\n    +--Wd--+-Sp*i+--TO-+-I*t-+-MVp+    +--VJrpi\u003e+--MVp-+---Js-\u003e+\n    |      |     |     |     |    |    |        |      |       |\nLEFT-WALL I.p want.v to.r look.v at and.j-v listen.v to.r everything\n```\nThe above really wants to have a `Js` link from 'at' to 'everything',\nbut this `Js` link crosses (clashes with - marked by xxx) the link\nto the conjunction.  Other examples suggest that one should\nallow most links to cross over the down-links to conjunctions.\n\nThe planarity-maintaining worked-around is to split the `Js` link into\ntwo: a `Jj` part and a `Jk` part; the two are used together to cross\nover the conjunction. This is currently implemented in the English\ndictionary, and it works.\n\nThis work-around is in fact completely generic, and can be extended\nto any kind of link crossing. For this to work, a better notation would\nbe convenient; perhaps `uJs-` instead of `Jj-` and `vJs-` instead of\n`Jk-`, or something like that ... (TODO: invent better notation.)\n(NB: This is a kind of re-invention of \"fat links\", but in the\ndictionary, not in the code.)\n\nLandmark Transitivity: Theory\n-----------------------------\nGiven that non-planar parses can be enabled without any changes to the\nparser algorithm, all that is required is to understand what sort of\ntheory describes link-crossing in a coherent grounding. That theory is\nDick Hudson's Landmark Transitivity, explained here.\n\nThis mechanism works as follows:\n\n * First, every link must be directional, with a head and a dependent.\nThat is, we are concerned with directional-LG links, which are\nof the form x--A--\u003ey or y\u003c--A--x for words x,y and LG link type A.\n\n * Given either the directional-LG relation x--A--\u003ey or y\u003c--A--x,\ndefine the dependency relation x--\u003ey.  That is, ignore the link-type\nlabel.\n\n * Heads are landmarks for dependents. If the dependency relation\nx--\u003ey holds, then x is said to be a landmark for y, and the\npredicate land(x,y) is true, while the predicate land(y,x) is false.\nHere, x and y are words, while --\u003e is the landmark relation.\n\n * Although the basic directional-LG links form landmark relations,\nthe total set of landmark relations is extended by transitive closure.\nThat is, if land(x,y) and land(y,z) then land(x,z).  That is, the\nbasic directional-LG links are \"generators\" of landmarks; they\ngenerate by means of transitivity.  Note that the transitive closure\nis unique.\n\n * In addition to the above landmark relation, there are two additional\nrelations: the before and after landmark relations. (In English,\nthese correspond to left and right; in Hebrew, the opposite).\nThat is, since words come in chronological order in a sentence,\nthe dependency relation can point either left or right.  The\npreviously-defined landmark relation only described the dependency\norder; we now introduce the word-sequence order. Thus, there are\nare land-before() and land-after() relations that capture both\nthe dependency relation, and the word-order relation.\n\n * Notation: the before-landmark relation land-B(x,y) corresponds to\nx--\u003ey (in English, reversed in right-left languages such as Hebrew),\nwhereas the after-landmark relation land-A(x,y) corresponds to y\u003c--x.\nThat is, land(x,y) == land-B(x,y) or land-A(x,y) holds as a statement\nabout the predicate form of the relations.\n\n * As before, the full set of directional landmarks are obtained by\ntransitive closure applied to the directional-LG links.  Two\ndifferent rules are used to perform this closure:\n```\n-- land-B(x,y) and land(y,z) ==\u003e land-B(x,y)\n-- land-A(x,y) and land(y,z) ==\u003e land-A(x,y)\n```\nParsing is then performed by joining LG connectors in the usual manner,\nto form a directional link. The transitive closure of the directional\nlandmarks are then computed. Finally, any parse that does not conclude\nwith the \"left wall\" being the upper-most landmark is discarded.\n\nHere is an example where landmark transitivity provides a natural\nsolution to a (currently) broken parse. The \"to.r\" has a disjunct\n\"I+ \u0026 MVi-\" which allows \"What is there to do?\" to parse correctly.\nHowever, it also allows the incorrect parse \"He is going to do\".\nThe fix would be to force \"do\" to take an object; however, a link\nfrom \"do\" to \"what\" is not allowed, because link-crossing would\nprevent it.\n\nFixing this requires only a fix to the dictionary, and not to the\nparser itself.\n\nLink-crossing Examples\n----------------------\nExamples where the no-links-cross constraint seems to be violated,\nin English:\n```text\n  \"He is either in the 105th or the 106th battalion.\"\n  \"He is in either the 105th or the 106th battalion.\"\n```\nBoth seem to be acceptable in English, but the ambiguity of the\n\"in-either\" temporal ordering requires two different parse trees, if\nthe no-links-cross rule is to be enforced. This seems un-natural.\nSimilarly:\n```text\n  \"He is either here or he is there.\"\n  \"He either is here or he is there.\"\n```\n\nA different example involves a crossing to the left wall.  That is, the\nlinks *LEFT-WALL--remains* crosses over *here--found*:\n```text\n  \"Here the remains can be found.\"\n```\n\nOther examples, per And Rosta:\n\nThe *allowed--by* link crosses *cake--that*:\n```text\nHe had been allowed to eat a cake by Sophy that she had made him specially\n```\n\n*a--book*, *very--indeed*\n```text\n\"a very much easier book indeed\"\n```\n\n*an--book*, *easy--to*\n```text\n\"an easy book to read\"\n```\n\n*a--book*, *more--than*\n```text\n\"a more difficult book than that one\"\n```\n\n*that--have* crosses *remains--of*\n```text\n\"It was announced that remains have been found of the ark of the covenant\"\n```\n\nThere is a natural crossing, driven by conjunctions:\n```text\n\"I was in hell yesterday and heaven on Tuesday.\"\n```\n\nthe \"natural\" linkage is to use MV links to connect \"yesterday\" and \"on\nTuesday\" to the verb. However, if this is done, then these must cross\nthe links from the conjunction \"and\" to \"heaven\" and \"hell\".  This can\nbe worked around partly as follows:\n```text\n              +--------\u003eJu---------\u003e+\n              |    +\u003c------SJlp\u003c----+\n+\u003c-SX\u003c-+-\u003ePp-\u003e+    +--\u003eMpn-\u003e+       +-\u003eSJru-\u003e+-\u003eMp-\u003e+-\u003eJs-\u003e+\n|      |      |    |        |       |        |      |      |\nI     was    in  hell   yesterday  and    heaven    on  Tuesday\n```\nbut the desired MV links from the verb to the time-prepositions\n\"yesterday\" and \"on Tuesday\" are missing -- whereas they are present,\nwhen the individual sentences \"I was in hell yesterday\" and\n\"I was in heaven on Tuesday\" are parsed.  Using a conjunction should\nnot wreck the relations that get used; but this requires link-crossing.\n\n```text\n\"Sophy wondered up to whose favorite number she should count\"\n```\nHere, \"up_to\" must modify \"number\", and not \"whose\". There's no way to\ndo this without link-crossing.\n\n\nType Theory\n-----------\nLink Grammar can be understood in the context of type theory.\nA simple introduction to type theory can be found in chapter 1\nof the [HoTT book](https://homotopytypetheory.org/book/).\nThis book is freely available online and strongly recommended if\nyou are interested in types.\n\nLink types can be mapped to types that appear in categorial grammars.\nThe nice thing about link-grammar is that the link types form a type\nsystem that is much easier to use and comprehend than that of categorial\ngrammar, and yet can be directly converted to that system!  That is,\nlink-grammar is completely compatible with categorial grammar, and is\neasier-to-use. See the paper\n[\u003ci\u003e\"Combinatory Categorial Grammar and Link Grammar are Equivalent\"\u003c/i\u003e](https://github.com/opencog/atomspace/raw/master/opencog/sheaf/docs/ccg.pdf)\nfor details.\n\nThe foundational LG papers make comments to this effect; however, see\nalso work by Bob Coecke on category theory and grammar.  Coecke's\ndiagrammatic approach is essentially identical to the diagrams given in\nthe foundational LG papers; it becomes abundantly clear that the\ncategory theoretic approach is equivalent to Link Grammar. See, for\nexample, this introductory sketch\nhttp://www.cs.ox.ac.uk/people/bob.coecke/NewScientist.pdf\nand observe how the diagrams are essentially identical to the LG\njigsaw-puzzle piece diagrams of the foundational LG publications.\n\n\nADDRESSES\n---------\nIf you have any questions, please feel free to send a note to the\n[mailing list](http://groups.google.com/group/link-grammar).\n\nThe source code of link-parser and the link-grammar library is located at\n[GitHub](https://github.com/opencog/link-grammar).\u003cbr\u003e\nFor bug reports, please open an **issue** there.\n\nAlthough all messages should go to the mailing list, the current\nmaintainers can be contacted at:\n```text\n  Linas Vepstas - \u003clinasvepstas@gmail.com\u003e\n  Amir Plivatsky - \u003camirpli@gmail.com\u003e\n  Dom Lachowicz - \u003cdomlachowicz@gmail.com\u003e\n```\nA complete list of authors and copyright holders can be found in the\nAUTHORS file.  The original authors of the Link Grammar parser are:\n```text\n  Daniel Sleator                    sleator@cs.cmu.edu\n  Computer Science Department       412-268-7563\n  Carnegie Mellon University        www.cs.cmu.edu/~sleator\n  Pittsburgh, PA 15213\n\n  Davy Temperley                    dtemp@theory.esm.rochester.edu\n  Eastman School of Music           716-274-1557\n  26 Gibbs St.                      www.link.cs.cmu.edu/temperley\n  Rochester, NY 14604\n\n  John Lafferty                     lafferty@cs.cmu.edu\n  Computer Science Department       412-268-6791\n  Carnegie Mellon University        www.cs.cmu.edu/~lafferty\n  Pittsburgh, PA 15213\n```\n\n\nTODO -- Working Notes\n---------------------\nSome working notes.\n\nEasy to fix: provide a more uniform API to the constituent tree.\ni.e provide word index.   Also, provide a better word API,\nshowing word extent, subscript, etc.\n\n### Capitalized first words:\nThere are subtle technical issues for handling capitalized first\nwords. This needs to be fixed. In addition, for now these words are\nshown uncapitalized in the result linkages. This can be fixed.\n\nMaybe capitalization could be handled in the same way that a/an\ncould be handled!  After all, it's essentially a nearest-neighbor\nphenomenon!\n\nSee also [issue 690](https://github.com/opencog/link-grammar/issues/690)\n\n#### Capitalization-mark tokens:\nThe proximal issue is to add a cost, so that Bill gets a lower\ncost than bill.n when parsing \"Bill went on a walk\".  The best\nsolution would be to add a 'capitalization-mark token' during\ntokenization; this token precedes capitalized words. The\ndictionary then explicitly links to this token, with rules similar\nto the a/an phonetic distinction.  The point here is that this\nmoves capitalization out of ad-hoc C code and into the dictionary,\nwhere it can be handled like any other language feature.\nThe tokenizer includes experimental code for that.\n\n### Corpus-statistics-based parse ranking:\nThe old for parse ranking via corpus statistics needs to be revived.\nThe issue can be illustrated with these example sentences:\n```text\n\"Please the customer, bring in the money\"\n\"Please, turn off the lights\"\n```\nIn the first sentence, the comma acts as a conjunction of two\ndirectives (imperatives). In the second sentence, it is much too\neasy to mistake \"please\" for a verb, the comma for a conjunction,\nand come to the conclusion that one should please some unstated\nobject, and then turn off the lights. (Perhaps one is pleasing\nby turning off the lights?)\n\n### Bad grammar:\nWhen a sentence fails to parse, look for:\n * confused words: its/it's, there/their/they're, to/too, your/you're ...\n   These could be added at high cost to the dicts.\n * missing apostrophes in possessives: \"the peoples desires\"\n * determiner agreement errors: \"a books\"\n * aux verb agreement errors: \"to be hooks up\"\n\nPoor agreement might be handled by giving a cost to mismatched\nlower-case connector letters.\n\n### Elision/ellipsis/zero/phantom words:\nAn common phenomenon in English is that some words that one might\nexpect to \"properly\" be present can disappear under various conditions.\nBelow is a sampling of these. Some possible solutions are given below.\n\nExpressions such as \"Looks good\" have an implicit \"it\" (also called\na zero-it or phantom-it) in them; that is, the sentence should really\nparse as \"(it) looks good\".  The dictionary could be simplified by\nadmitting such phantom words explicitly, rather than modifying the\ngrammar rules to allow such constructions.  Other examples, with the\nphantom word in parenthesis, include:\n * I ate all (of) the cookies.\n * I've known him only (for) a week.\n * I taught him (how) to swim.\n * I told him (that) it was gone.\n * It stopped me (from) flying off the cliff.\n * (It) looks good.\n * (You) go home!\n * (You) do tell (me).\n * (That is) enough!\n * (I) heard that he's giving a test.\n * (Are) you all right?\n * He opened the door and (he) went in.\n * Emma was the younger (daughter) of two daughters.\n\nThis can extend to elided/unvoiced syllables:\n * (I'm a)'fraid so.\n\nElided punctuation:\n * God (,) give me strength.\n\nNormally, the subjects of imperatives must always be offset by a comma:\n\"John, give me the hammer\", but here, in muttering an oath, the comma\nis swallowed (unvoiced).\n\nSome complex phantom constructions:\n * They play billiards but (they do) not (play) snooker.\n * I know Ringo, but (I do) not (know) his brother.\n * She likes Indian food, but (she does) not (like) Chinese (food).\n * If this is true, then (you should) do it.\n * Perhaps he will (do it), if he sees enough of her.\n\nSee also [GitHub issue #224](https://github.com/opencog/link-grammar/issues/224).\n\nActual ellipsis:\n * At first, it seemed like ...\n * It became clear that ...\n\nHere, the ellipsis stands for a subordinate clause, which attaches\nwith not one, but two links: `C+ \u0026 CV+`, and thus requires two words,\nnot one. There is no way to have the ellipsis word to sink two\nconnectors starting from the same word, and so some more complex\nmechanism is needed. The solution is to infer a second phantom ellipsis:\n\n * It became clear that ... (...)\n\nwhere the first ellipsis is a stand in for the subject of a subordinate\nclause, and the second stands in for an unknown verb.\n\n#### Elision of syllables\nMany (unstressed) syllables can be elided; in modern English, this occurs\nmost commonly in the initial unstressed syllable:\n* (a)'ccount (a)'fraid (a)'gainst (a)'greed (a)'midst (a)'mongst\n* (a)'noint (a)'nother (a)'rrest (at)'tend\n* (be)'fore (be)'gin (be)'havior (be)'long (be)'twixt\n* (con)'cern (e)'scape (e)'stablish\nAnd so on.\n\n#### Punctuation, zero-copula, zero-that:\nPoorly punctuated sentences cause problems:  for example:\n```text\n\"Mike was not first, nor was he last.\"\n\"Mike was not first nor was he last.\"\n```\nThe one without the comma currently fails to parse.  How can we\ndeal with this in a simple, fast, elegant way?  Similar questions\nfor zero-copula and zero-that sentences.\n\n#### Context-dependent zero phrases.\nConsider an argument between a professor and a dean, and the dean\nwants the professor to write a brilliant review. At the end of the\nargument, the dean exclaims: \"I want the review brilliant!\"  This\nis a predicative adjective; clearly it means \"I want the review\n[that you write to be] brilliant.\"  However, taken out of context,\nsuch a construction is ungrammatical, as the predictiveness is not\nat all apparent, and it reads just as incorrectly as would\n\"*Hey Joe, can you hand me that review brilliant?\"\n\n#### Imperatives as phantoms:\n```text\n\"Push button\"\n\"Push button firmly\"\n```\nThe subject is a phantom; the subject is \"you\".\n\n#### Handling zero/phantom words by explicitly inserting them:\nOne possible solution is to perform a one-point compactification.\nThe dictionary contains the phantom words, and their connectors.\nOrdinary disjuncts can link to these, but should do so using\na special initial lower-case letter (say, 'z', in addition to\n'h' and 'd' as is currently implemented).  The parser, as it\nworks, examines the initial letter of each connector: if it is\n'z', then the usual pruning rules no longer apply, and one or\nmore phantom words are selected out of the bucket of phantom words.\n(This bucket is kept out-of-line, it is not yet placed into\nsentence word sequence order, which is why the usual pruning rules\nget modified.)  Otherwise, parsing continues as normal. At the end\nof parsing, if there are any phantom words that are linked, then\nall of the connectors on the disjunct must be satisfied (of course!)\nelse the linkage is invalid. After parsing, the phantom words can\nbe inserted into the sentence, with the location deduced from link\nlengths.\n\n#### Handling zero/phantom words as re-write rules.\nA more principled approach to fixing the phantom-word issue is to\nborrow the idea of re-writing from the theory of\n[operator grammar](https://en.wikipedia.org/wiki/Operator_grammar).\nThat is, certain phrases and constructions can be (should be)\nre-written into their \"proper form\", prior to parsing. The re-writing\nstep would insert the missing words, then the parsing proceeds. One\nappeal of such an approach is that re-writing can also handle other\n\"annoying\" phenomena, such as typos (missing apostrophes, e.g. \"lets\"\nvs. \"let's\", \"its\" vs. \"it's\") as well as multi-word rewrites (e.g.\n\"let's\" vs. \"let us\", or \"it's\" vs. \"it is\").\n\nExactly how to implement this is unclear.  However, it seems to open\nthe door to more abstract, semantic analysis. Thus, for example, in\nMeaning-Text Theory (MTT), one must move between SSynt to DSynt\nstructures.  Such changes require a graph re-write from the surface\nsyntax parse (e.g. provided by link-grammar) to the deep-syntactic\nstructure.  By contrast, handling phantom words by graph re-writing\nprior to parsing inverts the order of processing. This suggests that\na more holistic approach is needed to graph rewriting: it must somehow\nbe performed \"during\" parsing, so that parsing can both guide the\ninsertion of the phantom words, and, simultaneously guide the deep\nsyntactic rewrites.\n\nAnother interesting possibility arises with regards to tokenization.\nThe current tokenizer is clever, in that it splits not only on\nwhitespace, but can also strip off prefixes, suffixes, and perform\ncertain limited kinds of morphological splitting. That is, it currently\nhas the ability to re-write single-words into sequences of words. It\ncurrently does so in a conservative manner; the letters that compose\na word are preserved, with a few exceptions, such as making spelling\ncorrection suggestions. The above considerations suggest that the\nboundary between tokenization and parsing needs to become both more\nfluid, and more tightly coupled.\n\n### Poor linkage choices:\nCompare \"she will be happier than before\" to \"she will be more happy\nthan before.\" Current parser makes \"happy\" the head word, and \"more\"\na modifier w/EA link.  I believe the correct solution would be to\nmake \"more\" the head (link it as a comparative), and make \"happy\"\nthe dependent.  This would harmonize rules for comparatives... and\nwould eliminate/simplify rules for less,more.\n\nHowever, this idea needs to be double-checked against, e.g. Hudson's\nword grammar.  I'm confused on this issue ...\n\n### Stretchy links:\nCurrently, some links can act at \"unlimited\" length, while others\ncan only be finite-length.  e.g. determiners should be near the\nnoun that they apply to.  A better solution might be to employ\na 'stretchiness' cost to some connectors: the longer they are, the\nhigher the cost. (This eliminates the \"unlimited_connector_set\"\nin the dictionary).\n\n### Opposing (repulsing) parses:\nSometimes, the existence of one parse should suggest\nthat another parse must surely be wrong: if one parse is possible,\nthen the other parses must surely be unlikely. For example: the\nconjunction and.j-g allows the \"The Great Southern and Western\nRailroad\" to be parsed as the single name of an entity. However,\nit also provides a pattern match for \"John and Mike\" as a single\nentity, which is almost certainly wrong. But \"John and Mike\" has\nan alternative parse, as a conventional-and -- a list of two people,\nand so the existence of this alternative (and correct) parse suggests\nthat perhaps the entity-and is really very much the wrong parse.\nThat is, the mere possibility of certain parses should strongly\ndisfavor other possible parses. (Exception: Ben \u0026 Jerry's ice\ncream; however, in this case, we could recognize Ben \u0026 Jerry as the\nname of a proper brand; but this is outside of the \"normal\"\ndictionary (?) (but maybe should be in the dictionary!))\n\nMore examples: \"high water\" can have the connector A joining high.a\nand AN joining high.n; these two should either be collapsed into\none, or one should be eliminated.\n\n\n### WordNet hinting:\nUse WordNet to reduce the number for parses for sentences containing\ncompound verb phrases, such as \"give up\", \"give off\", etc.\n\n### Sliding-window (Incremental) parsing:\nTo avoid a combinatorial explosion of parses, it would be nice to\nhave an incremental parsing, phrase by phrase, using a sliding window\nalgorithm to obtain the parse. Thus, for example, the parse of the\nlast half of a long, run-on sentence should not be sensitive to the\nparse of the beginning of the sentence.\n\nDoing so would help with combinatorial explosion. So, for example,\nif the first half of a sentence has 4 plausible parses, and the\nlast half has 4 more, then currently, the parser reports 16 parses\ntotal.  It would be much more useful if it could instead report the\nfactored results: i.e. the four plausible parses for the first half,\nand the four plausible parses for the last half.  This would ease\nthe burden on downstream users of link-grammar.\n\nThis approach has at psychological support. Humans take long sentences\nand split them into smaller chunks that \"hang together\" as phrase-\nstructures, viz compounded sentences. The most likely parse is the\none where each of the quasi sub-sentences is parsed correctly.\n\nThis could be implemented by saving dangling right-going connectors\ninto a parse context, and then, when another sentence fragment\narrives, use that context in place of the left-wall.\n\nThis somewhat resembles the application of construction grammar\nideas to the link-grammar dictionary. It also somewhat resembles\nViterbi parsing to some fixed depth. Viz. do a full backward-forward\nparse for a phrase, and then, once this is done, take a Viterbi-step.\nThat is, once the phrase is done, keep only the dangling connectors\nto the phrase, place a wall, and then step to the next part of the\nsentence.\n\nCaution: watch out for garden-path sentences:\n``` text\n  The horse raced past the barn fell.\n  The old man the boat.\n  The cotton clothing is made of grows in Mississippi.\n```\nThe current parser parses these perfectly; a viterbi parser could trip on these.\n\nOther benefits of a Viterbi decoder:\n* Less sensitive to sentence boundaries: this would allow longer,\n  run-on sentences to be parsed far more quickly.\n* Could do better with slang, hip-speak.\n* Support for real-time dialog (parsing of half-uttered sentences).\n* Parsing of multiple streams, e.g. from play/movie scripts.\n* Would enable (or simplify) co-reference resolution across sentences\n  (resolve referents of pronouns, etc.)\n* Would allow richer state to be passed up to higher layers:\n  specifically, alternate parses for fractions of a sentence,\n  alternate reference resolutions.\n* Would allow plug-in architecture, so that plugins, employing\n  some alternate, higher-level logic, could disambiguate (e.g.\n  by making use of semantic content).\n* Eliminate many of the hard-coded array sizes in the code.\n\nOne may argue that Viterbi is a more natural, biological way of\nworking with sequences.  Some experimental, psychological support\nfor this can be found at\nhttp://www.sciencedaily.com/releases/2012/09/120925143555.htm\nper Morten Christiansen, Cornell professor of psychology.\n\n\n### Registers, sociolects, dialects (cost vectors):\nConsider the sentence \"Thieves rob bank\" -- a typical newspaper\nheadline. LG currently fails to parse this, because the determiner\nis missing (\"bank\" is a count noun, not a mass noun, and thus\nrequires a determiner. By contrast, \"thieves rob water\" parses\njust fine.) A fix for this would be to replace mandatory\ndeterminer links by (D- or {[[()]] \u0026 headline-flag}) which allows\nthe D link to be omitted if the headline-flag bit is set.\nHere, \"headline-flag\" could be a new link-type, but one that is\nnot subject to planarity constraints.\n\nNote that this is easier said than done: if one simply adds a\nhigh-cost null link, and no headline-flag, then all sorts of\nungrammatical sentences parse, with strange parses; while some\ngrammatical sentences, which should parse, but currently don't,\nbecome parsable, but with crazy results.\n\nMore examples, from And Rosta:\n```text\n   \"when boy meets girl\"\n   \"when bat strikes ball\"\n   \"both mother and baby are well\"\n```\n\nA natural approach would be to replace fixed costs by formulas.\nThis would allow the dialect/sociolect to be dynamically\nchangeable.  That is, rather than having a binary headline-flag,\nthere would be a formula for the cost, which could be changed\noutside of the parsing loop.  Such formulas could be used to\nenable/disable parsing specific to different dialects/sociolects,\nsimply by altering the network of link costs.\n\nA simpler alternative would be to have labeled costs (a cost vector),\nso that different dialects assign different costs to various links.\nA dialect would be specified during the parse, thus causing the costs\nfor that dialect to be employed during parse ranking.\n\nThis has been implemented; what's missing is a practical tutorial on\nhow this might be used.\n\n### Hand-refining verb patterns:\nA good reference for refining verb usage patterns is:\n\"COBUILD GRAMMAR PATTERNS 1: VERBS from THE COBUILD SERIES\",\nfrom THE BANK OF ENGLISH, HARPER COLLINS.  Online at\nhttps://arts-ccr-002.bham.ac.uk/ccr/patgram/ and\nhttp://www.corpus.bham.ac.uk/publications/index.shtml\n\n\n### Quotations:\n   Currently tokenize.c tokenizes double-quotes and some UTF8 quotes\n   (see the RPUNC/LPUNC class in en/4.0.affix - the QUOTES class is\n   not used for that, but for capitalization support), with some very\n   basic support in the English dictionary (see \"% Quotation marks.\"\n   there).  However, it does not do this for the various \"curly\" UTF8\n   quotes, such as ‘these’ and “these”.  This results is some ugly\n   parsing for sentences containing such quotes. (Note that these are\n   in 4.0.affix).\n\n   A mechanism is needed to disentangle the quoting from the quoted\n   text, so that each can be parsed appropriately.  It's somewhat\n   unclear how to handle this within link-grammar. This is somewhat\n   related to the problem of morphology (parsing words as if they\n   were \"mini-sentences\",) idioms (phrases that are treated as if\n   they were single words), set-phrase structures (if ... then ... not\n   only... but also ...) which have a long-range structure similar to\n   quoted text (he said ...).\n\n   See also [GitHub issue #42](https://github.com/opencog/link-grammar/issues/42).\n\n### Semantification of the dictionary:\n  \"to be fishing\": Link grammar offers four parses of \"I was fishing for\n  evidence\", two of which are given low scores, and two are given\n  high scores. Of the two with high scores, one parse is clearly bad.\n  Its links \"to be fishing.noun\" as opposed to the correct\n  \"to be fishing.gerund\". That is, I can be happy, healthy and wise,\n  but I certainly cannot be fishing.noun.  This is perhaps not\n  just a bug in the structure of the dictionary, but is perhaps\n  deeper: link-grammar has little or no concept of lexical units\n  (i.e. collocations, idioms, institutional phrases), which thus\n  allows parses with bad word-senses to sneak in.\n\n  The goal is to introduce more knowledge of lexical units into LG.\n\n  Different word senses can have different grammar rules (and thus,\n  the links employed reveal the sense of the word): for example:\n  \"I tend to agree\" vs. \"I tend to the sheep\" -- these employ two\n  different meanings for the verb \"tend\", and the grammatical\n  constructions allowed for one meaning are not the same as those\n  allowed for the other. Yet, the link rules for \"tend.v\" have\n  to accommodate both senses, thus making the rules rather complex.\n  Worse, it potentially allows for non-sense constructions.\n  If, instead, we allowed the dictionary to contain different\n  rules for \"tend.meaning1\" and \"tend.meaning2\", the rules would\n  simplify (at the cost of inflating the size of the dictionary).\n\n  Another example: \"I fear so\" -- the word \"so\" is only allowed\n  with some, but not all, lexical senses of \"fear\". So e.g.\n  \"I fear so\" is in the same semantic class as \"I think so\" or\n  \"I hope so\", although other meanings of these verbs are\n  otherwise quite different.\n\n  [Sin2004] \"New evidence, new priorities, new attitudes\" in J.\n  Sinclair, (ed) (2004) How to use corpora in language teaching,\n  Amsterdam: John Benjamins\n\n  See also: Pattern Grammar: A Corpus-Driven Approach to the Lexical\n  Grammar of English\u003cbr\u003e\n  Susan Hunston and Gill Francis (University of Birmingham)\u003cbr\u003e\n  Amsterdam: John Benjamins (Studies in corpus linguistics,\n  edited by Elena Tognini-Bonelli, volume 4), 2000\u003cbr\u003e\n  [Book review](http://www.aclweb.org/anthology/J01-2013).\n\n  “The Molecular Level of Lexical Semantics”, EA Nida, (1997)\n  International Journal of Lexicography, 10(4): 265–274.\n  [Online](https://www.academia.edu/36534355/The_Molecular_Level_of_Lexical_Semantics_by_EA_Nida)\n\n### \"holes\" in collocations (aka \"set phrases\" of \"phrasemes\"):\n  The link-grammar provides several mechanisms to support\n  circumpositions or even more complicated multi-word structures.\n  One mechanism is by ordinary links; see the V, XJ and RJ links.\n  The other mechanism is by means of post-processing rules.\n  (For example, the \"filler-it\" SF rules use post-processing.)\n  However, rules for many common forms have not yet been written.\n  The general problem is of supporting structures that have \"holes\"\n  in the middle, that require \"lacing\" to tie them together.\n\n  For a general theory, see\n  [catena](http://en.wikipedia.org/wiki/Catena_(linguistics)).\n\n  For example, the adposition:\n```text\n... from [xxx] on.\n    \"He never said another word from then on.\"\n    \"I promise to be quiet from now on.\"\n    \"Keep going straight from that point on.\"\n    \"We went straight from here on.\"\n\n... from there on.\n    \"We went straight, from the house on to the woods.\"\n    \"We drove straight, from the hill onwards.\"\n```\nNote that multiple words can fit in the slot [xxx].\nNote the tangling of another prepositional phrase:\n`\"... from [xxx] on to [yyy]\"`\n\nMore complicated collocations with holes include\n```text\n \"First.. next...\"\n \"If ... then ...\"\n```\n'Then' is optional ('then' is a 'null word'), for example:\n```text\n\"If it is raining, stay inside!\"\n\"If it is raining, [then] stay inside!\"\n\n\n\"if ... only ...\" \"If there were only more like you!\"\n\"... not only, ... but also ...\"\n\n\n\"As ..., so ...\"  \"As it was commanded, so it shall be done\"\n\n\n\"Either ... or ...\"\n\"Both ... and  ...\"  \"Both June and Tom are coming\"\n\"ought ... if ...\" \"That ought to be the case, if John is not lying\"\n\n\n\"Someone ... who ...\"\n\"Someone is outside who wants to see you\"\n\n\n\"... for ... to ...\"\n\"I need for you to come to my party\"\n```\nThe above are not currently supported. An example that is supported\nis the \"non-referential it\", e.g.\n```\n\"It ... that ...\"\n\"It seemed likely that John would go\"\n```\nThe above is supported by means of special disjuncts for 'it' and\n'that', which must occur in the same post-processing domain.\n\nSee also:\u003cbr\u003e\nhttp://www.phon.ucl.ac.uk/home/dick/enc2010/articles/extraposition.htm\u003cbr\u003e\nhttp://www.phon.ucl.ac.uk/home/dick/enc2010/articles/relative-clause.htm\n\n \"...from X and from Y\"\n \"By X, and by Y, ...\"\n Here, X and Y might be rather long phrases, containing other\n prepositions. In this case, the usual link-grammar linkage rules\n will typically conjoin \"and from Y\" to some preposition in X,\n instead of the correct link to \"from X\". Although adding a cost to\n keep the lengths of X and Y approximately equal can help, it would\n be even better to recognize the \"...from ... and from...\" pattern.\n\n The correct solution for the \"Either ... or ...\" appears to be this:\n```text\n---------------------------+---SJrs--+\n       +------???----------+         |\n       |     +Ds**c+--SJls-+    +Ds**+\n       |     |     |       |    |    |\n   either.r the lorry.n or.j-n the van.n\n```\n The wrong solution is\n```text\n--------------------------+\n     +-----Dn-----+       +---SJrs---+\n     |      +Ds**c+--SJn--+     +Ds**+\n     |      |     |       |     |    |\n neither.j the lorry.n nor.j-n the van.n\n```\n The problem with this is that \"neither\" must coordinate with \"nor\".\n That is, one cannot say \"either.. nor...\" \"neither ... or ... \"\n \"neither ...and...\" \"but ... nor ...\"  The way I originally solved\n the coordination problem was to invent a new link called Dn, and a\n link SJn and to make sure that Dn could only connect to SJn, and\n nothing else. Thus, the lower-case \"n\" was used to propagate the\n coordination across two links. This demonstrates how powerful the\n link-grammar theory is: with proper subscripts, constraints can be\n propagated along links over large distances. However, this also\n makes the dictionary more complex, and the rules harder to write:\n coordination requires a lot of different links to be hooked together.\n And so I think that creating a single, new link, called ???, will\n make the coordination easy and direct. That is why I like that idea.\n\n The ??? link should be the XJ link, which-see.\n\n\n More idiomatic than the above examples:\n \"...the chip on X's shoulder\"\n \"to do X a favour\"\n \"to give X a look\"\n\n The above are all examples of \"set phrases\" or \"phrasemes\", and are\n most commonly discussed in the context of MTT or Meaning-Text Theory\n of Igor Mel'cuk et al (search for \"MTT Lexical Function\" for more\n info). Mel'cuk treats set phrases as lexemes, and, for parsing, this\n is not directly relevant. However, insofar as phrasemes have a high\n mutual information content, they can dominate the syntactic\n structure of a sentence.\n\n### Preposition linking:\n The current parse of \"he wanted to look at and listen to everything.\"\n is inadequate: the link to \"everything\" needs to connect to \"and\", so\n that \"listen to\" and \"look at\" are treated as atomic verb phrases.\n\n### Lexical functions:\n MTT suggests that perhaps the correct way to understand the contents\n of the post-processing rules is as an implementation of 'lexical\n functions' projected onto syntax.  That is, the post-processing\n rules allow only certain syntactical constructions, and these are\n the kinds of constructions one typically sees in certain kinds\n of lexical functions.\n\n Alternately, link-grammar suffers from a combinatoric explosion\n of possible parses of a given sentence. It would seem that lexical\n functions could be used to rule out many of these parses.  On the\n other hand, the results are likely to be similar to that of\n statistical parse ranking (which presumably captures such\n quasi-idiomatic collocations at least weakly).\n\n Ref. I. Mel'cuk: \"Collocations and Lexical Functions\", in ''Phraseology:\n theory, analysis, and applications'' Ed. Anthony Paul Cowie (1998)\n Oxford University Press pp. 23-54.\n\n More generally, all of link-grammar could benefit from a MTT-izing\n of infrastructure.\n\n### Morphology:\nCompare the above commentary on lexical functions to Hebrew morphological\nanalysis. To quote Wikipedia:\n\n   \u003e This distinction between the word as a unit of speech and the\n   \u003e root as a unit of meaning is even more important in the case of\n   \u003e languages where roots have many different forms when used in\n   \u003e actual words, as is the case in Semitic languages. In these,\n   \u003e roots are formed by consonants alone, and different words\n   \u003e (belonging to different parts of speech) are derived from the\n   \u003e same root by inserting vowels. For example, in Hebrew, the root\n   \u003e gdl represents the idea of largeness, and from it we have gadol\n   \u003e and gdola (masculine and feminine forms of the adjective \"big\"),\n   \u003e gadal \"he grew\", higdil \"he magnified\" and magdelet \"magnifier\",\n   \u003e along with many other words such as godel \"size\" and migdal\n   \u003e \"tower\".\n\n### Morphology printing:\n   Instead of hard-coding LL, declare which links are morpho links\n   in the dict.\n\n### Assorted minor cleanup:\n   * Should provide a query that returns compile-time consts,\n      e.g. the max number of characters in a word, or max words\n      in a sentence.\n   * Should remove compile-time constants, e.g. max words, max\n      length etc.\n\n### Version 6.0 TODO list:\nVersion 6.0 will change `Sentence` to `Sentence*,` `Linkage` to\n`Linkage*` in the API.  But perhaps this is a bad idea...\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencog%2Flink-grammar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopencog%2Flink-grammar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencog%2Flink-grammar/lists"}