Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/opencog/relex

English Dependency Relationship Extractor
https://github.com/opencog/relex

Last synced: about 1 year ago
JSON representation

English Dependency Relationship Extractor

Awesome Lists containing this project

README 

          

             Stanford Parser Compatibility Mode

             ----------------------------------

                      October 2009



The Stanford parser provides a defacto standard definition of dependency

relations used in parsing.  Relex has a compatibility mode which

attempts to generate the same output as the Stanford parser.  This is

an interesting thing to do, for two reasons:



   1) It provides a way of double-checking parse results,

   2) RelEx is more than three times faster than the "factored"

      version of the Stanford parser, although it is slower than the

      PCFG version.



The compatibility mode is currently in "beta": it is mostly implemented,

but still has a number of rough spots that need work.



Compatibility mode is enabled with the --stanford flag, passed to

RelationExtractor. So for example, edit "relation-extractor.sh"

and add "--stanford" to the command line there.



A set of test sentences, testing operation of the current parser,

are documented below. An automated test case, testing compatibility,

can be run by issueing the command "ant test", which will compile and

run src/java_test/relex/test/TestStanford.java.



The current RelEx implementation, and the notes below, are all relative

to the version of the Stanford parser that is available at the time of

this writing.  RelEx is attempting to be functionally compatible, while

using the documentation as a guide.  There are a number of areas where

the actual operation of the Stanford parser does not match the

documentation for it -- it either fails to generate the markup as

claimed, or it generates undocumented dependencies.  Thus, creating

"compatibility" is somewhat of an inexact science. The status of

various test sentences are reviewed below.



Note that compatibility is strongly influenced by the parse ranking

algorithm. The Link-Grammar parser often generates several candidate

parses for a sentence, and then ranks them according to liklihood.

It can be configured to use one of several different ranking algorithms.

Thus, the "top-most" parse, used by the test case, is subject to change

depending on the ranking algorithm. That is, precision (but not recall)

is strongly dependent on the parse ranking.



What its not

------------

This mode is NOT an API compatibility. The API's are completely

different. The compatibility is in the dependencies generated.



Missing at this time is compatible part-of-speech tagging. At this

time, RelEx generates much simpler part-of-speech tags, and there has

been no effort made to ensure their accuracy.



RelEx cannot be trained to learn other languages. RelEx is

hand-crafted, and handles English only. (We have vague plans to do

better someday...)



Issues:

-------

The stanford parser has several different operational modes, wherein

it generates different relations, dependeing on the mode. A fair number

of the relations, including csubjpass, infmod, pcomp, possessive, punct,

purpcl, tmod and xcomp, are not generated in the default mode.  At this

time RelEx is attempting to be compatible only with the default mode.



At this time, link-grammar handles conjunctions in a novel way.

These are not percolated up to RelEx in a proper manner, and are

thus are mostly not handled.



Some other rough spots/issues are marked with XXX below.



Version

-------

The current Stanford parser version is labelled as:



    Stanford Lexicalized Parser v1.6.1 - 19 Oct 2008.



The ruling documentation is taken to be:



   Stanford typed dependencies manual, Marie-Catherine de Marneffe

   and Christopher D. Manning September 2008, available at:

   http://nlp.stanford.edu/software/dependencies_manual.pdf



Documentation for the Penn Tagset can be found at:

ftp://ftp.cis.upenn.edu/pub/treebank/doc/tagguide.ps.gz

http://www.ims.uni-stuttgart.de/projekte/CorpusWorkbench/CQP-HTMLDemo/PennTreebankTS.html

http://www.mozart-oz.org/mogul/doc/lager/brill-tagger/penn.html

http://www.comp.leeds.ac.uk/ccalas/tagsets/upenn.html

http://bulba.sdsu.edu/jeanette/thesis/PennTags.html



Test sentences

--------------

A list of test sentences, and status. Many/most of these sentences are

taken from the examples provided in the Stanford documentation.

Additional test sentences can be found in "TestStanford.java".



The parsing of these sentences are tested automatically by running

"ant test", which runs the test case "TestStanford.java".



Who invented sliced bread?

    Stanford output:

       nsubj(invented-2, Who-1)

       amod(bread-4, sliced-3)

       dobj(invented-2, bread-4)

    RelEx status: OK



Jim runs quickly.

    Stanford output:

       nsubj(runs-2, Jim-1)

       advmod(runs-2, quickly-3)

    RelEx status: OK



The bird, a robin, sang sweetly.

    Stanford output:

       det(bird-2, The-1)

       nsubj(sang-7, bird-2)

       det(robin-5, a-4)

       appos(bird-2, robin-5)

       advmod(sang-7, sweetly-8)

    RelEx status: OK



There is a place we can go.

    Stanford output:

       expl(is-2, There-1)

       det(place-4, a-3)

       nsubj(is-2, place-4)

       nsubj(go-7, we-5)

       aux(go-7, can-6)

       dep(is-2, go-7)

    RelEx status:

       Not testing this sentence, since stanford outputs a

       "dep" which is not documented in thier manual.

       Other than that, the parse works.



The linebacker gave the quarterback a push.

    Stanford output:

       det(linebacker-2, The-1)

       nsubj(gave-3, linebacker-2)

       det(quarterback-5, the-4)

       iobj(gave-3, quarterback-5)

       det(push-7, a-6)

       dobj(gave-3, push-7)

    RelEx status: OK



He stood at the goal line.

    Stanford output:

       nsubj(stood-2, He-1)

       det(line-6, the-4)

       nn(line-6, goal-5)

       prep_at(stood-2, line-6)

    RelEx status: OK



She looks very beautiful.

    Stanford output:

       nsubj(looks-2, She-1)

       advmod(beautiful-4, very-3)

       acomp(looks-2, beautiful-4)

    RelEx status: OK



The accident happened as the night was falling.

    Stanford output:

       det(accident-2, The-1)

       nsubj(happened-3, accident-2)

       mark(falling-8, as-4)

       det(night-6, the-5)

       nsubj(falling-8, night-6)

       aux(falling-8, was-7)

       advcl(happened-3, falling-8)

    RelEx status: OK



If you know who did it, you should tell the teacher.

    Stanford output:

       mark(know-3, If-1)

       nsubj(know-3, you-2)

       advcl(tell-10, know-3)

       nsubj(did-5, who-4)

       ccomp(know-3, did-5)

       dobj(did-5, it-6)

       nsubj(tell-10, you-8)

       aux(tell-10, should-9)

       det(teacher-12, the-11)

       dobj(tell-10, teacher-12)

    RelEx status: OK



The man has been killed by the police.

    Stanford output:

       det(man-2, The-1)

       nsubjpass(killed-5, man-2)

       aux(killed-5, has-3)

       auxpass(killed-5, been-4)

       det(police-8, the-7)

       agent(killed-5, police-8)

    Relex status: OK.

    Hmm ... but the above seems broken -- it leaves out "by" from the

    parse, using "agent" instead. Very strange! But that's what we'll

    do too.



Effects caused by the protein are important.

    Stanford output:

       nsubj(important-7, Effects-1)

       partmod(Effects-1, caused-2)

       det(protein-5, the-4)

       agent(caused-2, protein-5)

       cop(important-7, are-6)

    RelEx status: OK



Sam, my brother, has arrived.

    Stanford output:

       nsubj(arrived-7, Sam-1)

       poss(brother-4, my-3)

       appos(Sam-1, brother-4)

       aux(arrived-7, has-6)

    RelEx status: OK



What is that?

    Stanford output:

       attr(is-2, What-1)

       nsubj(is-2, that-3)

    RelEx status: OK



Reagan has died.

    Stanford output:

       nsubj(died-3, Reagan-1)

       aux(died-3, has-2)

    RelEx status: OK



He should leave.

    Stanford output:

       nsubj(leave-3, He-1)

       aux(leave-3, should-2)

    RelEx status: OK



Kennedy has been killed.

    Stanford output:

       nsubjpass(killed-4, Kennedy-1)

       aux(killed-4, has-2)

       auxpass(killed-4, been-3)

    RelEx status: OK



Kennedy was killed.

    Stanford output:

       nsubjpass(killed-3, Kennedy-1)

       auxpass(killed-3, was-2)

    RelEx status: OK



Kennedy got killed.

    Stanford output:

       nsubjpass(killed-3, Kennedy-1)

       auxpass(killed-3, got-2)

    RelEx status: OK



He says that you like to swim.

    Stanford output:

       nsubj(says-2, He-1)

       complm(like-5, that-3)

       nsubj(like-5, you-4)

       ccomp(says-2, like-5)

       nsubj(swim-7, to-6)

       ccomp(like-5, swim-7)

    RelEx status: Not testing. The "nsubj(swim, to)" seems insane to me

      RelEx currently generates aux(swim-7, to-6) which seems

      reasonable to me.



Bill is big.

    Stanford output:

       nsubj(big-3, Bill-1)

       cop(big-3, is-2)

    RelEx status: OK

       (This is an example where the use of the predicative adjective

       would have been more conceptually correct!)



Bill is an honest man.

    Stanford output:

       nsubj(man-5, Bill-1)

       cop(man-5, is-2)

       det(man-5, an-3)

       amod(man-5, honest-4)

    RelEx status: OK



What she said makes sense.

    Stanford output:

       dobj(said-3, What-1)

       nsubj(said-3, she-2)

       csubj(makes-4, said-3)

       dobj(makes-4, sense-5)

    RelEx status: OK



What she said is not true.

    Stanford output:

       dobj(said-3, What-1)

       nsubj(said-3, she-2)

       csubj(true-6, said-3)

       cop(true-6, is-4)

       neg(true-6, not-5)

    RelEx status: OK



That she lied was suspected by everyone.

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate csubjpass.

    RelEx status: nominal



Which book do you prefer?

    Stanford output:

       det(book-2, Which-1)

       dobj(prefer-5, book-2)

       aux(prefer-5, do-3)

       nsubj(prefer-5, you-4)

    RelEx status: OK



There is a ghost in the room.

    Stanford output:

       expl(is-2, There-1)

       det(ghost-4, a-3)

       nsubj(is-2, ghost-4)

       det(room-7, the-6)

       prep_in(is-2, room-7)

    RelEx status: OK



I don’t have anything to say.

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate infmod.

    RelEx status: nominal



She gave me a raise.

    Stanford output:

       nsubj(gave-2, She-1)

       iobj(gave-2, me-3)

       det(raise-5, a-4)

       dobj(gave-2, raise-5)

    RelEx status: OK



Forces engaged in fighting after insurgents attacked.

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate mark.

    RelEx status: nominal



The director is 65 years old.

    Stanford output:

       det(director-2, The-1)

       nsubj(old-6, director-2)

       cop(old-6, is-3)

       num(years-5, 65-4)

       measure(old-6, years-5)

    RelEx status: OK



Sam eats 3 sheep.

    Stanford output:

       nsubj(eats-2, Sam-1)

       num(sheep-4, 3-3)

       dobj(eats-2, sheep-4)

    RelEx status: OK



I lost $ 3.2 billion.

    Stanford output:

       nsubj(lost-2, I-1)

       dobj(lost-2, $-3)

       number($-3, 3.2-4)

       number($-3, billion-5)



    RelEx status: nominal

       I don't get it. There's a subtle distinction between num and

       number that Stanford is making that I dont' understand.  RelEx

       currently generates:

          number(billion-5, 3.2-4)

          num($-3, billion-5)

       which I think is more correct.



Truffles picked during the spring are tasty.

    Stanford output:

       nsubj(tasty-7, Truffles-1)

       partmod(Truffles-1, picked-2)

       det(spring-5, the-4)

       prep_during(picked-2, spring-5)

       cop(tasty-7, are-6)

    RelEx status: OK



We have no information on whether users are at risk.

They heard about you missing classes.

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate pcomp.

    RelEx status: nominal



I sat on the chair.

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate pobj.

    RelEx status: nominal

       Relex is currently producing the same results as Stanford.



We went to their offices to get Bill's clothes.

    Stanford output:

       nsubj(went-2, We-1)

       xsubj(get-7, We-1)

       poss(offices-5, their-4)

       prep_to(went-2, offices-5)

       aux(get-7, to-6)

       xcomp(went-2, get-7)

       poss(clothes-10, Bill-8)

       dobj(get-7, clothes-10)

    RelEx status:

Under construction:

       xsubj(get-7, We-1)

       xcomp(went-2, get-7)

prep_to2(went-2, get-7)



All the boys are here.

    Stanford output:

       predet(boys-3, All-1)

       det(boys-3, the-2)

       nsubj(are-4, boys-3)

       advmod(are-4, here-5)

    RelEx status:

       Predet is generated, as described in stanford docs. However,

       the copular structure is different. More generally, link

       grammar treats predeterminers differently (AL link) and

       this needs some additional resolution. Also, it is not clear

       to me why Stanford doesn't treat this as being copular, i.e.

          nsubj(here-5, boys-3)

          cop(here-5, are-4)

       the way that it treats other copulas (e.g. "The garage is

       next to the house." gets nsubj(next-4, garage-2) cop(next-4,

       is-3).



I saw a cat in a hat.

I saw a cat with a telescope.

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate prep. Instead,

       Stanford is producing a collapsed prep. Worse, these

       sentences are ambiguous -- is the cat holding a telescope,

       or am I looking through one?  The top-ranked parses for

       RelEx indicate that the seeing is doine through the telescope

       (and that the seeing is done while wearing a hat). By contrast,

       Stanford currently presumes that the cat has a hat, and a

       telescope -- these parses are generated by RelEx, but are

       ranked lower.

    RelEx status: OK



He is responsible for meals.

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate prep. Instead,

       stanford is producing a collapsed prep.

    RelEx status: OK

       i.e. RelEx is producing the same thing as Stanford at this time.



They shut down the station.

    Stanford output:

       nsubj(shut-2, They-1)

       prt(shut-2, down-3)

       det(station-5, the-4)

       dobj(shut-2, station-5)

    RelEx status: OK



Go home!

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate punct.

    RelEx status: nominal



He talked to him in order to secure the account.

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate purpcl.

    RelEx status: nominal

        xxx relex gets xcomp wrong here, and needs an xsubj



About 200 people came to the party.

    Stanford output:

       quantmod(200-2, About-1)

       num(people-3, 200-2)

       nsubj(came-4, people-3)

       det(party-7, the-6)

       prep_to(came-4, party-7)

    RelEx status: OK



I saw the book which you bought.

    Stanford output:

       Stanford is not outputting what the documentation says it

       should -- it was supposed to illustrate ref. Instead, it

       is generating rel, exactly the same as below. No surprise,

       since syntactically, these sentences are identical.

    RelEx status: OK

       (I.e. relex is generatng exactly the same thing as the

       stanford parser currently is.)



I saw the man who you love.

    Stanford output:

       nsubj(saw-2, I-1)

       det(man-4, the-3)

       dobj(saw-2, man-4)

       dobj(love-7, man-4)

       rel(love-7, who-5)

       nsubj(love-7, you-6)

       rcmod(man-4, love-7)

    RelEx status: OK



I saw the man whose wife you love.

    Stanford output:

       nsubj(saw-2, I-1)

       det(man-4, the-3)

       dobj(saw-2, man-4)

       poss(wife-6, whose-5)

       dobj(love-8, wife-6)

       rel(love-8, wife-6)

       nsubj(love-8, you-7)

       rcmod(man-4, love-8)

    RelEx status: Incomplete

      Not yet generating the rel and the rcmod. under construction.

      shouldn't be hard ...



Last night, I swam in the pool.

    Stanford output:

       (Stanford is not outputting what the documentation says it

        should -- it was supposed to illustrate tmod.)

    RelEx status: nominal



He says that you like to swim.

    Stanford output:

       (Stanford is not outputting what the documentation says it

        should -- it was supposed to illustrate xcomp.)

    RelEx status: nominal



I am ready to leave.

    Stanford output:

       nsubj(ready-3, I-1)

       cop(ready-3, am-2)

       aux(leave-5, to-4)

       xcomp(ready-3, leave-5)

    RelEx status: OK



Tom likes to eat fish.

    Stanford output:

       nsubj(likes-2, Tom-1)

       xsubj(eat-4, Tom-1)

       aux(eat-4, to-3)

       xcomp(likes-2, eat-4)

       dobj(eat-4, fish-5)

    RelEx status: OK



Under construction:



The garage is next to the house.

    Stanford output:

       det(garage-2, The-1)

       nsubj(next-4, garage-2)

       cop(next-4, is-3)

       det(house-7, the-6)

       prep_to(next-4, house-7)