https://github.com/percyliang/sempre
Semantic Parser with Execution
https://github.com/percyliang/sempre
Last synced: 6 months ago
JSON representation
Semantic Parser with Execution
- Host: GitHub
- URL: https://github.com/percyliang/sempre
- Owner: percyliang
- License: other
- Created: 2013-09-18T06:59:41.000Z (almost 13 years ago)
- Default Branch: master
- Last Pushed: 2023-05-01T19:48:10.000Z (about 3 years ago)
- Last Synced: 2025-05-19T10:16:49.746Z (about 1 year ago)
- Language: Java
- Size: 10.9 MB
- Stars: 835
- Watchers: 69
- Forks: 296
- Open Issues: 88
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
- awesome-java - SEMPRE
README
# SEMPRE 2.4: Semantic Parsing with Execution
## What is semantic parsing?
A semantic parser maps natural language utterances into an intermediate logical
form, which is "executed" to produce a denotation that is useful for some task.
A simple arithmetic task:
- Utterance: *What is three plus four?*
- Logical form: `(+ 3 4)`
- Denotation: `7`
A question answering task:
- Utterance: *Where was Obama born?*
- Logical form: `(place_of_birth barack_obama)`
- Denotation: `Honolulu`
A virtual travel agent task:
- Utterance: *Show me flights to Montreal leaving tomorrow.*
- Logical form: `(and (type flight) (destination montreal) (departure_date 2014.12.09))`
- Denotation: `(list ...)`
By parsing utterances into logical forms, we obtain a rich representation that
enables much deeper, context-aware understanding beyond the words. With the
rise of natural language interfaces, semantic parsers are becoming increasingly
more powerful and useful.
## What is SEMPRE?
SEMPRE is a toolkit that makes it easy to develop semantic parsers for new
tasks. The main paradigm is to learn a feature-rich discriminative semantic
parser from a set of utterance-denotation pairs. One can also quickly
prototype rule-based systems, learn from other forms of supervision, and
combine any of the above.
If you use SEMPRE in your work, please cite:
@inproceedings{berant2013freebase,
author = {J. Berant and A. Chou and R. Frostig and P. Liang},
booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
title = {Semantic Parsing on {F}reebase from Question-Answer Pairs},
year = {2013},
}
SEMPRE has been used in the following papers:
- J. Berant and A. Chou and R. Frostig and P. Liang. [Semantic parsing on
Freebase from question-answer
pairs](http://cs.stanford.edu/~pliang/papers/freebase-emnlp2013.pdf). EMNLP,
2013.
This paper introduced SEMPRE 1.0, applied it to question answering on
Freebase, and created the WebQuestions dataset. The paper focuses on scaling
up semantic parsing via alignment and bridging, and does not talk about the
SEMPRE framework at all. To reproduce those results, check out SEMPRE 1.0.
- J. Berant and P. Liang. [Semantic Parsing via
Paraphrasing](http://cs.stanford.edu/~pliang/papers/paraphrasing-acl2014.pdf).
ACL, 2014.
This paper also used SEMPRE 1.0. The paraphrasing model is somewhat of a
offshoot, and does not use many of the core learning and parsing utiltiies in
SEMPRE. To reproduce those results, check out SEMPRE 1.0.
Please refer to the [project page](https://nlp.stanford.edu/software/sempre/) for a more complete list.
## Where do I go next?
- If you're new to semantic parsing, you can learn more from the [background
reading section of the tutorial](TUTORIAL.md).
- Install SEMPRE using the instructions under **Installation** below.
- Walk through the [tutorial](TUTORIAL.md)
to get a hands-on introduction to semantic parsing through SEMPRE.
- Read the complete [documentation](DOCUMENTATION.md)
to learn about the different components in SEMPRE.
# Installation
## Requirements
You must have the following already installed on your system.
- Java 8 (not 7)
- Ant 1.8.2
- Ruby 1.8.7 or 1.9
- wget
- make (for compiling fig and Virtuoso)
- zip (for unzip downloaded dependencies)
Other dependencies will be downloaded as you need them. SEMPRE has been tested
on Ubuntu Linux 12.04 and MacOS X. Your mileage will vary depending on how
similar your system is.
## Easy setup
1. Clone the GitHub repository:
git clone https://github.com/percyliang/sempre
2. Download the minimal core dependencies (all dependencies will be placed in `lib`):
ruby ./pull-dependencies core
3. Compile the source code (this produces `libsempre/sempre-core.jar`):
ant core
If there is any issue during compilation, try deleting the directories `lib` and `fig`
and rerunning `./pull-dependencies core`
4. Run an interactive shell:
ruby ./run @mode=simple
You should be able to type the following into the shell and get the answer `(number 7)`:
(execute (call + (number 3) (number 4)))
To go further, check out the [tutorial](TUTORIAL.md) and then the [full
documentation](DOCUMENTATION.md).
## Virtuoso graph database
If you will be using natural language to query databases (e.g., Freebase), then
you will also need to setup your own Virtuoso database (unless someone already
has done this for you):
For Ubuntu, follow this:
sudo apt-get install -y automake gawk gperf libtool bison flex libssl-dev
# Clone the repository
./pull-dependencies virtuoso
# Make and install
cd virtuoso-opensource
./autogen.sh
./configure --prefix=$PWD/install
make
make install
cd ..
on OS/X you can install virtuoso using homebrew by following the instructions
[here](http://carsten.io/virtuoso-os-on-mac-os/)
To have SEMPRE interact with Virtuoso, the required modules need to be compiled as follow:
./pull-dependencies core corenlp freebase
ant freebase
# Contribute
To contribute code or resource to SEMPRE:
- Create a fork of the repository. If you already have a fork,
it is a good idea to sync with the upstream repository first.
- Push your changes to a new branch in your fork.
- Start a pull request: go to your branch on the GitHub website,
then click "New pull request". Please specify the `develop` branch
of the upstream repository.
# ChangeLog
Changes from SEMPRE 1.0 to SEMPRE 2.0:
- Updated tutorial and documentation.
- Refactored into a core part for building semantic parsers in general;
interacting with Freebase and Stanford CoreNLP are just different modules.
- Removed fbalignment (EMNLP 2013) and paraphrase (ACL 2014) components to
avoid confusion. If you want to reproduce those systems, use SEMPRE 1.0.
Changes from SEMPRE 2.0 to SEMPRE 2.1:
- Added the `tables` package for the paper *Compositional semantic parsing on semi-structured tables* (ACL 2015).
- Add and `overnight` package for the paper *Building a semantic parser overnight* (ACL 2015).
Changes from SEMPRE 2.1 to SEMPRE 2.2:
- Added code for the paper *Inferring Logical Forms From Denotations* (ACL 2016).
Changes from SEMPRE 2.2 to SEMPRE 2.3:
- Added the `interactive` package for the paper *Naturalizing a programming language through interaction* (ACL 2017).
Changes from SEMPRE 2.3 to SEMPRE 2.3.1:
- Modified the `tables` module to resemble SEMPRE 2.1, effectively making it work again.
Changes from SEMPRE 2.3.1 to SEMPRE 2.4:
- Added the `cprune` package for the paper *Macro Grammars and Holistic Triggering for Efficient Semantic Parsing* (EMNLP 2017).