https://github.com/prokil/anlp-grading-tools
https://github.com/prokil/anlp-grading-tools
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/prokil/anlp-grading-tools
- Owner: ProKil
- License: mit
- Created: 2021-09-28T17:19:58.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2022-09-22T01:58:57.000Z (over 3 years ago)
- Last Synced: 2025-04-25T12:45:55.234Z (about 1 year ago)
- Language: Python
- Size: 45.9 KB
- Stars: 3
- Watchers: 1
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Grading tools for Advanced NLP (11-711)
## Installation
You'll need `docker` and `unzip` to use this repo. For `docker`, visit the [official guide](https://docs.docker.com/get-started/) to get started. For `unzip`, you can install it on ubuntu via `sudo apt-get install unzip`.
Install the python package by
```
git clone https://github.com/ProKil/anlp-grading-tools
cd anlp-grading-tools
pip install -e .
```
## Usage
To evaluate your code, you'll need to change the environment variables in test.sh.
`ANLP_TMP_DIR`: `mkdir` a new folder, e.g. `mkdir tmp`, and point this variable to the absolute path of the tmp folder.
`SUBMISSION_DIR`: this should point to the folder containing your submission zip file. Note that the toolkit will automatically evaluate all zip files in the folder.
`SCORES_DIR`: this should point to an empty folder. Your score will be logged in a text file there.
`DATA_DIR`: this should point to the data folder of `minbert-assignment`.
Example code to prepare the folders:
```
mkdir tmp
mkdir scores
cp -r path/to/minbert-assignment/data ./
mkdir submission
cp your/submission.zip submission
```
Now you can evaluate your code through `bash test.sh`, after which your scores are at `SCORES_DIR/andrewid`.
## Troubleshooting
1. You may find writing files inside `ANLP_TMP_DIR` and `SCORE_DIR` requiring permission. You can either use `sudo` or log into docker through `docker run -v FOLDER_TO_WRITE:/mnt -it --entrypoint /bin/bash anlp` and `cd /mnt` to write those files.
2. You may experience other permission issues with docker. Please refer to [this page](https://docs.docker.com/engine/install/linux-postinstall/) to use docker without sudo.