https://github.com/farishijazi/legalgpt-legaldata

https://github.com/farishijazi/legalgpt-legaldata

Last synced: 29 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/farishijazi/legalgpt-legaldata
• Owner: FarisHijazi
• Created: 2024-04-22T14:57:52.000Z (8 months ago)
• Default Branch: master
• Last Pushed: 2024-08-07T22:35:35.000Z (5 months ago)
• Last Synced: 2024-10-15T01:23:22.008Z (2 months ago)
• Language: Jupyter Notebook
• Size: 9.28 MB
• Stars: 0
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# ArabLegalEval: A Multitask Benchmark for Assessing Arabic Legal Knowledge in Large Language Models

## Structure

- `data/` is handled using `DVC`, details can be found in [data/README.md](data/README.md)

- `RAG/` here you can find code for Retrieval Augmented Generation (RAG) and benchmarking. This is a modification of [https://github.com/predlico/ARAGOG](https://github.com/predlico/ARAGOG)

The ArLegalEval dataset is composed of 3 main parent tasks. Evaluation code for each of these tasks can be found in its directory as follows:

- `[benchmarkQA/](benchmarkQA/)` contains the question-answering task

- `[benchmarkArLegalBench/](benchmarkArLegalBench/)` contains the translated subset of LegalBench

- `[benchmarkMCQs/](benchmarkMCQs/)` contains the multiple-choice questions task

In this folder `data/processed/ArabLegalEval` you will find the benchmark as follows:

---

Tested with Python 3.12

## Citing this work

Please include all citations below, which credit all sources ArabLegalEval draws on.

```text

@inproceedings{

      anonymous2024arablegaleval,

      title={ArabLegalEval: A Multitask Benchmark for Assessing Arabic Legal Knowledge in Large Language Models},

      author={Anonymous},

      booktitle={The Second Arabic Natural Language Processing Conference},

      year={2024},

      url={https://openreview.net/forum?id=3EHYXqKKLA}

}

```