Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/foxminchan/LawKnowledge
A legal knowledge search and Q&A application based on Vietnam's Legal Code and legal document database ⚖️
https://github.com/foxminchan/LawKnowledge
generative-ai microservice natural-language-processing nlp nx searching semantic-search
Last synced: 26 days ago
JSON representation
A legal knowledge search and Q&A application based on Vietnam's Legal Code and legal document database ⚖️
- Host: GitHub
- URL: https://github.com/foxminchan/LawKnowledge
- Owner: foxminchan
- License: mit
- Created: 2023-11-17T04:10:19.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-03-17T04:30:53.000Z (9 months ago)
- Last Synced: 2024-08-03T16:08:25.561Z (4 months ago)
- Topics: generative-ai, microservice, natural-language-processing, nlp, nx, searching, semantic-search
- Language: TypeScript
- Homepage: https://foxminchan.github.io/LawKnowledge/
- Size: 174 MB
- Stars: 93
- Watchers: 1
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: .github/CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
- Citation: CITATION.cff
- Codeowners: .github/CODEOWNERS
- Security: .github/SECURITY.md
- Support: .github/SUPPORT.md
Awesome Lists containing this project
- awesome-nestjs - LawKnowledge
README
Law Knowlege
A legal knowledge search and Q&A application based on Vietnam's Legal Code and legal document database
Explore the docs »
🐛 Report Bug
|
✈️ Request Feature
|
💬 Join Our Telegram
|
📚 Read Wiki
|
📋 Roadmap
⚙️ This project used **SonarCloud** to analyze the code quality of the project. **Pulumi** for infrastructure as code (IaC). The badge below shows the quality of the code. Click on the badge to see the details.
Features
- 📖 Document Organization and Linking
- 📦 Terminology Extraction
- 🔍 Quick Search for Related Content
- 🤖 User Query Support
Table of Contents
Expand contents
- [What is Law Knowledge?](#what-is-law-knowledge)
- [Tentative technologies](#tentative-technologies)
- [Building blocks](#building-blocks)
- [Getting Started](#getting-started)
- [🛠️ Prerequisites](#️-prerequisites)
- [Infrastucture](#infrastucture)
- [Services](#services)
- [😄 HuggingFace Resources](#-huggingface-resources)
- [🧑💻 Setup](#-setup)
- [🚀 Running](#-running)
- [🧪 Testing](#-testing)
- [🧩 Other](#-other)
- [Dependency Graph](#dependency-graph)
- [CI/CD](#cicd)
- [Contributing](#contributing)
- [📖 Contributing Guidelines](#-contributing-guidelines)
- [💁 Want to Help?](#-want-to-help)
- [🫂 Code of Conduct](#-code-of-conduct)
- [Support and Organization](#support-and-organization)
- [License](#license)
# What is Law Knowledge?
Law Knowledge is an app designed to provide quick access to Vietnam's legal information, including laws and legal documents. It's ideal for professionals, students, or anyone needing insights into Vietnamese legal codes and regulations.
[](https://stars.medv.io/foxminchan/LawKnowledge)
Law Knowledge is a legal knowledge search and Q&A application based on Vietnam's Legal Code and legal document database. The application is built on the Microservice Architecture and uses Generative AI to extract legal terms and Question Answering to answer user questions. The application is built with Open Source technologies and is deployed on Cloud platforms.
If you want to find out more about the contest, please visit the [VFOSSA website](https://vfossa.vn/tin-tuc/de-thi-phan-mem-nguon-mo-olp-2023-688.html).
# Tentative technologies
- [React](https://reactjs.org/)
- [NestJS](https://nestjs.com/)
- [Python](https://www.python.org/)
- [Pulumi](https://www.pulumi.com/)
- [Kafka](https://kafka.apache.org/)
- [ArgoCD](https://argoproj.github.io/argo-cd/)
- [Jenkins](https://www.jenkins.io/)
- [Kubernetes](https://kubernetes.io/)
- [OpenTelemetry](https://opentelemetry.io/)
- [Grafana](https://grafana.com/), [Prometheus](https://prometheus.io/), [Zipkin](https://zipkin.io/), [Fluentd](https://www.fluentd.org/)
# Building blocks
We used Microservice Architecture to build this project to make it easier to scale and maintain. The following diagram shows the architecture of the project.
# Getting Started
## 🛠️ Prerequisites
### Infrastucture
-
Nx - Nx is a set of extensible dev tools for monorepos.
-
Nodejs - Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine.
-
pnpm - Fast, disk space efficient package manager.
-
Python - Python is a programming language that lets you work quickly and integrate systems more effectively.
-
Poetry - Poetry helps you declare, manage and install dependencies of Python projects, ensuring you have the right stack everywhere.
-
Pulumi - Pulumi is a manage infrastructure, secrets, and configurations intuitively on any cloud.
-
Docker (Kubernetes Enabled) - Docker is an open platform for developing, shipping, and running applications.
-
Helm - Helm is the best way to find, share, and use software built for Kubernetes.
### Services
-
AWS - An Amazon Web Services account is required to deploy the infrastructure.
-
HuggingFace - A HuggingFace account is required to download and upload the dataset and model.
## 😄 HuggingFace Resources
You can download the dataset from the following link:
- [Law Knowledge Data](https://huggingface.co/datasets/foxminchan/law-knowledge): The dataset from [Pháp điển](https://phapdien.moj.gov.vn/) and [Văn bản pháp luật](https://vbpl.vn/).
- [Law Knowledge Model](https://huggingface.co/foxminchan/law-knowledge): A fine-tuned model from [PhoBERT](https://huggingface.co/vinai/phobert-base-v2).
## 🧑💻 Setup
First, clone the repository to your local machine:
```bash
git clone https://github.com/foxminchan/LawKnowledge
```
Next, navigate to the root directory of the project and install the dependencies:
```bash
pnpm install --force
```
## 🚀 Running
For the website, you can run the following command:
```bash
npx nx serve website --prod
```
For the API, you can run the following command:
```bash
# For the API Gateway
npx nx serve api-gateway --prod
# For the Auth Service
npx nx serve auth-svc --prod
# For the Law Service
npx nx serve law-svc --prod
# For the Search Service, navigate to the search-svc directory
npx nx build search-svc && cd apps/api/search-svc/search-svc && python main.py
# For the Chat Service, navigate to the chat-svc directory
npx nx build chat-svc && cd apps/api/chat-svc/chat-svc && python main.py
```
To traning the model, you can run the following command:
```bash
npx nx build bert && cd apps/nlp/bert/bert && python main.py
```
To set up the infrastructure, you can run the following command:
```bash
npx nx up pulumi
```
> [!IMPORTANT]
>
>
> If you want to run with Nx, make sure you have installed `poetry` globally. Python version must be `>=3.9.1 <=3.12`.
>
## 🧪 Testing
For the website, you can run the following command:
```bash
npx nx test website-e2e
```
For the API, you can run the following command:
```bash
# For the Auth Service
npx nx test auth-svc-e2e
# For the Law Service
npx nx test law-svc-e2e
# For the Search Service
npx nx test search-svc-e2e
# For the Chat Service
npx nx test chat-svc
```
To test the model, you can run the following command:
```bash
npx nx test bert
```
## 🧩 Other
To run the tooling for processing the dataset, you can run the following command:
```bash
npx nx build processor && cd apps/nlp/processor/processor && python main.py
```
For running documentation, you can run the following command:
```bash
npx nx serve docs
```
For running the Diagram as Code, you can run the following command:
```bash
npx nx build dac && cd assets/dac/dac && python main.py
```
Some useful scripts:
```bash
# Navigate to the scripts directory
cd tools/scripts && ls
# + scripts
# +-- build.sh --> Build the project
# +-- clean.sh --> Clean the docker image
# +-- init.sh --> Initialize the project
# +-- prepare.sh --> Prepare the project
# +-- start.sh --> Start the project
# +-- start-o11y.sh --> Start the observability stack
# +-- stop-o11y.sh --> Stop the observability stack
# Run the script
./
````
# Dependency Graph
You can see the dependency graph of the project by running the following command:
```bash
npx nx dep-graph
```
Here is the dependency graph of the project:
# CI/CD
# Contributing
## 📖 Contributing Guidelines
We are excited that you are interested in contributing to this project! Before submitting your contribution, please make sure to take a moment and read through the following guidelines:
Read through our [contributing guidelines](.github/CONTRIBUTING.md) to learn about our submission process, coding rules, and more.
## 💁 Want to Help?
Want to report a bug, contribute some code, or improve the documentation? Excellent! Read up on our guidelines for [contributing](.github/CONTRIBUTING.md) and then check out one of our issues labeled as [help wanted](https://github.com/foxminchan/LawKnowledge/labels/help%20wanted) or [good first issue](https://github.com/foxminchan/LawKnowledge/labels/good%20first%20issue).
## 🫂 Code of Conduct
Help us keep Law Knowledge open and inclusive. Please read and follow our [Code of Conduct](.github/CODE_OF_CONDUCT.md).
# Support and Organization
# License
This project is licensed under the terms of the [MIT](LICENSE) license.