{"id":24500663,"url":"https://github.com/inseefrlab/codif-ape-nace-revision","last_synced_at":"2025-03-15T07:23:06.306Z","repository":{"id":256948515,"uuid":"777694366","full_name":"InseeFrLab/codif-ape-nace-revision","owner":"InseeFrLab","description":"Centralized repo on nace revision","archived":false,"fork":false,"pushed_at":"2025-03-14T12:48:39.000Z","size":114,"stargazers_count":1,"open_issues_count":5,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-14T13:38:47.469Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/InseeFrLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-26T10:43:18.000Z","updated_at":"2025-03-14T12:48:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"93f85539-764d-4323-93de-91810c466993","html_url":"https://github.com/InseeFrLab/codif-ape-nace-revision","commit_stats":null,"previous_names":["inseefrlab/codif-ape-nace-revision"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InseeFrLab%2Fcodif-ape-nace-revision","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InseeFrLab%2Fcodif-ape-nace-revision/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InseeFrLab%2Fcodif-ape-nace-revision/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InseeFrLab%2Fcodif-ape-nace-revision/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/InseeFrLab","download_url":"https://codeload.github.com/InseeFrLab/codif-ape-nace-revision/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243697351,"owners_count":20332936,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-21T22:21:32.615Z","updated_at":"2025-03-15T07:23:06.301Z","avatar_url":"https://github.com/InseeFrLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 📌 CAG vs RAG: Centralized Repository for NACE Revision\n\n## 📑 Table of Contents\n\n- [📌 CAG vs RAG: Centralized Repository for NACE Revision](#-cag-vs-rag-centralized-repository-for-nace-revision)\n  - [📑 Table of Contents](#-table-of-contents)\n  - [📖 Overview](#-overview)\n  - [🚀 Getting Started](#-getting-started)\n    - [🛠 Installation](#-installation)\n    - [🏗 Pre-commit Setup](#-pre-commit-setup)\n    - [Cache LLM model from S3 Bucket (Optionnal)](#cache-llm-model-from-s3-bucket-optionnal)\n  - [📜 Running the Scripts](#-running-the-scripts)\n    - [✅ 1. Build Vector Database (if you are in the RAG case)](#-1-build-vector-database-if-you-are-in-the-rag-case)\n    - [🏷 2. Encode Business Activity Codes](#-2-encode-business-activity-codes)\n    - [🔬 3. Evaluate Classification Strategies](#-3-evaluate-classification-strategies)\n    - [📊 4. Build the NACE 2025 Dataset](#-4-build-the-nace-2025-dataset)\n  - [📡 LLM Integration](#-llm-integration)\n  - [🏗 Argo Workflows](#-argo-workflows)\n  - [📄 License](#-license)\n\n\n## 📖 Overview\nThis repository is dedicated to the revision of the **Nomenclature statistique des Activités économiques dans la Communauté Européenne (NACE)**.\n\nIt provides tools for **automated classification and evaluation of business activity codes** using **Large Language Models (LLMs)** and vector-based retrieval systems.\n\n\n## 🚀 Getting Started\n\n### 🛠 Installation\nEnsure you have **Python 3.12+** and **uv** or **pip** installed, then install the required dependencies:\n\n```bash\nuv pip install -r requirements.txt\n```\n\nor\n\n```bash\nuv pip install -r pyproject.toml\n```\n\n### 🏗 Pre-commit Setup\nSet up linting and formatting checks using `pre-commit`:\n\n```bash\npre-commit install\n```\n\n### Cache LLM model from S3 Bucket (Optionnal)\nIf you want to use a model available in the SSPCloud you can execute this command:\n\n```bash\nMODEL_NAME=mistralai/Ministral-8B-Instruct-2410\nLOCAL_PATH=~/.cache/huggingface/hub\n\n./bash/fetch_model_s3.sh $MODEL_NAME $LOCAL_PATH\n```\n\n## 📜 Running the Scripts\n\n### ✅ 1. Build Vector Database (if you are in the RAG case)\n\nTo create a searchable database of NACE 2025 codes:\n\n```bash\npython build-vector-db.py\n```\n\n### 🏷 2. Encode Business Activity Codes\n\nFor **unambiguous** classification:\n\n```bash\npython encode-univoque.py\n```\n\nFor **ambiguous** classification using an LLM:\n\n```bash\npython encode-multivoque.py --experiment_name NACE2025_DATASET --llm_name mistralai/Mistral-7B-Instruct\n```\n\n### 🔬 3. Evaluate Classification Strategies\n\nCompare different classification models:\n\n```bash\npython evaluate_strategies.py\n```\n\n### 📊 4. Build the NACE 2025 Dataset\n\nOnce all unique ambiguous cases have been recoded using the best strategy, you can rebuild the entire dataset with NACE 2025 labels:\n\n```bash\npython build_nace2025_sirene4.py\n```\n\n---\n\n## 📡 LLM Integration\nThis repository leverages **Large Language Models (LLMs)** to assist in classifying business activities. The supported models include:\n\n- `Qwen/Qwen2.5-32B-Instruct`\n- `mistralai/Mistral-Small-Instruct-2409`\n- `hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4`\n\nThese models help improve classification accuracy for ambiguous business activity cases.\n\n\n## 🏗 Argo Workflows\nThis project supports **automated workflows** via [Argo Workflows](https://argoproj.github.io/argo-workflows/).\nTo trigger a workflow, execute:\n\n```yaml\nargo submit argo-workflows/relabel-naf08-to-naf25.yaml\n```\n\nOr use the **Argo Workflow UI**.\n\n\n## 📄 License\nThis project is licensed under the **Apache License 2.0**. See the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finseefrlab%2Fcodif-ape-nace-revision","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finseefrlab%2Fcodif-ape-nace-revision","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finseefrlab%2Fcodif-ape-nace-revision/lists"}