https://github.com/alchemine/intent-classification-analysis
A comparative analysis of encoder only and decoder only models in intent classication
https://github.com/alchemine/intent-classification-analysis
decoder encoder huggingface intent-classification nlp openaiapi python
Last synced: about 1 year ago
JSON representation
A comparative analysis of encoder only and decoder only models in intent classication
- Host: GitHub
- URL: https://github.com/alchemine/intent-classification-analysis
- Owner: alchemine
- Created: 2024-07-22T14:08:36.000Z (almost 2 years ago)
- Default Branch: develop
- Last Pushed: 2024-07-30T13:13:06.000Z (over 1 year ago)
- Last Synced: 2025-01-16T07:15:39.166Z (over 1 year ago)
- Topics: decoder, encoder, huggingface, intent-classification, nlp, openaiapi, python
- Language: Python
- Homepage:
- Size: 1.91 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# intent-classification-analysis
A comparative analysis of encoder only and decoder only models in intent classication.
## Reference
- [A Comparative Analysis of Encoder Only and Decoder Only Models in Intent Classification and Sentiment Analysis: Navigating the Trade-Offs in Model Size and Performance](https://www.researchgate.net/publication/377467915_A_Comparative_Analysis_of_Encoder_Only_and_Decoder_Only_Models_in_Intent_Classification_and_Sentiment_Analysis_Navigating_the_Trade-Offs_in_Model_Size_and_Performance)
```
Abstract
Intent classification and sentiment analysis stand as pivotal tasks in natural language processing, with applications ranging from virtual assistants to customer service.
The advent of transformerbased models has significantly enhanced the performance of various NLP tasks, with encoder-only architectures gaining prominence for their effectiveness.
More recently, there has been a surge in the development of larger and more powerful decoder-only models, traditionally employed for text generation tasks.
This paper aims to answer the question of whether the colossal scale of newer decoder-only language models is essential for real-world applications by comparing their performance to the well established encoder-only models, in the domains of intent classification and sentiment analysis.
Our results shows that for such natural language understanding tasks, encoder-only models in general provide better performance than decoder-only models, at a fraction of the computational demands.
```
## Dataset
- [AI Hub/용도별 목적대화 데이터/shopping](https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=&topMenu=&aihubDataSe=data&dataSetSn=544)
1. **Training**
- Label: `TL_1.shopping.zip`: `data/training/label`
- Raw: `TS_1.shopping.zip`: `data/training/raw`
2. **Validation**
- Label: `VL_1.shopping.zip`: `data/validation/label`
- Raw: `VS_1.shopping.zip`: `data/validation/raw`