{"id":23960578,"url":"https://github.com/ivanfioravanti/wine_variety_classification","last_synced_at":"2025-04-23T04:22:26.544Z","repository":{"id":271129640,"uuid":"912477289","full_name":"ivanfioravanti/wine_variety_classification","owner":"ivanfioravanti","description":"Examples on how to use various LLM providers with a Wine Classification problem","archived":false,"fork":false,"pushed_at":"2025-03-17T08:01:56.000Z","size":1426,"stargazers_count":28,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-19T06:53:46.816Z","etag":null,"topics":["llm","mlx","ollama","openai"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ivanfioravanti.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-05T17:22:33.000Z","updated_at":"2025-04-14T18:54:46.000Z","dependencies_parsed_at":"2025-01-05T18:37:38.267Z","dependency_job_id":null,"html_url":"https://github.com/ivanfioravanti/wine_variety_classification","commit_stats":null,"previous_names":["ivanfioravanti/wine_variety_classification"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivanfioravanti%2Fwine_variety_classification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivanfioravanti%2Fwine_variety_classification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivanfioravanti%2Fwine_variety_classification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivanfioravanti%2Fwine_variety_classification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ivanfioravanti","download_url":"https://codeload.github.com/ivanfioravanti/wine_variety_classification/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250367411,"owners_count":21418881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llm","mlx","ollama","openai"],"created_at":"2025-01-06T19:30:06.760Z","updated_at":"2025-04-23T04:22:26.528Z","avatar_url":"https://github.com/ivanfioravanti.png","language":"Python","funding_links":[],"categories":["Full list","LLM \u0026 Inference"],"sub_categories":["Scientific Research"],"readme":"# Wine Variety Prediction with LLMs\n\n![Wine Variety Prediction](./images/guess_the_wine.webp)\n\nThis project demonstrates how to use various Large Language Models (LLMs) to predict wine varieties based on wine reviews and characteristics. It's inspired by [OpenAI's model distillation cookbook](https://cookbook.openai.com/examples/leveraging_model_distillation_to_fine-tune_a_model) but focuses on comparing different models' performance without distillation.\n\nThe goal is using native API for each provider where possible for this classification task.\nMain focus is on local models through Ollama with llama.cpp and LM Studio with MLX.\n\n## Results first\nI ran the tests on Italian wines first and then French wines later to see if there was any difference in performance.\nAnd...\n- French wines are easier to predict than Italian wines!\n- Anthropic models are the best performing classification models out there! Including Claude 3.5 Haiku.\n\nHere is the chart that compares Italian and French results:\n![Comparison chart](./images/italy_france_comparison.png)\n\nHere is the detailed chart for Italian wines:\n![Italian Wine Results](./results/accuracy_chart_20250105_095642.png) \n\nHere is the detailed chart for French wines:\n![French Wine Results](./results/accuracy_chart_20250105_115642.png)\n\n## Overview\n\nThe project uses a dataset of Italian wines to test different LLMs' ability to predict grape varieties based on wine descriptions, regions, and other characteristics. It showcases the use of Structured Outputs with various LLM providers and compares their performance.\n\n## Features\n\n- Wine variety prediction using multiple LLM providers:\n  - [Ollama](https://ollama.ai)\n  - [OpenAI](https://openai.com)\n  - [Google Gemini](https://gemini.google.com)\n  - [LM Studio](https://lmstudio.ai)\n  - [OpenRouter](https://openrouter.ai)\n  - [DeepSeek](https://deepseek.com)\n  - [Anthropic](https://anthropic.com)\n  - [MLX Omni Server](https://github.com/madroidmaq/mlx-omni-server)\n- Structured Output implementation for consistent responses (where possible)\n- Performance comparison between different models\n- Support for parallel processing with some providers\n\n## Prerequisites\n\n- Python 3.12+\n- Jupyter Notebook\n- API keys for various providers (store in `.env` file):\n  - OPENAI_API_KEY\n  - GEMINI_API_KEY\n  - OPENROUTER_API_KEY\n  - DEEPSEEK_API_KEY\n  - ANTHROPIC_API_KEY\n\n## Recommendations\n\n### Customizing Model Selection\nThe default configuration in `wine_all.py` includes some large models that may not run on all systems. To adapt for your hardware:\n\n1. Edit `wine_all.py` to use models suitable for your system:\n   - For Ollama: Use smaller models like \"llama3.2\"\n   - For LM Studio: Stick to 3B-7B models with 4-bit quantization\n   - Cloud models (OpenAI, Anthropic, etc.) don't have local hardware requirements\n\n2. In individual provider files (e.g., `wine_ollama.py`), adjust model selections similarly\n\nExample model substitutions for lower-end hardware:\n- Replace \"qwen2.5:72b-instruct\" with \"llama3.2\"\n- Remove \"Llama3.3\" \nIn general feel free to add models available locally to Ollama or LM Studio.\n\n### Performance vs Resource Trade-offs\n- Smaller models (1B-3B) run faster but may have lower accuracy\n- Mid-size models (7B-14B) offer good balance of performance and resource usage\n- Largest models (\u003e30B) provide best accuracy but require significant resources\n\n## Dataset\n\nThe project uses the [Wine Reviews dataset](https://www.kaggle.com/datasets/zynicide/wine-reviews) from Kaggle. Download and place it in a `data` folder in your project directory.\n\n## Project Structure\n\n- `wine.ipynb` - Main Jupyter notebook with code and explanations\n- `wine_all.py` - Implementation using all providers\n\nIn the providers folder you will find the individual implementations for each provider.\n- `wine_openrouter.py` - Implementation using OpenRouter API\n- `wine_ollama.py` - Implementation using Ollama\n- `wine_gemini.py` - Implementation using Google Gemini\n- `wine_lmstudio.py` - Implementation using LM Studio\n- `wine_deepseek.py` - Implementation using DeepSeek\n- `wine_anthropic.py` - Implementation using Anthropic\n- `wine_openai.py` - Implementation using OpenAI (Structured)\n- `wine_openai_unstructured.py` - Implementation using OpenAI with unstructured processing.\n\n- `wine_mlx_omni_server.py` - Implementation using MLX Omni Server\n\n## Usage\n\n1. Clone the repository\n2. Install dependencies:\n   ```bash\n   pip install ollama numpy pandas tqdm pydantic\n   ```\n3. Set up your environment variables in `.env` file\n4. Download the dataset and place it in the `data` folder\n5. Run the Jupyter notebook or individual Python scripts\n\n### Running Individual Providers\n\nYou can run individual provider modules directly using Python's module syntax:\n\n```bash\n# Run MLX Omni Server provider\npython -m providers.wine_mlx_omni_server\n\n# Run Ollama provider\npython -m providers.wine_ollama\n\n# Run OpenAI provider (Structured)\npython -m providers.wine_openai\n\n# Run OpenAI Unstructured provider\npython -m providers.wine_openai_unstructured\n\n\n# Run Anthropic provider\npython -m providers.wine_anthropic\n\n# Run other providers similarly:\npython -m providers.wine_gemini\npython -m providers.wine_deepseek\npython -m providers.wine_lmstudio\npython -m providers.wine_openrouter\n\n```\n\nTo run all providers at once:\n```bash\npython wine_all.py\n```\n\nAvailable command-line options:\n```bash\npython wine_all.py --generate-chart                           # Generate chart from most recent results without running new tests\npython wine_all.py --generate-chart --summary SUMMARY_FILE    # Generate chart from specific summary file (e.g., summary_20250105_095642.csv)\npython wine_all.py --no-provider-csv                         # Run tests but don't save individual provider results to CSV files\n```\n\n## Fine-tuning models with MLX\nSee [LORA.md](LORA.md) for instructions on how to fine-tune models using LoRA with MLX.\n\n## TODO\n\n- [ ] Test google-genai python package for Gemini\n\n## DONE\n\n- [x] Try [MLX Omni Server](https://github.com/madroidmaq/mlx-omni-server) for Apple MLX tests\n- [x] Fine tune models with MLX with distillation (Phi-3.5-mini-instruct) \n\n## Contributing\n\nFeel free to open issues or submit pull requests with improvements.\n\n## License\n\n[MIT License](LICENSE)\n\n## Acknowledgments\n\n- Inspired by [OpenAI's model distillation cookbook](https://cookbook.openai.com/examples/leveraging_model_distillation_to_fine-tune_a_model)\n- Uses the Kaggle Wine Reviews dataset ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivanfioravanti%2Fwine_variety_classification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fivanfioravanti%2Fwine_variety_classification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivanfioravanti%2Fwine_variety_classification/lists"}