An open API service indexing awesome lists of open source software.

https://github.com/prajwalsrinvas/advent_of_haystack_2024

My solutions for Advent of Haystack 2024 challenges by Deepset.
https://github.com/prajwalsrinvas/advent_of_haystack_2024

advent-of-code haystack llm rag

Last synced: 4 months ago
JSON representation

My solutions for Advent of Haystack 2024 challenges by Deepset.

Awesome Lists containing this project

README

        

# Advent of Haystack 2024 Solutions 🎄

My solutions for [Advent of Haystack 2024](https://haystack.deepset.ai/advent-of-haystack) challenges by Deepset.

Read more about the event in their [announcement blog](https://haystack.deepset.ai/blog/announcing-advent-2024).

## What is Advent of Haystack?

Advent of Haystack 2024 is a festive coding challenge series organized by Deepset, creators of Haystack - an open-source framework for building production-ready AI applications. The event features:

- 10 holiday-themed challenges throughout December 2024
- Focus on practical Generative AI use cases and RAG applications
- Integration with industry-leading tools through partnerships with:
- Weaviate (Vector Database)
- AssemblyAI (Audio/Speech Models)
- NVIDIA (NGC)
- Arize (Phoenix)
- MongoDB (Atlas)

Each challenge is designed to showcase different aspects of building AI applications with Haystack, from basic pipelines to advanced use cases, making it suitable for both beginners and experienced AI developers.

## Solutions

| Day | Challenge | Concepts | NBSanity | Colab | GitHub |
|-----|-----------|-----------|----------|--------|---------|
| 1 | [Fetching Holiday Wisdom for Santa 📖](https://haystack.deepset.ai/advent-of-haystack/day-1#challenge) | - LinkContentFetcher for URL content extraction
- HTML to Document conversion
- Document chunking & ranking
- RAG pipeline orchestration | [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_1_Enhancing_Pipeline.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_1_Enhancing_Pipeline.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_1_Enhancing_Pipeline.ipynb) |
| 2 | [Help the Elves Find What They've Lost 🎄](https://haystack.deepset.ai/advent-of-haystack/day-2#challenge) | - Weaviate vector store integration
- Document storage and retrieval
- Vector search with metadata filtering | [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_2_Weaviate_Day.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_2_Weaviate_Day.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_2_Weaviate_Day.ipynb) |
| 3 | [Making Answers Bright with Multi-Query RAG Magic ✨](https://haystack.deepset.ai/advent-of-haystack/day-3#challenge) | - Multi-query retrieval strategies
- Custom component creation
- Query expansion techniques| [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_3_Multi_Query_Retrieval.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_3_Multi_Query_Retrieval.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_3_Multi_Query_Retrieval.ipynb) |
| 4 | [Sharing the Christmas Magic with Gen Z 🎶](https://haystack.deepset.ai/advent-of-haystack/day-4#challenge) | - AssemblyAI integration
- Speech-to-text conversion
- Text summarization
- Content style adaptation | [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_4_AssemblyAI.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_4_AssemblyAI.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_4_AssemblyAI.ipynb) |
| 5 | [Elves' Secret for Faster Development 💨](https://haystack.deepset.ai/advent-of-haystack/day-5#challenge) | - Deepset Studio utilization
- RAG pipeline templates
- Visual pipeline building
- Model deployment workflows | [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_5_Deepset_Studio.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_5_Deepset_Studio.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_5_Deepset_Studio.ipynb) |
| 6 | [Keep the Holiday Spirit Alive 🛷](https://haystack.deepset.ai/advent-of-haystack/day-6#challenge) | - NVIDIA NIM microservices
- NvidiaRanker implementation
- NvidiaTextEmbedder usage
- DocumentMRREvaluator | [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_6_NVIDIA_NIM.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_6_NVIDIA_NIM.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_6_NVIDIA_NIM.ipynb) |
| 7 | [Judging Toys, Tracing Joy 🧑‍⚖️](https://haystack.deepset.ai/advent-of-haystack/day-7#challenge) | - Arize Phoenix integration
- Pipeline tracing & monitoring
- LLM-as-Judge evaluation
- Performance metrics tracking | [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_7_Arize_Phoenix.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_7_Arize_Phoenix.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_7_Arize_Phoenix.ipynb) |
| 8 | [Ultimate Inventory Agent to Control Holiday Chaos 🛠️ 🎄](https://haystack.deepset.ai/advent-of-haystack/day-8#challenge) | - Agent implementation
- Tool creation & orchestration
- OpenAIChatGenerator usage
- ToolInvoker implementation | [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_8_Agents_and_Tools.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_8_Agents_and_Tools.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_8_Agents_and_Tools.ipynb) |
| 9 | [Santa's Self-Reflecting Gift Agent 🎁](https://haystack.deepset.ai/advent-of-haystack/day-9#challenge) | - MongoDB Atlas vector search
- Self-reflecting agent design
- Meta field embeddings | [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_9_Haystack_MongoDB.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_9_Haystack_MongoDB.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_9_Haystack_MongoDB.ipynb) |
| 10 | [Jingle Metrics All the Way 🔔](https://haystack.deepset.ai/advent-of-haystack/day-10#challenge) | - RAGEvaluationHarness usage
- Pipeline evaluation metrics
- FaithfulnessEvaluator
- Hybrid retrieval optimization | [![Open In NBSanity](https://nbsanity.com/assets/icon.png)](https://nbsanity.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_Ten_Evaluation_Harness.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_Ten_Evaluation_Harness.ipynb) | [![GitHub](https://cdn-icons-png.flaticon.com/32/270/270798.png)](https://github.com/Prajwalsrinvas/advent_of_haystack_2024/blob/main/Day_Ten_Evaluation_Harness.ipynb) |