{"id":16817292,"url":"https://github.com/parths007/multimedia-information-retrieval","last_synced_at":"2026-07-03T10:05:46.122Z","repository":{"id":244663377,"uuid":"815897927","full_name":"ParthS007/Multimedia-Information-Retrieval","owner":"ParthS007","description":null,"archived":false,"fork":false,"pushed_at":"2025-07-05T17:58:05.000Z","size":56807,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-20T06:49:57.584Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ParthS007.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-06-16T13:14:19.000Z","updated_at":"2025-07-05T17:58:09.000Z","dependencies_parsed_at":"2024-06-16T14:46:38.725Z","dependency_job_id":"09ae1c69-01d8-4e33-b9e9-a351ee1a07bd","html_url":"https://github.com/ParthS007/Multimedia-Information-Retrieval","commit_stats":null,"previous_names":["parths007/multimedia-information-retrieval"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ParthS007/Multimedia-Information-Retrieval","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthS007%2FMultimedia-Information-Retrieval","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthS007%2FMultimedia-Information-Retrieval/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthS007%2FMultimedia-Information-Retrieval/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthS007%2FMultimedia-Information-Retrieval/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ParthS007","download_url":"https://codeload.github.com/ParthS007/Multimedia-Information-Retrieval/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthS007%2FMultimedia-Information-Retrieval/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35081268,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-03T02:00:05.635Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T10:46:46.285Z","updated_at":"2026-07-03T10:05:46.104Z","avatar_url":"https://github.com/ParthS007.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multimedia Information Retrieval\n\n_This repository contains code implementations for understanding information retrieval practically and applying NLP techniques._\n\n## 📚 Table of Contents\n\n- [Overview](#overview)\n- [Course Structure](#course-structure)\n- [Technologies Used](#technologies-used)\n- [Exercises](#exercises)\n  - [Sheet 0: Text Retrieval Fundamentals](#sheet-0-text-retrieval-fundamentals)\n  - [Sheet 1: Video Shot Detection Performance](#sheet-1-video-shot-detection-performance)\n  - [Sheet 2: Lucene-based Movie Search](#sheet-2-lucene-based-movie-search)\n  - [Sheet 3: Language Detection \u0026 Subword Tokenization](#sheet-3-language-detection--subword-tokenization)\n  - [Sheet 4: Semantic Search with Sentence Transformers](#sheet-4-semantic-search-with-sentence-transformers)\n  - [Sheet 5: Image Processing \u0026 Skin Detection](#sheet-5-image-processing--skin-detection)\n- [Setup \u0026 Installation](#setup--installation)\n- [Usage](#usage)\n- [License](#license)\n\n## 🎯 Overview\n\nThis repository is a comprehensive collection of practical exercises covering various aspects of **Multimedia Information Retrieval (MIR)**. The projects span across text retrieval, natural language processing, computer vision, and machine learning techniques applied to multimedia data.\n\nThe exercises demonstrate both **classical** and **modern** approaches to information retrieval, including:\n- Traditional text indexing and search (Apache Lucene)\n- Video shot boundary detection\n- Language detection algorithms\n- Semantic search using transformer models\n- Image processing and classification\n\n## 📖 Course Structure\n\nThe course is structured into 6 main exercise sheets (Sheet 0-5), each focusing on different aspects of multimedia information retrieval:\n\n## 🛠 Technologies Used\n\n- **Languages**: Python, Java\n- **Libraries \u0026 Frameworks**:\n  - **Python**: pandas, scikit-learn, matplotlib, seaborn, opencv-python, nltk, sentence-transformers, numpy\n  - **Java**: Apache Lucene, Apache Commons CSV\n- **Tools**: Jupyter Notebooks, BeautifulSoup, PIL (Python Imaging Library)\n- **Datasets**: IMDB movies, SimpleWiki, language detection datasets, skin detection datasets\n\n## 📋 Exercises\n\n### Sheet 0: Text Retrieval Fundamentals\n\n**Topics**: Classical text retrieval with Apache Lucene\n\n**Files**:\n- `classical_text_retrieval.ipynb` - Jupyter notebook with Lucene implementation\n- `Advanced Text Retrieval-NLP.pdf` - Theoretical background\n- `Classical Text Retrieval.pdf` - Course material\n- `Performance of Video Shot Detection.ipynb` - Video analysis implementation\n- `nlp_*.py` - Python scripts for NLP tasks\n\n**Key Concepts**:\n- Document indexing and retrieval\n- Boolean queries and scoring\n- Text analysis and tokenization\n- Search optimization techniques\n\n### Sheet 1: Video Shot Detection Performance\n\n**Topics**: Performance evaluation of video shot boundary detection algorithms\n\n**Files**:\n- `exercise_1.ipynb` - Main implementation notebook\n- `Exercise 1 - Improved.txt` - Improved algorithm results\n- `Exercise 1 - Naive.txt` - Naive algorithm results\n\n**Key Concepts**:\n- **Confusion Matrix Analysis**: Calculating TP, TN, FP, FN for shot detection\n- **ROC Curve Analysis**: Plotting and interpreting receiver operating characteristics\n- **Performance Metrics**: Sensitivity, specificity, accuracy, AUC calculations\n- **Threshold Optimization**: Finding optimal decision thresholds\n- **Comparative Analysis**: Naive vs improved detection algorithms\n\n### Sheet 2: Lucene-based Movie Search\n\n**Topics**: Building a search engine for movie data using Apache Lucene\n\n**Files**:\n- `CSVIndexer.java` - Lucene indexing implementation\n- `MovieSearch.java` - Search functionality\n- `imdb_top_1000.csv` - Movie dataset\n- `exercise_2.zip` - Complete project archive\n\n**Key Concepts**:\n- CSV data indexing\n- Fuzzy search implementation\n- Boolean query construction\n- Document scoring and ranking\n\n### Sheet 3: Language Detection \u0026 Subword Tokenization\n\n**Topics**: Text analysis, language detection, and similarity matching\n\n**Files**:\n- `exercise_3.ipynb` - Main implementation\n- `imdb.csv` - Movie titles dataset\n- `ld.csv` - Language detection dataset\n\n**Key Concepts**:\n- **Language Detection**: Using stopwords and NLTK for language identification\n- **Subword Tokenization**: N-gram based text analysis (2-grams, 3-grams, 4-grams)\n- **Similarity Calculation**: Jaccard similarity for text matching\n- **Semantic Search**: Using transformer models (SentenceTransformers)\n- **Performance Benchmarking**: Comparing different embedding models\n\n### Sheet 4: Semantic Search with Sentence Transformers\n\n**Topics**: Advanced semantic search using modern NLP techniques\n\n**Files**:\n- `exercise_4.ipynb` - Semantic search implementation\n- `data/simplewiki-2020-11-01.jsonl.gz` - Wikipedia dataset\n\n**Key Concepts**:\n- **Data Processing**: Parsing compressed JSON datasets\n- **Sentence Embeddings**: Using pre-trained transformer models\n- **Semantic Similarity**: Cosine similarity for sentence matching\n- **Context Expansion**: Enhancing search results with neighboring content\n- **Question Answering**: Building QA systems with context retrieval\n\n### Sheet 5: Image Processing \u0026 Skin Detection\n\n**Topics**: Computer vision and image classification\n\n**Files**:\n- `exercise_5.ipynb` - Image processing implementation\n- `face.jpg`, `MK.jpg` - Sample images\n- `result.png` - Processing results\n- `Skin_NonSkin.txt` - Skin detection dataset\n- `skin_dataset/` - Image classification dataset\n\n**Key Concepts**:\n- **Color Space Conversion**: BGR to XYZ transformations\n- **Principal Component Analysis (PCA)**: Dimensionality reduction for visualization\n- **Decision Tree Classification**: Machine learning for skin detection\n- **Image Preprocessing**: Resizing, normalization, and feature extraction\n- **Performance Evaluation**: Classification accuracy and model assessment\n\n## 🚀 Setup \u0026 Installation\n\n### Prerequisites\n\n- **Python 3.8+**\n- **Java 11+**\n- **Jupyter Notebook**\n\n### Python Dependencies\n\n```bash\npip install pandas numpy matplotlib seaborn scikit-learn opencv-python nltk sentence-transformers pillow unidecode tqdm\n```\n\n### Java Dependencies\n\nFor Lucene-based exercises, ensure you have:\n- Apache Lucene 9.7.0\n- Apache Commons CSV\n\n### NLTK Data\n\nDownload required NLTK data:\n```python\nimport nltk\nnltk.download('stopwords')\nnltk.download('punkt')\nnltk.download('punkt_tab')\n```\n\n## 💻 Usage\n\n### Running Jupyter Notebooks\n\n```bash\n# Navigate to the repository\ncd Multimedia-Information-Retrieval\n\n# Start Jupyter Notebook\njupyter notebook\n\n# Open any .ipynb file to run the exercises\n```\n\n### Running Java Applications\n\n```bash\n# Navigate to Sheet_2\ncd Sheet_2\n\n# Compile Java files\njavac -cp \".:lib/*\" *.java\n\n# Run the indexer\njava -cp \".:lib/*\" CSVIndexer\n```\n\n### Example Usage\n\n1. **Video Shot Detection Analysis** (Sheet 1):\n   ```python\n   # Load and analyze shot detection performance\n   python -c \"import pandas as pd; data = pd.read_csv('Sheet_1/Exercise 1 - Improved.txt', delimiter='\\t'); print(data.head())\"\n   ```\n\n2. **Movie Search** (Sheet 2):\n   ```bash\n   # Index movies and perform search\n   cd Sheet_2\n   java CSVIndexer\n   # Follow interactive prompts\n   ```\n\n3. **Language Detection** (Sheet 3):\n   ```python\n   # Detect language from text\n   from Sheet_3.exercise_3 import detect_language\n   result = detect_language(\"Hola mundo\", [\"english\", \"spanish\", \"french\"])\n   ```\n\n## 📊 Key Results \u0026 Insights\n\n- **ROC Analysis**: Improved shot detection algorithm achieved AUC of 0.9711 vs 0.9420 for naive approach\n- **Semantic Search**: Sentence transformers significantly outperform traditional n-gram approaches\n- **Language Detection**: Stopword-based detection achieves high accuracy for European languages\n- **Image Classification**: PCA visualization reveals clear clustering of skin vs non-skin samples\n\n## 🤝 Contributing\n\nThis repository is primarily for educational purposes. If you find issues or have improvements:\n\n1. Fork the repository\n2. Create a feature branch\n3. Submit a pull request\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n---\n\n**Note**: This repository contains coursework for academic purposes. Datasets and some implementations may require appropriate citations when used in research or commercial applications.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparths007%2Fmultimedia-information-retrieval","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparths007%2Fmultimedia-information-retrieval","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparths007%2Fmultimedia-information-retrieval/lists"}