{"id":25939246,"url":"https://github.com/raphaelmos/movie-scrapper","last_synced_at":"2025-08-07T22:36:06.304Z","repository":{"id":279881671,"uuid":"940311354","full_name":"Raphaelmos/movie-scrapper","owner":"Raphaelmos","description":"Movie reviews scrapper","archived":false,"fork":false,"pushed_at":"2025-02-28T01:10:23.000Z","size":2803,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-28T09:10:36.965Z","etag":null,"topics":["movie","python3","review-scraper"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Raphaelmos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-28T00:45:07.000Z","updated_at":"2025-02-28T01:46:26.000Z","dependencies_parsed_at":"2025-02-28T09:10:41.819Z","dependency_job_id":"44e31ed9-95c6-45f4-9a70-7d77a9465a5f","html_url":"https://github.com/Raphaelmos/movie-scrapper","commit_stats":null,"previous_names":["raphaelmos/movie-scrapper"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Raphaelmos%2Fmovie-scrapper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Raphaelmos%2Fmovie-scrapper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Raphaelmos%2Fmovie-scrapper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Raphaelmos%2Fmovie-scrapper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Raphaelmos","download_url":"https://codeload.github.com/Raphaelmos/movie-scrapper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241780495,"owners_count":20019061,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["movie","python3","review-scraper"],"created_at":"2025-03-04T04:16:05.217Z","updated_at":"2025-03-04T04:16:06.008Z","avatar_url":"https://github.com/Raphaelmos.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Movie Scrapper\n\n## Overview\nThe Movie Scrapper is a Python-based project designed to scrape movie reviews and perform sentiment analysis on them. This repository contains scripts that fetch reviews from specified URLs, analyze the sentiments of the reviews, and store the results in CSV files for further analysis.\n\n## Features\n- **Review Scraping**: Extracts movie reviews from specified websites.\n- **Sentiment Analysis**: Uses a trained neural network model to classify reviews as positive or negative.\n- **Data Storage**: Saves scraped reviews and sentiment results into CSV files for easy access and analysis.\n\n## Files Included\n- `scrappresse.py`: Script for scraping reviews from a specific movie review site, handling pagination and storing data in a CSV file.\n- `scrapes.py`: A more structured script for scraping reviews, designed to avoid duplicates and manage multiple URLs effectively. Outputs data to a CSV file.\n- `predic.py`: Implements a sentiment analysis model using PyTorch. Trains on predefined datasets and predicts sentiments of scraped reviews.\n\n## Requirements\nTo run this project, you need the following Python packages installed:\n- `requests`\n- `beautifulsoup4`\n- `torch`\n- `pandas`\n- `scikit-learn`\n\nYou can install the necessary packages using pip:\n\n```bash\npip install requests beautifulsoup4 torch pandas scikit-learn\n```\n## Usage\n1. **Set Up URLs**: Create a file named `url.txt` in the root directory and list the URLs of movie reviews you want to scrape.\n2. **Run the Scraping Scripts**: Execute either `scrappresse.py` or `scrapes.py` to begin scraping reviews.\n   ```bash\n   python scrappresse.py\n   ```\n   ```bash\n   python scrapes.py\n   ```\n3. **Train the Sentiment Model**: Run `predic.py` to train the sentiment analysis model using the predefined dataset.\n   ```bash\n   python predic.py\n   ```\n4. **Analyze Results**: The results will be saved in `criticrate.csv` and `films-critic.csv` for your review.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraphaelmos%2Fmovie-scrapper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fraphaelmos%2Fmovie-scrapper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraphaelmos%2Fmovie-scrapper/lists"}