{"id":20931451,"url":"https://github.com/pramodkondur/redbusdatascrapinganddynamicfiltering","last_synced_at":"2026-04-11T07:45:13.695Z","repository":{"id":248125797,"uuid":"827808353","full_name":"pramodkondur/RedbusDataScrapingandDynamicFiltering","owner":"pramodkondur","description":"This project leverages Selenium for web scraping from the RedBus Website, Python for data manipulation, SQL for data storage, and Streamlit for creating an interactive application to visualize and filter the data.","archived":false,"fork":false,"pushed_at":"2024-07-16T06:46:52.000Z","size":14,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-30T14:35:39.592Z","etag":null,"topics":["data-visualization","python","selenium","sql","streamlit","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pramodkondur.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-12T12:22:07.000Z","updated_at":"2025-01-26T03:45:12.000Z","dependencies_parsed_at":"2024-07-16T09:12:37.038Z","dependency_job_id":null,"html_url":"https://github.com/pramodkondur/RedbusDataScrapingandDynamicFiltering","commit_stats":null,"previous_names":["pramodkondur/redbusdatascrapinganddynamicfiltering"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pramodkondur/RedbusDataScrapingandDynamicFiltering","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pramodkondur%2FRedbusDataScrapingandDynamicFiltering","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pramodkondur%2FRedbusDataScrapingandDynamicFiltering/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pramodkondur%2FRedbusDataScrapingandDynamicFiltering/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pramodkondur%2FRedbusDataScrapingandDynamicFiltering/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pramodkondur","download_url":"https://codeload.github.com/pramodkondur/RedbusDataScrapingandDynamicFiltering/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pramodkondur%2FRedbusDataScrapingandDynamicFiltering/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31673067,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-10T17:19:37.612Z","status":"online","status_checked_at":"2026-04-11T02:00:05.776Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-visualization","python","selenium","sql","streamlit","web-scraping"],"created_at":"2024-11-18T21:43:09.433Z","updated_at":"2026-04-11T07:45:13.675Z","avatar_url":"https://github.com/pramodkondur.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Redbus Data Scraping with Selenium \u0026 Dynamic Filtering using Streamlit\n\n## Overview\n\nThe **Redbus Data Scraping and Filtering with Streamlit Application** project is designed to transform the transportation industry by automating the collection, analysis, and visualization of bus travel data. This project leverages Selenium for web scraping, Python for data manipulation, SQL for data storage, and Streamlit for creating an interactive application to visualize and filter the data.\n\n## Skills and Technologies\n\n- **Web Scraping**: Selenium\n- **Programming Language**: Python\n- **Data Analysis \u0026 Visualization**: Streamlit\n- **Data Storage**: SQL\n- **Domain**: Transportation\n\n## Problem Statement\n\nThe project aims to streamline the process of collecting and analyzing bus travel information from the Redbus website. By automating data extraction and providing a user-friendly interface for data exploration, the project seeks to improve operational efficiency, support market analysis, and enhance customer service in the transportation industry.\n\n## Business Use Cases\n\nThis solution can be applied in various scenarios including:\n\n- **Travel Aggregators**: Offering real-time bus schedules and seat availability for customers.\n- **Market Analysis**: Studying travel patterns and preferences for strategic planning.\n- **Customer Service**: Providing customized travel options and improving user experience.\n- **Competitor Analysis**: Comparing pricing and service levels with other providers.\n\n## Approach\n\n### Data Scraping\n\n- **Tool**: Selenium\n- **Objective**: Automate the extraction of detailed bus travel information from Redbus.\n- **Data Collected**: Bus routes, schedules, prices, seat availability, etc.\n\n### Data Storage\n\n- **Tool**: SQL\n- **Objective**: Store scraped data in a structured SQL database for efficient querying and analysis.\n\n### Streamlit Application\n\n- **Tool**: Streamlit\n- **Objective**: Develop an interactive web application to filter and display the scraped data.\n- **Features**:\n  - Filters for bus type, route, price range, star rating, and availability.\n  - Data visualization and interactive user interface.\n\n### Data Analysis/Filtering\n\n- **Tool**: SQL queries within Streamlit\n- **Objective**: Retrieve and filter data based on user inputs.\n\n## Results\n\nThe project aims to achieve the following objectives:\n\n- **Scrape Data**: Collect data from at least 10 Government State Bus Transport routes from the Redbus website, along with private bus information.\n- **Store Data**: Organize data in a structured SQL database.\n- **Develop Application**: Create a functional and user-friendly Streamlit application for data filtering.\n- **User Experience**: Ensure the application is intuitive and responsive.\n\n## Project Evaluation Metrics\n\n- **Data Scraping Accuracy**: Completeness and correctness of the scraped data.\n- **Database Design**: Effectiveness of the database schema for data storage and retrieval.\n- **Application Usability**: User experience and functionality of the Streamlit application.\n- **Filter Functionality**: Efficiency and responsiveness of the data filters.\n- **Code Quality**: Adherence to coding standards and best practices.\n\n## Technical Tags\n\n- Web Scraping\n- Selenium\n- Streamlit\n- SQL\n- Data Analysis\n- Python\n- Interactive Application\n\n## Data Set\n\n- **Source**: Redbus website [https://www.redbus.in/](https://www.redbus.in/)\n- **Format**: SQL database\n- **Fields**:\n  - **Bus Routes Name**: Start and end locations of the bus journey.\n  - **Bus Routes Link**: Detailed link for the route.\n  - **Bus Name**: Name of the bus or service provider.\n  - **Bus Type**: Type of bus (e.g., Sleeper, Seater, AC, Non-AC).\n  - **Departing Time**: Scheduled departure time.\n  - **Duration**: Total journey time.\n  - **Reaching Time**: Expected arrival time.\n  - **Star Rating**: Passenger rating of the bus service.\n  - **Price**: Ticket price.\n  - **Seat Availability**: Number of available seats.\n\n## Database Schema\n\n| Column Name      | Data Type | Description                                      |\n|------------------|----------|--------------------------------------------------|\n| id               | INT      | Primary Key (Auto-increment)                    |\n| route_name       | TEXT     | Bus Route information                           |\n| route_link       | TEXT     | Link to the route details                        |\n| busname           | TEXT     | Name of the bus                                 |\n| bustype          | TEXT     | Type of the bus (Sleeper/Seater/AC/Non-AC)      |\n| departing_time   | DATETIME | Departure time                                   |\n| duration         | TEXT     | Duration of the journey                          |\n| reaching_time    | DATETIME | Arrival time                                     |\n| star_rating      | FLOAT    | Rating of the bus                                |\n| price            | DECIMAL   | Ticket price                                     |\n| seats_available  | INT      | Number of seats available                       |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpramodkondur%2Fredbusdatascrapinganddynamicfiltering","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpramodkondur%2Fredbusdatascrapinganddynamicfiltering","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpramodkondur%2Fredbusdatascrapinganddynamicfiltering/lists"}