Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

Projects in Awesome Lists tagged with beautifulsoup

A curated list of projects in awesome lists tagged with beautifulsoup .

https://github.com/jhleeeme/todays-brunch

Today's brunch scraping & Issue posting (with github-actions)

beautifulsoup github-actions python3 scraper selenium

Last synced: 17 Jan 2025

https://github.com/prash5t/scrap-election-commission-nepal

Scraps voter information from Election Commission Nepal's website

beautifulsoup beautifulsoup4 csv data-extraction nepal

Last synced: 07 Jan 2025

https://github.com/antodata/webscraperallmusic

Simple example of a web scrapper using python. In this case, we ask the user using the console for the name of a band/artist and using selenium webdriver and beautifulsoup we print information about the discography of that artist/band

allmusic beautiful-soup beautifulsoup beautifulsoup4 chrome-webdriver discography html html-parser html-parsing music music-information-retrieval parser python python3 selenium selenium-python selenium-webdriver webdriver webscraper webscraping

Last synced: 08 Dec 2024

https://github.com/antodata/on_page_basic_seo_checker

This project provides methods and utils to make basic checks in the SEO of an instance of a page using the URL of this page or a webdriver instance that is browsing that page at the moment

beautifulsoup beautifulsoup-library beautifulsoup4 decorators html html-parser python python3 qa qa-checks qa-tools requests requests-html selenium seo seo-analysis seo-report seo-tags seotools webdriver

Last synced: 08 Dec 2024

https://github.com/imsahil007/gsoc-scraper

Script for creating a simple Html page filtering orgs on basis of tech stack

beautifulsoup gsoc gsoc-2020 payload payload-scraping python3 rich websc webscraper

Last synced: 24 Dec 2024

https://github.com/vgvr0/supermarket-mercadona-scraper

Supermarket-Mercadona-Scraper is an efficient and user-friendly tool that automatically scrapes and organizes product data from the Mercadona website, enabling you to analyze and compare product prices and features in real-time.

beautifulsoup chromedriver mercadona mercadona-scraper mercadona-scraping python-scraping selenium

Last synced: 26 Nov 2024

https://github.com/hanan-nawaz/daraz-price-comparator

Daraz Price Comparator 🌐🛍️ Compare prices effortlessly! 🔄 Find deals in 🇵🇰 Pakistan, 🇧🇩 Bangladesh, 🇳🇵 Nepal, and 🇱🇰 Sri Lanka. Python, Requests, BeautifulSoup, and Airflow power smart shopping. Save time, shop savvy! 🛒💡

beautifulsoup bs4 darazpk python requests-python telemart webscraping

Last synced: 28 Nov 2024

https://github.com/rohan-bhautoo/python-web-scraper

A python web scaper to extract content and data from a website.

beautifulsoup python python2 scraping webscraper

Last synced: 20 Jan 2025

https://github.com/bijoy-sust/simple-web-scraping-in-python

A list of resources and introductory notebooks for Web Scraping in Python using BeautifulSoup.

beautifulsoup machine-learning notebook python-3 web-scraping

Last synced: 25 Nov 2024

https://github.com/mousazourob/essayresearcher

A website that streamlines the research process by showing articles with excerpts based on a particular topic and group of keywords

beautifulsoup bootstrap css flask html javascript jquery python web-scraping

Last synced: 22 Dec 2024

https://github.com/irfanchahyadi/scraping-manga

Scrape manga with python + BeautifulSoup

beautifulsoup flask manga python scraping web-gui

Last synced: 27 Dec 2024

https://github.com/jwc20/waasuapi

A scraper API for fetching available jobs from YCombinator's workatastartup.com

beautifulsoup jobsearch python scraper scraper-api selenium

Last synced: 25 Nov 2024

https://github.com/nemeslaszlo/product_pricetracking

Simple script to check a product price, if the price went down to our border price send a mail about it and you can check it. (Beautiful Soup usage and practise)

beautifulsoup mail product-price product-pricetracking requests smtplib

Last synced: 01 Dec 2024

https://github.com/somada141/mangatown-downloader

Simple scripts that download mangatown manga chapters through the URL of the first page.

beautifulsoup compression downloader manga mangatown python requests scraper zip

Last synced: 19 Jan 2025

https://github.com/imvickykumar999/dark-web-scraping

https://www.upgrad.com/learn/

beautifulsoup bs4 webscraping

Last synced: 01 Dec 2024

https://github.com/safaa-p/scraping-linkedin

Scraping LinkedIn using Beautiful Soup and Selenium libraries,.

beautifulsoup linkedin nlp-machine-learning scraping selenium

Last synced: 16 Jan 2025

https://github.com/kevinam99/callie

Web implementation of the Instagram Like Bot in Flask. I though of calling it Renee since my previous project is reborn through this. However, I have a fetish for names ending with the "e" sound (yes, the syllable).

beautifulsoup bot flask flask-application instagram python selenium selenium-python webscraper webscraping

Last synced: 12 Jan 2025

https://github.com/samridhisainii/scrap-it

Scrapper for magicbricks.com

beautifulsoup python selenium web-scraper

Last synced: 29 Nov 2024

https://github.com/naveenaidu/ieee-webscraper

A cli Scrapping tool to gather all the IEEE Journals

beautifulsoup ieee scrapping-python webscraper webscraper-website

Last synced: 18 Jan 2025

https://github.com/michaelsdavid/xss_vulnerability_scanner

Capstone project from the Python and Ethical Hacking course by ZSecurity

beautifulsoup ethical-hacking python python27 vulnerability-scanner xss xss-vulnerability

Last synced: 19 Jan 2025

https://github.com/adriantomin/email-scraper

📧 A Python-based web scraping tool that recursively collects email addresses from websites. 🌐 It follows links to multiple pages and extracts emails using BeautifulSoup and regex.

automation beautifulsoup emailextractor python regex requests webscraping

Last synced: 13 Jan 2025

https://github.com/elizabethsiegle/parse-html-for-top-goodreads-quotes-with-twilio-whatsapp

Parse HTML of Goodreads quotes page with Twilio's WhatsAPP API https://www.twilio.com/whatsapp/api

beautifulsoup python3 twilio whatsapp-api

Last synced: 05 Dec 2024

https://github.com/barbaracalderon/web-scrapping-with-python

Study notes on Web Scrapping using Python, Beautiful Soup and Requests. Notas de estudo sobre Web Scrapping usando Python, Beautiful Soup e Requests.

beautifulsoup python requests scraper scraping

Last synced: 28 Nov 2024

https://github.com/elizabethsiegle/cf-ai-lora-news-summarizer

Python webapp that summarizes news with Cloudflare Workers AI LoRA, Mistral, Beautifulsoup, and Streamlit

beautifulsoup beautifulsoup4 cloudflare cloudflare-ai cloudflare-workers cloudflare-workers-ai lora mistral-7b python streamlit

Last synced: 05 Dec 2024

https://github.com/udityamerit/python-librearies-for-data-science

Python libraries for data science enable efficient data manipulation, analysis, and modeling. Key libraries include NumPy for numerical computing, pandas for data handling, Matplotlib for visualization, Scikit-learn for machine learning, TensorFlow for deep learning, and BeautifulSoup/requests for web scraping. These libraries simplify complex data

beautifulsoup data data-science data-science-libraries machine-learning matplotlib numpy pandas requests scikit-learn scikitlearn-machine-learning tensorflow

Last synced: 08 Nov 2024

https://github.com/programmer-rd-ai/moviedatascraper

Explore the cinematic universe with our IMDb web scraping project! Dive into movie data with ease, uncovering insights from cast to critical reviews. With dynamic visualizations and reliable data, let's journey through the world of movies like never before. Lights, camera, analysis!

beautifulsoup beautifulsoup4 data data-analysis jupyter-notebook matplotlib numpy pandas programming python python3 scraping seaborn software web

Last synced: 12 Jan 2025

https://github.com/raedaddala/scraping-imdb

This Python script extracts comprehensive movie data from IMDB, focusing on top-grossing movies from 1960 to 2024. The scraper collects detailed information including box office performance, cast & crew, awards, and other key metrics.

beautifulsoup crawling data-mining dataset imdb imdb-dataset imdb-webscrapping movies python python-scraper scraping selenium webscraping

Last synced: 03 Jan 2025

https://github.com/cunningdj/webscrape

Playing with web scraping. screen_scraper main() method allows image harvesting, linkedin parse_url() grabs job descriptions, indeed accesses the Indeed API. Use politely.

beautifulsoup python scrape scraper scraping scraping-websites

Last synced: 26 Dec 2024

https://github.com/gursv/url-summ

A URL summarizer, which summarizes the content of a URL with proper formatting. It uses 'sshleifer/distilbart-cnn-12-6', which is a distilled version of the BART model, specifically optimized for text summarization tasks, including CNN summarization.

ai beautifulsoup chunking formatted-text huggingface-models python3 smtp star-rating streamlit text-extraction text-summarization transformers url-summarization

Last synced: 26 Dec 2024

https://github.com/faizanmohd5/web-scraping-iphone-11-reviews

This is a web scraping project that extracts customer reviews for the iPhone 11 from Flipkart.com using Python and BeautifulSoup. The extracted data is saved in a CSV file for further analysis. Use it as a starting point for your own web scraping projects or for analyzing customer reviews of the iPhone 11.

beautifulsoup csv data-visualization dataanalysis dataextraction datainsights datamining datapreprocessing ecommerce-website ipython-notebook jupyter-notebook python reviews reviewscrapper webscraping

Last synced: 12 Jan 2025

https://github.com/adityasworks/websudoku-solver

This Python script automates solving Sudoku puzzles from websudoku.com using browser automation with Selenium and data extraction with BeautifulSoup.

automation beautifulsoup python selenium sudoku sudoku-solver web-scraping webscraping

Last synced: 17 Jan 2025

https://github.com/easonlai/yahoo_news_searching_and_text_analytics

This is a demo repo to demonstrate how to use Python Requests and Beautiful Soup 4 libraries to search and scrape result data. And save scraping results into Pandas dataframe.

azure azure-cognitive-services azure-text-analytics beautifulsoup beautifulsoup4 microsoft-cognitive-services news-scraper news-scraping newsscraper python python3 requests sentiment-analysis web-scraping webscraping yahoo

Last synced: 10 Nov 2024

https://github.com/dmarks84/coursework_project_banks-web-scraping-sql

Project for IBM Data Engineering & Python course on ETL & Big Data -- Scraped website data and made API calls for additional data; wrangled and transformed this data and loaded into a SQL database.

apis beautifulsoup databases elt etl nosql numpy pandas pipelines python sql sqlite web-scraping

Last synced: 23 Dec 2024

https://github.com/ljnath/getdomainage

getDomainAge is a web application which can provide the age of a given domain name. The intention of this project is to demonstrate flask for API development, SQLAlchemy for ORM and beautifulsoup for HTML parsing.

beautifulsoup docker domain flask flask-api flask-application flask-restful flask-server flask-sqlalchemy flask-web job-queue job-scheduler producer-consumer python python3 scraping-websites sqlalchemy sqlalchemy-database sqllite whois

Last synced: 10 Jan 2025

https://github.com/trallard/webdata_python

Course materials for working with webdata in Python. Originally developed for a 1 full-day course at the Methods Institute @Sheffield

api beautifulsoup json python requests webscraper website xml

Last synced: 09 Nov 2024

https://github.com/cleversamer/imdb-scraping

A python scraping bot for extracting data from IMDB website and write it to an Excel file

beautifulsoup excel python script web-scraping

Last synced: 14 Jan 2025

https://github.com/idanhoro/nasa-heat-maps-prediction

In this project we research the correlations between different weather conditions and try to predict future scenarios by using image processing and traditional machine learning algorithms

beautifulsoup crawler machine-learning pillow prediction python sklearn

Last synced: 20 Jan 2025

https://github.com/carloocchiena/python_url_crawler

A script that starting from a webpage, iterate thru all its link, appending them in a list. Sort of proxy to get all pages in a website

beautifulsoup crawler python python3

Last synced: 28 Nov 2024

https://github.com/pydevcasts/resumerover

This project is a tool for extracting emails from websites. Users can input a list of links, and the tool will gather emails from those pages. It also allows users to send resumes to the extracted emails

asyncio beautifulsoup bs4 cache email-sender email-template job-scheduler jobsearch oop python request resume-template selenium threading unittest

Last synced: 11 Nov 2024

https://github.com/henrylin03/video-games

Using Python and SQL to clean, analyse and visualise video games' data from Metacritic. Includes scraping using BeautifulSoup.

analysis beautifulsoup beautifulsoup4 data data-analysis data-science eda jupyter-notebook pandas python sql sqlite3 video-game video-games

Last synced: 14 Jan 2025

https://github.com/keosariel/ramby

Ramby is a simple way to setup a webscraper

beautifulsoup crawler python3 webscraping

Last synced: 06 Dec 2024

https://github.com/rahulvictor12/the-movie-database-data-scrapper

A Python web scraper that collects movie data from The Movie Database (TMDB). It uses `requests`, `BeautifulSoup`, and `pandas` to extract titles, ratings, genres, and cast details from multiple pages. The data is structured into DataFrames and saved as a CSV, perfect for analysis or integration into projects.

beautifulsoup colab-notebook dataframes numpy pandas python requests testing webscraping

Last synced: 30 Dec 2024

https://github.com/vincent-g-van/timed-web-scraper

Script takes in a url, class type, and class name and uses this information to scrape a list. In time intervals, the script will check to see if the site has updated that particular list.

beautifulsoup timer webscraper

Last synced: 03 Dec 2024

https://github.com/r00tman/gmailnotify

Simple GMail new message notifications through Atom feed

beautifulsoup gmail libnotify linux python requests

Last synced: 09 Dec 2024

https://github.com/nanorobocop/beautifulsoup-for-confluence

Patch for BeautifulSoup to work with Confluence's namespaced tags and attributes

beautifulsoup confluence docker python3 xml

Last synced: 09 Dec 2024

https://github.com/kenjyco/parse-helper

Helpers to fetch & parse text on pages with requests, lxml, & beautifulsoup4

beautifulsoup cli duckduckgo kenjyco lxml parse python requests

Last synced: 28 Nov 2024

https://github.com/tj2904/pp-api

An API providing sentiment-scored news for PositivePress using the Vader NLP algorithm. A consuming frontend can be found at: https://github.com/tj2904/positive-press

beautifulsoup fastapi feedparser nltk vader-sentiment-analysis

Last synced: 28 Dec 2024

https://github.com/nelcifranmagalhaes/web_crawler

A web crawler for all Naruto characters

anime beautifulsoup characters crawler naruto python

Last synced: 03 Dec 2024

https://github.com/luissevillano/scraping-examples

A couple of scraping examples in Python

beautifulsoup python scraping scrapy

Last synced: 30 Dec 2024

https://github.com/abhimanyu-gaurav/webscraping

WebScraping using Beautiful Soup .

beautifulsoup bs4 html5 python requests

Last synced: 20 Jan 2025

https://github.com/antodata/coronaviruswebscraper

Web scraper that gets information about the corona virus spread from https://www.worldometers.info/coronavirus/ and world population in from https://www.worldometers.info/world-population/population-by-country/ and generates three maps with folium that represent the ratio between cases and population, between recovered people and cases and between death people and cases using circles

beautifulsoup big-data bs4 corona-virus covid-19 covid-19-tracker covid19 covid19-data data-science folium html-parser maps pandas python python-requests python3 requests web-scraper web-scraping webscraper

Last synced: 08 Dec 2024

https://github.com/hjsblogger/async-io-python

Demonstration of asyncio in Python using a string of different usecases (or test scenarios)

asyncio automation beautifulsoup beautifulsoup4 python-asynchronous python-asyncio requests selenium selenium-python web-scraping webscraping

Last synced: 11 Oct 2024

https://github.com/dmickelson/newssentimentanalyzerproject

Analyzes top current headlines from major RSS news feed, Uses DistilBERT model fine-tuned for sentiment analysis. Provides a GradioUI for ease of use.

beautifulsoup gradio huggingface logging pinecone pytest python pytorch rss sentiment-analysis tqdm webscraping yaml

Last synced: 27 Nov 2024

https://github.com/cau777/weather-app

A service made with Flask and Python to help you find the weather of your favorite cities.

beautifulsoup flask jinja2 python weather weather-api weather-app website

Last synced: 02 Dec 2024

https://github.com/aurelienmorgan/french_text_sentiment

Sentiment Analysis in texts written in French language using Tensorflow/Keras (and using XGBoost for hyperparameters optimization)

beautifulsoup dask fasttext french gru hyperparameters-optimization jupyter-notebook keras multiprocessing nlp python rnn scikit-learn sentiment-analysis tensorflow transfer-learning web-scraping xgboost

Last synced: 15 Dec 2024

https://github.com/flaiers/github-members

The tabular view of members at GitHub

beautifulsoup django github

Last synced: 26 Nov 2024

https://github.com/ubeydgur/job-posting-scraper

Scraping job postings from the website with BeautifulSoup.

beautifulsoup excel pandas python requests scraper web-scraping

Last synced: 13 Jan 2025

https://github.com/baxi19/python-web-scraping

Python example of web scraping :alien:

autopep8 beautifulsoup python

Last synced: 14 Jan 2025

https://github.com/kenwuqianghao/internship-opportunity-web-scraper

A web scraper that finds opportunities and internships for highschoolers

beautifulsoup python webscraper

Last synced: 27 Dec 2024

https://github.com/rookedsysc/curriculum-scraping

Curriculum Scraping is a tool for scraping (crawling) curriculum data from online education platforms.

beautifulsoup python scraping scrapper selenium

Last synced: 22 Nov 2024

https://github.com/walidhabbach/car-rental-desktop-app

this project is a user-friendly car rental management system that simplifies vehicle renting. It offers essential features for tracking availability, managing bookings, and more !

beautifulsoup pyqt5 python

Last synced: 22 Nov 2024

https://github.com/mengyaohuang/web-scraping-in-python

Use BeautifulSoup/Selenium to extract IMDb director information and analyze

beautifulsoup imdb-webscrapping python3

Last synced: 05 Dec 2024

https://github.com/gourab-sinha/instabot

This project is all about web scraping using Selenium, BeatifulSoup and Python Packages.

beautifulsoup matplotlib numpy pandas python3 selenium-python

Last synced: 25 Nov 2024

https://github.com/mittal-parth/freadom

A simple tool to scrape articles. Works even on websites that dont allow users to read the content without a login.

beautifulsoup css django html python scraping web-development

Last synced: 21 Dec 2024

https://github.com/canayter/web-scraping-data-collection

Identifying id and class attributes from websites, and use this knowledge to extract information via automated browsing with Splinter and HTML parsing with Beautiful Soup.

beautifulsoup css html webscraping

Last synced: 09 Jan 2025

https://github.com/sayakpaul/scraping-datacamp-courses-and-projects

Contains code to scrape the names of DataCamp courses and projects and gather them in .csv files

beautifulsoup jupyter-notebooks python web-scraping

Last synced: 13 Jan 2025

https://github.com/pregismond/python-project-for-data-engineering

Final Project Submission: Acquiring and processing information on world's largest banks

beautifulsoup coursera data-engineering etl ibm-skills-network jupyter-notebook python sqlite

Last synced: 07 Dec 2024

https://github.com/gamemann/web-extract-with-chatgpt

A Python application that extracts web data using Selenium and BeautifulSoup, sends the web data through the ChatGPT API, and outputs the ChatGPT response somewhere.

api beautifulsoup beautifulsoup4 bs4 chatgpt discourse extract post python request selenium web-extractor web-scrape web-scraper

Last synced: 21 Dec 2024

https://github.com/ejw-data/web-scraping-proteins

Webscrape of Pubmed publication data that is used in a single webpage with multiple plotly charts. The basic structure of the website is updated with an excel spreadsheet to help those who don't know how to code.

beautifulsoup excel html-css-javascript pandas plotly python splinter

Last synced: 21 Nov 2024

https://github.com/jerrytheo/jdoc-scraper

Set of scripts that crawl the online Java documentation to scrape information about the methods and constructors of each class, organised by package names.

beautifulsoup python-3-6 webscraper

Last synced: 06 Dec 2024

https://github.com/eliasbnk/web-scraping-sandbox

Python scripts for scraping data from Scrapethissite.com

aiohttp beautifulsoup beautifulsoup4 python sandbox webscraping

Last synced: 20 Nov 2024

https://github.com/algorithmalchemy/combot-telegram-channel-parser

Парсер telegram / каналов и супергрупп / с сайта Combot. Написан с использованием библиотек Requests и SQLite3.

beautifulsoup bs4 combot combot-api requests urlib

Last synced: 24 Nov 2024

https://github.com/mmrosatab/etl-brasileirao-2021-project

Application for make ETL with social midia data

beautifulsoup pandas python3 selenium-webdriver

Last synced: 21 Dec 2024

https://github.com/thevickypedia/crypto-monitor

Python script to monitor crypto currencies

beautifulsoup cryptocurrency smtp threadpool yfinance-api

Last synced: 20 Dec 2024

https://github.com/harshpatel44/website-rank-finder

This program finds the rank of the Website provided. It finds the rank in Google Search Engine

beautifulsoup finder google mechanize python rank website

Last synced: 24 Nov 2024

https://github.com/nothingnothings/zap-scraper

Zap Imóveis Website Scraper Built with Python

beautifulsoup beautifulsoup4 docker python scraper selenium sql zap-imoveis

Last synced: 22 Dec 2024

https://github.com/mrqadeer/alphax-entertainments

This project is a Movie/Series/Drama Recognizer and Recommender that uses dialogue excerpts to identify media titles, genres, cast, directors, and other key information. It also recommends related titles based on the recognized media. The system leverages external APIs and web scraping to fetch details like IMDb rating, number of votes, and poster.

beautifulsoup genai-poc langchain openai pillow python requests streamlit

Last synced: 21 Nov 2024

https://github.com/mdimado/medraga

MedRaga is a medical assistance application aimed at providing accurate and personalized medical information to healthcare professionals. It uses RAG technology to retrieve the latest medical research from trusted sources, augment it with patient data, and deliver personalized diagnoses and treatment plans.

arxiv beautifulsoup cohere fastapi firebase firebase-auth gemini-api langchain langchain-python playwright qdrant qdrant-vector-database rag reactjs selenium

Last synced: 22 Dec 2024

https://github.com/rahulguptagzb09/scraping-real-estate-property-data-from-web-using-python

Scraping Real Estate Property Data From Web Using Python, BeautifulSoup

beautifulsoup pandas python real-estate requests scraping-websites

Last synced: 20 Nov 2024

https://github.com/mynameiscarsten/seo-analyzer

Basic Django project to emulate the meta analysis for a submitted website. Data is stored within a table and subsequently displayed.

beautifulsoup django httplib2 python requests

Last synced: 29 Dec 2024

https://github.com/adithya-s-k/discoverydino

a robust and scalable system that proactively identifies new generally available (GA) software products and checks their availability on the G2 software marketplace. The goal is to compile a list of products that are not yet listed on G2, simplifying the process of onboarding them onto the platform

aws beautifulsoup fastapi nextjs serveless webscraping

Last synced: 22 Dec 2024

https://github.com/preetvadaliya/weather-app

Simple python program to get real time weather data of your location.

beautifulsoup python python3 weather-app webscraping

Last synced: 05 Dec 2024

https://github.com/letelete/easy-documentation

Automatize the GitHub repository documentation process with this CLI! Made for Google Code-In ‘18

beautifulsoup cli github-api google-sheets-api halo python

Last synced: 20 Nov 2024

https://github.com/code-alchemist01/intellifist-ai

An artificial intelligence model that randomly generates questions according to class, course and subject for the educational application we will make.

api beautifulsoup fastapi gemini python random-generation scraping-python scraping-websites token tokenization

Last synced: 31 Oct 2024