https://github.com/memirdogan/web-scraping-with-beautiful-soup-and-selenium

This repository offers a guide to web scraping with Beautiful Soup and Selenium. It covers data extraction, HTML basics, XPath, and CSS selectors. Includes a project for scraping Trendyol. Ideal for beginners wanting to learn web scraping.
https://github.com/memirdogan/web-scraping-with-beautiful-soup-and-selenium

beautifulsoup beautifulsoup4 html python selenium selenium-webdriver webscraping

Last synced: 4 months ago
JSON representation

This repository offers a guide to web scraping with Beautiful Soup and Selenium. It covers data extraction, HTML basics, XPath, and CSS selectors. Includes a project for scraping Trendyol. Ideal for beginners wanting to learn web scraping.

• Host: GitHub
• URL: https://github.com/memirdogan/web-scraping-with-beautiful-soup-and-selenium
• Owner: Memirdogan
• Created: 2024-07-09T06:55:13.000Z (7 months ago)
• Default Branch: main
• Last Pushed: 2024-07-22T07:40:10.000Z (6 months ago)
• Last Synced: 2024-09-27T14:40:55.497Z (4 months ago)
• Topics: beautifulsoup, beautifulsoup4, html, python, selenium, selenium-webdriver, webscraping
• Language: Python
• Homepage:
• Size: 2.24 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

        
![Star Badge](https://img.shields.io/static/v1?label=%F0%9F%8C%9F&message=If%20Useful&style=style=flat&color=BC4E99)

![Open Source Love](https://badges.frapsoft.com/os/v1/open-source.svg?v=103)

[![View My Profile](https://img.shields.io/badge/View-My_Profile-green?logo=GitHub)](https://github.com/Memirdogan)

[![View Repositories](https://img.shields.io/badge/View-My_Repositories-blue?logo=GitHub)](https://github.com/Memirdogan?tab=repositories)

# Web Scraping with Beautiful Soup and Selenium

Welcome to the Web Scraping with Beautiful Soup and Selenium repository! This repository provides a comprehensive guide to web scraping, from basic concepts to practical implementations.

## Table of Contents

- [Introduction](#introduction)

- [Prerequisites](#prerequisites)

- [Installation](#installation)

- [Usage](#usage)

- [Project Structure](#project-structure)

- [Topics Covered](#topics-covered)

- [Contributing](#contributing)

## Introduction

This repository is designed to help beginners understand the basics of web scraping using Beautiful Soup and Selenium. You'll learn how to extract data from websites and perform various tasks such as navigating HTML structures, using XPath, and applying CSS selectors. The repository includes a practical project for scraping data from the Trendyol website.

## Prerequisites

Before you begin, make sure you have the following installed:

- Python 3.x

- pip (Python package installer)

## Installation

To install the required libraries

```

beautifulsoup4

selenium

requests

```

## Usage

Here's a brief overview of how to use the scripts in this repository:

1. Clone the repository:

    ```bash

    git clone https://github.com/Memirdogan/Web-Scraping-with-Beautiful-Soup-and-Selenium.git

    ```

2. Navigate to the project directory:

    ```bash

    cd Web-Scraping-with-Beautiful-Soup-and-Selenium

    ```

    

## Project Structure

```plaintext

Web-Scraping-with-Beautiful-Soup-and-Selenium/

├── .idea/

├── Beautiful Soup’a Giriş/

├── Bot Tespiti ve Proxy/

├── HTML Temelleri/

│   └── Uygulama

├── Selenium'a Giriş/

├── TrendyolPhoneData/

│   ├── TrendyolPhoneData.py

│   ├── main.py

│   ├── DataClear.py

│   ├── datasetanalyser.py

│   └── ...

└── XPath ve CSS Seçicilere Giriş/

```

## Topics Covered

- Introduction to Beautiful Soup

- Bot Detection and Proxy

- HTML Basics

- Introduction to Selenium

- TrendyolPhoneData Project

- Introduction to XPath and CSS Selectors

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.