https://github.com/ismaelfi/scrape-linkedin-posts

Automated LinkedIn Post Scraper using Python, Selenium, and BeautifulSoup. Extract content, reactions, and comments from LinkedIn user posts and save them to a CSV file.
https://github.com/ismaelfi/scrape-linkedin-posts

beautifulsoup linkedin linkedin-scraper python3 script selenium selenium-python

Last synced: about 1 month ago
JSON representation

Automated LinkedIn Post Scraper using Python, Selenium, and BeautifulSoup. Extract content, reactions, and comments from LinkedIn user posts and save them to a CSV file.

• Host: GitHub
• URL: https://github.com/ismaelfi/scrape-linkedin-posts
• Owner: ismaelfi
• License: mit
• Created: 2024-07-31T08:48:30.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-07-31T08:54:40.000Z (5 months ago)
• Last Synced: 2024-10-15T07:26:02.537Z (3 months ago)
• Topics: beautifulsoup, linkedin, linkedin-scraper, python3, script, selenium, selenium-python
• Language: Python
• Homepage:
• Size: 4.88 KB
• Stars: 4
• Watchers: 1
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Linkedin Post Scraper 

## DISCLAIMER

This script is provided for educational purposes only. LinkedIn's terms of service prohibit scraping or any form of automated data collection.

Using this script to scrape LinkedIn's data is against their terms of service and can result in your account being banned.

Use this script at your own risk. The author is not responsible for any misuse of this script.

## Overview

This script uses Selenium and BeautifulSoup to scrape the latest posts from a specified LinkedIn user's activity page.

It extracts the content, reactions, and comments from each post and saves the data to a CSV file.

## Requirements

To run this script, you need to install the following Python packages and tools:

- Selenium: For web browser automation.

- BeautifulSoup: For parsing HTML and extracting data.

- Pandas: For data manipulation and exporting to CSV.

- ChromeDriver: The WebDriver for Google Chrome.

- Google Chrome Browser: The actual browser used by ChromeDriver.

## Installation Guide

### Step 1: Install Python Packages

You can install the required Python packages using pip. Run the following command in your terminal:

 ```sh

 pip install selenium beautifulsoup4 pandas

 ```

### Step 2: Install Google Chrome

Download and install Google Chrome from the official website.

### Step 3: Install ChromeDriver

Download ChromeDriver: Go to the ChromeDriver download page and download the version that matches your installed version of Chrome.

Install ChromeDriver:

Windows: Extract the downloaded file and place chromedriver.exe in a directory that's included in your system's PATH, or specify the path in your script.

macOS and Linux: Extract the downloaded file and place chromedriver in a directory that's included in your system's PATH, or specify the path in your script.

### Step 4: Create the Cookies File

Create a file named your_linkedin_cookies.txt in the same directory as the script.

This file should contain your LinkedIn cookies in Netscape format.

You can use browser extensions like "EditThisCookie" to export cookies from your LinkedIn session.