{"id":13674789,"url":"https://github.com/joeyism/linkedin_scraper","last_synced_at":"2026-04-10T05:01:38.003Z","repository":{"id":38898735,"uuid":"108595093","full_name":"joeyism/linkedin_scraper","owner":"joeyism","description":"A library that scrapes Linkedin for user data","archived":false,"fork":false,"pushed_at":"2025-05-23T16:45:59.000Z","size":4218,"stargazers_count":3507,"open_issues_count":130,"forks_count":827,"subscribers_count":48,"default_branch":"master","last_synced_at":"2025-12-02T20:54:56.608Z","etag":null,"topics":["chrome","company","driver","firefox","linkedin","linkedin-profile","linkedin-scraper","linkedin-url","profile","scraper","scrapes-linkedin","users"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joeyism.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-27T20:58:54.000Z","updated_at":"2025-12-01T16:33:01.000Z","dependencies_parsed_at":"2024-09-29T17:00:52.609Z","dependency_job_id":"0e4a2f17-46a7-4484-9e46-faa0b91d642c","html_url":"https://github.com/joeyism/linkedin_scraper","commit_stats":{"total_commits":160,"total_committers":24,"mean_commits":6.666666666666667,"dds":"0.36250000000000004","last_synced_commit":"5771fe863fd02259e0b16b1dde586046214dd261"},"previous_names":["joeyism/linkedin_user_scraper"],"tags_count":48,"template":false,"template_full_name":null,"purl":"pkg:github/joeyism/linkedin_scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joeyism%2Flinkedin_scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joeyism%2Flinkedin_scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joeyism%2Flinkedin_scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joeyism%2Flinkedin_scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joeyism","download_url":"https://codeload.github.com/joeyism/linkedin_scraper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joeyism%2Flinkedin_scraper/sbom","scorecard":{"id":527555,"data":{"date":"2025-08-11","repo":{"name":"github.com/joeyism/linkedin_scraper","commit":"44eafb893e691732474e37a20123c5cc9007e0ad"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.4,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":5,"reason":"Found 7/12 approved changesets -- score normalized to 5","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"1 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU General Public License v3.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 25 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"13 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-55x5-fj6c-h6m8","Warn: Project is vulnerable to: PYSEC-2014-9 / GHSA-57qw-cc2g-pv5p","Warn: Project is vulnerable to: PYSEC-2021-19 / GHSA-jq4v-f5q6-mjqq","Warn: Project is vulnerable to: GHSA-pgww-xf46-h92r","Warn: Project is vulnerable to: PYSEC-2022-230 / GHSA-wrxv-2j5q-m38w","Warn: Project is vulnerable to: PYSEC-2018-12 / GHSA-xp26-p53h-6h2p","Warn: Project is vulnerable to: PYSEC-2014-14 / GHSA-652x-xj99-gmcc","Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-9wx4-h78v-vm56","Warn: Project is vulnerable to: PYSEC-2014-13 / GHSA-cfj3-7x9c-4p3h","Warn: Project is vulnerable to: PYSEC-2018-28 / GHSA-x84v-xcm2-53pg","Warn: Project is vulnerable to: PYSEC-2022-43167","Warn: Project is vulnerable to: PYSEC-2023-206"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-20T04:49:33.479Z","repository_id":38898735,"created_at":"2025-08-20T04:49:33.479Z","updated_at":"2025-08-20T04:49:33.479Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27714241,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-13T02:00:09.769Z","response_time":147,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chrome","company","driver","firefox","linkedin","linkedin-profile","linkedin-scraper","linkedin-url","profile","scraper","scrapes-linkedin","users"],"created_at":"2024-08-02T11:01:01.477Z","updated_at":"2026-04-10T05:01:37.995Z","avatar_url":"https://github.com/joeyism.png","language":"Python","funding_links":[],"categories":["Python","🔧 Utilities \u0026 Miscellaneous","HarmonyOS"],"sub_categories":["Windows Manager"],"readme":"# LinkedIn Scraper\n\n[![PyPI version](https://badge.fury.io/py/linkedin-scraper.svg)](https://badge.fury.io/py/linkedin-scraper)\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\nAsync LinkedIn scraper built with Playwright for extracting profile, company, and job data from LinkedIn.\n\n## ⚠️ Breaking Changes in v3.0.0\n\n**Version 3.0.0 introduces breaking changes and is NOT backwards compatible with previous versions.**\n\n### What Changed:\n- **Playwright instead of Selenium** - Complete rewrite using Playwright for better performance and reliability\n- **Async/await throughout** - All methods are now async and require `await`\n- **New package structure** - Imports have changed (e.g., `from linkedin_scraper import PersonScraper`)\n- **Updated data models** - Using Pydantic models instead of simple objects\n- **Different API** - Method signatures and return types have changed\n\n### Migration Guide:\n\n**Before (v2.x with Selenium):**\n```python\nfrom linkedin_scraper import Person\n\nperson = Person(\"https://linkedin.com/in/username\", driver=driver)\nprint(person.name)\n```\n\n**After (v3.0+ with Playwright):**\n```python\nimport asyncio\nfrom linkedin_scraper import BrowserManager, PersonScraper\n\nasync def main():\n    async with BrowserManager() as browser:\n        await browser.load_session(\"session.json\")\n        scraper = PersonScraper(browser.page)\n        person = await scraper.scrape(\"https://linkedin.com/in/username\")\n        print(person.name)\n\nasyncio.run(main())\n```\n\n**If you need the old Selenium-based version:**\n```bash\npip install linkedin-scraper==2.11.2\n```\n## Quick Testing\n\nTo test that this works, you can clone this repo, install dependencies with\n```\ngit clone https://github.com/joeyism/linkedin_scraper.git\ncd linkedin_scraper\npip3 install -e .\n```\nthen run\n```\npython3 samples/create_session.py\npython3 samples/scrape_company.py\npython3 samples/scrape_person.py\n```\nand you will see the scraping in action.\n\n---\n\n## Features\n\n- **Person Profiles** - Scrape comprehensive profile information\n  - Basic info (name, headline, location, about)\n  - Work experience with details\n  - Education history\n  - Skills and accomplishments\n  \n- **Company Pages** - Extract company information\n  - Company overview and details\n  - Industry and size\n  - Headquarters location\n  \n- **Company Posts** - Scrape posts from company pages\n  - Post content and text\n  - Reactions, comments, reposts counts\n  - Posted date and images\n  \n- **Job Listings** - Scrape job postings\n  - Job details and requirements\n  - Company information\n  - Application links\n\n- **Async/Await** - Modern async Python with Playwright\n- **Type Safety** - Full Pydantic models for all data\n- **Progress Callbacks** - Track scraping progress\n- **Session Management** - Reuse authenticated sessions\n\n## Installation\n\n```bash\npip install linkedin-scraper\n```\n\n### Install Playwright browsers:\n\n```bash\nplaywright install chromium\n```\n\n## Quick Start\n\n### Basic Usage\n\n```python\nimport asyncio\nfrom linkedin_scraper import BrowserManager, PersonScraper\n\nasync def main():\n    # Initialize browser\n    async with BrowserManager(headless=False) as browser:\n        # Load authenticated session\n        await browser.load_session(\"session.json\")\n        \n        # Create scraper\n        scraper = PersonScraper(browser.page)\n        \n        # Scrape a profile\n        person = await scraper.scrape(\"https://linkedin.com/in/williamhgates/\")\n        \n        # Access data\n        print(f\"Name: {person.name}\")\n        print(f\"Headline: {person.headline}\")\n        print(f\"Location: {person.location}\")\n        print(f\"Experiences: {len(person.experiences)}\")\n        print(f\"Education: {len(person.educations)}\")\n\nasyncio.run(main())\n```\n\n### Company Scraping\n\n```python\nfrom linkedin_scraper import CompanyScraper\n\nasync def scrape_company():\n    async with BrowserManager(headless=False) as browser:\n        await browser.load_session(\"session.json\")\n        \n        scraper = CompanyScraper(browser.page)\n        company = await scraper.scrape(\"https://linkedin.com/company/microsoft/\")\n        \n        print(f\"Company: {company.name}\")\n        print(f\"Industry: {company.industry}\")\n        print(f\"Size: {company.company_size}\")\n        print(f\"About: {company.about_us[:200]}...\")\n\nasyncio.run(scrape_company())\n```\n\n### Job Scraping\n\n```python\nfrom linkedin_scraper import JobSearchScraper\n\nasync def search_jobs():\n    async with BrowserManager(headless=False) as browser:\n        await browser.load_session(\"session.json\")\n        \n        scraper = JobSearchScraper(browser.page)\n        jobs = await scraper.search(\n            keywords=\"Python Developer\",\n            location=\"San Francisco\",\n            limit=10\n        )\n        \n        for job in jobs:\n            print(f\"{job.title} at {job.company}\")\n            print(f\"Location: {job.location}\")\n            print(f\"Link: {job.linkedin_url}\")\n            print(\"---\")\n\nasyncio.run(search_jobs())\n```\n\n### Company Posts Scraping\n\n```python\nfrom linkedin_scraper import BrowserManager, CompanyPostsScraper\n\nasync def scrape_company_posts():\n    async with BrowserManager(headless=False) as browser:\n        await browser.load_session(\"session.json\")\n        \n        scraper = CompanyPostsScraper(browser.page)\n        posts = await scraper.scrape(\n            \"https://linkedin.com/company/microsoft/\",\n            limit=10\n        )\n        \n        for post in posts:\n            print(f\"Posted: {post.posted_date}\")\n            print(f\"Text: {post.text[:200]}...\")\n            print(f\"Reactions: {post.reactions_count}\")\n            print(f\"Comments: {post.comments_count}\")\n            print(f\"URL: {post.linkedin_url}\")\n            print(\"---\")\n\nasyncio.run(scrape_company_posts())\n```\n\n## Authentication\n\nLinkedIn requires authentication. You need to create a session file first:\n\n### Option 1: Manual Login Script\n\n```python\nfrom linkedin_scraper import BrowserManager, wait_for_manual_login\n\nasync def create_session():\n    async with BrowserManager(headless=False) as browser:\n        # Navigate to LinkedIn\n        await browser.page.goto(\"https://www.linkedin.com/login\")\n        \n        # Wait for manual login (opens browser)\n        print(\"Please log in to LinkedIn...\")\n        await wait_for_manual_login(browser.page, timeout=300)\n        \n        # Save session\n        await browser.save_session(\"session.json\")\n        print(\"✓ Session saved!\")\n\nasyncio.run(create_session())\n```\n\n### Option 2: Programmatic Login\n\n```python\nfrom linkedin_scraper import BrowserManager, login_with_credentials\nimport os\n\nasync def login():\n    async with BrowserManager(headless=False) as browser:\n        # Login with credentials\n        await login_with_credentials(\n            browser.page,\n            username=os.getenv(\"LINKEDIN_EMAIL\"),\n            password=os.getenv(\"LINKEDIN_PASSWORD\")\n        )\n        \n        # Save session for reuse\n        await browser.save_session(\"session.json\")\n\nasyncio.run(login())\n```\n\n## Progress Tracking\n\nTrack scraping progress with callbacks:\n\n```python\nfrom linkedin_scraper import ConsoleCallback, PersonScraper\n\nasync def scrape_with_progress():\n    callback = ConsoleCallback()  # Prints progress to console\n    \n    async with BrowserManager(headless=False) as browser:\n        await browser.load_session(\"session.json\")\n        \n        scraper = PersonScraper(browser.page, callback=callback)\n        person = await scraper.scrape(\"https://linkedin.com/in/williamhgates/\")\n\nasyncio.run(scrape_with_progress())\n```\n\n### Custom Callbacks\n\n```python\nfrom linkedin_scraper import ProgressCallback\n\nclass MyCallback(ProgressCallback):\n    async def on_start(self, scraper_type: str, url: str):\n        print(f\"Starting {scraper_type} scraping: {url}\")\n    \n    async def on_progress(self, message: str, percent: int):\n        print(f\"[{percent}%] {message}\")\n    \n    async def on_complete(self, scraper_type: str, url: str):\n        print(f\"Completed {scraper_type}: {url}\")\n    \n    async def on_error(self, error: Exception):\n        print(f\"Error: {error}\")\n```\n\n## Data Models\n\nAll scraped data is returned as Pydantic models:\n\n### Person\n\n```python\nclass Person(BaseModel):\n    name: str\n    headline: Optional[str]\n    location: Optional[str]\n    about: Optional[str]\n    linkedin_url: str\n    experiences: List[Experience]\n    educations: List[Education]\n    skills: List[str]\n    accomplishments: Optional[Accomplishment]\n```\n\n### Company\n\n```python\nclass Company(BaseModel):\n    name: str\n    industry: Optional[str]\n    company_size: Optional[str]\n    headquarters: Optional[str]\n    founded: Optional[str]\n    specialties: List[str]\n    about: Optional[str]\n    linkedin_url: str\n```\n\n### Job\n\n```python\nclass Job(BaseModel):\n    title: str\n    company: str\n    location: Optional[str]\n    description: Optional[str]\n    employment_type: Optional[str]\n    seniority_level: Optional[str]\n    linkedin_url: str\n```\n\n### Post\n\n```python\nclass Post(BaseModel):\n    linkedin_url: Optional[str]\n    urn: Optional[str]\n    text: Optional[str]\n    posted_date: Optional[str]\n    reactions_count: Optional[int]\n    comments_count: Optional[int]\n    reposts_count: Optional[int]\n    image_urls: List[str]\n```\n\n## Advanced Usage\n\n### Browser Configuration\n\n```python\nbrowser = BrowserManager(\n    headless=False,  # Show browser window\n    slow_mo=100,     # Slow down operations (ms)\n    viewport={\"width\": 1920, \"height\": 1080},\n    user_agent=\"Custom User Agent\"\n)\n```\n\n### Error Handling\n\n```python\nfrom linkedin_scraper import (\n    AuthenticationError,\n    RateLimitError,\n    ProfileNotFoundError\n)\n\ntry:\n    person = await scraper.scrape(url)\nexcept AuthenticationError:\n    print(\"Not logged in - session expired\")\nexcept RateLimitError:\n    print(\"Rate limited by LinkedIn\")\nexcept ProfileNotFoundError:\n    print(\"Profile not found or private\")\n```\n\n## Best Practices\n\n1. **Rate Limiting** - Add delays between requests\n   ```python\n   import asyncio\n   await asyncio.sleep(2)  # 2 second delay\n   ```\n\n2. **Session Reuse** - Save and reuse sessions to avoid frequent logins\n\n3. **Error Handling** - Always handle exceptions (rate limits, auth errors, etc.)\n\n4. **Headless Mode** - Use `headless=False` during development, `True` for production\n\n5. **Respect LinkedIn** - Don't scrape aggressively, respect rate limits\n\n## Requirements\n\n- Python 3.8+\n- Playwright\n- Pydantic 2.0+\n- aiofiles\n- python-dotenv (optional, for credentials)\n\n## License\n\nApache License 2.0 - see [LICENSE](LICENSE) file for details.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## Disclaimer\n\nThis tool is for educational purposes only. Make sure to comply with LinkedIn's Terms of Service and use responsibly. The authors are not responsible for any misuse of this tool.\n\n## Links\n\n- [GitHub Repository](https://github.com/joeyism/linkedin_scraper)\n- [Issue Tracker](https://github.com/joeyism/linkedin_scraper/issues)\n- [PyPI Package](https://pypi.org/project/linkedin-scraper/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoeyism%2Flinkedin_scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoeyism%2Flinkedin_scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoeyism%2Flinkedin_scraper/lists"}