https://github.com/thewebscrapingclub/articleindex
https://github.com/thewebscrapingclub/articleindex
Last synced: 12 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/thewebscrapingclub/articleindex
- Owner: TheWebScrapingClub
- Created: 2025-06-24T20:00:09.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2025-06-24T20:03:18.000Z (12 months ago)
- Last Synced: 2025-06-24T21:22:43.874Z (12 months ago)
- Size: 19.5 KB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# 📚 Article Index by Tag
## 🏷️ AI
| Title | Date | Link |
|-------|------|------|
| THE LAB #86: Querying Web Data using GPT-Like Web Interface | 2025-06-05 | [THE LAB #86: Querying Web Data using GPT-Like Web Interface](articles/THE-LAB-#86-Querying-Web-Data-using-GPT-Like-Web-Interface.md) |
| Scrape like a pro... but not like an AI company | 2025-05-20 | [Scrape like a pro... but not like an AI company](articles/Scrape-like-a-pro...-but-not-like-an-AI-company.md) |
| AI and data: different faces of the same coin | 2025-05-20 | [AI and data: different faces of the same coin](articles/AI-and-data-different-faces-of-the-same-coin.md) |
| How AI is changing the web scraping industry | 2025-05-20 | [How AI is changing the web scraping industry](articles/How-AI-is-changing-the-web-scraping-industry.md) |
| The AI-Powered web scraping tools landscape | 2025-05-20 | [The AI-Powered web scraping tools landscape](articles/The-AI-Powered-web-scraping-tools-landscape.md) |
| Building a custom GPT using Firecrawl | 2025-05-20 | [Building a custom GPT using Firecrawl](articles/Building-a-custom-GPT-using-Firecrawl.md) |
| About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra | 2025-05-20 | [About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra](articles/About-LLMs,-AI-and-Web-Scraping---by-Pierluigi-Vinciguerra.md) |
| Building a generic scraper for multiple websites | 2025-05-20 | [Building a generic scraper for multiple websites](articles/Building-a-generic-scraper-for-multiple-websites.md) |
| Use Cursor as web scraping assistant with MCP servers | 2025-05-20 | [Use Cursor as web scraping assistant with MCP servers](articles/Use-Cursor-as-web-scraping-assistant-with-MCP-servers.md) |
| Build your web scraping assistant with Claude and Cursor | 2025-05-20 | [Build your web scraping assistant with Claude and Cursor](articles/Build-your-web-scraping-assistant-with-Claude-and-Cursor.md) |
| Are LLMs capable of replacing traditional scrapers? | 2025-05-20 | [Are LLMs capable of replacing traditional scrapers?](articles/Are-LLMs-capable-of-replacing-traditional-scrapers.md) |
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | [Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base](articles/Evolution-from-RAG-to-MCP-A-Breakthrough-for-LLM-Dynamic-Knowledge-Base.md) |
| Build a RAG Application with ScraperAPI, Gemini, and FAISS | 2025-04-02 | [Build a RAG Application with ScraperAPI, Gemini, and FAISS](articles/Build-a-RAG-Application-with-ScraperAPI,-Gemini,-and-FAISS.md) |
| Rethinking the web browser - by Katie Hallett | 2025-01-21 | [Rethinking the web browser - by Katie Hallett](articles/Rethinking-the-web-browser---by-Katie-Hallett.md) |
| Is Web Scraping Dead? - by Pierluigi Vinciguerra | 2024-02-25 | [Is Web Scraping Dead? - by Pierluigi Vinciguerra](articles/Is-Web-Scraping-Dead---by-Pierluigi-Vinciguerra.md) |
| Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra | 2023-10-13 | [Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra](articles/Are-CAPTCHAs-still-a-thing---by-Pierluigi-Vinciguerra.md) |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | [Web Scraping experts: Is AI stealing our job?](articles/Web-Scraping-experts-Is-AI-stealing-our-job.md) |
| How to create a web scraper with ChatGPT | 2023-10-13 | [How to create a web scraper with ChatGPT](articles/How-to-create-a-web-scraper-with-ChatGPT.md) |
| The state of web scraping and AI - by Pierluigi Vinciguerra | 2023-10-13 | [The state of web scraping and AI - by Pierluigi Vinciguerra](articles/The-state-of-web-scraping-and-AI---by-Pierluigi-Vinciguerra.md) |
## 🏷️ API
| Title | Date | Link |
|-------|------|------|
| THE LAB #26: From internal API to insights. | 2024-10-31 | [THE LAB #26: From internal API to insights.](articles/THE-LAB-#26-From-internal-API-to-insights..md) |
## 🏷️ AWS
| Title | Date | Link |
|-------|------|------|
| THE LAB #74: Running scrapers on GitHub Actions | 2025-05-20 | [THE LAB #74: Running scrapers on GitHub Actions](articles/THE-LAB-#74-Running-scrapers-on-GitHub-Actions.md) |
| The Lab #53: Bypassing AWS WAF - by Pierluigi Vinciguerra | 2025-05-20 | [The Lab #53: Bypassing AWS WAF - by Pierluigi Vinciguerra](articles/The-Lab-#53-Bypassing-AWS-WAF---by-Pierluigi-Vinciguerra.md) |
| The Lab #48: Scraping with AWS Lambda | 2024-10-18 | [The Lab #48: Scraping with AWS Lambda](articles/The-Lab-#48-Scraping-with-AWS-Lambda.md) |
## 🏷️ Airbnb
| Title | Date | Link |
|-------|------|------|
| THE LAB #66: How to properly scrape a booking website | 2025-05-20 | [THE LAB #66: How to properly scrape a booking website](articles/THE-LAB-#66-How-to-properly-scrape-a-booking-website.md) |
| The Lab #5 - Scraping Airbnb.com using GraphQL | 2023-05-29 | [The Lab #5 - Scraping Airbnb.com using GraphQL](articles/The-Lab-#5---Scraping-Airbnb.com-using-GraphQL.md) |
## 🏷️ Airflow
| Title | Date | Link |
|-------|------|------|
| Scheduling Scrapers with Airflow - by Pierluigi Vinciguerra | 2025-05-20 | [Scheduling Scrapers with Airflow - by Pierluigi Vinciguerra](articles/Scheduling-Scrapers-with-Airflow---by-Pierluigi-Vinciguerra.md) |
## 🏷️ Akamai
| Title | Date | Link |
|-------|------|------|
| THE LAB #30: How to bypass Akamai protected website when nothing else works | 2025-06-09 | [THE LAB #30: How to bypass Akamai protected website when nothing else works](articles/THE-LAB-#30-How-to-bypass-Akamai-protected-website-when-nothing-else-works.md) |
| THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies | 2025-05-29 | [THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies](articles/THE-LAB-#85-Bypass-Akamai-Bot-Protection-by-Chaining-Proxies.md) |
| Scraping Akamai-protected websites with Scrapy | 2025-05-20 | [Scraping Akamai-protected websites with Scrapy](articles/Scraping-Akamai-protected-websites-with-Scrapy.md) |
| Scraping Cloudflare websites using an API | 2025-05-20 | [Scraping Cloudflare websites using an API](articles/Scraping-Cloudflare-websites-using-an-API.md) |
| Scraping Akamai protected websites | 2024-09-08 | [Scraping Akamai protected websites](articles/Scraping-Akamai-protected-websites.md) |
| THE LAB 32: hRequests vs anti-bots: a full benchmark | 2023-11-30 | [THE LAB 32: hRequests vs anti-bots: a full benchmark](articles/THE-LAB-32-hRequests-vs-anti-bots-a-full-benchmark.md) |
| hRequests: bypass Akamai with Python requests | 2023-11-12 | [hRequests: bypass Akamai with Python requests](articles/hRequests-bypass-Akamai-with-Python-requests.md) |
## 🏷️ AlexsandrasSulzenko
| Title | Date | Link |
|-------|------|------|
| Interview #6: Aleksandras Šulženko - Oxylabs | 2023-10-13 | [Interview #6: Aleksandras Šulženko - Oxylabs](articles/Interview-#6-Aleksandras-Šulženko---Oxylabs.md) |
| Three web scraping tools just discovered on GitHub | 2023-10-08 | [Three web scraping tools just discovered on GitHub](articles/Three-web-scraping-tools-just-discovered-on-GitHub.md) |
## 🏷️ Algolia
| Title | Date | Link |
|-------|------|------|
| The Lab #54: Scraping from Algolia APIs | 2025-05-20 | [The Lab #54: Scraping from Algolia APIs](articles/The-Lab-#54-Scraping-from-Algolia-APIs.md) |
| Algolia and web scraping: an introduction | 2023-12-10 | [Algolia and web scraping: an introduction](articles/Algolia-and-web-scraping-an-introduction.md) |
## 🏷️ AlternativeData
| Title | Date | Link |
|-------|------|------|
| THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools | 2025-06-20 | [THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools](articles/THE-LAB-#87-Bypassing-ReCAPTCHAs-with-open-source-and-commercial-tools.md) |
| Creating a dataset for investors with web scraping: Tesla (TSLA) | 2025-03-30 | [Creating a dataset for investors with web scraping: Tesla (TSLA)](articles/Creating-a-dataset-for-investors-with-web-scraping-Tesla-(TSLA).md) |
| Web scraping and alternative data for financial markets | 2023-10-13 | [Web scraping and alternative data for financial markets](articles/Web-scraping-and-alternative-data-for-financial-markets.md) |
## 🏷️ Amazon
| Title | Date | Link |
|-------|------|------|
| How to Scrape E-Commerce Websites With Python | 2024-08-02 | [How to Scrape E-Commerce Websites With Python](articles/How-to-Scrape-E-Commerce-Websites-With-Python.md) |
## 🏷️ AntiDetectBrowsers
| Title | Date | Link |
|-------|------|------|
| The Anti-Detect Browser Royal Rumble - updated with notes | 2025-05-20 | [The Anti-Detect Browser Royal Rumble - updated with notes](articles/The-Anti-Detect-Browser-Royal-Rumble---updated-with-notes.md) |
| The Browser Automation Landscape in 2025 | 2025-05-20 | [The Browser Automation Landscape in 2025](articles/The-Browser-Automation-Landscape-in-2025.md) |
| The Lab #36: Bypassing Cloudflare with anti-detect browsers | 2025-04-16 | [The Lab #36: Bypassing Cloudflare with anti-detect browsers](articles/The-Lab-#36-Bypassing-Cloudflare-with-anti-detect-browsers.md) |
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | [In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers](articles/In-Depth-Pricing-Comparison-of-Anti-Detect-Browsers-for-Web-Scrapers.md) |
| The Anti-Detect Browser Royal Rumble - Fingerprint tests | 2024-04-23 | [The Anti-Detect Browser Royal Rumble - Fingerprint tests](articles/The-Anti-Detect-Browser-Royal-Rumble---Fingerprint-tests.md) |
| How Can Multi-Accounting Browsers Help with Web Scraping? | 2024-04-16 | [How Can Multi-Accounting Browsers Help with Web Scraping?](articles/How-Can-Multi-Accounting-Browsers-Help-with-Web-Scraping.md) |
| Behind the scenes of anti-detect browsers - by Tamas Deak | 2024-03-05 | [Behind the scenes of anti-detect browsers - by Tamas Deak](articles/Behind-the-scenes-of-anti-detect-browsers---by-Tamas-Deak.md) |
| The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2 | 2024-01-19 | [The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2](articles/The-Lab-#37-Bypassing-Cloudflare-with-anti-detect-browsers---Part-2.md) |
| The rise of antidetect browsers - by Pierluigi Vinciguerra | 2023-10-13 | [The rise of antidetect browsers - by Pierluigi Vinciguerra](articles/The-rise-of-antidetect-browsers---by-Pierluigi-Vinciguerra.md) |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | [How to by-pass Kasada bot mitigation?](articles/How-to-by-pass-Kasada-bot-mitigation.md) |
| From Traditional Browsers to AI-Powered Web Scraping | 2023-10-13 | [From Traditional Browsers to AI-Powered Web Scraping](articles/From-Traditional-Browsers-to-AI-Powered-Web-Scraping.md) |
## 🏷️ Antrophic
| Title | Date | Link |
|-------|------|------|
| Scrape like a pro... but not like an AI company | 2025-05-20 | [Scrape like a pro... but not like an AI company](articles/Scrape-like-a-pro...-but-not-like-an-AI-company.md) |
| AI and data: different faces of the same coin | 2025-05-20 | [AI and data: different faces of the same coin](articles/AI-and-data-different-faces-of-the-same-coin.md) |
## 🏷️ Apify
| Title | Date | Link |
|-------|------|------|
| THE LAB #15: Deep diving into Apify world | 2023-10-13 | [THE LAB #15: Deep diving into Apify world](articles/THE-LAB-#15-Deep-diving-into-Apify-world.md) |
## 🏷️ Automotive
| Title | Date | Link |
|-------|------|------|
| Web data and automotive industry - by Pierluigi Vinciguerra | 2025-05-20 | [Web data and automotive industry - by Pierluigi Vinciguerra](articles/Web-data-and-automotive-industry---by-Pierluigi-Vinciguerra.md) |
## 🏷️ AvivBesinky
| Title | Date | Link |
|-------|------|------|
| Interview #7: Aviv Besinsky - Bright Data | 2023-10-13 | [Interview #7: Aviv Besinsky - Bright Data](articles/Interview-#7-Aviv-Besinsky---Bright-Data.md) |
## 🏷️ BearerToken
| Title | Date | Link |
|-------|------|------|
| Scraping APIs with Bearer Token - by Pierluigi Vinciguerra | 2025-05-20 | [Scraping APIs with Bearer Token - by Pierluigi Vinciguerra](articles/Scraping-APIs-with-Bearer-Token---by-Pierluigi-Vinciguerra.md) |
## 🏷️ Botasaurus
| Title | Date | Link |
|-------|------|------|
| THE LAB #73: How to Bypass Cloudflare in 2025 | 2025-05-20 | [THE LAB #73: How to Bypass Cloudflare in 2025](articles/THE-LAB-#73-How-to-Bypass-Cloudflare-in-2025.md) |
| Testing the new Botasaurus 4 - by Pierluigi Vinciguerra | 2025-05-20 | [Testing the new Botasaurus 4 - by Pierluigi Vinciguerra](articles/Testing-the-new-Botasaurus-4---by-Pierluigi-Vinciguerra.md) |
| Open source Python libraries for your web scraping projects | 2025-05-20 | [Open source Python libraries for your web scraping projects](articles/Open-source-Python-libraries-for-your-web-scraping-projects.md) |
| Botasaurus: an anti-ban web scraping framework | 2024-03-10 | [Botasaurus: an anti-ban web scraping framework](articles/Botasaurus-an-anti-ban-web-scraping-framework.md) |
## 🏷️ BrightData
| Title | Date | Link |
|-------|------|------|
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | [The Great Web Unblocker Benchmark - Cloudflare Edition](articles/The-Great-Web-Unblocker-Benchmark---Cloudflare-Edition.md) |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | [The Great Web Unblocker Benchmark: Kasada edition](articles/The-Great-Web-Unblocker-Benchmark-Kasada-edition.md) |
| Bypassing Kasada for web scraping 2024 edition | 2024-09-30 | [Bypassing Kasada for web scraping 2024 edition](articles/Bypassing-Kasada-for-web-scraping-2024-edition.md) |
| The state of public web data in 2024 | 2024-05-05 | [The state of public web data in 2024](articles/The-state-of-public-web-data-in-2024.md) |
| The Great Web Unblocker Benchmark: March 2024 | 2024-03-19 | [The Great Web Unblocker Benchmark: March 2024](articles/The-Great-Web-Unblocker-Benchmark-March-2024.md) |
| Testing the Bright Data Web Unblocker proxy | 2023-12-08 | [Testing the Bright Data Web Unblocker proxy](articles/Testing-the-Bright-Data-Web-Unblocker-proxy.md) |
| Scraping Kasada protected websites | 2023-10-13 | [Scraping Kasada protected websites](articles/Scraping-Kasada-protected-websites.md) |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | [How to by-pass Kasada bot mitigation?](articles/How-to-by-pass-Kasada-bot-mitigation.md) |
## 🏷️ Browser
| Title | Date | Link |
|-------|------|------|
| The Browser Automation Landscape in 2025 | 2025-05-20 | [The Browser Automation Landscape in 2025](articles/The-Browser-Automation-Landscape-in-2025.md) |
| Web Unblocker vs. Browser as a service for scraping | 2025-05-20 | [Web Unblocker vs. Browser as a service for scraping](articles/Web-Unblocker-vs.-Browser-as-a-service-for-scraping.md) |
| Rethinking the web browser - by Katie Hallett | 2025-01-21 | [Rethinking the web browser - by Katie Hallett](articles/Rethinking-the-web-browser---by-Katie-Hallett.md) |
| THE LAB #20 - AI powered web scrapers with Nimble Browser | 2023-10-13 | [THE LAB #20 - AI powered web scrapers with Nimble Browser](articles/THE-LAB-#20---AI-powered-web-scrapers-with-Nimble-Browser.md) |
## 🏷️ BrowserAPI
| Title | Date | Link |
|-------|------|------|
| Google has exclusive access to a browser API | 2025-05-20 | [Google has exclusive access to a browser API](articles/Google-has-exclusive-access-to-a-browser-API.md) |
## 🏷️ BrowserFingerprint
| Title | Date | Link |
|-------|------|------|
| Browser Fingerprinting 101 - What it is and how it works | 2025-05-20 | [Browser Fingerprinting 101 - What it is and how it works](articles/Browser-Fingerprinting-101---What-it-is-and-how-it-works.md) |
| Making Playwright scrapers undetected with open source solutions | 2025-05-20 | [Making Playwright scrapers undetected with open source solutions](articles/Making-Playwright-scrapers-undetected-with-open-source-solutions.md) |
| The Lab #55: Checking your browser fingerprint | 2025-05-20 | [The Lab #55: Checking your browser fingerprint](articles/The-Lab-#55-Checking-your-browser-fingerprint.md) |
| Google has exclusive access to a browser API | 2025-05-20 | [Google has exclusive access to a browser API](articles/Google-has-exclusive-access-to-a-browser-API.md) |
| The Lab #46: Fingerprint injection in Playwright | 2025-01-26 | [The Lab #46: Fingerprint injection in Playwright](articles/The-Lab-#46-Fingerprint-injection-in-Playwright.md) |
| The latest papers in 2023 about browser fingerprinting | 2024-02-11 | [The latest papers in 2023 about browser fingerprinting](articles/The-latest-papers-in-2023-about-browser-fingerprinting.md) |
| THE LAB 33: Fingerprinting at different connection layers | 2023-11-30 | [THE LAB 33: Fingerprinting at different connection layers](articles/THE-LAB-33-Fingerprinting-at-different-connection-layers.md) |
| What is device fingerprinting? A deep dive | 2023-10-13 | [What is device fingerprinting? A deep dive](articles/What-is-device-fingerprinting-A-deep-dive.md) |
| Browser fingerprinting and web scraping | 2023-10-13 | [Browser fingerprinting and web scraping](articles/Browser-fingerprinting-and-web-scraping.md) |
| Browser API: an introduction - by Pierluigi Vinciguerra | 2023-10-13 | [Browser API: an introduction - by Pierluigi Vinciguerra](articles/Browser-API-an-introduction---by-Pierluigi-Vinciguerra.md) |
| Is web scraping becoming harder? - by Pierluigi Vinciguerra | 2023-10-13 | [Is web scraping becoming harder? - by Pierluigi Vinciguerra](articles/Is-web-scraping-becoming-harder---by-Pierluigi-Vinciguerra.md) |
| From Traditional Browsers to AI-Powered Web Scraping | 2023-10-13 | [From Traditional Browsers to AI-Powered Web Scraping](articles/From-Traditional-Browsers-to-AI-Powered-Web-Scraping.md) |
| THE LAB #19: How to mask the device fingerprint | 2023-09-11 | [THE LAB #19: How to mask the device fingerprint](articles/THE-LAB-#19-How-to-mask-the-device-fingerprint.md) |
## 🏷️ BrowserForge
| Title | Date | Link |
|-------|------|------|
| Bypassing Cloudflare with open source repositories | 2024-09-18 | [Bypassing Cloudflare with open source repositories](articles/Bypassing-Cloudflare-with-open-source-repositories.md) |
## 🏷️ Business
| Title | Date | Link |
|-------|------|------|
| Stuck? More of the Same Won’t Do - by Andrea Squatrito | 2025-05-20 | [Stuck? More of the Same Won’t Do - by Andrea Squatrito](articles/Stuck-More-of-the-Same-Won’t-Do---by-Andrea-Squatrito.md) |
| The importance of scraping inventory levels data in the retail industry | 2025-05-20 | [The importance of scraping inventory levels data in the retail industry](articles/The-importance-of-scraping-inventory-levels-data-in-the-retail-industry.md) |
| Is web scraping a profitable industry? | 2025-05-20 | [Is web scraping a profitable industry?](articles/Is-web-scraping-a-profitable-industry.md) |
| Three ways to make money with web scraping as a freelancer | 2025-05-20 | [Three ways to make money with web scraping as a freelancer](articles/Three-ways-to-make-money-with-web-scraping-as-a-freelancer.md) |
| THE LAB #31: Scraping location data using a world grid | 2025-01-16 | [THE LAB #31: Scraping location data using a world grid](articles/THE-LAB-#31-Scraping-location-data-using-a-world-grid.md) |
| How We Scraped Global Hotel Data to Track Economic Trends | 2024-12-17 | [How We Scraped Global Hotel Data to Track Economic Trends](articles/How-We-Scraped-Global-Hotel-Data-to-Track-Economic-Trends.md) |
| How Scraping the Web Became an Expensive Business | 2024-12-10 | [How Scraping the Web Became an Expensive Business](articles/How-Scraping-the-Web-Became-an-Expensive-Business.md) |
| Scraping The Inflation - by Andrea Squatrito | 2024-12-03 | [Scraping The Inflation - by Andrea Squatrito](articles/Scraping-The-Inflation---by-Andrea-Squatrito.md) |
| THE LAB #26: From internal API to insights. | 2024-10-31 | [THE LAB #26: From internal API to insights.](articles/THE-LAB-#26-From-internal-API-to-insights..md) |
| Web Scraping from 0 to hero: kickstart your career in web scraping | 2024-05-26 | [Web Scraping from 0 to hero: kickstart your career in web scraping](articles/Web-Scraping-from-0-to-hero-kickstart-your-career-in-web-scraping.md) |
| 10 years of web scraping: a perspective about selling web data | 2024-03-24 | [10 years of web scraping: a perspective about selling web data](articles/10-years-of-web-scraping-a-perspective-about-selling-web-data.md) |
| The Lab #43: Scraping inventory data: why, how and where | 2024-02-29 | [The Lab #43: Scraping inventory data: why, how and where](articles/The-Lab-#43-Scraping-inventory-data-why,-how-and-where.md) |
| How to monetize web scraping skills on Data Boutique? | 2024-02-08 | [How to monetize web scraping skills on Data Boutique?](articles/How-to-monetize-web-scraping-skills-on-Data-Boutique.md) |
| Monetize your web scraping skills: a brief guide | 2024-01-14 | [Monetize your web scraping skills: a brief guide](articles/Monetize-your-web-scraping-skills-a-brief-guide.md) |
| From 0 to 2 Billion Prices scraped per months | 2023-10-13 | [From 0 to 2 Billion Prices scraped per months](articles/From-0-to-2-Billion-Prices-scraped-per-months.md) |
| THE LAB #28: Deep dive on inventory levels tracking | 2023-09-28 | [THE LAB #28: Deep dive on inventory levels tracking](articles/THE-LAB-#28-Deep-dive-on-inventory-levels-tracking.md) |
| THE LAB #27: Scraping stock level data to estimate revenues | 2023-09-13 | [THE LAB #27: Scraping stock level data to estimate revenues](articles/THE-LAB-#27-Scraping-stock-level-data-to-estimate-revenues.md) |
## 🏷️ CAPTCHA
| Title | Date | Link |
|-------|------|------|
| Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra | 2023-10-13 | [Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra](articles/Are-CAPTCHAs-still-a-thing---by-Pierluigi-Vinciguerra.md) |
## 🏷️ CDP
| Title | Date | Link |
|-------|------|------|
| The Lab #57: Improving your Playwright scraper and avoid CDP detection | 2025-05-20 | [The Lab #57: Improving your Playwright scraper and avoid CDP detection](articles/The-Lab-#57-Improving-your-Playwright-scraper-and-avoid-CDP-detection.md) |
## 🏷️ CSS
| Title | Date | Link |
|-------|------|------|
| XPATH and CSS Selectors in Web Scraping | 2024-04-28 | [XPATH and CSS Selectors in Web Scraping](articles/XPATH-and-CSS-Selectors-in-Web-Scraping.md) |
| XPath vs CSS selectors: a comparison | 2023-10-13 | [XPath vs CSS selectors: a comparison](articles/XPath-vs-CSS-selectors-a-comparison.md) |
## 🏷️ Camoufox
| Title | Date | Link |
|-------|------|------|
| THE LAB #65: Scraping Datadome protected websites with Camoufox | 2025-05-20 | [THE LAB #65: Scraping Datadome protected websites with Camoufox](articles/THE-LAB-#65-Scraping-Datadome-protected-websites-with-Camoufox.md) |
| THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 | 2025-05-20 | [THE LAB #76: Bypassing Kasada With Open Source Tools In 2025](articles/THE-LAB-#76-Bypassing-Kasada-With-Open-Source-Tools-In-2025.md) |
| THE LAB #83: Camoufox as a containerized server | 2025-05-19 | [THE LAB #83: Camoufox as a containerized server](articles/THE-LAB-#83-Camoufox-as-a-containerized-server.md) |
## 🏷️ Castle
| Title | Date | Link |
|-------|------|------|
| Scraping APIs with Bearer Token - by Pierluigi Vinciguerra | 2025-05-20 | [Scraping APIs with Bearer Token - by Pierluigi Vinciguerra](articles/Scraping-APIs-with-Bearer-Token---by-Pierluigi-Vinciguerra.md) |
## 🏷️ ChangeDetection
| Title | Date | Link |
|-------|------|------|
| Change detection for web scraping: tools and techniques | 2023-10-15 | [Change detection for web scraping: tools and techniques](articles/Change-detection-for-web-scraping-tools-and-techniques.md) |
## 🏷️ Changedetectionio
| Title | Date | Link |
|-------|------|------|
| Change detection for web scraping: tools and techniques | 2023-10-15 | [Change detection for web scraping: tools and techniques](articles/Change-detection-for-web-scraping-tools-and-techniques.md) |
## 🏷️ ChatGPT
| Title | Date | Link |
|-------|------|------|
| Scrape like a pro... but not like an AI company | 2025-05-20 | [Scrape like a pro... but not like an AI company](articles/Scrape-like-a-pro...-but-not-like-an-AI-company.md) |
| Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared | 2025-05-20 | [Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared](articles/Writing-scrapers-with-LLMs-GPT4,-LLama3.1,-Mistral-compared.md) |
| No-Code Web Scraping with Make.com | 2025-05-20 | [No-Code Web Scraping with Make.com](articles/No-Code-Web-Scraping-with-Make.com.md) |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | [Web Scraping experts: Is AI stealing our job?](articles/Web-Scraping-experts-Is-AI-stealing-our-job.md) |
| Writing a web scraper with ChatGPT. Is it a good idea? | 2023-10-13 | [Writing a web scraper with ChatGPT. Is it a good idea?](articles/Writing-a-web-scraper-with-ChatGPT.-Is-it-a-good-idea.md) |
| How to create a web scraper with ChatGPT | 2023-10-13 | [How to create a web scraper with ChatGPT](articles/How-to-create-a-web-scraper-with-ChatGPT.md) |
## 🏷️ Ciphers
| Title | Date | Link |
|-------|------|------|
| THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting | 2023-05-29 | [THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting](articles/THE-LAB-#6-Changing-Ciphers-in-Scrapy-to-avoid-bans-by-TLS-Fingerprinting.md) |
## 🏷️ Claude
| Title | Date | Link |
|-------|------|------|
| Build your web scraping assistant with Claude and Cursor | 2025-05-20 | [Build your web scraping assistant with Claude and Cursor](articles/Build-your-web-scraping-assistant-with-Claude-and-Cursor.md) |
## 🏷️ Cloudflare
| Title | Date | Link |
|-------|------|------|
| THE LAB #3: Scraping Cloudflare protected websites | 2025-06-07 | [THE LAB #3: Scraping Cloudflare protected websites](articles/THE-LAB-#3-Scraping-Cloudflare-protected-websites.md) |
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | [The Great Web Unblocker Benchmark - Cloudflare Edition](articles/The-Great-Web-Unblocker-Benchmark---Cloudflare-Edition.md) |
| THE LAB #73: How to Bypass Cloudflare in 2025 | 2025-05-20 | [THE LAB #73: How to Bypass Cloudflare in 2025](articles/THE-LAB-#73-How-to-Bypass-Cloudflare-in-2025.md) |
| Scraping Cloudflare websites using an API | 2025-05-20 | [Scraping Cloudflare websites using an API](articles/Scraping-Cloudflare-websites-using-an-API.md) |
| Testing the new Botasaurus 4 - by Pierluigi Vinciguerra | 2025-05-20 | [Testing the new Botasaurus 4 - by Pierluigi Vinciguerra](articles/Testing-the-new-Botasaurus-4---by-Pierluigi-Vinciguerra.md) |
| THE LAB #62: Bypassing Cloudflare with Nodriver | 2025-05-20 | [THE LAB #62: Bypassing Cloudflare with Nodriver](articles/THE-LAB-#62-Bypassing-Cloudflare-with-Nodriver.md) |
| The Lab #36: Bypassing Cloudflare with anti-detect browsers | 2025-04-16 | [The Lab #36: Bypassing Cloudflare with anti-detect browsers](articles/The-Lab-#36-Bypassing-Cloudflare-with-anti-detect-browsers.md) |
| Bypassing Cloudflare with open source repositories | 2024-09-18 | [Bypassing Cloudflare with open source repositories](articles/Bypassing-Cloudflare-with-open-source-repositories.md) |
| The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2 | 2024-01-19 | [The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2](articles/The-Lab-#37-Bypassing-Cloudflare-with-anti-detect-browsers---Part-2.md) |
| THE LAB 32: hRequests vs anti-bots: a full benchmark | 2023-11-30 | [THE LAB 32: hRequests vs anti-bots: a full benchmark](articles/THE-LAB-32-hRequests-vs-anti-bots-a-full-benchmark.md) |
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | [Can Undetected Chromedriver bypass Cloudflare or Datadome?](articles/Can-Undetected-Chromedriver-bypass-Cloudflare-or-Datadome.md) |
| Cloudflare Turnstile: what is that and how it works? | 2023-10-13 | [Cloudflare Turnstile: what is that and how it works?](articles/Cloudflare-Turnstile-what-is-that-and-how-it-works.md) |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | [THE LAB #21 - Bypass anti-bot challenges with AI](articles/THE-LAB-#21---Bypass-anti-bot-challenges-with-AI.md) |
| THE LAB #29: Bypass Cloudflare Bot Protection with Scrapy | 2023-10-12 | [THE LAB #29: Bypass Cloudflare Bot Protection with Scrapy](articles/THE-LAB-#29-Bypass-Cloudflare-Bot-Protection-with-Scrapy.md) |
| Scraping Cloudflare Protected Websites (early 2023 version) | 2023-06-10 | [Scraping Cloudflare Protected Websites (early 2023 version)](articles/Scraping-Cloudflare-Protected-Websites-(early-2023-version).md) |
| THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin | 2023-05-29 | [THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin](articles/THE-LAB-#10-Bypass-Cloudflare-Bot-Protection-with-GoLogin.md) |
## 🏷️ Cloudscraper
| Title | Date | Link |
|-------|------|------|
| THE LAB #73: How to Bypass Cloudflare in 2025 | 2025-05-20 | [THE LAB #73: How to Bypass Cloudflare in 2025](articles/THE-LAB-#73-How-to-Bypass-Cloudflare-in-2025.md) |
## 🏷️ Codex
| Title | Date | Link |
|-------|------|------|
| THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools | 2025-05-22 | [THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools](articles/THE-LAB-#84-AI-Driven-Web-Scraping-OpenAI-Codex-vs-Cursor-vs-AI-Scraping-Tools.md) |
## 🏷️ Consultancy
| Title | Date | Link |
|-------|------|------|
| Stuck? More of the Same Won’t Do - by Andrea Squatrito | 2025-05-20 | [Stuck? More of the Same Won’t Do - by Andrea Squatrito](articles/Stuck-More-of-the-Same-Won’t-Do---by-Andrea-Squatrito.md) |
## 🏷️ Container
| Title | Date | Link |
|-------|------|------|
| THE LAB #83: Camoufox as a containerized server | 2025-05-19 | [THE LAB #83: Camoufox as a containerized server](articles/THE-LAB-#83-Camoufox-as-a-containerized-server.md) |
## 🏷️ Costs
| Title | Date | Link |
|-------|------|------|
| The Unit Economics of Proxy Providers - by Abed | 2025-05-27 | [The Unit Economics of Proxy Providers - by Abed](articles/The-Unit-Economics-of-Proxy-Providers---by-Abed.md) |
| Analyzing the cost of a web scraping project | 2025-05-20 | [Analyzing the cost of a web scraping project](articles/Analyzing-the-cost-of-a-web-scraping-project.md) |
| THE LAB #61: Evaluating your proxy provider | 2025-05-20 | [THE LAB #61: Evaluating your proxy provider](articles/THE-LAB-#61-Evaluating-your-proxy-provider.md) |
| Optimizing Proxy Usage for Large-Scale Scraping | 2025-05-20 | [Optimizing Proxy Usage for Large-Scale Scraping](articles/Optimizing-Proxy-Usage-for-Large-Scale-Scraping.md) |
| Optimizing costs for large-scale scraping operations | 2025-05-20 | [Optimizing costs for large-scale scraping operations](articles/Optimizing-costs-for-large-scale-scraping-operations.md) |
| The Web Unblocker Cost Benchmark - by Pierluigi Vinciguerra | 2025-02-14 | [The Web Unblocker Cost Benchmark - by Pierluigi Vinciguerra](articles/The-Web-Unblocker-Cost-Benchmark---by-Pierluigi-Vinciguerra.md) |
| How Scraping the Web Became an Expensive Business | 2024-12-10 | [How Scraping the Web Became an Expensive Business](articles/How-Scraping-the-Web-Became-an-Expensive-Business.md) |
| Scrapoxy, the super proxy aggregator, how it works? | 2024-02-21 | [Scrapoxy, the super proxy aggregator, how it works?](articles/Scrapoxy,-the-super-proxy-aggregator,-how-it-works.md) |
| How scraping a single website costed thousands of dollars in proxy | 2024-01-28 | [How scraping a single website costed thousands of dollars in proxy](articles/How-scraping-a-single-website-costed-thousands-of-dollars-in-proxy.md) |
| The true costs of a web scraping project | 2023-11-25 | [The true costs of a web scraping project](articles/The-true-costs-of-a-web-scraping-project.md) |
| The costs of web scraping - by Pierluigi Vinciguerra | 2023-10-13 | [The costs of web scraping - by Pierluigi Vinciguerra](articles/The-costs-of-web-scraping---by-Pierluigi-Vinciguerra.md) |
## 🏷️ Crawlee
| Title | Date | Link |
|-------|------|------|
| The most interesting GitHub Repositories about web scraping (2023) | 2023-10-13 | [The most interesting GitHub Repositories about web scraping (2023)](articles/The-most-interesting-GitHub-Repositories-about-web-scraping-(2023).md) |
## 🏷️ Cursor
| Title | Date | Link |
|-------|------|------|
| THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools | 2025-05-22 | [THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools](articles/THE-LAB-#84-AI-Driven-Web-Scraping-OpenAI-Codex-vs-Cursor-vs-AI-Scraping-Tools.md) |
| Use Cursor as web scraping assistant with MCP servers | 2025-05-20 | [Use Cursor as web scraping assistant with MCP servers](articles/Use-Cursor-as-web-scraping-assistant-with-MCP-servers.md) |
| Build your web scraping assistant with Claude and Cursor | 2025-05-20 | [Build your web scraping assistant with Claude and Cursor](articles/Build-your-web-scraping-assistant-with-Claude-and-Cursor.md) |
## 🏷️ DataQuality
| Title | Date | Link |
|-------|------|------|
| THE LAB #69: Building a dashboard for your scrapers with Grafana | 2025-05-20 | [THE LAB #69: Building a dashboard for your scrapers with Grafana](articles/THE-LAB-#69-Building-a-dashboard-for-your-scrapers-with-Grafana.md) |
| Web Scraping from 0 to hero: data cleaning processes | 2024-05-12 | [Web Scraping from 0 to hero: data cleaning processes](articles/Web-Scraping-from-0-to-hero-data-cleaning-processes.md) |
| Ensuring data quality in web scraping projects | 2023-10-13 | [Ensuring data quality in web scraping projects](articles/Ensuring-data-quality-in-web-scraping-projects.md) |
## 🏷️ Datadoma
| Title | Date | Link |
|-------|------|------|
| Testing the new Botasaurus 4 - by Pierluigi Vinciguerra | 2025-05-20 | [Testing the new Botasaurus 4 - by Pierluigi Vinciguerra](articles/Testing-the-new-Botasaurus-4---by-Pierluigi-Vinciguerra.md) |
| Web Scraping Idealista and Bypass Idealista Blockers | 2024-08-06 | [Web Scraping Idealista and Bypass Idealista Blockers](articles/Web-Scraping-Idealista-and-Bypass-Idealista-Blockers.md) |
## 🏷️ Datadome
| Title | Date | Link |
|-------|------|------|
| THE LAB #65: Scraping Datadome protected websites with Camoufox | 2025-05-20 | [THE LAB #65: Scraping Datadome protected websites with Camoufox](articles/THE-LAB-#65-Scraping-Datadome-protected-websites-with-Camoufox.md) |
| THE LAB #82: How to scrape Vinted using their internal APIs | 2025-05-20 | [THE LAB #82: How to scrape Vinted using their internal APIs](articles/THE-LAB-#82-How-to-scrape-Vinted-using-their-internal-APIs.md) |
| THE LAB #2: scraping data from a website with Datadome and xsrf tokens | 2025-03-28 | [THE LAB #2: scraping data from a website with Datadome and xsrf tokens](articles/THE-LAB-#2-scraping-data-from-a-website-with-Datadome-and-xsrf-tokens.md) |
| Botasaurus: an anti-ban web scraping framework | 2024-03-10 | [Botasaurus: an anti-ban web scraping framework](articles/Botasaurus-an-anti-ban-web-scraping-framework.md) |
| Bypassing Datadome with Web Scraping - End of 2023 Version | 2023-12-06 | [Bypassing Datadome with Web Scraping - End of 2023 Version](articles/Bypassing-Datadome-with-Web-Scraping---End-of-2023-Version.md) |
| THE LAB 32: hRequests vs anti-bots: a full benchmark | 2023-11-30 | [THE LAB 32: hRequests vs anti-bots: a full benchmark](articles/THE-LAB-32-hRequests-vs-anti-bots-a-full-benchmark.md) |
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | [Can Undetected Chromedriver bypass Cloudflare or Datadome?](articles/Can-Undetected-Chromedriver-bypass-Cloudflare-or-Datadome.md) |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | [THE LAB #21 - Bypass anti-bot challenges with AI](articles/THE-LAB-#21---Bypass-anti-bot-challenges-with-AI.md) |
| How to scrape Datadome protected websites (early 2023 version) | 2023-05-29 | [How to scrape Datadome protected websites (early 2023 version)](articles/How-to-scrape-Datadome-protected-websites-(early-2023-version).md) |
## 🏷️ Datasets
| Title | Date | Link |
|-------|------|------|
| THE LAB #86: Querying Web Data using GPT-Like Web Interface | 2025-06-05 | [THE LAB #86: Querying Web Data using GPT-Like Web Interface](articles/THE-LAB-#86-Querying-Web-Data-using-GPT-Like-Web-Interface.md) |
| Creating a dataset for investors with web scraping: Tesla (TSLA) | 2025-03-30 | [Creating a dataset for investors with web scraping: Tesla (TSLA)](articles/Creating-a-dataset-for-investors-with-web-scraping-Tesla-(TSLA).md) |
| How to monetize web scraping skills on Data Boutique? | 2024-02-08 | [How to monetize web scraping skills on Data Boutique?](articles/How-to-monetize-web-scraping-skills-on-Data-Boutique.md) |
## 🏷️ Deals
| Title | Date | Link |
|-------|------|------|
| Club Deals - by Pierluigi Vinciguerra | 2025-06-13 | [Club Deals - by Pierluigi Vinciguerra](articles/Club-Deals---by-Pierluigi-Vinciguerra.md) |
## 🏷️ Decodo
| Title | Date | Link |
|-------|------|------|
| Hands On #4: Testing the new Smartproxy Site Unblocker | 2023-10-13 | [Hands On #4: Testing the new Smartproxy Site Unblocker](articles/Hands-On-#4-Testing-the-new-Smartproxy-Site-Unblocker.md) |
| Tik Tok Scraping: how to do it properly | 2023-10-13 | [Tik Tok Scraping: how to do it properly](articles/Tik-Tok-Scraping-how-to-do-it-properly.md) |
## 🏷️ Discounts
| Title | Date | Link |
|-------|------|------|
| Club Deals - by Pierluigi Vinciguerra | 2025-06-13 | [Club Deals - by Pierluigi Vinciguerra](articles/Club-Deals---by-Pierluigi-Vinciguerra.md) |
## 🏷️ E-commerce
| Title | Date | Link |
|-------|------|------|
| Web scraping and journalism: the Chiara Ferragni case | 2025-05-20 | [Web scraping and journalism: the Chiara Ferragni case](articles/Web-scraping-and-journalism-the-Chiara-Ferragni-case.md) |
| Scraping E-Commerce websites 101 - by Pierluigi Vinciguerra | 2023-10-13 | [Scraping E-Commerce websites 101 - by Pierluigi Vinciguerra](articles/Scraping-E-Commerce-websites-101---by-Pierluigi-Vinciguerra.md) |
## 🏷️ F5
| Title | Date | Link |
|-------|------|------|
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | [Can Undetected Chromedriver bypass Cloudflare or Datadome?](articles/Can-Undetected-Chromedriver-bypass-Cloudflare-or-Datadome.md) |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | [THE LAB #21 - Bypass anti-bot challenges with AI](articles/THE-LAB-#21---Bypass-anti-bot-challenges-with-AI.md) |
## 🏷️ FabianoSileo
| Title | Date | Link |
|-------|------|------|
| Interview #8 - Fabiano Sileo - by Pierluigi Vinciguerra | 2023-10-13 | [Interview #8 - Fabiano Sileo - by Pierluigi Vinciguerra](articles/Interview-#8---Fabiano-Sileo---by-Pierluigi-Vinciguerra.md) |
## 🏷️ Fiddler
| Title | Date | Link |
|-------|------|------|
| The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 | 2025-05-20 | [The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2](articles/The-Lab-#59-Bypassing-certificate-pinning-with-Frida-and-Fiddler---part-2.md) |
## 🏷️ Firecrawl
| Title | Date | Link |
|-------|------|------|
| Building a custom GPT using Firecrawl | 2025-05-20 | [Building a custom GPT using Firecrawl](articles/Building-a-custom-GPT-using-Firecrawl.md) |
## 🏷️ GPT
| Title | Date | Link |
|-------|------|------|
| THE LAB #86: Querying Web Data using GPT-Like Web Interface | 2025-06-05 | [THE LAB #86: Querying Web Data using GPT-Like Web Interface](articles/THE-LAB-#86-Querying-Web-Data-using-GPT-Like-Web-Interface.md) |
| THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools | 2025-05-22 | [THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools](articles/THE-LAB-#84-AI-Driven-Web-Scraping-OpenAI-Codex-vs-Cursor-vs-AI-Scraping-Tools.md) |
| Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared | 2025-05-20 | [Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared](articles/Writing-scrapers-with-LLMs-GPT4,-LLama3.1,-Mistral-compared.md) |
| Building a custom GPT using Firecrawl | 2025-05-20 | [Building a custom GPT using Firecrawl](articles/Building-a-custom-GPT-using-Firecrawl.md) |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | [Web Scraping experts: Is AI stealing our job?](articles/Web-Scraping-experts-Is-AI-stealing-our-job.md) |
| The state of web scraping and AI - by Pierluigi Vinciguerra | 2023-10-13 | [The state of web scraping and AI - by Pierluigi Vinciguerra](articles/The-state-of-web-scraping-and-AI---by-Pierluigi-Vinciguerra.md) |
## 🏷️ Geofencing
| Title | Date | Link |
|-------|------|------|
| Bypassing Geo-fencing While Scraping | 2024-03-25 | [Bypassing Geo-fencing While Scraping](articles/Bypassing-Geo-fencing-While-Scraping.md) |
| Buy cheaper plane tickets using a VPN: truth or myth? | 2023-09-11 | [Buy cheaper plane tickets using a VPN: truth or myth?](articles/Buy-cheaper-plane-tickets-using-a-VPN-truth-or-myth.md) |
## 🏷️ GermanasLatvaitis
| Title | Date | Link |
|-------|------|------|
| Interview #10 - Germanas Latvaitis | 2023-10-13 | [Interview #10 - Germanas Latvaitis](articles/Interview-#10---Germanas-Latvaitis.md) |
## 🏷️ GhostCursor
| Title | Date | Link |
|-------|------|------|
| Mouse movements in Playwright with Ghost Cursor | 2024-10-13 | [Mouse movements in Playwright with Ghost Cursor](articles/Mouse-movements-in-Playwright-with-Ghost-Cursor.md) |
| Bypassing Datadome with Web Scraping - End of 2023 Version | 2023-12-06 | [Bypassing Datadome with Web Scraping - End of 2023 Version](articles/Bypassing-Datadome-with-Web-Scraping---End-of-2023-Version.md) |
## 🏷️ Github
| Title | Date | Link |
|-------|------|------|
| THE LAB #74: Running scrapers on GitHub Actions | 2025-05-20 | [THE LAB #74: Running scrapers on GitHub Actions](articles/THE-LAB-#74-Running-scrapers-on-GitHub-Actions.md) |
## 🏷️ Glovo
| Title | Date | Link |
|-------|------|------|
| Scraping food delivery data - by Pierluigi Vinciguerra | 2025-05-20 | [Scraping food delivery data - by Pierluigi Vinciguerra](articles/Scraping-food-delivery-data---by-Pierluigi-Vinciguerra.md) |
## 🏷️ GoLogin
| Title | Date | Link |
|-------|------|------|
| Scraping Kasada protected websites | 2023-10-13 | [Scraping Kasada protected websites](articles/Scraping-Kasada-protected-websites.md) |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | [How to by-pass Kasada bot mitigation?](articles/How-to-by-pass-Kasada-bot-mitigation.md) |
| Scraping Cloudflare Protected Websites (early 2023 version) | 2023-06-10 | [Scraping Cloudflare Protected Websites (early 2023 version)](articles/Scraping-Cloudflare-Protected-Websites-(early-2023-version).md) |
| THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin | 2023-05-29 | [THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin](articles/THE-LAB-#10-Bypass-Cloudflare-Bot-Protection-with-GoLogin.md) |
## 🏷️ Google
| Title | Date | Link |
|-------|------|------|
| The Scriptwall: Why Google is hiding its SERP content behind Javascript | 2025-05-20 | [The Scriptwall: Why Google is hiding its SERP content behind Javascript](articles/The-Scriptwall-Why-Google-is-hiding-its-SERP-content-behind-Javascript.md) |
## 🏷️ Grafana
| Title | Date | Link |
|-------|------|------|
| THE LAB #69: Building a dashboard for your scrapers with Grafana | 2025-05-20 | [THE LAB #69: Building a dashboard for your scrapers with Grafana](articles/THE-LAB-#69-Building-a-dashboard-for-your-scrapers-with-Grafana.md) |
## 🏷️ HTTPToolkit
| Title | Date | Link |
|-------|------|------|
| How to Scrape Data from Mobile Apps using HTTP Toolkit | 2025-05-20 | [How to Scrape Data from Mobile Apps using HTTP Toolkit](articles/How-to-Scrape-Data-from-Mobile-Apps-using-HTTP-Toolkit.md) |
| Scraping food delivery data - by Pierluigi Vinciguerra | 2025-05-20 | [Scraping food delivery data - by Pierluigi Vinciguerra](articles/Scraping-food-delivery-data---by-Pierluigi-Vinciguerra.md) |
| HTTP Toolkit, your best friend for network inspection | 2025-05-20 | [HTTP Toolkit, your best friend for network inspection](articles/HTTP-Toolkit,-your-best-friend-for-network-inspection.md) |
## 🏷️ HistoricalData
| Title | Date | Link |
|-------|------|------|
| Scraping Historical Data From the Wayback Machine | 2025-05-20 | [Scraping Historical Data From the Wayback Machine](articles/Scraping-Historical-Data-From-the-Wayback-Machine.md) |
## 🏷️ Hotel
| Title | Date | Link |
|-------|------|------|
| How We Scraped Global Hotel Data to Track Economic Trends | 2024-12-17 | [How We Scraped Global Hotel Data to Track Economic Trends](articles/How-We-Scraped-Global-Hotel-Data-to-Track-Economic-Trends.md) |
## 🏷️ Hrequests
| Title | Date | Link |
|-------|------|------|
| THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools | 2025-06-20 | [THE LAB #87: Bypassing ReCAPTCHAs with open source and commercial tools](articles/THE-LAB-#87-Bypassing-ReCAPTCHAs-with-open-source-and-commercial-tools.md) |
| THE LAB 32: hRequests vs anti-bots: a full benchmark | 2023-11-30 | [THE LAB 32: hRequests vs anti-bots: a full benchmark](articles/THE-LAB-32-hRequests-vs-anti-bots-a-full-benchmark.md) |
| hRequests: bypass Akamai with Python requests | 2023-11-12 | [hRequests: bypass Akamai with Python requests](articles/hRequests-bypass-Akamai-with-Python-requests.md) |
| HTTP requests in Python explained | 2023-10-13 | [HTTP requests in Python explained](articles/HTTP-requests-in-Python-explained.md) |
## 🏷️ IKEA
| Title | Date | Link |
|-------|------|------|
| The Kallax Index - Scraping Ikea websites | 2023-10-13 | [The Kallax Index - Scraping Ikea websites](articles/The-Kallax-Index---Scraping-Ikea-websites.md) |
## 🏷️ Idealista
| Title | Date | Link |
|-------|------|------|
| Web Scraping Idealista and Bypass Idealista Blockers | 2024-08-06 | [Web Scraping Idealista and Bypass Idealista Blockers](articles/Web-Scraping-Idealista-and-Bypass-Idealista-Blockers.md) |
## 🏷️ Incognition
| Title | Date | Link |
|-------|------|------|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | [In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers](articles/In-Depth-Pricing-Comparison-of-Anti-Detect-Browsers-for-Web-Scrapers.md) |
## 🏷️ Infatica
| Title | Date | Link |
|-------|------|------|
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | [The Great Web Unblocker Benchmark - Cloudflare Edition](articles/The-Great-Web-Unblocker-Benchmark---Cloudflare-Edition.md) |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | [The Great Web Unblocker Benchmark: Kasada edition](articles/The-Great-Web-Unblocker-Benchmark-Kasada-edition.md) |
| Hands On #6: Testing the Infatica web scraper | 2023-10-05 | [Hands On #6: Testing the Infatica web scraper](articles/Hands-On-#6-Testing-the-Infatica-web-scraper.md) |
## 🏷️ Infrastructure
| Title | Date | Link |
|-------|------|------|
| THE LAB #72: Advanced logging in Playwright | 2025-05-20 | [THE LAB #72: Advanced logging in Playwright](articles/THE-LAB-#72-Advanced-logging-in-Playwright.md) |
| Analyzing the cost of a web scraping project | 2025-05-20 | [Analyzing the cost of a web scraping project](articles/Analyzing-the-cost-of-a-web-scraping-project.md) |
| THE LAB #69: Building a dashboard for your scrapers with Grafana | 2025-05-20 | [THE LAB #69: Building a dashboard for your scrapers with Grafana](articles/THE-LAB-#69-Building-a-dashboard-for-your-scrapers-with-Grafana.md) |
| THE LAB #74: Running scrapers on GitHub Actions | 2025-05-20 | [THE LAB #74: Running scrapers on GitHub Actions](articles/THE-LAB-#74-Running-scrapers-on-GitHub-Actions.md) |
| THE LAB #71: Sending Scrapy logs to RabbitMQ | 2025-05-20 | [THE LAB #71: Sending Scrapy logs to RabbitMQ](articles/THE-LAB-#71-Sending-Scrapy-logs-to-RabbitMQ.md) |
| THE LAB #66: How to properly scrape a booking website | 2025-05-20 | [THE LAB #66: How to properly scrape a booking website](articles/THE-LAB-#66-How-to-properly-scrape-a-booking-website.md) |
| THE LAB #70: Advanced logging in Scrapy | 2025-05-20 | [THE LAB #70: Advanced logging in Scrapy](articles/THE-LAB-#70-Advanced-logging-in-Scrapy.md) |
| Web DRAGON - LLM-powered web scraping on a distributed cloud | 2023-12-19 | [Web DRAGON - LLM-powered web scraping on a distributed cloud](articles/Web-DRAGON---LLM-powered-web-scraping-on-a-distributed-cloud.md) |
| The costs of web scraping - by Pierluigi Vinciguerra | 2023-10-13 | [The costs of web scraping - by Pierluigi Vinciguerra](articles/The-costs-of-web-scraping---by-Pierluigi-Vinciguerra.md) |
| THE LAB #4: Scrapyd - how to manage and schedule a fleet of scrapers | 2023-05-29 | [THE LAB #4: Scrapyd - how to manage and schedule a fleet of scrapers](articles/THE-LAB-#4-Scrapyd---how-to-manage-and-schedule-a-fleet-of-scrapers.md) |
## 🏷️ Interview
| Title | Date | Link |
|-------|------|------|
| Interview #2: Neil Emeigh - Rayobyte | 2023-10-13 | [Interview #2: Neil Emeigh - Rayobyte](articles/Interview-#2-Neil-Emeigh---Rayobyte.md) |
| Interview #5: Veritas - The anti obfuscation master | 2023-10-13 | [Interview #5: Veritas - The anti obfuscation master](articles/Interview-#5-Veritas---The-anti-obfuscation-master.md) |
| Interview with Uriel Knorovich of Nimble | 2023-10-13 | [Interview with Uriel Knorovich of Nimble](articles/Interview-with-Uriel-Knorovich-of-Nimble.md) |
| Interview #7: Aviv Besinsky - Bright Data | 2023-10-13 | [Interview #7: Aviv Besinsky - Bright Data](articles/Interview-#7-Aviv-Besinsky---Bright-Data.md) |
| Interview #3: Ondra Urban - Apify | 2023-10-13 | [Interview #3: Ondra Urban - Apify](articles/Interview-#3-Ondra-Urban---Apify.md) |
| Interview #4: Martin Ganchev - Smartproxy | 2023-10-13 | [Interview #4: Martin Ganchev - Smartproxy](articles/Interview-#4-Martin-Ganchev---Smartproxy.md) |
| Interview #10 - Germanas Latvaitis | 2023-10-13 | [Interview #10 - Germanas Latvaitis](articles/Interview-#10---Germanas-Latvaitis.md) |
| Interview #6: Aleksandras Šulženko - Oxylabs | 2023-10-13 | [Interview #6: Aleksandras Šulženko - Oxylabs](articles/Interview-#6-Aleksandras-Šulženko---Oxylabs.md) |
| Interview #1: Neha Setia - Zyte - by Pierluigi Vinciguerra | 2023-10-13 | [Interview #1: Neha Setia - Zyte - by Pierluigi Vinciguerra](articles/Interview-#1-Neha-Setia---Zyte---by-Pierluigi-Vinciguerra.md) |
| Interview #8 - Fabiano Sileo - by Pierluigi Vinciguerra | 2023-10-13 | [Interview #8 - Fabiano Sileo - by Pierluigi Vinciguerra](articles/Interview-#8---Fabiano-Sileo---by-Pierluigi-Vinciguerra.md) |
## 🏷️ InventoryData
| Title | Date | Link |
|-------|------|------|
| The importance of scraping inventory levels data in the retail industry | 2025-05-20 | [The importance of scraping inventory levels data in the retail industry](articles/The-importance-of-scraping-inventory-levels-data-in-the-retail-industry.md) |
| THE LAB #28: Deep dive on inventory levels tracking | 2023-09-28 | [THE LAB #28: Deep dive on inventory levels tracking](articles/THE-LAB-#28-Deep-dive-on-inventory-levels-tracking.md) |
| THE LAB #27: Scraping stock level data to estimate revenues | 2023-09-13 | [THE LAB #27: Scraping stock level data to estimate revenues](articles/THE-LAB-#27-Scraping-stock-level-data-to-estimate-revenues.md) |
## 🏷️ JSON
| Title | Date | Link |
|-------|------|------|
| How to Parse JSON with Python: A Beginner-Friendly Guide | 2025-05-20 | [How to Parse JSON with Python: A Beginner-Friendly Guide](articles/How-to-Parse-JSON-with-Python-A-Beginner-Friendly-Guide.md) |
## 🏷️ JWT
| Title | Date | Link |
|-------|------|------|
| THE LAB #64: JWT Tokens and API scraping | 2025-05-20 | [THE LAB #64: JWT Tokens and API scraping](articles/THE-LAB-#64-JWT-Tokens-and-API-scraping.md) |
## 🏷️ Ja3Proxy
| Title | Date | Link |
|-------|------|------|
| THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies | 2025-05-29 | [THE LAB #85: Bypass Akamai Bot Protection by Chaining Proxies](articles/THE-LAB-#85-Bypass-Akamai-Bot-Protection-by-Chaining-Proxies.md) |
## 🏷️ Javascript
| Title | Date | Link |
|-------|------|------|
| The Scriptwall: Why Google is hiding its SERP content behind Javascript | 2025-05-20 | [The Scriptwall: Why Google is hiding its SERP content behind Javascript](articles/The-Scriptwall-Why-Google-is-hiding-its-SERP-content-behind-Javascript.md) |
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | [Web Scraping and Coding: Five Programming Languages to Check Out](articles/Web-Scraping-and-Coding-Five-Programming-Languages-to-Check-Out.md) |
## 🏷️ Kameleo
| Title | Date | Link |
|-------|------|------|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | [In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers](articles/In-Depth-Pricing-Comparison-of-Anti-Detect-Browsers-for-Web-Scrapers.md) |
| The Lab #46: Fingerprint injection in Playwright | 2025-01-26 | [The Lab #46: Fingerprint injection in Playwright](articles/The-Lab-#46-Fingerprint-injection-in-Playwright.md) |
| Behind the scenes of anti-detect browsers - by Tamas Deak | 2024-03-05 | [Behind the scenes of anti-detect browsers - by Tamas Deak](articles/Behind-the-scenes-of-anti-detect-browsers---by-Tamas-Deak.md) |
| The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2 | 2024-01-19 | [The Lab #37: Bypassing Cloudflare with anti-detect browsers - Part 2](articles/The-Lab-#37-Bypassing-Cloudflare-with-anti-detect-browsers---Part-2.md) |
## 🏷️ Kasada
| Title | Date | Link |
|-------|------|------|
| THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 | 2025-05-20 | [THE LAB #76: Bypassing Kasada With Open Source Tools In 2025](articles/THE-LAB-#76-Bypassing-Kasada-With-Open-Source-Tools-In-2025.md) |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | [The Great Web Unblocker Benchmark: Kasada edition](articles/The-Great-Web-Unblocker-Benchmark-Kasada-edition.md) |
| Testing the new Botasaurus 4 - by Pierluigi Vinciguerra | 2025-05-20 | [Testing the new Botasaurus 4 - by Pierluigi Vinciguerra](articles/Testing-the-new-Botasaurus-4---by-Pierluigi-Vinciguerra.md) |
| Bypassing Kasada for web scraping 2024 edition | 2024-09-30 | [Bypassing Kasada for web scraping 2024 edition](articles/Bypassing-Kasada-for-web-scraping-2024-edition.md) |
| Botasaurus: an anti-ban web scraping framework | 2024-03-10 | [Botasaurus: an anti-ban web scraping framework](articles/Botasaurus-an-anti-ban-web-scraping-framework.md) |
| Scraping Kasada protected websites | 2023-10-13 | [Scraping Kasada protected websites](articles/Scraping-Kasada-protected-websites.md) |
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | [Can Undetected Chromedriver bypass Cloudflare or Datadome?](articles/Can-Undetected-Chromedriver-bypass-Cloudflare-or-Datadome.md) |
| Wanted a parka and got an Error 429: Too many requests | 2023-10-13 | [Wanted a parka and got an Error 429: Too many requests](articles/Wanted-a-parka-and-got-an-Error-429-Too-many-requests.md) |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | [How to by-pass Kasada bot mitigation?](articles/How-to-by-pass-Kasada-bot-mitigation.md) |
| What is Kasada bot mitigation? - by Pierluigi Vinciguerra | 2023-10-13 | [What is Kasada bot mitigation? - by Pierluigi Vinciguerra](articles/What-is-Kasada-bot-mitigation---by-Pierluigi-Vinciguerra.md) |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | [THE LAB #21 - Bypass anti-bot challenges with AI](articles/THE-LAB-#21---Bypass-anti-bot-challenges-with-AI.md) |
## 🏷️ LLM
| Title | Date | Link |
|-------|------|------|
| THE LAB #77: Building a Web Scraping Knowledge Assistant with RAG | 2025-05-20 | [THE LAB #77: Building a Web Scraping Knowledge Assistant with RAG](articles/THE-LAB-#77-Building-a-Web-Scraping-Knowledge-Assistant-with-RAG.md) |
| How AI is changing the web scraping industry | 2025-05-20 | [How AI is changing the web scraping industry](articles/How-AI-is-changing-the-web-scraping-industry.md) |
| The AI-Powered web scraping tools landscape | 2025-05-20 | [The AI-Powered web scraping tools landscape](articles/The-AI-Powered-web-scraping-tools-landscape.md) |
| Use Cursor as web scraping assistant with MCP servers | 2025-05-20 | [Use Cursor as web scraping assistant with MCP servers](articles/Use-Cursor-as-web-scraping-assistant-with-MCP-servers.md) |
| THE LAB #75: Building self healing scrapers with AI | 2025-05-20 | [THE LAB #75: Building self healing scrapers with AI](articles/THE-LAB-#75-Building-self-healing-scrapers-with-AI.md) |
| THE LAB #78: Building a Web Scraping Knowledge Assistant with RAG - Part2 | 2025-05-20 | [THE LAB #78: Building a Web Scraping Knowledge Assistant with RAG - Part2](articles/THE-LAB-#78-Building-a-Web-Scraping-Knowledge-Assistant-with-RAG---Part2.md) |
| How LLMs are affecting the costs of web scraping | 2025-05-20 | [How LLMs are affecting the costs of web scraping](articles/How-LLMs-are-affecting-the-costs-of-web-scraping.md) |
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | [Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base](articles/Evolution-from-RAG-to-MCP-A-Breakthrough-for-LLM-Dynamic-Knowledge-Base.md) |
| Is Web Scraping Dead? - by Pierluigi Vinciguerra | 2024-02-25 | [Is Web Scraping Dead? - by Pierluigi Vinciguerra](articles/Is-Web-Scraping-Dead---by-Pierluigi-Vinciguerra.md) |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | [Web Scraping experts: Is AI stealing our job?](articles/Web-Scraping-experts-Is-AI-stealing-our-job.md) |
| How to create a web scraper with ChatGPT | 2023-10-13 | [How to create a web scraper with ChatGPT](articles/How-to-create-a-web-scraper-with-ChatGPT.md) |
## 🏷️ LLMScraping
| Title | Date | Link |
|-------|------|------|
| Scrape like a pro... but not like an AI company | 2025-05-20 | [Scrape like a pro... but not like an AI company](articles/Scrape-like-a-pro...-but-not-like-an-AI-company.md) |
| Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared | 2025-05-20 | [Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared](articles/Writing-scrapers-with-LLMs-GPT4,-LLama3.1,-Mistral-compared.md) |
| The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1 | 2025-05-20 | [The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1](articles/The-Lab-#52-Scraping-with-LLMs-and-ScrapeGraphAi---part-1.md) |
| How AI is changing the web scraping industry | 2025-05-20 | [How AI is changing the web scraping industry](articles/How-AI-is-changing-the-web-scraping-industry.md) |
| The AI-Powered web scraping tools landscape | 2025-05-20 | [The AI-Powered web scraping tools landscape](articles/The-AI-Powered-web-scraping-tools-landscape.md) |
| Building a custom GPT using Firecrawl | 2025-05-20 | [Building a custom GPT using Firecrawl](articles/Building-a-custom-GPT-using-Firecrawl.md) |
| About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra | 2025-05-20 | [About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra](articles/About-LLMs,-AI-and-Web-Scraping---by-Pierluigi-Vinciguerra.md) |
| Building a generic scraper for multiple websites | 2025-05-20 | [Building a generic scraper for multiple websites](articles/Building-a-generic-scraper-for-multiple-websites.md) |
| Use Cursor as web scraping assistant with MCP servers | 2025-05-20 | [Use Cursor as web scraping assistant with MCP servers](articles/Use-Cursor-as-web-scraping-assistant-with-MCP-servers.md) |
| THE LAB #75: Building self healing scrapers with AI | 2025-05-20 | [THE LAB #75: Building self healing scrapers with AI](articles/THE-LAB-#75-Building-self-healing-scrapers-with-AI.md) |
| Build your web scraping assistant with Claude and Cursor | 2025-05-20 | [Build your web scraping assistant with Claude and Cursor](articles/Build-your-web-scraping-assistant-with-Claude-and-Cursor.md) |
| Are LLMs capable of replacing traditional scrapers? | 2025-05-20 | [Are LLMs capable of replacing traditional scrapers?](articles/Are-LLMs-capable-of-replacing-traditional-scrapers.md) |
| How LLMs are affecting the costs of web scraping | 2025-05-20 | [How LLMs are affecting the costs of web scraping](articles/How-LLMs-are-affecting-the-costs-of-web-scraping.md) |
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | [Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base](articles/Evolution-from-RAG-to-MCP-A-Breakthrough-for-LLM-Dynamic-Knowledge-Base.md) |
| Is Web Scraping Dead? - by Pierluigi Vinciguerra | 2024-02-25 | [Is Web Scraping Dead? - by Pierluigi Vinciguerra](articles/Is-Web-Scraping-Dead---by-Pierluigi-Vinciguerra.md) |
| Web DRAGON - LLM-powered web scraping on a distributed cloud | 2023-12-19 | [Web DRAGON - LLM-powered web scraping on a distributed cloud](articles/Web-DRAGON---LLM-powered-web-scraping-on-a-distributed-cloud.md) |
| Web Scraping experts: Is AI stealing our job? | 2023-10-13 | [Web Scraping experts: Is AI stealing our job?](articles/Web-Scraping-experts-Is-AI-stealing-our-job.md) |
| The state of web scraping and AI - by Pierluigi Vinciguerra | 2023-10-13 | [The state of web scraping and AI - by Pierluigi Vinciguerra](articles/The-state-of-web-scraping-and-AI---by-Pierluigi-Vinciguerra.md) |
## 🏷️ Lambda
| Title | Date | Link |
|-------|------|------|
| The Lab #48: Scraping with AWS Lambda | 2024-10-18 | [The Lab #48: Scraping with AWS Lambda](articles/The-Lab-#48-Scraping-with-AWS-Lambda.md) |
## 🏷️ LeadGeneration
| Title | Date | Link |
|-------|------|------|
| Web Scraping for Lead Generation and Prospecting | 2025-03-12 | [Web Scraping for Lead Generation and Prospecting](articles/Web-Scraping-for-Lead-Generation-and-Prospecting.md) |
## 🏷️ Legal
| Title | Date | Link |
|-------|------|------|
| AI and data: different faces of the same coin | 2025-05-20 | [AI and data: different faces of the same coin](articles/AI-and-data-different-faces-of-the-same-coin.md) |
| Is web scraping legal? - by Pierluigi Vinciguerra | 2025-03-12 | [Is web scraping legal? - by Pierluigi Vinciguerra](articles/Is-web-scraping-legal---by-Pierluigi-Vinciguerra.md) |
| The X vs Bright Data case - by Sanaea Daruwalla | 2024-07-09 | [The X vs Bright Data case - by Sanaea Daruwalla](articles/The-X-vs-Bright-Data-case---by-Sanaea-Daruwalla.md) |
| Legal Zyte-geist #4: Overview of the EU AI Act | 2024-05-28 | [Legal Zyte-geist #4: Overview of the EU AI Act](articles/Legal-Zyte-geist-#4-Overview-of-the-EU-AI-Act.md) |
| Is Web Scraping Dead? - by Pierluigi Vinciguerra | 2024-02-25 | [Is Web Scraping Dead? - by Pierluigi Vinciguerra](articles/Is-Web-Scraping-Dead---by-Pierluigi-Vinciguerra.md) |
| Legal Zyte-geist #3: What the court’s ruling in the Meta v Bright Data case really means for web scrapers | 2024-02-13 | [Legal Zyte-geist #3: What the court’s ruling in the Meta v Bright Data case really means for web scrapers](articles/Legal-Zyte-geist-#3-What-the-court’s-ruling-in-the-Meta-v-Bright-Data-case-reall.md) |
| Legal Zyte-geist #2: Web Scraping and AI 2023 Legal Wrap-Up | 2024-01-09 | [Legal Zyte-geist #2: Web Scraping and AI 2023 Legal Wrap-Up](articles/Legal-Zyte-geist-#2-Web-Scraping-and-AI-2023-Legal-Wrap-Up.md) |
| Legal Zyte-geist #1: Step-by-Step Guide to Compliant Web Scraping | 2023-12-05 | [Legal Zyte-geist #1: Step-by-Step Guide to Compliant Web Scraping](articles/Legal-Zyte-geist-#1-Step-by-Step-Guide-to-Compliant-Web-Scraping.md) |
| Can I scrape any public data? - by Pierluigi Vinciguerra | 2023-10-13 | [Can I scrape any public data? - by Pierluigi Vinciguerra](articles/Can-I-scrape-any-public-data---by-Pierluigi-Vinciguerra.md) |
| Is it legal to scrape social networks like Facebook or Instagram? | 2023-10-13 | [Is it legal to scrape social networks like Facebook or Instagram?](articles/Is-it-legal-to-scrape-social-networks-like-Facebook-or-Instagram.md) |
| Web Scraping Legal Context - by Andrea Squatrito | 2023-10-13 | [Web Scraping Legal Context - by Andrea Squatrito](articles/Web-Scraping-Legal-Context---by-Andrea-Squatrito.md) |
## 🏷️ Lightpanda
| Title | Date | Link |
|-------|------|------|
| Rethinking the web browser - by Katie Hallett | 2025-01-21 | [Rethinking the web browser - by Katie Hallett](articles/Rethinking-the-web-browser---by-Katie-Hallett.md) |
## 🏷️ LocationData
| Title | Date | Link |
|-------|------|------|
| THE LAB #31: Scraping location data using a world grid | 2025-01-16 | [THE LAB #31: Scraping location data using a world grid](articles/THE-LAB-#31-Scraping-location-data-using-a-world-grid.md) |
## 🏷️ MCP
| Title | Date | Link |
|-------|------|------|
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | [Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base](articles/Evolution-from-RAG-to-MCP-A-Breakthrough-for-LLM-Dynamic-Knowledge-Base.md) |
## 🏷️ MachineLearning
| Title | Date | Link |
|-------|------|------|
| Machine learning models for detecting bot detection triggers | 2025-06-15 | [Machine learning models for detecting bot detection triggers](articles/Machine-learning-models-for-detecting-bot-detection-triggers.md) |
## 🏷️ Make
| Title | Date | Link |
|-------|------|------|
| No-Code Web Scraping with Make.com | 2025-05-20 | [No-Code Web Scraping with Make.com](articles/No-Code-Web-Scraping-with-Make.com.md) |
## 🏷️ MarketResearch
| Title | Date | Link |
|-------|------|------|
| Web scraping in market research and competitive analysis | 2025-03-12 | [Web scraping in market research and competitive analysis](articles/Web-scraping-in-market-research-and-competitive-analysis.md) |
## 🏷️ MartinGanchev
| Title | Date | Link |
|-------|------|------|
| Interview #4: Martin Ganchev - Smartproxy | 2023-10-13 | [Interview #4: Martin Ganchev - Smartproxy](articles/Interview-#4-Martin-Ganchev---Smartproxy.md) |
## 🏷️ Mistral
| Title | Date | Link |
|-------|------|------|
| Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared | 2025-05-20 | [Writing scrapers with LLMs: GPT4, LLama3.1, Mistral compared](articles/Writing-scrapers-with-LLMs-GPT4,-LLama3.1,-Mistral-compared.md) |
## 🏷️ MobileApp
| Title | Date | Link |
|-------|------|------|
| How to Scrape Data from Mobile Apps using HTTP Toolkit | 2025-05-20 | [How to Scrape Data from Mobile Apps using HTTP Toolkit](articles/How-to-Scrape-Data-from-Mobile-Apps-using-HTTP-Toolkit.md) |
| Scraping food delivery data - by Pierluigi Vinciguerra | 2025-05-20 | [Scraping food delivery data - by Pierluigi Vinciguerra](articles/Scraping-food-delivery-data---by-Pierluigi-Vinciguerra.md) |
| HTTP Toolkit, your best friend for network inspection | 2025-05-20 | [HTTP Toolkit, your best friend for network inspection](articles/HTTP-Toolkit,-your-best-friend-for-network-inspection.md) |
| The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 | 2025-05-20 | [The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2](articles/The-Lab-#59-Bypassing-certificate-pinning-with-Frida-and-Fiddler---part-2.md) |
| The Lab #58: Intercepting traffic from an App - part 1 | 2025-05-20 | [The Lab #58: Intercepting traffic from an App - part 1](articles/The-Lab-#58-Intercepting-traffic-from-an-App---part-1.md) |
| THE LAB #1: Scraping data from an app | 2024-12-26 | [THE LAB #1: Scraping data from an app](articles/THE-LAB-#1-Scraping-data-from-an-app.md) |
| THE LAB #12: Reverse-engineering Mobile API | 2023-05-29 | [THE LAB #12: Reverse-engineering Mobile API](articles/THE-LAB-#12-Reverse-engineering-Mobile-API.md) |
## 🏷️ MobileProxy
| Title | Date | Link |
|-------|------|------|
| Comparing Residential And Mobile Proxies for Anti-Bot Evasion | 2025-06-01 | [Comparing Residential And Mobile Proxies for Anti-Bot Evasion](articles/Comparing-Residential-And-Mobile-Proxies-for-Anti-Bot-Evasion.md) |
| Building an in-house mobile proxy farm | 2025-05-20 | [Building an in-house mobile proxy farm](articles/Building-an-in-house-mobile-proxy-farm.md) |
| How I've built my home made mobile proxy | 2023-10-13 | [How I've built my home made mobile proxy](articles/How-I've-built-my-home-made-mobile-proxy.md) |
## 🏷️ MouseMovements
| Title | Date | Link |
|-------|------|------|
| THE LAB #8: Using Bezier curves for human-like mouse movements | 2023-05-29 | [THE LAB #8: Using Bezier curves for human-like mouse movements](articles/THE-LAB-#8-Using-Bezier-curves-for-human-like-mouse-movements.md) |
## 🏷️ Multilogin
| Title | Date | Link |
|-------|------|------|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | [In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers](articles/In-Depth-Pricing-Comparison-of-Anti-Detect-Browsers-for-Web-Scrapers.md) |
## 🏷️ NFT
| Title | Date | Link |
|-------|------|------|
| THE LAB #9: Scraping OpenSea NFT's data | 2023-05-29 | [THE LAB #9: Scraping OpenSea NFT's data](articles/THE-LAB-#9-Scraping-OpenSea-NFT's-data.md) |
## 🏷️ NSTBrowser
| Title | Date | Link |
|-------|------|------|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | [In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers](articles/In-Depth-Pricing-Comparison-of-Anti-Detect-Browsers-for-Web-Scrapers.md) |
## 🏷️ NehaSetia
| Title | Date | Link |
|-------|------|------|
| Interview #1: Neha Setia - Zyte - by Pierluigi Vinciguerra | 2023-10-13 | [Interview #1: Neha Setia - Zyte - by Pierluigi Vinciguerra](articles/Interview-#1-Neha-Setia---Zyte---by-Pierluigi-Vinciguerra.md) |
## 🏷️ NeilEmeigh
| Title | Date | Link |
|-------|------|------|
| Interview #2: Neil Emeigh - Rayobyte | 2023-10-13 | [Interview #2: Neil Emeigh - Rayobyte](articles/Interview-#2-Neil-Emeigh---Rayobyte.md) |
## 🏷️ NetNut
| Title | Date | Link |
|-------|------|------|
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | [The Great Web Unblocker Benchmark: Kasada edition](articles/The-Great-Web-Unblocker-Benchmark-Kasada-edition.md) |
## 🏷️ News
| Title | Date | Link |
|-------|------|------|
| A brief wrap up of the latest news on web scraping | 2023-10-13 | [A brief wrap up of the latest news on web scraping](articles/A-brief-wrap-up-of-the-latest-news-on-web-scraping.md) |
| The 2022 recap for the Web Scraping industry | 2023-10-13 | [The 2022 recap for the Web Scraping industry](articles/The-2022-recap-for-the-Web-Scraping-industry.md) |
## 🏷️ Nimble
| Title | Date | Link |
|-------|------|------|
| Hands on #3: Building a price comparison tool with Nimble APIs | 2023-10-13 | [Hands on #3: Building a price comparison tool with Nimble APIs](articles/Hands-on-#3-Building-a-price-comparison-tool-with-Nimble-APIs.md) |
| THE LAB #20 - AI powered web scrapers with Nimble Browser | 2023-10-13 | [THE LAB #20 - AI powered web scrapers with Nimble Browser](articles/THE-LAB-#20---AI-powered-web-scrapers-with-Nimble-Browser.md) |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | [THE LAB #21 - Bypass anti-bot challenges with AI](articles/THE-LAB-#21---Bypass-anti-bot-challenges-with-AI.md) |
| From Traditional Browsers to AI-Powered Web Scraping | 2023-10-13 | [From Traditional Browsers to AI-Powered Web Scraping](articles/From-Traditional-Browsers-to-AI-Powered-Web-Scraping.md) |
## 🏷️ NoCode
| Title | Date | Link |
|-------|------|------|
| No-Code Web Scraping with Make.com | 2025-05-20 | [No-Code Web Scraping with Make.com](articles/No-Code-Web-Scraping-with-Make.com.md) |
## 🏷️ Nodriver
| Title | Date | Link |
|-------|------|------|
| THE LAB #62: Bypassing Cloudflare with Nodriver | 2025-05-20 | [THE LAB #62: Bypassing Cloudflare with Nodriver](articles/THE-LAB-#62-Bypassing-Cloudflare-with-Nodriver.md) |
## 🏷️ Octobrowser
| Title | Date | Link |
|-------|------|------|
| In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers | 2025-03-25 | [In-Depth Pricing Comparison of Anti-Detect Browsers for Web Scrapers](articles/In-Depth-Pricing-Comparison-of-Anti-Detect-Browsers-for-Web-Scrapers.md) |
## 🏷️ OndraUrban
| Title | Date | Link |
|-------|------|------|
| Interview #3: Ondra Urban - Apify | 2023-10-13 | [Interview #3: Ondra Urban - Apify](articles/Interview-#3-Ondra-Urban---Apify.md) |
## 🏷️ OpenAI
| Title | Date | Link |
|-------|------|------|
| THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools | 2025-05-22 | [THE LAB #84: AI-Driven Web Scraping: OpenAI Codex vs Cursor vs AI Scraping Tools](articles/THE-LAB-#84-AI-Driven-Web-Scraping-OpenAI-Codex-vs-Cursor-vs-AI-Scraping-Tools.md) |
| AI and data: different faces of the same coin | 2025-05-20 | [AI and data: different faces of the same coin](articles/AI-and-data-different-faces-of-the-same-coin.md) |
| No-Code Web Scraping with Make.com | 2025-05-20 | [No-Code Web Scraping with Make.com](articles/No-Code-Web-Scraping-with-Make.com.md) |
| Building a custom GPT using Firecrawl | 2025-05-20 | [Building a custom GPT using Firecrawl](articles/Building-a-custom-GPT-using-Firecrawl.md) |
| How to create a web scraper with ChatGPT | 2023-10-13 | [How to create a web scraper with ChatGPT](articles/How-to-create-a-web-scraper-with-ChatGPT.md) |
## 🏷️ OpenSea
| Title | Date | Link |
|-------|------|------|
| THE LAB #9: Scraping OpenSea NFT's data | 2023-05-29 | [THE LAB #9: Scraping OpenSea NFT's data](articles/THE-LAB-#9-Scraping-OpenSea-NFT's-data.md) |
## 🏷️ Oxylabs
| Title | Date | Link |
|-------|------|------|
| The Great Web Unblocker Benchmark - Cloudflare Edition | 2025-05-20 | [The Great Web Unblocker Benchmark - Cloudflare Edition](articles/The-Great-Web-Unblocker-Benchmark---Cloudflare-Edition.md) |
| The Great Web Unblocker Benchmark: Kasada edition | 2025-05-20 | [The Great Web Unblocker Benchmark: Kasada edition](articles/The-Great-Web-Unblocker-Benchmark-Kasada-edition.md) |
| THE LAB #63: Oxymouse and Playwright | 2025-05-20 | [THE LAB #63: Oxymouse and Playwright](articles/THE-LAB-#63-Oxymouse-and-Playwright.md) |
| How to Scrape E-Commerce Websites With Python | 2024-08-02 | [How to Scrape E-Commerce Websites With Python](articles/How-to-Scrape-E-Commerce-Websites-With-Python.md) |
| The Great Web Unblocker Benchmark: March 2024 | 2024-03-19 | [The Great Web Unblocker Benchmark: March 2024](articles/The-Great-Web-Unblocker-Benchmark-March-2024.md) |
| Hands On #5: Testing the Oxylabs Web Unblocker | 2023-10-13 | [Hands On #5: Testing the Oxylabs Web Unblocker](articles/Hands-On-#5-Testing-the-Oxylabs-Web-Unblocker.md) |
| Bypassing Perimeterx in 2023 with code and examples | 2023-09-11 | [Bypassing Perimeterx in 2023 with code and examples](articles/Bypassing-Perimeterx-in-2023-with-code-and-examples.md) |
## 🏷️ Oxymouse
| Title | Date | Link |
|-------|------|------|
| THE LAB #63: Oxymouse and Playwright | 2025-05-20 | [THE LAB #63: Oxymouse and Playwright](articles/THE-LAB-#63-Oxymouse-and-Playwright.md) |
## 🏷️ PHP
| Title | Date | Link |
|-------|------|------|
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | [Web Scraping and Coding: Five Programming Languages to Check Out](articles/Web-Scraping-and-Coding-Five-Programming-Languages-to-Check-Out.md) |
## 🏷️ Patchwright
| Title | Date | Link |
|-------|------|------|
| THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 | 2025-05-20 | [THE LAB #76: Bypassing Kasada With Open Source Tools In 2025](articles/THE-LAB-#76-Bypassing-Kasada-With-Open-Source-Tools-In-2025.md) |
## 🏷️ PerimeterX
| Title | Date | Link |
|-------|------|------|
| The Lab #56: Bypassing PerimeterX 3 | 2025-05-20 | [The Lab #56: Bypassing PerimeterX 3](articles/The-Lab-#56-Bypassing-PerimeterX-3.md) |
| Bypassing PerimeterX without a browser automation tool | 2024-11-15 | [Bypassing PerimeterX without a browser automation tool](articles/Bypassing-PerimeterX-without-a-browser-automation-tool.md) |
| The Lab #35: Bypassing PerimeterX with Python and Playwright | 2023-12-21 | [The Lab #35: Bypassing PerimeterX with Python and Playwright](articles/The-Lab-#35-Bypassing-PerimeterX-with-Python-and-Playwright.md) |
| Can Undetected Chromedriver bypass Cloudflare or Datadome? | 2023-10-13 | [Can Undetected Chromedriver bypass Cloudflare or Datadome?](articles/Can-Undetected-Chromedriver-bypass-Cloudflare-or-Datadome.md) |
| THE LAB #21 - Bypass anti-bot challenges with AI | 2023-10-13 | [THE LAB #21 - Bypass anti-bot challenges with AI](articles/THE-LAB-#21---Bypass-anti-bot-challenges-with-AI.md) |
| Bypassing Perimeterx in 2023 with code and examples | 2023-09-11 | [Bypassing Perimeterx in 2023 with code and examples](articles/Bypassing-Perimeterx-in-2023-with-code-and-examples.md) |
| THE LAB #7: Scraping PerimeterX protected websites | 2023-05-29 | [THE LAB #7: Scraping PerimeterX protected websites](articles/THE-LAB-#7-Scraping-PerimeterX-protected-websites.md) |
## 🏷️ PixelWhispererAPI
| Title | Date | Link |
|-------|------|------|
| Scraping Cloudflare websites using an API | 2025-05-20 | [Scraping Cloudflare websites using an API](articles/Scraping-Cloudflare-websites-using-an-API.md) |
## 🏷️ Playwright
| Title | Date | Link |
|-------|------|------|
| THE LAB #72: Advanced logging in Playwright | 2025-05-20 | [THE LAB #72: Advanced logging in Playwright](articles/THE-LAB-#72-Advanced-logging-in-Playwright.md) |
| How to start with Scrapy and Playwright - Part 2 | 2025-05-20 | [How to start with Scrapy and Playwright - Part 2](articles/How-to-start-with-Scrapy-and-Playwright---Part-2.md) |
| THE LAB #76: Bypassing Kasada With Open Source Tools In 2025 | 2025-05-20 | [THE LAB #76: Bypassing Kasada With Open Source Tools In 2025](articles/THE-LAB-#76-Bypassing-Kasada-With-Open-Source-Tools-In-2025.md) |
| Making Playwright scrapers undetected with open source solutions | 2025-05-20 | [Making Playwright scrapers undetected with open source solutions](articles/Making-Playwright-scrapers-undetected-with-open-source-solutions.md) |
| THE LAB #63: Oxymouse and Playwright | 2025-05-20 | [THE LAB #63: Oxymouse and Playwright](articles/THE-LAB-#63-Oxymouse-and-Playwright.md) |
| THE LAB #73: How to Bypass Cloudflare in 2025 | 2025-05-20 | [THE LAB #73: How to Bypass Cloudflare in 2025](articles/THE-LAB-#73-How-to-Bypass-Cloudflare-in-2025.md) |
| The Lab #56: Bypassing PerimeterX 3 | 2025-05-20 | [The Lab #56: Bypassing PerimeterX 3](articles/The-Lab-#56-Bypassing-PerimeterX-3.md) |
| The Lab #55: Checking your browser fingerprint | 2025-05-20 | [The Lab #55: Checking your browser fingerprint](articles/The-Lab-#55-Checking-your-browser-fingerprint.md) |
| The 2025 web scraping tech stack - by Pierluigi Vinciguerra | 2025-05-20 | [The 2025 web scraping tech stack - by Pierluigi Vinciguerra](articles/The-2025-web-scraping-tech-stack---by-Pierluigi-Vinciguerra.md) |
| The Lab #53: Bypassing AWS WAF - by Pierluigi Vinciguerra | 2025-05-20 | [The Lab #53: Bypassing AWS WAF - by Pierluigi Vinciguerra](articles/The-Lab-#53-Bypassing-AWS-WAF---by-Pierluigi-Vinciguerra.md) |
| The Lab #57: Improving your Playwright scraper and avoid CDP detection | 2025-05-20 | [The Lab #57: Improving your Playwright scraper and avoid CDP detection](articles/The-Lab-#57-Improving-your-Playwright-scraper-and-avoid-CDP-detection.md) |
| The Lab #46: Fingerprint injection in Playwright | 2025-01-26 | [The Lab #46: Fingerprint injection in Playwright](articles/The-Lab-#46-Fingerprint-injection-in-Playwright.md) |
| THE LAB #11: The Anti-Detect Anti-Bot matrix | 2025-01-01 | [THE LAB #11: The Anti-Detect Anti-Bot matrix](articles/THE-LAB-#11-The-Anti-Detect-Anti-Bot-matrix.md) |
| Mouse movements in Playwright with Ghost Cursor | 2024-10-13 | [Mouse movements in Playwright with Ghost Cursor](articles/Mouse-movements-in-Playwright-with-Ghost-Cursor.md) |
| Bypassing Kasada for web scraping 2024 edition | 2024-09-30 | [Bypassing Kasada for web scraping 2024 edition](articles/Bypassing-Kasada-for-web-scraping-2024-edition.md) |
| Scraping the dark web with Playwright and Brave | 2024-03-07 | [Scraping the dark web with Playwright and Brave](articles/Scraping-the-dark-web-with-Playwright-and-Brave.md) |
| Web Scraping from 0 to hero: tips and tricks for Microsoft Playwright | 2024-02-18 | [Web Scraping from 0 to hero: tips and tricks for Microsoft Playwright](articles/Web-Scraping-from-0-to-hero-tips-and-tricks-for-Microsoft-Playwright.md) |
| Web Scraping from 0 to hero: our first scraper with Microsoft Playwright | 2024-02-04 | [Web Scraping from 0 to hero: our first scraper with Microsoft Playwright](articles/Web-Scraping-from-0-to-hero-our-first-scraper-with-Microsoft-Playwright.md) |
| Web scraping from 0 to hero: Microsoft Playwright | 2024-01-21 | [Web scraping from 0 to hero: Microsoft Playwright](articles/Web-scraping-from-0-to-hero-Microsoft-Playwright.md) |
| The Lab #35: Bypassing PerimeterX with Python and Playwright | 2023-12-21 | [The Lab #35: Bypassing PerimeterX with Python and Playwright](articles/The-Lab-#35-Bypassing-PerimeterX-with-Python-and-Playwright.md) |
| Bypassing Datadome with Web Scraping - End of 2023 Version | 2023-12-06 | [Bypassing Datadome with Web Scraping - End of 2023 Version](articles/Bypassing-Datadome-with-Web-Scraping---End-of-2023-Version.md) |
| Scraping Kasada protected websites | 2023-10-13 | [Scraping Kasada protected websites](articles/Scraping-Kasada-protected-websites.md) |
| Selenium vs Playwright, a comparison | 2023-10-13 | [Selenium vs Playwright, a comparison](articles/Selenium-vs-Playwright,-a-comparison.md) |
| HTTP requests in Python explained | 2023-10-13 | [HTTP requests in Python explained](articles/HTTP-requests-in-Python-explained.md) |
| What do I need for web scraping? - by Pierluigi Vinciguerra | 2023-10-13 | [What do I need for web scraping? - by Pierluigi Vinciguerra](articles/What-do-I-need-for-web-scraping---by-Pierluigi-Vinciguerra.md) |
| The starter toolkit for a python web scraping developer (2022) | 2023-10-13 | [The starter toolkit for a python web scraping developer (2022)](articles/The-starter-toolkit-for-a-python-web-scraping-developer-(2022).md) |
| Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra | 2023-10-13 | [Are CAPTCHAs still a thing? - by Pierluigi Vinciguerra](articles/Are-CAPTCHAs-still-a-thing---by-Pierluigi-Vinciguerra.md) |
| How to by-pass Kasada bot mitigation? | 2023-10-13 | [How to by-pass Kasada bot mitigation?](articles/How-to-by-pass-Kasada-bot-mitigation.md) |
| What is Playwright? - by Pierluigi Vinciguerra | 2023-10-13 | [What is Playwright? - by Pierluigi Vinciguerra](articles/What-is-Playwright---by-Pierluigi-Vinciguerra.md) |
| Is web scraping becoming harder? - by Pierluigi Vinciguerra | 2023-10-13 | [Is web scraping becoming harder? - by Pierluigi Vinciguerra](articles/Is-web-scraping-becoming-harder---by-Pierluigi-Vinciguerra.md) |
| 5 Playwright useful features for web scraping | 2023-09-30 | [5 Playwright useful features for web scraping](articles/5-Playwright-useful-features-for-web-scraping.md) |
| Bypassing Perimeterx in 2023 with code and examples | 2023-09-11 | [Bypassing Perimeterx in 2023 with code and examples](articles/Bypassing-Perimeterx-in-2023-with-code-and-examples.md) |
| THE LAB #19: How to mask the device fingerprint | 2023-09-11 | [THE LAB #19: How to mask the device fingerprint](articles/THE-LAB-#19-How-to-mask-the-device-fingerprint.md) |
| Buy cheaper plane tickets using a VPN: truth or myth? | 2023-09-11 | [Buy cheaper plane tickets using a VPN: truth or myth?](articles/Buy-cheaper-plane-tickets-using-a-VPN-truth-or-myth.md) |
| Scraping Cloudflare Protected Websites (early 2023 version) | 2023-06-10 | [Scraping Cloudflare Protected Websites (early 2023 version)](articles/Scraping-Cloudflare-Protected-Websites-(early-2023-version).md) |
| THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin | 2023-05-29 | [THE LAB #10: Bypass Cloudflare Bot Protection with GoLogin](articles/THE-LAB-#10-Bypass-Cloudflare-Bot-Protection-with-GoLogin.md) |
| How to scrape Datadome protected websites (early 2023 version) | 2023-05-29 | [How to scrape Datadome protected websites (early 2023 version)](articles/How-to-scrape-Datadome-protected-websites-(early-2023-version).md) |
| THE LAB #8: Using Bezier curves for human-like mouse movements | 2023-05-29 | [THE LAB #8: Using Bezier curves for human-like mouse movements](articles/THE-LAB-#8-Using-Bezier-curves-for-human-like-mouse-movements.md) |
| THE LAB #9: Scraping OpenSea NFT's data | 2023-05-29 | [THE LAB #9: Scraping OpenSea NFT's data](articles/THE-LAB-#9-Scraping-OpenSea-NFT's-data.md) |
## 🏷️ PriceMonitoring
| Title | Date | Link |
|-------|------|------|
| Web Scraping in Price Monitoring and Dynamic Pricing | 2025-03-12 | [Web Scraping in Price Monitoring and Dynamic Pricing](articles/Web-Scraping-in-Price-Monitoring-and-Dynamic-Pricing.md) |
## 🏷️ Proxies
| Title | Date | Link |
|-------|------|------|
| Comparing Residential And Mobile Proxies for Anti-Bot Evasion | 2025-06-01 | [Comparing Residential And Mobile Proxies for Anti-Bot Evasion](articles/Comparing-Residential-And-Mobile-Proxies-for-Anti-Bot-Evasion.md) |
| The Unit Economics of Proxy Providers - by Abed | 2025-05-27 | [The Unit Economics of Proxy Providers - by Abed](articles/The-Unit-Economics-of-Proxy-Providers---by-Abed.md) |
| Analyzing the cost of a web scraping project | 2025-05-20 | [Analyzing the cost of a web scraping project](articles/Analyzing-the-cost-of-a-web-scraping-project.md) |
| How to start with Scrapy and Playwright - Part 2 | 2025-05-20 | [How to start with Scrapy and Playwright - Part 2](articles/How-to-start-with-Scrapy-and-Playwright---Part-2.md) |
| THE LAB #61: Evaluating your proxy provider | 2025-05-20 | [THE LAB #61: Evaluating your proxy provider](articles/THE-LAB-#61-Evaluating-your-proxy-provider.md) |
| Optimizing Proxy Usage for Large-Scale Scraping | 2025-05-20 | [Optimizing Proxy Usage for Large-Scale Scraping](articles/Optimizing-Proxy-Usage-for-Large-Scale-Scraping.md) |
| Building an in-house mobile proxy farm | 2025-05-20 | [Building an in-house mobile proxy farm](articles/Building-an-in-house-mobile-proxy-farm.md) |
| How to start with Scrapy and Playwright - Part 1 | 2025-05-20 | [How to start with Scrapy and Playwright - Part 1](articles/How-to-start-with-Scrapy-and-Playwright---Part-1.md) |
| The Dirty Little Secret of Internet's Data | 2025-05-17 | [The Dirty Little Secret of Internet's Data](articles/The-Dirty-Little-Secret-of-Internet's-Data.md) |
| Web Scraping with Proxies: How Many IPs Do You Really Need? | 2025-04-29 | [Web Scraping with Proxies: How Many IPs Do You Really Need?](articles/Web-Scraping-with-Proxies-How-Many-IPs-Do-You-Really-Need.md) |
| Five Secrets of the Proxy Industry - by Julia Levi | 2025-03-18 | [Five Secrets of the Proxy Industry - by Julia Levi](articles/Five-Secrets-of-the-Proxy-Industry---by-Julia-Levi.md) |
| What is a residential proxy? - by Pierluigi Vinciguerra | 2025-03-13 | [What is a residential proxy? - by Pierluigi Vinciguerra](articles/What-is-a-residential-proxy---by-Pierluigi-Vinciguerra.md) |
| Where do proxy companies take residential IPs from? | 2025-02-24 | [Where do proxy companies take residential IPs from?](articles/Where-do-proxy-companies-take-residential-IPs-from.md) |
| Web Scraping from 0 to hero: Everything about proxies | 2024-04-14 | [Web Scraping from 0 to hero: Everything about proxies](articles/Web-Scraping-from-0-to-hero-Everything-about-proxies.md) |
| Scrapoxy, the super proxy aggregator, how it works? | 2024-02-21 | [Scrapoxy, the super proxy aggregator, how it works?](articles/Scrapoxy,-the-super-proxy-aggregator,-how-it-works.md) |
| How scraping a single website costed thousands of dollars in proxy | 2024-01-28 | [How scraping a single website costed thousands of dollars in proxy](articles/How-scraping-a-single-website-costed-thousands-of-dollars-in-proxy.md) |
| The costs of web scraping - by Pierluigi Vinciguerra | 2023-10-13 | [The costs of web scraping - by Pierluigi Vinciguerra](articles/The-costs-of-web-scraping---by-Pierluigi-Vinciguerra.md) |
| What's a proxy server? - by Pierluigi Vinciguerra | 2023-10-13 | [What's a proxy server? - by Pierluigi Vinciguerra](articles/What's-a-proxy-server---by-Pierluigi-Vinciguerra.md) |
| On choosing the right proxy provider for scraping | 2023-10-13 | [On choosing the right proxy provider for scraping](articles/On-choosing-the-right-proxy-provider-for-scraping.md) |
| The most interesting GitHub Repositories about web scraping (2023) | 2023-10-13 | [The most interesting GitHub Repositories about web scraping (2023)](articles/The-most-interesting-GitHub-Repositories-about-web-scraping-(2023).md) |
| Buy cheaper plane tickets using a VPN: truth or myth? | 2023-09-11 | [Buy cheaper plane tickets using a VPN: truth or myth?](articles/Buy-cheaper-plane-tickets-using-a-VPN-truth-or-myth.md) |
## 🏷️ Puppeteer
| Title | Date | Link |
|-------|------|------|
| How to Improve the Performance of Puppeteer Stealth Evasions | 2024-04-02 | [How to Improve the Performance of Puppeteer Stealth Evasions](articles/How-to-Improve-the-Performance-of-Puppeteer-Stealth-Evasions.md) |
## 🏷️ Pyppetteer
| Title | Date | Link |
|-------|------|------|
| THE LAB #11: The Anti-Detect Anti-Bot matrix | 2025-01-01 | [THE LAB #11: The Anti-Detect Anti-Bot matrix](articles/THE-LAB-#11-The-Anti-Detect-Anti-Bot-matrix.md) |
## 🏷️ Python
| Title | Date | Link |
|-------|------|------|
| Scraping Through Tor for Increased Anonymity | 2025-05-25 | [Scraping Through Tor for Increased Anonymity](articles/Scraping-Through-Tor-for-Increased-Anonymity.md) |
| Optimizing Python Scripts for High-Traffic Websites | 2025-05-20 | [Optimizing Python Scripts for High-Traffic Websites](articles/Optimizing-Python-Scripts-for-High-Traffic-Websites.md) |
| How to Parse JSON with Python: A Beginner-Friendly Guide | 2025-05-20 | [How to Parse JSON with Python: A Beginner-Friendly Guide](articles/How-to-Parse-JSON-with-Python-A-Beginner-Friendly-Guide.md) |
| The Lab #47: Scraping real time data with Python | 2025-03-14 | [The Lab #47: Scraping real time data with Python](articles/The-Lab-#47-Scraping-real-time-data-with-Python.md) |
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | [Web Scraping and Coding: Five Programming Languages to Check Out](articles/Web-Scraping-and-Coding-Five-Programming-Languages-to-Check-Out.md) |
| Botasaurus: an anti-ban web scraping framework | 2024-03-10 | [Botasaurus: an anti-ban web scraping framework](articles/Botasaurus-an-anti-ban-web-scraping-framework.md) |
| HTTP requests in Python explained | 2023-10-13 | [HTTP requests in Python explained](articles/HTTP-requests-in-Python-explained.md) |
## 🏷️ R
| Title | Date | Link |
|-------|------|------|
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | [Web Scraping and Coding: Five Programming Languages to Check Out](articles/Web-Scraping-and-Coding-Five-Programming-Languages-to-Check-Out.md) |
## 🏷️ RAG
| Title | Date | Link |
|-------|------|------|
| THE LAB #77: Building a Web Scraping Knowledge Assistant with RAG | 2025-05-20 | [THE LAB #77: Building a Web Scraping Knowledge Assistant with RAG](articles/THE-LAB-#77-Building-a-Web-Scraping-Knowledge-Assistant-with-RAG.md) |
| THE LAB #78: Building a Web Scraping Knowledge Assistant with RAG - Part2 | 2025-05-20 | [THE LAB #78: Building a Web Scraping Knowledge Assistant with RAG - Part2](articles/THE-LAB-#78-Building-a-Web-Scraping-Knowledge-Assistant-with-RAG---Part2.md) |
| Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base | 2025-04-08 | [Evolution from RAG to MCP: A Breakthrough for LLM Dynamic Knowledge Base](articles/Evolution-from-RAG-to-MCP-A-Breakthrough-for-LLM-Dynamic-Knowledge-Base.md) |
## 🏷️ RabbitMQ
| Title | Date | Link |
|-------|------|------|
| THE LAB #72: Advanced logging in Playwright | 2025-05-20 | [THE LAB #72: Advanced logging in Playwright](articles/THE-LAB-#72-Advanced-logging-in-Playwright.md) |
| THE LAB #71: Sending Scrapy logs to RabbitMQ | 2025-05-20 | [THE LAB #71: Sending Scrapy logs to RabbitMQ](articles/THE-LAB-#71-Sending-Scrapy-logs-to-RabbitMQ.md) |
## 🏷️ RaspberryPI
| Title | Date | Link |
|-------|------|------|
| How I've built my home made mobile proxy | 2023-10-13 | [How I've built my home made mobile proxy](articles/How-I've-built-my-home-made-mobile-proxy.md) |
## 🏷️ RealEstate
| Title | Date | Link |
|-------|------|------|
| Web Scraping Idealista and Bypass Idealista Blockers | 2024-08-06 | [Web Scraping Idealista and Bypass Idealista Blockers](articles/Web-Scraping-Idealista-and-Bypass-Idealista-Blockers.md) |
## 🏷️ Reddit
| Title | Date | Link |
|-------|------|------|
| THE LAB #18: How to scrape Reddit with Scrapy | 2023-09-11 | [THE LAB #18: How to scrape Reddit with Scrapy](articles/THE-LAB-#18-How-to-scrape-Reddit-with-Scrapy.md) |
## 🏷️ Report
| Title | Date | Link |
|-------|------|------|
| The state of public web data in 2024 | 2024-05-05 | [The state of public web data in 2024](articles/The-state-of-public-web-data-in-2024.md) |
## 🏷️ Requests
| Title | Date | Link |
|-------|------|------|
| Scraping Through Tor for Increased Anonymity | 2025-05-25 | [Scraping Through Tor for Increased Anonymity](articles/Scraping-Through-Tor-for-Increased-Anonymity.md) |
| Optimizing Python Scripts for High-Traffic Websites | 2025-05-20 | [Optimizing Python Scripts for High-Traffic Websites](articles/Optimizing-Python-Scripts-for-High-Traffic-Websites.md) |
## 🏷️ ResidentialProxies
| Title | Date | Link |
|-------|------|------|
| Comparing Residential And Mobile Proxies for Anti-Bot Evasion | 2025-06-01 | [Comparing Residential And Mobile Proxies for Anti-Bot Evasion](articles/Comparing-Residential-And-Mobile-Proxies-for-Anti-Bot-Evasion.md) |
## 🏷️ Ruby
| Title | Date | Link |
|-------|------|------|
| Web Scraping and Coding: Five Programming Languages to Check Out | 2024-05-21 | [Web Scraping and Coding: Five Programming Languages to Check Out](articles/Web-Scraping-and-Coding-Five-Programming-Languages-to-Check-Out.md) |
## 🏷️ SEO
| Title | Date | Link |
|-------|------|------|
| Web Scraping for SEO and content marketing | 2025-03-12 | [Web Scraping for SEO and content marketing](articles/Web-Scraping-for-SEO-and-content-marketing.md) |
## 🏷️ SERP
| Title | Date | Link |
|-------|------|------|
| The Scriptwall: Why Google is hiding its SERP content behind Javascript | 2025-05-20 | [The Scriptwall: Why Google is hiding its SERP content behind Javascript](articles/The-Scriptwall-Why-Google-is-hiding-its-SERP-content-behind-Javascript.md) |
## 🏷️ SSLPinning
| Title | Date | Link |
|-------|------|------|
| The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 | 2025-05-20 | [The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2](articles/The-Lab-#59-Bypassing-certificate-pinning-with-Frida-and-Fiddler---part-2.md) |
## 🏷️ ScrapeGraphAI
| Title | Date | Link |
|-------|------|------|
| The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1 | 2025-05-20 | [The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1](articles/The-Lab-#52-Scraping-with-LLMs-and-ScrapeGraphAi---part-1.md) |
| About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra | 2025-05-20 | [About LLMs, AI and Web Scraping - by Pierluigi Vinciguerra](articles/About-LLMs,-AI-and-Web-Scraping---by-Pierluigi-Vinciguerra.md) |
| Building a generic scraper for multiple websites | 2025-05-20 | [Building a generic scraper for multiple websites](articles/Building-a-generic-scraper-for-multiple-websites.md) |
| Open source Python libraries for your web scraping projects | 2025-05-20 | [Open source Python libraries for your web scraping projects](articles/Open-source-Python-libraries-for-your-web-scraping-projects.md) |
| Build a RAG Application with ScraperAPI, Gemini, and FAISS | 2025-04-02 | [Build a RAG Application with ScraperAPI, Gemini, and FAISS](articles/Build-a-RAG-Application-with-ScraperAPI,-Gemini,-and-FAISS.md) |
## 🏷️ ScrapeOps
| Title | Date | Link |
|-------|------|------|
| THE LAB #13: Managing a fleet of scrapers with Scrapeops | 2023-06-10 | [THE LAB #13: Managing a fleet of scrapers with Scrapeops](articles/THE-LAB-#13-Managing-a-fleet-of-scrapers-with-Scrapeops.md) |
## 🏷️ ScrapegraphAI
| Title | Date | Link |
|-------|------|------|
| Building a generic scraper for multiple websites | 2025-05-20 | [Building a generic scraper for multiple websites](articles/Building-a-generic-scraper-for-multiple-websites.md) |
## 🏷️ Scraping
| Title | Date | Link |
|-------|------|------|
| Machine learning models for detecting bot detection triggers | 2025-06-15 | [Machine learning models for detecting bot detection triggers](articles/Machine-learning-models-for-detecting-bot-detection-triggers.md) |
| Analyzing the cost of a web scraping project | 2025-05-20 | [Analyzing the cost of a web scraping project](articles/Analyzing-the-cost-of-a-web-scraping-project.md) |
| The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1 | 2025-05-20 | [The Lab #52: Scraping with LLMs and ScrapeGraphAi - part 1](articles/The-Lab-#52-Scraping-with-LLMs-and-ScrapeGraphAi---part-1.md) |
| THE LAB #81: Scraping Zillow for fun and profit | 2025-05-20 | [THE LAB #81: Scraping Zillow for fun and profit](articles/THE-LAB-#81-Scraping-Zillow-for-fun-and-profit.md) |
| The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2 | 2025-05-20 | [The Lab #59: Bypassing certificate pinning with Frida and Fiddler - part 2](articles/The-Lab-#59-Bypassing-certificate-pinning-with-Frida-and-Fiddler---part-2.md) |
| Web scraping and journalism: the Chiara Ferragni case | 2025-05-20 | [Web scraping and journalism: the Chiara Ferragni case](articles/Web-scraping-and-journalism-the-Chiara-Ferragni-case.md) |
| The Lab #58: Intercepting traffic from an App - part 1 | 2025-05-20 | [The Lab #58: Intercepting traffic from an App - part 1](articles/The-Lab-#58-Intercepting-traffic-from-an-App---part-1.md) |
| THE LAB #66: How to properly scrape a booking website | 2025-05-20 | [THE LAB #66: How to properly scrape a booking website](articles/THE-LAB-#66-How-to-properly-scrape-a-booking-website.md) |
| THE LAB #67: Scraping Telegram using its APIs | 2025-05-20 | [THE LAB #67: Scraping Telegram using its APIs](articles/THE-LAB-#67-Scraping-Telegram-using-its-APIs.md) |
| Web data and automotive industry - by Pierluigi Vinciguerra | 2025-05-20 | [Web data and automotive industry - by Pierluigi Vinciguerra](articles/Web-data-and-automotive-industry---by-Pierluigi-Vinciguerra.md) |
| THE LAB #64: JWT Tokens and API scraping | 2025-05-20 | [THE LAB #64: JWT Tokens and API scraping](articles/THE-LAB-#64-JWT-Tokens-and-API-scrapin