https://github.com/rusq/chromedl

Go library for scraping or downloading files bypassing Cloudflare protection and browser checks
https://github.com/rusq/chromedl

automation bypass bypass-cloudflare chromedp cloudflare download downloader scraping

Last synced: about 2 months ago
JSON representation

Go library for scraping or downloading files bypassing Cloudflare protection and browser checks

• Host: GitHub
• URL: https://github.com/rusq/chromedl
• Owner: rusq
• License: mit
• Created: 2021-05-03T11:05:07.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2021-05-08T22:07:11.000Z (over 3 years ago)
• Last Synced: 2024-07-27T03:35:39.345Z (about 2 months ago)
• Topics: automation, bypass, bypass-cloudflare, chromedp, cloudflare, download, downloader, scraping
• Language: Go
• Homepage:
• Size: 50.8 KB
• Stars: 25
• Watchers: 3
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst

Awesome Lists containing this project

README

        
========================

 Chrome File Downloader

========================

.. image:: https://pkg.go.dev/badge/github.com/rusq/chromedl.svg

   :alt: Go Reference

   :target: https://pkg.go.dev/github.com/rusq/chromedl 

.. contents::

   :depth: 2

The sole purpose of this package is to download files from the Internets with

headless Chrome bypassing the Cloudflare and maybe some other annoying browser

checks.

It does so by implementing the solutions posted in "`bypass headless chrome

detection issue`_" for chromedp_.

This library may help you if the other download methods don't work, i.e. curl or

the standard `http.Get()`.

The implementation is based on this `chromedp example`_.

Thanks to `@ZekeLu`_ for huge help in getting this going.

Compatibility

-------------

Tested with:

* Chrome (stable) v90.0.4430.93.

* github.com/chromedp/chromedp v0.6.12

* github.com/chromedp/cdproto v0.0.0-20210323015217-0942afbea50e

Newer versions of Chrome will require some code changes, as described in `this

issue`_, as it uses calls that are deprecated in newer protocol version in order

to be compatible with current stable version of Chrome (see above).

When using headless-shell docker image, please use the following tag::

  FROM chromedp/headless-shell:90.0.4430.93

LICENCES

--------

chromedp_: Copyright (c) 2016-2020 Kenneth Shaw

.. _`this issue`: https://github.com/chromedp/chromedp/issues/807

.. _`chromedp example`: https://github.com/chromedp/examples/tree/master/download_file

.. _`@ZekeLu`: https://github.com/ZekeLu

.. _chromedp: https://github.com/chromedp/chromedp

.. _`bypass headless chrome detection issue`: https://github.com/chromedp/chromedp/issues/396