https://github.com/scrape-do/dotnet-example
Best Rotating Proxy & Scraping API Alternative. C# Example.
https://github.com/scrape-do/dotnet-example
captcha captcha-solver crawler crawlers crawling data-mining data-science data-scraping free free-proxy free-proxy-list proxy proxy-list proxylist rotating-proxy scraper scraping scraping-api scraping-tool
Last synced: 6 days ago
JSON representation
Best Rotating Proxy & Scraping API Alternative. C# Example.
- Host: GitHub
- URL: https://github.com/scrape-do/dotnet-example
- Owner: scrape-do
- Created: 2023-08-08T10:49:34.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2023-08-08T11:02:13.000Z (almost 3 years ago)
- Last Synced: 2023-08-08T12:43:54.531Z (almost 3 years ago)
- Topics: captcha, captcha-solver, crawler, crawlers, crawling, data-mining, data-science, data-scraping, free, free-proxy, free-proxy-list, proxy, proxy-list, proxylist, rotating-proxy, scraper, scraping, scraping-api, scraping-tool
- Language: C#
- Homepage: https://scrape.do
- Size: 1000 Bytes
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## Scrape.do C# Example
### Web Scraper API
- Best Rotating Proxy & [Scraping API](https://scrape.do) Alternative.
- ⚠ Please read the [documents](https://docs.scrape.do) first.
#### You can send request to any webpages with proxy gateway & web api provided by scrape.do. As you can see from the example, this takes only few lines of code
### You can see in example ([Program.cs](/Program.cs))
```C#
private static string CreateRequestUrl(string targetUrl) {
if (string.IsNullOrEmpty(_apiToken)) throw new Exception("API_TOKEN cannot be empty!");
string url = $ "http://api.scrape.do?token={_apiToken}";
if (_jsRender) url += "&render=true";
if (_superProxy) url += "&super=true";
if (!string.IsNullOrEmpty(_geoCode)) url += $ "&geoCode={_geoCode}";
if (!string.IsNullOrEmpty(_session)) url += $ "&session={_session}";
return $ "{url}&url={targetUrl}";
}
```
```C#
static void Main(string[] args) {
HttpClientHandler handler = new HttpClientHandler() {
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
var client = new HttpClient(handler);
string requestUrl = CreateRequestUrl("https://example.com");
var response = client.GetAsync(requestUrl).Result;
Console.WriteLine("Status Code : {0} ", response.StatusCode);
Console.WriteLine("HTML : \n{0}", response.Content.ReadAsStringAsync().Result);
}
```