{"id":20504487,"url":"https://github.com/2naive/angrycurl","last_synced_at":"2025-06-19T05:05:09.433Z","repository":{"id":2509647,"uuid":"3484976","full_name":"2naive/AngryCurl","owner":"2naive","description":"AngryCurl - Anonymized Rolling Curl class, used for parsing information from remote resourse using user-predefined amount of simultaneous connections over proxies-list.","archived":false,"fork":false,"pushed_at":"2015-08-01T14:19:39.000Z","size":299,"stargazers_count":110,"open_issues_count":9,"forks_count":49,"subscribers_count":24,"default_branch":"master","last_synced_at":"2025-06-19T05:04:38.955Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://stupid.su/php-curl_multi/","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/2naive.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-02-19T11:40:43.000Z","updated_at":"2025-06-09T13:46:48.000Z","dependencies_parsed_at":"2022-08-30T09:11:06.547Z","dependency_job_id":null,"html_url":"https://github.com/2naive/AngryCurl","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/2naive/AngryCurl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/2naive%2FAngryCurl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/2naive%2FAngryCurl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/2naive%2FAngryCurl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/2naive%2FAngryCurl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/2naive","download_url":"https://codeload.github.com/2naive/AngryCurl/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/2naive%2FAngryCurl/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260690825,"owners_count":23047098,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T19:38:23.238Z","updated_at":"2025-06-19T05:05:04.415Z","avatar_url":"https://github.com/2naive.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AngryCurl\n- used for parsing information from remote resourse using user-predefined amount of simultaneous connections over proxies-list.\n\n## Basic information\n\n### Depencies:\n\n* PHP 5 \u003e= 5.1.0\n* RollingCurl\n* cURL\n \n### Use cases:\n\n* multi-threaded parsing over proxy\n* overcoming simple parsing protection by using User-Agent header and proxy-lists\n* proxy list checking\n* validating proxies' response\n \n### Main features\n\n* loading proxy-list from file or array\n* removing duplicates\n* filtering alive proxies\n* checking if proxy given response content is correct\n* loading useragent-list from file or array\n* changing proxy/useragent \"on the fly\"\n* preventing direct connections without any proxy/useragent if such options are set\n* multi-thread connections\n* callback functions\n* working with chains of requests\n* web-console mode\n* logging\n\n## Documentation\n\n### Preferred environment configuration\n\n* PHP as Apache module\n* safe_mode Off\n* open_basedir is NOT set\n* PHP cURL installed\n* gzip Off\n\n### Basic usage\n\n```php\nrequire(\"RollingCurl.class.php\");\nrequire(\"AngryCurl.class.php\");\n\nfunction my_callback($response, $info, $request)\n{\n    // callback function here\n}\n\n// sending callback function name as param\n$AC = new AngryCurl('my_callback');\n// initializing console-style output\n$AC-\u003einit_console();\n\n\n// Importing proxy and useragent lists, setting regexp, proxy type and target url for proxy check\n// You may also import proxy from an array as simple as $AC-\u003eload_proxy_list($proxy array);\n$AC-\u003eload_proxy_list(\n    // path to proxy-list file\n    'proxy_list.txt',\n    // optional: number of threads\n    200,\n    // optional: proxy type\n    'http',\n    // optional: target url to check\n    'http://google.com',\n    // optional: target regexp to check\n    'title\u003eG[o]{2}gle'\n);\n// You may also import useragents from an array as simple as $AC-\u003eload_useragent_list($proxy array);\n$AC-\u003eload_useragent_list('useragent_list.txt');\n\nwhile(/* */)\n{\n    $url = /**/;\n    // adding URL to queue\n    $AC-\u003eget($url);\n    \n    // you may also use \n    // $AC-\u003epost($url, $post_data = null, $headers = null, $options = null);\n    // $AC-\u003eget($url, $headers = null, $options = null);\n    // $AC-\u003erequest($url, $method = \"GET\", $post_data = null, $headers = null, $options = null);\n    // as well\n}\n\n// setting amount of threads and starting connections\n$AC-\u003eexecute(200);\n\n// if console_mode is off\n//AngryCurl::print_debug(); \n\nunset($AC);\n```\n\n### cURL options\n\nYou may also pass cURL options for each url before adding to queue like here:\n```php\n// Define HTTP headers (CURLOPT_HTTPHEADER) if needed, or just set to NULL\n$headers = array('Content-type: text/plain', 'Content-length: 100');\n// Define cURL options (will be passed through curl_setopt_array) if needed, or just set to NULL\n$options = array(CURLOPT_HEADER =\u003e true, CURLOPT_NOBODY =\u003e true);\n// Define post-data array to send in case of POST method, or just set to NULL\n$post_data = array('param' =\u003e 'value');\n\n// Add request\n$AC-\u003eget($url, $headers, $options);\n// or\n$AC-\u003epost($url, $post_data, $headers, $options) ;\n// or\n$AC-\u003erequest($url, $method = \"GET\", $post_data, $headers, $options);\n\n// ATTENTION: temporary \"on-the-fly\" proxy/useragents lists are not\n// working with AngryCurlRequest. Keep it in mind if you will use code below\n// as alternative to written above.\n\n$request = new AngryCurlRequest($url);\n// $url, $method, $post_data, $headers, $options - public properties of AngryCurlRequest\n$request-\u003eoptions = array(CURLOPT_HEADER =\u003e true, CURLOPT_NOBODY =\u003e true);\n$AC-\u003eadd($request);\n```\n\nBecause this class is kind of extension of RollingCurl class you may use any constructions RollingCurl has.\nFor other information read here:\nhttp://code.google.com/p/rolling-curl/source/browse/trunk/\n\n## TODO\n* chains of requests\n* stop on error_limit exceed\n* better documentation and examples\n\n## Credits\nYou may join this class discussion here:\nhttp://stupid.su/php-curl_multi/\nAny questions, change requests and other things you may send to my email written in class comments.\n\nThank you for reading.\n- naive\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F2naive%2Fangrycurl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F2naive%2Fangrycurl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F2naive%2Fangrycurl/lists"}