{"id":27016097,"url":"https://github.com/kytiken/euros","last_synced_at":"2025-08-15T15:26:27.942Z","repository":{"id":57497906,"uuid":"113959161","full_name":"kytiken/euros","owner":"kytiken","description":"Euros web-spider framework","archived":false,"fork":false,"pushed_at":"2019-03-10T09:31:25.000Z","size":56,"stargazers_count":2,"open_issues_count":13,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-02T17:02:47.644Z","etag":null,"topics":["elixir"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kytiken.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-12T07:41:42.000Z","updated_at":"2018-06-09T10:05:39.000Z","dependencies_parsed_at":"2022-09-06T12:10:26.512Z","dependency_job_id":null,"html_url":"https://github.com/kytiken/euros","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kytiken%2Feuros","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kytiken%2Feuros/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kytiken%2Feuros/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kytiken%2Feuros/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kytiken","download_url":"https://codeload.github.com/kytiken/euros/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247198472,"owners_count":20900082,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir"],"created_at":"2025-04-04T15:17:57.326Z","updated_at":"2025-04-04T15:17:57.885Z","avatar_url":"https://github.com/kytiken.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Euros\n\nEuros is a web spider framework that can spider a domain and collect useful information about the pages it visits.\nIt is versatile, allowing you to write your own specialized spider tasks quickly and easily.\n\n## Installation\n\n```elixir\ndef deps do\n  [\n    {:euros, \"~\u003e 0.3.0\"}\n  ]\nend\n```\n\n## Usage\n\n```elixir\niex\u003e url = \"https://euros-test.blogspot.jp/\"\niex\u003e Euros.Core.crawl(url, fn(page) -\u003e IO.puts(inspect(page)) end)\n%HTTPoison.Response{body: \"\u003c!DOCTYPE html\u003e\u003chtml dir='ltr'\u003e\u003chead\u003e\u003cmeta content='width=device-width, initial-scale=1' name='viewport'/\u003e\u003ctitle\u003eeuros test blog\u003c/title\u003e....\n```\n\n### set depth limit\n```elixir\niex\u003e url = \"https://euros-test.blogspot.jp/\" \niex\u003e option = %Euros.CrawlOption{depth_limit: 2}\niex\u003e Euros.Core.crawl(url, fn(page) -\u003e page |\u003e inspect |\u003e IO.puts end, option)\n```\n\n### set cookie\n\n```elixir\niex\u003e url = \"https://euros-test.blogspot.jp/\" \niex\u003e http_option = %Euros.HTTPOption{cookie: \"foo=bar;\"}\niex\u003e option = %Euros.CrawlOption{http_option: http_option}\niex\u003e Euros.Core.crawl(url, fn(page) -\u003e page |\u003e inspect |\u003e IO.puts end, option)\n```\n\n### cusotm url pattern\n\n```elixir\niex\u003e url = \"https://euros-test.blogspot.jp/\" \niex\u003e option = %Euros.HTTPOption{cookie: \"foo=bar;\"}\niex\u003e pattern = ~r/test1/\niex\u003e option = %Euros.CrawlOption{http_option: http_option, pattern: pattern}\niex\u003e Euros.Core.crawl(url, fn(page) -\u003e page |\u003e inspect |\u003e IO.puts end, option, pattern)\n```\n\n## License\n```\nCopyright (c) 2017 kytiken\n\nReleased under the MIT license\nhttps://github.com/kytiken/euros/blob/master/LICENSE\n```\n\nDocumentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_doc)\nand published on [HexDocs](https://hexdocs.pm). Once published, the docs can\nbe found at [https://hexdocs.pm/euros](https://hexdocs.pm/euros).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkytiken%2Feuros","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkytiken%2Feuros","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkytiken%2Feuros/lists"}