{"id":15859472,"url":"https://github.com/nakabonne/netsurfer","last_synced_at":"2025-04-01T19:48:38.897Z","repository":{"id":57607683,"uuid":"109475552","full_name":"nakabonne/netsurfer","owner":"nakabonne","description":"netsurfer is a very lightweight scraping framework","archived":false,"fork":false,"pushed_at":"2018-02-21T14:03:01.000Z","size":12,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-27T03:11:14.743Z","etag":null,"topics":["crawler","go","library","scraping"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nakabonne.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-04T07:23:08.000Z","updated_at":"2017-11-07T04:27:39.000Z","dependencies_parsed_at":"2022-08-27T10:40:53.899Z","dependency_job_id":null,"html_url":"https://github.com/nakabonne/netsurfer","commit_stats":null,"previous_names":["ryonakao/netsurfer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nakabonne%2Fnetsurfer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nakabonne%2Fnetsurfer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nakabonne%2Fnetsurfer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nakabonne%2Fnetsurfer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nakabonne","download_url":"https://codeload.github.com/nakabonne/netsurfer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246705855,"owners_count":20820783,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","go","library","scraping"],"created_at":"2024-10-05T21:20:56.964Z","updated_at":"2025-04-01T19:48:38.867Z","avatar_url":"https://github.com/nakabonne.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# netsurfer\n\nnetsurfer is a very lightweight scraping framework.  \nThis has only the minimum function necessary for scraping.\n\n# What can be done\n\n- You can know the ranking of the web site.\n- You can know the URL displayed when searching with the google engine.\n- You can get HTML.\n\n# Installation\n\n```\n$ go get -u github.com/nakabonne/netsurfer\n```\n\n# Usage\n\nIf you want to know more, please read [examples](https://github.com/ryonakao/netsurfer/tree/master/examples).\n\n```go\nimport (\n\t\"fmt\"\n\t\"net/url\"\n\t\"github.com/ryonakao/netsurfer\"\n)\n// If you want to know the rank of a page, please write the following code.\nu, _ := url.Parse(\"https://qiita.com/ryonakao\")\nrank, _ := netsurfer.GetRank(u, \"ryonakao\", 2)\nfmt.Println(\"Rank is \", rank)\n\n// If you want to know the search result URL, please write following\nurls, _ := netsurfer.OrganicSearch(\"ruby\", 3)\n\nfor _, url := range urls {\n\t// If you want to know the title, please write the following\n\ttitle, _ := netsurfer.GetTitle(url.String())\n\tfmt.Println(\"Title is \", title)\n}\n```\n\n# dependencies\n\n- [goquery](https://github.com/PuerkitoBio/goquery)\n\n# License\n\n`netsurfer` source code is available under the MIT [License](https://github.com/ryonakao/netsurfer/blob/master/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnakabonne%2Fnetsurfer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnakabonne%2Fnetsurfer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnakabonne%2Fnetsurfer/lists"}