{"id":13494471,"url":"https://github.com/mislav/nibbler","last_synced_at":"2025-03-28T14:31:13.421Z","repository":{"id":700782,"uuid":"346420","full_name":"mislav/nibbler","owner":"mislav","description":"A cute HTML scraper / data extraction tool in under 70 lines of code","archived":true,"fork":false,"pushed_at":"2016-06-06T10:02:29.000Z","size":330,"stargazers_count":143,"open_issues_count":0,"forks_count":6,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-25T03:28:44.633Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mislav.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2009-10-22T19:53:48.000Z","updated_at":"2024-10-31T08:32:27.000Z","dependencies_parsed_at":"2022-07-18T04:00:44.343Z","dependency_job_id":null,"html_url":"https://github.com/mislav/nibbler","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mislav%2Fnibbler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mislav%2Fnibbler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mislav%2Fnibbler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mislav%2Fnibbler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mislav","download_url":"https://codeload.github.com/mislav/nibbler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246045917,"owners_count":20714872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T19:01:25.365Z","updated_at":"2025-03-28T14:31:13.057Z","avatar_url":"https://github.com/mislav.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"Nibbler\n=======\n\n*Nibbler* is a small little tool (~100 LOC) that helps you map data structures to objects that you define.\n\nIt can be used for HTML screen scraping:\n\n~~~ ruby\nrequire 'nibbler'\nrequire 'open-uri'\n\nclass BlogScraper \u003c Nibbler\n  element :title\n\n  elements 'div.hentry' =\u003e :articles do\n    element 'h2' =\u003e :title\n    element 'a/@href' =\u003e :url\n  end\nend\n\nblog = BlogScraper.parse open('http://example.com')\n\nblog.title\n#=\u003e \"My blog title\"\n\nblog.articles.first.title\n#=\u003e \"First article title\"\n\nblog.articles.first.url\n#=\u003e \"http://example.com/article\"\n~~~\n\nFor mapping XML API payloads:\n\n~~~ ruby\nclass Movie \u003c Nibbler\n  element './title/@regular' =\u003e :name\n  element './box_art/@large' =\u003e :poster_large\n  element 'release_year' =\u003e :year, :with =\u003e lambda { |node| node.text.to_i }\n  element './/link[@title=\"web page\"]/@href' =\u003e :url\nend\n\nresponse = Net::HTTP.get_response URI('http://example.com/movie.xml')\nmovie = Movie.parse response.body\n\nmovie.name  #=\u003e \"Toy Story 3\"\nmovie.year  #=\u003e 2010\n~~~\n\nOr even for JSON:\n\n~~~ ruby\nrequire 'json'\nrequire 'nibbler/json'\n\nclass Movie \u003c NibblerJSON\n  element :title\n  element :year\n  elements :genres\n  # JSONPath selectors:\n  element '.links.alternate' =\u003e :url\n  element '.ratings.critics_score' =\u003e :critics_score\nend\n\nmovie = Movie.parse json_string\n~~~\n\nThere are sample scripts in the \"examples/\" directory:\n\n    ruby -Ilib -rubygems examples/delicious.rb\n    ruby -Ilib -rubygems examples/tweetburner.rb \u003e output.csv\n\n[See the wiki][wiki] for more on how to use *Nibbler*.\n\nRequirements\n------------\n\n*None*. Well, [Nokogiri][] is a requirement if you pass in an HTML string for parsing, like in the example above. Otherwise you can initialize the scraper with an\nHpricot document or anything else that implements `at(selector)` and `search(selector)` methods.\n\nNibblerJSON needs a JSON parser if string content is passed, so \"json\" library should be installed on Ruby 1.8.\n\n\n[wiki]: http://wiki.github.com/mislav/nibbler\n[nokogiri]: http://nokogiri.rubyforge.org/nokogiri/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmislav%2Fnibbler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmislav%2Fnibbler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmislav%2Fnibbler/lists"}