https://github.com/sergeypedan/integral-web-scraper

Downloads a set of web pages from your URLs list, saves them into a local SQLite file and calls your transformer class to deal with them, saving the results into the DB. Can further export transformed HTML or create an ePub.
https://github.com/sergeypedan/integral-web-scraper

epub web-scraper web-scraping

Last synced: about 2 months ago
JSON representation

Downloads a set of web pages from your URLs list, saves them into a local SQLite file and calls your transformer class to deal with them, saving the results into the DB. Can further export transformed HTML or create an ePub.

• Host: GitHub
• URL: https://github.com/sergeypedan/integral-web-scraper
• Owner: sergeypedan
• Created: 2020-02-19T18:24:09.000Z (almost 5 years ago)
• Default Branch: master
• Last Pushed: 2023-03-31T13:56:11.000Z (almost 2 years ago)
• Last Synced: 2024-03-21T09:16:42.730Z (10 months ago)
• Topics: epub, web-scraper, web-scraping
• Language: Ruby
• Size: 41 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 6
• Metadata Files:
  • Readme: README.md
  • Changelog: Changelog.md

Awesome Lists containing this project

README

        
# Integral::WebScraper

Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/integral/web_scraper`. To experiment with that code, run `bin/console` for an interactive prompt.

TODO: Delete this and the text above, and describe your gem

## Installation

Add this line to your application's Gemfile:

```ruby

gem 'integral-web_scraper'

```

And then execute:

```sh

bundle install

```

## Usage

TODO: Write usage instructions here

## Development

After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/integral-web_scraper.