https://github.com/stav/wgrep

Web grep: search all rendered resources used by a URI
https://github.com/stav/wgrep

grep-like node puppeteer web

Last synced: 22 days ago
JSON representation

Web grep: search all rendered resources used by a URI

• Host: GitHub
• URL: https://github.com/stav/wgrep
• Owner: stav
• License: apache-2.0
• Created: 2018-08-08T20:42:26.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2024-02-29T14:34:39.000Z (4 months ago)
• Last Synced: 2024-03-03T19:32:26.110Z (4 months ago)
• Topics: grep-like, node, puppeteer, web
• Language: JavaScript
• Size: 618 KB
• Stars: 76
• Watchers: 4
• Forks: 7
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Lists

• awesome-starred - stav/wgrep - Web grep: search all rendered resources used by a URI (web)

README

        
# wgrep

Web grep: search all rendered resources used by a URI

[![Coverage][cov-image]][cov-url]

[![Releases][rel-image]][rel-url]

[![Build][build-image]][build-url]

This `node` command-line utility uses a headless browser (Puppeteer) to render

a webpage and download all resources it may need. These resources including the

original HTML are all saved locally which it searches one-by-one for a text

string.

*Since we are downloading all resources it is easy to determine the total

download size.*

## Features

* Search using regular expressions

* A screen capture is created (not configurable)

## Installation

	$ git clone https://github.com/stav/wgrep.git

	$ cd wgrep

	$ npm install

## Usage example

Let's try to find the string "stav" from the repository website on GitHub:

	$ wgrep stav https://github.com/stav/wgrep

	Calling for "stav" in "output" from "https://github.com/stav/wgrep" with user "undefined"

	Looking in "output" for 'stav'

	Found 1 files

	[ 'output/stav/wgrep/index.html' ]

It was only found in the `index.html` page.

Now let's see what the total download size was:

	$ du -sh output

	1.4M    output

## Options

	$ wgrep --help

	Usage: wgrep [options]  

	Options:

	  -V, --version                output the version number

	  -d, --directory   The output directory (default: "output")

	  -u, --username     The user to authenticate as

	  -h, --help                   output usage information

## Tests

	$ npm test

	$ npm run test-e2e

## Contributing

Please file any issues you have.

If you fix a bug or add new features it would be great to have you fork this

repo and submit a pull request:

1. Fork it (`https://github.com/yourname/yourproject/fork`)

2. Create your feature branch (`git checkout -b feature/fooBar`)

3. Commit your changes (`git commit -am 'Add some fooBar'`)

4. Push to the branch (`git push origin feature/fooBar`)

5. Create a new Pull Request

## License

Apache 2.0

[rel-image]:   https://img.shields.io/github/release/stav/wgrep.svg

[cov-image]:   https://codecov.io/gh/stav/wgrep/branch/master/graph/badge.svg

[build-image]: https://github.com/stav/wgrep/actions/workflows/tests.yml/badge.svg

[rel-url]:     https://github.com/stav/wgrep/releases

[cov-url]:     https://codecov.io/github/stav/wgrep

[build-url]:   https://github.com/stav/wgrep/actions