https://github.com/entrptaher/puppeteer-har

https://github.com/entrptaher/puppeteer-har

Last synced: about 8 hours ago
JSON representation

• Host: GitHub
• URL: https://github.com/entrptaher/puppeteer-har
• Owner: entrptaher
• Created: 2024-08-15T13:34:03.000Z (3 months ago)
• Default Branch: main
• Last Pushed: 2024-08-15T13:46:06.000Z (3 months ago)
• Last Synced: 2024-08-15T15:42:36.127Z (3 months ago)
• Language: JavaScript
• Size: 17.6 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

        
Allows you to capture all of the requests in a HAR format so you can convert it to WARC or view in HAR viewer tool. Original version has some issues with initial request, so I have modified this.

```ts

const puppeteer = require("puppeteer");

const PuppeteerHar = require("@entrptaher/puppeteer-har");

const capture = async (url, filePathBase) => {

	const browser = await puppeteer.launch({

		userDataDir: "/tmp/" + filePathBase,

		args: ["--window-size=3840,2160"],

		defaultViewport: { width: 3840, height: 2160, deviceScaleFactor: 2 },

	});

	const page = await browser.newPage();

	const har = new PuppeteerHar(page);

	await har.start({ path: filePathBase + ".har", saveResponse: true });

	console.log("Loading start");

	await page.goto(url, {

		waitUntil: "domcontentloaded",

		timeout: 120000,

	});

	console.log("Loading done");

	await har.stop();

	await browser.close();

};

(async () => {

	await capture("https://crawlbase.com", "crawlbase1");

	await capture(

		"https://crawlbase.com/crawling-api-avoid-captchas-blocks",

		"crawlbase2",

	);

})();

```