https://github.com/jenslys/skrape-js

TypeScript/Node.js SDK to easily interact with the skrape.ai API
https://github.com/jenslys/skrape-js

ai ai-scraping llm scraper skrape skrapeai

Last synced: 3 days ago
JSON representation

TypeScript/Node.js SDK to easily interact with the skrape.ai API

• Host: GitHub
• URL: https://github.com/jenslys/skrape-js
• Owner: jenslys
• License: mit
• Created: 2024-11-24T13:10:41.000Z (about 2 months ago)
• Default Branch: main
• Last Pushed: 2024-11-24T15:44:16.000Z (about 1 month ago)
• Last Synced: 2024-12-24T14:21:29.251Z (15 days ago)
• Topics: ai, ai-scraping, llm, scraper, skrape, skrapeai
• Language: TypeScript
• Homepage: https://skrape.ai
• Size: 63.5 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# skrape-js

A TypeScript library for easily interacting with Skrape.ai API. Define your scraping schema using Zod and get type-safe results.

## Features

- 🛡️ **Type-safe**: Define your schemas using Zod and get fully typed results

- 🚀 **Simple API**: Just define a schema and get your data

- 🧩 **Minimal Dependencies**

## Installation

```bash

# Using npm

npm install skrape-js zod

# Using yarn, pnpm, etc.

yarn/pnpm/bun add skrape-js zod

```

Note: `zod` is a peer dependency and must be installed alongside `skrape-js`.

## Environment Setup

Setup your API key in `.env`:

```env

SKRAPE_API_KEY="your_api_key_here"

```

Get your API key on [Skrape.ai](https://skrape.ai)

## Quick Start

```typescript

import { Skrape } from "skrape-js";

import { z } from "zod";

// Initialize the client

const skrape = new Skrape({

  apiKey: process.env.SKRAPE_API_KEY,

});

// Define your schema using Zod

const newsSchema = z.object({

  topStories: z

    .array(

      z.object({

        title: z.string(),

        url: z.string(),

        score: z.number(),

        author: z.string(),

        commentCount: z.number(),

      })

    )

    .max(3),

});

// Extract data

const result = await skrape.extract(

  "https://news.ycombinator.com",

  newsSchema,

  { render_js: false }

);

console.log(result.topStories);

// [

//   {

//     title: "Example HN Post",

//     url: "https://example.com",

//     score: 100,

//     author: "user123",

//     commentCount: 25

//   },

//   ...

// ]

```

## Schema Definition

We leverage [Zod](https://zod.dev) for defining schemas. For a comprehensive understanding of all available options and advanced schema configurations, we recommend exploring Zod's documentation.

## API Options

When calling `extract()`, you can pass additional options:

```typescript

const result = await skrape.extract(url, schema, {

  render_js: true, // Enable JavaScript rendering

});

```

## Error Handling

The library throws typed errors that you can catch and handle:

```typescript

try {

  const result = await skrape.extract(url, schema);

} catch (error) {

  if (error instanceof SkrapeValidationError) {

    // Schema validation failed

    console.error("Data doesn't match schema:", error.message);

  } else if (error instanceof SkrapeAPIError) {

    // API request failed

    console.error("API error:", error.message);

  }

}