https://github.com/justmert/lens-project-crawler

https://github.com/justmert/lens-project-crawler

lens-protocol

Last synced: 25 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/justmert/lens-project-crawler
• Owner: justmert
• License: mit
• Created: 2024-01-10T15:33:26.000Z (12 months ago)
• Default Branch: master
• Last Pushed: 2024-01-10T15:35:00.000Z (12 months ago)
• Last Synced: 2024-04-14T16:13:11.142Z (9 months ago)
• Topics: lens-protocol
• Language: Python
• Homepage:
• Size: 4.88 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Electric Capital Crypto Ecosystems Crawler

## Overview

This crawler script, designed for the Electric Capital Crypto Ecosystems repository, automates the process of updating the list of LENS-related projects on GitHub. It identifies new projects using specific Lens Protocol dependencies, compares them with the existing list in lens-protocol.toml, and creates a pull request in the Electric Capital repository with new Lens protocol projects.

## Features

- **Automated GitHub Crawling:** Searches for specific LENS-related keywords in package.json, package-lock.json, yarn.lock, and .sol files.

- **Intelligent Filtering:** Adds only new projects not already listed in lens-protocol.toml.

- **Automated Pull Requests:** Creates a pull request with updates on a forked Electric Capital repository.

- **Rate Limit Handling:** Manages GitHub API rate limits.

## Requirements

- Python 3.x

- requirements.txt

- A GitHub personal access token

## Setup

1. **Environment Variables:** Set ```GITHUB_TOKEN``` and ```GITHUB_USERNAME``` in a .env file. (You can copy .env.example to .env, and edit it)

2. **Dependencies:** Install required Python libraries with ```pip install -r requirements.txt``` command.

## Usage

Run ```python3 main.py``` on the root folder. The script then,

1. Initializes and loads environment variables.

2. Searches GitHub for repositories with specified LENS-related criteria.

3. Compares and filters found repositories against lens-protocol.toml.

4. Synchronizes the forked repository with the upstream repository.

5. Creates a pull request with new entries in the forked repository.

## Output

- Logs of the process.

- A link to pull request in the forked repository with lens-protocol.toml updates.

## Error Handling

- Handles GitHub API rate limits and HTTP request failures.

## Contribution

Contribute by forking the repository, making changes, and submitting a pull request.

---

Note: Comply with GitHub's API policies and rate limits when using this script.