https://github.com/Doriandarko/RepoToTextForLLMs
Automate the analysis of GitHub repositories for LLMs with RepoToTextForLLMs. Fetch READMEs, structure, and non-binary files efficiently. Outputs include analysis prompts to aid in comprehensive repo evaluation
https://github.com/Doriandarko/RepoToTextForLLMs
Last synced: 2 months ago
JSON representation
Automate the analysis of GitHub repositories for LLMs with RepoToTextForLLMs. Fetch READMEs, structure, and non-binary files efficiently. Outputs include analysis prompts to aid in comprehensive repo evaluation
- Host: GitHub
- URL: https://github.com/Doriandarko/RepoToTextForLLMs
- Owner: Doriandarko
- Created: 2024-03-11T23:24:40.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-05-21T19:36:03.000Z (about 1 year ago)
- Last Synced: 2025-03-20T00:45:51.230Z (2 months ago)
- Language: Python
- Size: 19.5 KB
- Stars: 728
- Watchers: 10
- Forks: 94
- Open Issues: 7
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
- awesome - Doriandarko/RepoToTextForLLMs - Automate the analysis of GitHub repositories for LLMs with RepoToTextForLLMs. Fetch READMEs, structure, and non-binary files efficiently. Outputs include analysis prompts to aid in comprehensive repo evaluation (Python)
README
# RepoToTextForLLMs
Automates the analysis of GitHub repositories specifically tailored for usage with large context LLMs. This Python script efficiently fetches README files, repository structure, and non-binary file contents. Additionally, it provides structured outputs complete with pre-formatted prompts to guide further analysis of the repository's content.
## Features
- **README Retrieval:** Automatically extracts the content of README.md to provide an initial insight into the repository.
- **Structured Repository Traversal:** Maps out the repository's structure through an iterative traversal method, ensuring thorough coverage without the limitations of recursion.
- **Selective Content Extraction:** Retrieves text contents from files, intelligently skipping over binary files to streamline the analysis process.
## Prerequisites
To use **RepoToTextForLLMs**, you'll need:
- Python installed on your system.
- The `github` Python package.
- A GitHub Personal Access Token configured as an environment variable (`GITHUB_TOKEN`).
## Getting Started
1. Ensure Python and the required package (`PyGithub`) are installed:
```bash
pip install PyGithub tqdm
```
2. Set your GitHub Personal Access Token as an environment variable:
```python
GITHUB_TOKEN = os.getenv('GITHUB_TOKEN', 'YOUR TOKEN HERE')
```
## How to Use
1. Place the script in your desired directory.
2. Execute the script in your terminal:
```bash
python repototxt.py
```
3. Enter the GitHub repository URL when prompted. The script will process the repository and output its findings, including the README, structure, and file contents (excluding binary files), accompanied by analysis prompts.
## Contributing
Contributions to **RepoToTextForLLMs** are welcomed. Whether it's through submitting pull requests, reporting issues, or suggesting improvements, your input helps make this tool better for everyone.
## License
This project is licensed under the MIT License.