https://github.com/sisl/astra-rl
The Adaptive Stress Testing for Robust AI (ASTRA) toolbox provides tooling to support model developers and testing in the full life cycle of making more robust AI Systems through the application of adaptive stress testing and adversarial training.
https://github.com/sisl/astra-rl
ai-tools aisafe llms robust-ai stress-testing
Last synced: 23 days ago
JSON representation
The Adaptive Stress Testing for Robust AI (ASTRA) toolbox provides tooling to support model developers and testing in the full life cycle of making more robust AI Systems through the application of adaptive stress testing and adversarial training.
- Host: GitHub
- URL: https://github.com/sisl/astra-rl
- Owner: sisl
- License: mit
- Created: 2025-04-16T21:21:47.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2025-09-12T06:31:08.000Z (25 days ago)
- Last Synced: 2025-09-12T08:35:34.854Z (25 days ago)
- Topics: ai-tools, aisafe, llms, robust-ai, stress-testing
- Language: Python
- Homepage:
- Size: 1.66 MB
- Stars: 5
- Watchers: 7
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Contributing: docs/contributing.md
- License: LICENSE
Awesome Lists containing this project
README
ASTRA - Adaptive Stress Testing for Robust AI
----
# ASTRA-RL Toolbox
ASTRA-RL is a python toolbox for training and evaluating language models and generative AI systems that
use textual inputs. It provides a set of tools for training, evaluating, and analyzing
language models, with a focus on applying reinforcement learning based refinement techniques
to improve evaluator model performance.
## Installation
To install the ASTRA-RL toolbox, you can use pip. The package is available on PyPI, so you can install it directly from there.
```bash
pip install astra-rl
```
You can then import the library in your Python code:
```python
import astra_rl
# or
import astra_rl as astral
```
## Development
This section provides instructions for setting up the development environment and running tests.
To start, we _STRONGLY_ recommend using [uv](https://docs.astral.sh/uv/) to manage your Python environment. This will ensure that you have the correct dependencies and versions installed.
### Setting Up the Development Environment
1. Clone the repository:
```bash
git clone https://github.com/sisl/astra-rl.git
cd astra-rl
```
2. Sync package dependencies:
```bash
uv sync --dev
```
This will create a `.venv` directory in the project root with all the necessary dependencies installed.
3. Install pre-commit hooks:
```bash
uv run pre-commit install
```
This will ensure that the linter (`ruff`), formatter (`ruff`), and type checker (`mypy`) is happy with your code every time you commit.
### Running Tests
Assuming you've set up your environment using `uv`, you can run the tests using the following command:
```bash
pytest
```
or
```bash
uv run pytest
```
To generate local coverage reports, you can use:
```bash
uv run coverage run -m pytest
uv run coverage report # Generate CLI report
uv run coverage html # Generate HTML report
```
#### Running Tests with GPU
Some tests may require a GPU to run. You can enable GPU tests by passing the `--gpu` option:
```bash
uv run pytest --gpu
```
These tests will be _skipped_ by default unless you specify the `--gpu` option.
### Generating Documentation
To generate the documentation, you can use the following command:
```bash
uv run mkdocs serve
```
This will build the documentation and start a local server. You can then view the documentation in your web browser.