https://github.com/mathemage/preference-model-initial-assessment

solution for assessment at https://docs.google.com/forms/d/e/1FAIpQLSf4mqNST9kC6P41EDtaEYk65k0DptjuSyA_iRBoN10FwalSpw/viewform
https://github.com/mathemage/preference-model-initial-assessment

evaluation-framework large-language-models llama reinforcement-learning trasnformers

Last synced: about 1 month ago
JSON representation

solution for assessment at https://docs.google.com/forms/d/e/1FAIpQLSf4mqNST9kC6P41EDtaEYk65k0DptjuSyA_iRBoN10FwalSpw/viewform

• Host: GitHub
• URL: https://github.com/mathemage/preference-model-initial-assessment
• Owner: mathemage
• License: agpl-3.0
• Created: 2026-03-02T23:15:16.000Z (4 months ago)
• Default Branch: main
• Last Pushed: 2026-04-01T21:51:16.000Z (3 months ago)
• Last Synced: 2026-04-02T09:25:56.422Z (3 months ago)
• Topics: evaluation-framework, large-language-models, llama, reinforcement-learning, trasnformers
• Language: Python
• Homepage:
• Size: 37.1 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE
  • Agents: AGENTS.md

Awesome Lists containing this project

README

          
# Preference Model Initial Assessment

[![Validation](https://github.com/mathemage/preference-model-initial-assessment/actions/workflows/validation.yml/badge.svg)](https://github.com/mathemage/preference-model-initial-assessment/actions/workflows/validation.yml)

[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://choosealicense.com/licenses/agpl-3.0/)

Solution for the [technical assessment](https://docs.google.com/forms/d/e/1FAIpQLSf4mqNST9kC6P41EDtaEYk65k0DptjuSyA_iRBoN10FwalSpw/viewform).

Automated validation runs on every push, pull request, and manual workflow run.

It executes the

RL-style judge against `reference_solution/transformer_block.py` and runs

`pytest tests/ -v` to confirm the environment and reference implementation work

end to end.

## Environment: Implement a Llama-Style Transformer Decoder Block from Scratch

An RL environment where the LLM agent must implement five core components of

modern transformer architectures — **RMSNorm**, **Rotary Position Embeddings

(RoPE)**, **Grouped-Query Attention (GQA)**, **SwiGLU**, and a complete

**LlamaBlock** — entirely from scratch in PyTorch.

### Repository Structure

```

├── assessment_answers.md              # Full assessment answers

├── environment/

│   ├── prompt.md                      # Exact prompt given to the LLM agent

│   ├── judge.py                       # Automated judge (continuous score 0–1)

│   └── setup.sh                       # VM setup script

├── reference_solution/

│   └── transformer_block.py           # Working reference implementation

├── tests/

│   └── test_judge.py                  # Tests for the judge + reference solution

└── requirements.txt                   # Python dependencies

```

### Quick Start

```bash

pip install -r requirements.txt

# Run the judge against the reference solution

python environment/judge.py --solution reference_solution/transformer_block.py

# Run tests

pytest tests/ -v

```

### Assessment Answers

See [assessment_answers.md](assessment_answers.md) for detailed answers to all

assessment questions.