https://github.com/nachiket273/aora-light

Agreement-Aware Orthogonal Routing Attention (AORA-Light): lightweight dual-branch transformer attention for robustness to non-consensus signals.
https://github.com/nachiket273/aora-light

attention-mechanism deep-learning experimental-ml language-modeling neural-network pytorch research robust-ml scientific-machine-learning transformer

Last synced: 13 days ago
JSON representation

Agreement-Aware Orthogonal Routing Attention (AORA-Light): lightweight dual-branch transformer attention for robustness to non-consensus signals.

• Host: GitHub
• URL: https://github.com/nachiket273/aora-light
• Owner: nachiket273
• License: apache-2.0
• Created: 2026-05-04T16:26:58.000Z (about 1 month ago)
• Default Branch: main
• Last Pushed: 2026-05-13T10:05:39.000Z (about 1 month ago)
• Last Synced: 2026-05-13T12:13:32.724Z (about 1 month ago)
• Topics: attention-mechanism, deep-learning, experimental-ml, language-modeling, neural-network, pytorch, research, robust-ml, scientific-machine-learning, transformer
• Homepage:
• Size: 10.7 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# AORA-Light

A lightweight experimental transformer architecture introducing dual-branch attention:

- consensus attention

- exclusive attention

AORA-Light studies whether explicit separation of consensus and anti-consensus context improves robustness to noisy and minority signals.

## Current status

Early prototype.

## Planned experiments

- Tiny Shakespeare

- WikiText-2 subset

- Needle-in-noise benchmark

- contradiction benchmark

- scientific image benchmark

## Hardware

Designed for low-resource experimentation:

- RTX 3050 6GB

- Google Colab Free

## Citation

Coming soon.