{"id":30815517,"url":"https://github.com/codesignal/learn_simulation-transformers","last_synced_at":"2025-09-06T08:52:40.385Z","repository":{"id":313215186,"uuid":"1050020965","full_name":"CodeSignal/learn_simulation-transformers","owner":"CodeSignal","description":"Bespoke simulation about the Transformer architecture.","archived":false,"fork":false,"pushed_at":"2025-09-04T14:48:50.000Z","size":100,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-04T16:34:14.821Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CodeSignal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-03T20:41:00.000Z","updated_at":"2025-09-04T14:48:26.000Z","dependencies_parsed_at":"2025-09-04T16:34:21.887Z","dependency_job_id":null,"html_url":"https://github.com/CodeSignal/learn_simulation-transformers","commit_stats":null,"previous_names":["codesignal/learn_simulation-transformers"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/CodeSignal/learn_simulation-transformers","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSignal%2Flearn_simulation-transformers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSignal%2Flearn_simulation-transformers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSignal%2Flearn_simulation-transformers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSignal%2Flearn_simulation-transformers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CodeSignal","download_url":"https://codeload.github.com/CodeSignal/learn_simulation-transformers/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSignal%2Flearn_simulation-transformers/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273880196,"owners_count":25184429,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-06T08:52:39.000Z","updated_at":"2025-09-06T08:52:40.376Z","avatar_url":"https://github.com/CodeSignal.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Travel Through Transformers\n\nAn interactive web-based simulation that lets learners follow a single token step-by-step through every component of a Transformer encoder/decoder stack.\n\n## Features\n\n- **Component-focused visualization**: Click through different transformer components to see detailed internals\n- **Interactive parameters**: Adjust layers, model dimensions, attention heads, and sequence length in real-time\n- **Dual visualization modes**: Abstract shape view for understanding flow, or detailed numerical values\n- **Multi-head attention visualization**: See how different attention heads process information\n- **Event logging**: All interactions are logged for analytics\n\n## Quick Start\n\n### Prerequisites\n\n- Node.js 16+ and npm\n- Python 3.7+\n\n### Installation\n\n1. **Install dependencies**:\n   ```bash\n   npm install\n   ```\n\n2. **Build the application**:\n   ```bash\n   npm run build\n   ```\n\n3. **Start the server**:\n   ```bash\n   python server/server.py\n   ```\n\n4. **Open your browser**:\n   Navigate to `http://localhost:3000`\n\n### Development Mode\n\nFor development with hot reloading:\n\n```bash\n# Terminal 1 - Start the development server\nnpm run dev\n\n# Terminal 2 - Start the logging server\npython server/server.py\n```\n\nThen open `http://localhost:5173` (development) or `http://localhost:3000` (production).\n\n## Usage\n\n### Controls\n\n1. **Component Selection**: Click on transformer components in the diagram to explore their internals\n2. **Show Values Toggle**: Switch between abstract block view and actual numerical matrices\n3. **Model Parameters**: \n   - Adjust number of layers (1-6)\n   - Change model dimension (32-512)\n   - Set attention heads (1-8, must divide model dimension)\n   - Modify sequence length (3-10)\n   - Choose positional encoding type\n   - Enable dropout visualization\n\n### Understanding the Visualization\n\n#### Abstract Mode (Default)\n- Colored blocks represent matrices with dimensions shown\n- Different colors indicate different types of operations\n- Active components highlighted with distinctive colors\n- Attention heads shown in different colors\n\n#### Values Mode\n- Heat maps show actual numerical values\n- Color intensity represents magnitude\n- Hover for precise values\n- Active components highlighted\n\n## Architecture\n\n```\ntravel-through-transformers/\n├── src/\n│   ├── components/          # React UI components\n│   │   ├── MatrixVisualization.tsx # D3-powered matrix visualization\n│   │   ├── TokenVisualization.tsx  # Token display and interaction\n│   │   ├── SettingsMenu.tsx        # Parameter controls\n│   │   ├── ComponentDetailsPanel.tsx # Component detail exploration\n│   │   ├── HelpMenu.tsx           # Help system\n│   │   └── TransformerDiagram.tsx # Main architecture diagram\n│   ├── hooks/               # Custom React hooks\n│   │   ├── useTransformerMachine.ts # Main state management (XState)\n│   │   ├── useTransformerDiagram.ts # Diagram interaction logic\n│   │   └── useEventLogger.ts       # Analytics logging\n│   ├── utils/               # Utility functions\n│   │   ├── math.ts          # Matrix operations\n│   │   ├── randomWeights.ts # Seeded random generation\n│   │   ├── constants.ts     # Configuration and steps\n│   │   ├── data.ts          # Sample data generation\n│   │   ├── componentDataGenerator.ts # Component data creation\n│   │   └── componentTransformations.ts # Math transformations\n│   ├── state/               # State management\n│   │   └── transformerMachine.ts # XState machine definition\n│   └── types/               # TypeScript definitions\n│       └── events.d.ts      # Event and parameter types\n├── server/\n│   └── server.py           # Python logging server\n└── logs/                   # Event logs (generated)\n```\n\n## Educational Goals\n\nThis simulation helps learners understand:\n\n1. **Component Architecture**: How transformer components are organized and connected\n2. **Attention Mechanism**: How queries, keys, and values interact\n3. **Multi-Head Attention**: How different heads capture different patterns\n4. **Residual Connections**: How information flows around attention blocks\n5. **Layer Normalization**: How activations are normalized\n6. **Feed-Forward Networks**: How information is processed after attention\n7. **Positional Encoding**: How position information is added to tokens\n8. **Cross-Attention**: How decoder attends to encoder representations\n\n## Technical Details\n\n- **Frontend**: React + TypeScript + Vite\n- **State Management**: XState for complex state transitions\n- **Visualization**: D3.js for interactive SVG graphics\n- **Styling**: TailwindCSS with CodeSignal brand colors\n- **Math**: Custom lightweight tensor operations (no external ML libraries)\n- **Backend**: Simple Python HTTP server for logging\n- **Data**: Seeded random weights for reproducible results\n\n## Customization\n\n### Adding New Components\n\n1. Add component definition to transformer machine states\n2. Implement component logic in `componentDataGenerator.ts`\n3. Add appropriate visualizations in component files\n4. Update component transformations in `componentTransformations.ts`\n\n### Modifying Visualization\n\n- Matrix colors: Edit `COLORS` in `constants.ts`\n- D3 rendering: Modify `MatrixVisualization.tsx` \n- Component descriptions: Update transformer machine configuration\n\n### Analytics\n\nEvent logs are stored in `logs/simulation_log.jsonl` with schema:\n```json\n{\n  \"timestamp\": 1625239200,\n  \"event_type\": \"param_change\" | \"component_select\" | \"toggle\" | \"zoom_change\",\n  \"payload\": { /* event-specific data */ }\n}\n```\n\n## Browser Support\n\n- Chrome 90+\n- Firefox 88+\n- Safari 14+\n- Edge 90+\n\n## Performance\n\nOptimized for:\n- 60 FPS animations\n- Sequence length ≤ 10\n- Attention heads ≤ 8\n- Model dimension ≤ 512\n\n## License\n\nMIT License - see [LICENSE](LICENSE) for details.\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests if applicable\n5. Submit a pull request\n\n## Troubleshooting\n\n### Common Issues\n\n**\"Cannot find module\" errors**: Run `npm install`\n\n**Server won't start**: Check that port 3000 is available, or specify a different port: `python server/server.py 3001`\n\n**Visualization not updating**: Try refreshing the page or clearing browser cache\n\n**Performance issues**: Reduce model parameters (fewer layers, smaller dimensions)\n\n### Debug Mode\n\nEnable debug logging:\n```bash\nDEBUG=1 python server/server.py\n```\n\n## Educational Extensions\n\nFuture enhancements could include:\n\n- Real model weights from Hugging Face\n- Attention pattern analysis\n- Interactive quizzes between steps\n- Comparison with other architectures\n- Custom text input\n- Export/import configurations ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodesignal%2Flearn_simulation-transformers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodesignal%2Flearn_simulation-transformers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodesignal%2Flearn_simulation-transformers/lists"}