{"id":25349739,"url":"https://github.com/muhtasham/fp8-auto","last_synced_at":"2025-04-08T21:34:47.365Z","repository":{"id":274617562,"uuid":"923485778","full_name":"Muhtasham/fp8-auto","owner":"Muhtasham","description":"FP8 stochastic rounding","archived":false,"fork":false,"pushed_at":"2025-01-28T11:06:42.000Z","size":19,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-06T22:46:43.473Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Muhtasham.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-28T10:34:34.000Z","updated_at":"2025-01-28T11:06:46.000Z","dependencies_parsed_at":"2025-01-28T12:22:29.785Z","dependency_job_id":"11fbd77e-1af5-4ecd-aac4-8d14041e2668","html_url":"https://github.com/Muhtasham/fp8-auto","commit_stats":null,"previous_names":["muhtasham/fp8-auto"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Muhtasham%2Ffp8-auto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Muhtasham%2Ffp8-auto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Muhtasham%2Ffp8-auto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Muhtasham%2Ffp8-auto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Muhtasham","download_url":"https://codeload.github.com/Muhtasham/fp8-auto/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247933970,"owners_count":21020710,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-14T16:57:53.212Z","updated_at":"2025-04-08T21:34:47.331Z","avatar_url":"https://github.com/Muhtasham.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FP8 Auto-Converter\n\nA Work in Progress PyTorch implementation for converting between bfloat16 and FP8 formats, using **only native PyTorch and NumPy**.\n\n## Project Goals\n\nThis project aims to:\n\n1. **Convert Tensors**: \n   - bfloat16 -\u003e FP8 (stored as uint8) -\u003e back to bfloat16\n   - Using pure bit manipulation (no .to(torch.uint8))\n   - N-bit mantissa support (configurable)\n\n2. **Implement Stochastic Rounding**:\n   - Following [Higham's approach](https://nhigham.com/2020/07/07/what-is-stochastic-rounding/)\n   - Preserve statistical properties\n   - Mean preservation over multiple iterations\n\n3. **Handle Edge Cases**:\n   - NaN, Infinity\n   - Denormal numbers\n   - Zero values\n   - Sign preservation\n\n4. **Comprehensive Testing**:\n   - Statistical property validation\n   - Edge case verification\n   - Large-scale iteration tests\n\n\n\u003e 🚀 **Good Start, But Beware**: While the basic implementation is in place, there are critical issues with statistical properties and sign bit handling. Perfect for learning about FP8 and numerical methods, but not yet production-ready!\n\n## Current Status: Help Needed! 🚨\n\nThis implementation currently fails critical statistical tests. We're looking for help from the community to fix core issues:\n\n```python\n# Current behavior (problematic):\ntensor([  2.3594,  20.7500, -12.5000]) -\u003e\ntensor([ -2.3594, -20.7500, -12.5000])  # Complete sign flips!\n```\n\n### Key Issues to Solve\n\n1. **Sign Bit Flipping**:\n   - Values are getting their signs reversed during conversion\n   - Needs urgent fix in bit manipulation logic\n\n2. **Statistical Test Failure**:\n   ```python\n   # Test that fails:\n   assert torch.allclose(original, recovered_avg, rtol=1e-2)\n   # Running tensor through 100k iterations should preserve mean\n   ```\n\n3. **Stochastic Rounding Issues**:\n   - Current implementation doesn't preserve statistical properties\n   - Mean values drift significantly over multiple iterations\n\n## How You Can Help\n\n1. **Core Areas Needing Review**:\n   - Sign bit handling in `float_to_fp8` function\n   - Stochastic rounding implementation\n   - Statistical properties preservation\n\n2. **Testing**:\n   ```python\n   # Help us fix this test:\n   def test_statistical_properties(tensor_size=1000, num_runs=100000):\n       x = torch.randn(tensor_size, dtype=torch.bfloat16)\n       output_avg = average_over_runs(x, num_runs)\n       assert torch.allclose(x, output_avg, rtol=1e-2)  # Currently failing\n   ```\n\n3. **Contributions Welcome**:\n   - Bug fixes\n   - Code reviews\n   - Test cases\n   - Documentation improvements\n\n## Getting Started\n\n```python\n# Current implementation (needs fixing):\nfrom fp8_auto import round_to_fp8_represented_as_int8, undo_int8_fp8\n\n# Help us fix these functions\nfp8_tensor = round_to_fp8_represented_as_int8(tensor, n_mantissa=3)\nrecovered = undo_int8_fp8(fp8_tensor, n_mantissa=3, target_dt=torch.bfloat16)\n```\n\n## Contributing\n\n1. Fork the repository\n2. Create your feature branch\n3. Run the test suite\n4. Submit a pull request\n\nWe especially need help with:\n- [ ] Fixing sign bit preservation\n- [ ] Improving stochastic rounding accuracy\n- [ ] Adding comprehensive statistical tests\n- [ ] Optimizing performance\n\n## Community\n\n- Open an issue for discussion\n- Join [Discord](https://discord.gg/gpumode) \n- Share your expertise\n\nLet's make this work together! 🚀\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuhtasham%2Ffp8-auto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmuhtasham%2Ffp8-auto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuhtasham%2Ffp8-auto/lists"}