{"id":24493023,"url":"https://github.com/fastmachinelearning/nac-opt","last_synced_at":"2025-04-14T01:40:35.236Z","repository":{"id":206121648,"uuid":"705922627","full_name":"fastmachinelearning/nac-opt","owner":"fastmachinelearning","description":"Neural Architecture Codesign: A Model Optimization Toolkit for Physics","archived":false,"fork":false,"pushed_at":"2025-02-12T19:09:22.000Z","size":7818,"stargazers_count":7,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-27T15:52:14.870Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fastmachinelearning.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-17T00:54:39.000Z","updated_at":"2025-02-12T19:09:26.000Z","dependencies_parsed_at":"2023-12-01T23:23:29.960Z","dependency_job_id":"dde862ed-50dc-4fe0-b890-68178f3618c7","html_url":"https://github.com/fastmachinelearning/nac-opt","commit_stats":null,"previous_names":["lukemcdermotttt/morph","fastmachinelearning/morph","fastmachinelearning/nac-opt"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fastmachinelearning%2Fnac-opt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fastmachinelearning%2Fnac-opt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fastmachinelearning%2Fnac-opt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fastmachinelearning%2Fnac-opt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fastmachinelearning","download_url":"https://codeload.github.com/fastmachinelearning/nac-opt/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248809039,"owners_count":21164893,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-21T19:18:54.620Z","updated_at":"2025-04-14T01:40:35.219Z","avatar_url":"https://github.com/fastmachinelearning.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Neural Architecture Codesign for Fast Physics Applications\nThis repository contains the implementation of Neural Architecture Codesign (NAC), a framework for optimizing neural network architectures for physics applications with hardware efficiency in mind. NAC employs a two-stage optimization process to discover models that balance task performance with hardware constraints.\n\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.14618350.svg)](https://doi.org/10.5281/zenodo.14618350)\n\n## Overview\n\nNAC automates the design of deep learning models for physics applications while considering hardware constraints. The framework uses neural architecture search and network compression in a two-stage approach:\n\n1. Global Search Stage: Explores diverse architectures while considering hardware constraints\n2. Local Search Stage: Fine-tunes and compresses promising candidates\n3. FPGA Synthesis (*optional*): Converts optimized models to FPGA-deployable code\n\nThe framework is demonstrated through two case studies:\n- BraggNN: Fast X-ray Bragg peak analysis for materials science\n- Jet Classification: Deep Sets architecture for particle physics\n\n## Case Studies\n\nThe framework is demonstrated through two case studies:\n\n### BraggNN\n- Fast X-ray Bragg peak analysis for materials science\n- Convolutional architecture with attention mechanisms\n- Optimizes for peak position prediction accuracy and inference speed\n\n### Deep Sets for Jet Classification \n- Particle physics classification using permutation-invariant architectures\n- Optimizes classification accuracy and hardware efficiency\n\n\n## Installation\n\n1. Create a conda environment:\n```bash\nconda create --name NAC_env python=3.10.10\nconda activate NAC_env\n```\n\n2. Install dependencies:\n```bash\npip install -r requirements.txt\n```\n\n3. Download datasets:\n- For BraggNN:\n```bash\npython data/get_dataset.py\n```\n- For Deep Sets: Download `normalized_data3.zip` and extract to `/data/normalized_data3/`\n\n## Usage\n\n### Global Search\n\nRun architecture search for either BraggNN or Deep Sets:\n\n```bash\npython global_search.py\n```\n\nThe script will output results to `global_search.txt`. For the Deep Sets model, results will be in `Results/global_search.txt`.\n\n### Local Search\n\nRun model compression and optimization:\n\n```bash\npython local_search.py\n```\n\nResults will be saved in `Results/deepsets_search_results.txt` or `Results/bragg_search_results.txt`.\n\n## Directory Structure\n\n```\n.\n├── data/                         # Dataset handling\n├── examples/                     # Example configs and search spaces\n│   ├── BraggNN/\n│   └── DeepSets/\n├── models/                       # Model architectures\n├── utils/                        # Utility functions\n├── global_search.py             # Global architecture search\n├── local_search.py              # Local optimization\n└── requirements.txt\n```\n\n## Architecture Search Methodology\n\n### Global Search Stage\nThe global search explores a wide range of model architectures to find promising candidates that balance performance and hardware efficiency. This stage:\n\n1. **Example Model Starting Points**: \n   - Uses pre-defined model configurations in `*_model_example_configs.yaml` as initial reference points\n   - For BraggNN: includes baseline architectures like OpenHLS and original BraggNN\n   - For Deep Sets: includes baseline architectures of varying sizes (tiny to large)\n\n2. **Explores Architecture Space**:\n   - Search space defined in `*_search_space.yaml` specifies possible model variations\n   - For BraggNN: explores combinations of convolutional, attention, and MLP blocks\n   - For Deep Sets: varies network widths, aggregation functions, and MLP architectures\n\n3. **Multi-Objective Optimization**:\n   - Uses NSGA-II algorithm to optimize both task performance and hardware efficiency\n   - Evaluates models based on accuracy/mean distance and bit operations (BOPs)\n   - Maintains diverse population of candidate architectures\n\nRun global search with:\n```bash\npython global_search.py\n```\n\n### Local Search Stage\nThe local search takes promising architectures from the global search and optimizes them further through:\n\n1. **Training Optimization**:\n   - Fine-tunes hyperparameters using tree-structured Parzen estimation\n   - Optimizes learning rates, batch sizes, and regularization\n\n2. **Model Compression**:\n   - Quantization-aware training (4-32 bits)\n   - Iterative magnitude pruning (20 iterations, removing 20% parameters each time)\n   - Evaluates trade-offs between model size, accuracy, and hardware efficiency\n\n3. **Architecture Selection**:\n   - Identifies best models across different operating points\n   - Balances accuracy, latency, and resource utilization\n\nRun local search with:\n```bash\npython local_search.py\n```\n## Results\n\nThe framework achieves:\n\n### BraggNN\n- 0.5% improved accuracy with 5.9× fewer BOPs (large model)\n- 3% accuracy decrease for 39.2× fewer BOPs (small model)\n- 4.92 μs latency with \u003c10% FPGA resource utilization\n\n### Jet Classification\n- 1.06% improved accuracy with 7.2× fewer BOPs (medium model)\n- 2.8% accuracy decrease for 30.25× fewer BOPs (tiny model)\n- 70 ns latency with \u003c3% FPGA resource utilization\n\n\n## Citation\n\nIf you use this code in your research, please cite our paper:\n\n```bibtex\n@misc{weitz2025neuralarchitecturecodesignfast,\n      title={Neural Architecture Codesign for Fast Physics Applications}, \n      author={Jason Weitz and Dmitri Demler and Luke McDermott and Nhan Tran and Javier Duarte},\n      year={2025},\n      eprint={2501.05515},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG},\n      url={https://arxiv.org/abs/2501.05515}, \n}\n```\n\n## Docker\n\nYou can run this project using our pre-built Docker image:\n\n```bash\ndocker pull gitlab-registry.nrp-nautilus.io/dimademler/nac_image:latest\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffastmachinelearning%2Fnac-opt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffastmachinelearning%2Fnac-opt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffastmachinelearning%2Fnac-opt/lists"}