https://github.com/maneeshsit/pcie
Modify run:ai and other FOSS projects code for use with PCIe card-based AI accelerators for both inference and training
https://github.com/maneeshsit/pcie
cuda cxl cxl-mem distro exo k3s k8s kestra llamacpp llm-d mpi4py mpio onnxoptimizer opentelemetry-ebpf-profiler paxos-cluster pcie photonics-computing runai visualize vllm
Last synced: about 2 months ago
JSON representation
Modify run:ai and other FOSS projects code for use with PCIe card-based AI accelerators for both inference and training
- Host: GitHub
- URL: https://github.com/maneeshsit/pcie
- Owner: maneeshsit
- Created: 2025-02-10T04:03:53.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-08-11T13:30:13.000Z (about 2 months ago)
- Last Synced: 2025-08-11T15:22:26.062Z (about 2 months ago)
- Topics: cuda, cxl, cxl-mem, distro, exo, k3s, k8s, kestra, llamacpp, llm-d, mpi4py, mpio, onnxoptimizer, opentelemetry-ebpf-profiler, paxos-cluster, pcie, photonics-computing, runai, visualize, vllm
- Language: Python
- Homepage: https://medium.com/@maneeshsharma_68969/comparing-performance-of-ai-ml-hardware-a0d18cf657a0
- Size: 48.8 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# PCIe
Modify run.ai code for using a PCIe-based AI accelerator (such as a GPU, FPGA, or ASIC like NVIDIA, Xilinx, or Intel accelerators) for both inference and training
1. Set Up the PCIe AI Accelerator
Install the required drivers and SDKs for your PCIe accelerator. For example:
NVIDIA GPUs: Install CUDA and cuDNN.
Intel accelerators: Install OpenVINO Toolkit.
Xilinx FPGAs: Install Vitis AI runtime.
Ensure the PCIe device is visible via tools like lspci (Linux) or equivalent commands.
2. Modify Training Code
Use the appropriate deep learning framework and ensure device targeting is set to the PCIe accelerator. Examples:
3. and 4. code files
5. Use Accelerator-Specific Optimizations
For NVIDIA GPUs: Use TensorRT for inference optimization.
For Intel FPGAs: Use OpenVINO's optimized inference engine.
For Xilinx FPGAs: Use Vitis AI tools for quantization and deployment.
6. Monitor and Debug
Use monitoring tools to ensure efficient usage of the PCIe accelerator:
NVIDIA: nvidia-smi
Intel: OpenVINO Benchmark Tool
Xilinx: Vitis AI Profiler
# AI hardware accelerator-agnostic AI Platform Factory
