{"id":25707583,"url":"https://github.com/AndreasKaratzas/orin","last_synced_at":"2025-02-25T08:07:08.146Z","repository":{"id":222414822,"uuid":"757158820","full_name":"AndreasKaratzas/orin","owner":"AndreasKaratzas","description":"Setting up the NVIDIA Jetson Orin Nano Developer Kit","archived":false,"fork":false,"pushed_at":"2024-02-26T19:25:26.000Z","size":55,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-02-26T20:37:22.750Z","etag":null,"topics":["cuda","cudnn","jetpack6","nvidia-jetson","nvidia-sdkmanager","orin-nano"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AndreasKaratzas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-13T23:06:54.000Z","updated_at":"2024-02-14T02:27:10.000Z","dependencies_parsed_at":"2024-02-26T20:34:37.453Z","dependency_job_id":"d92a3faa-5dea-4009-99ad-75b2b4d9da52","html_url":"https://github.com/AndreasKaratzas/orin","commit_stats":null,"previous_names":["andreaskaratzas/orin"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AndreasKaratzas%2Forin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AndreasKaratzas%2Forin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AndreasKaratzas%2Forin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AndreasKaratzas%2Forin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AndreasKaratzas","download_url":"https://codeload.github.com/AndreasKaratzas/orin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240627959,"owners_count":19831599,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cuda","cudnn","jetpack6","nvidia-jetson","nvidia-sdkmanager","orin-nano"],"created_at":"2025-02-25T08:06:36.786Z","updated_at":"2025-02-25T08:07:08.134Z","avatar_url":"https://github.com/AndreasKaratzas.png","language":null,"funding_links":[],"categories":["Others"],"sub_categories":[],"readme":"\n# Setting up the NVIDIA Jetson Orin Nano\n\nThis tutorial was written for the NVIDIA Jetson Orin Nano Developer Kit (8 GB). The tutorial was written on `2024-2-13:19-42-5`. Details of the board are:\n\n| Key | Value |\n| --- | --- |\n| P-Number | p3767-0005 |\n| Module | NVIDIA Jetson Orin Nano (Developer kit) |\n| SoC | tegra234 |\n| CUDA Arch BIN | 8.7 |\n| L4T | 36.2.0 |\n| Jetpack | 6.0 DP |\n| Machine | aarch64 |\n| System | Linux |\n| Distribution | Ubuntu 22.04 Jammy Jellyfish |\n| Release | 5.15.122-tegra |\n| Python | 3.11.5 |\n| CUDA | 12.2.140 |\n| OpenCV | 4.8.0 |\n| OpenCV-Cuda | False |\n| cuDNN | 8.9.4.25 |\n| TensorRT | 8.6.2.3 |\n| VPI | 3.0.10 |\n| Vulkan | 1.3.204 |\n\n__IMPORTANT WARNING__: The host must be running on __Ubuntu 20.04__ to flash the board. \n\n\n### Flashing the Board\n\n\nThe board can be flashed using the SDK Manager. The SDK Manager can be downloaded from the [NVIDIA website](https://developer.nvidia.com/nvidia-sdk-manager). To flash the board, force the board into recovery mode by following the steps in [JetsonHacks Tutorial](https://www.youtube.com/watch?v=q4fGac-nrTI\u0026t=218s). Then, select the following options:\n\n- Host Machine: Ubuntu 20.04\n- Target Hardware: Jetson Orin Nano Developer Kit 8 GB\n- Target OS: Jetpack 6.0 DP\n- DeepStream: 6.0\n\nIn my case, I had already mounted an `NVME SSD` as a storage component. Most tutorials do SD cards, but I went with a mainstream 1 TB NVME SSD for performance reasons. Select the `pre-config` option to set the board and initialize it with a username and a password. For simplicity, the username and password are set to `nvidia`. The board will be flashed, and it will be ready to use.\n\n\n### Setting up the Board\n\nFirst, update the system:\n```bash\nsudo apt-get update\nsudo apt-get upgrade\nsudo reboot\n```\n\nThen, install the following packages:\n```bash\nsudo apt-get upgrade\nsudo apt-get install python3 python3-dev python3-distutils python3-venv python3-pip\nsudo apt-get install ssh firefox zlib1g software-properties-common lsb-release cmake build-essential libtool autoconf unzip wget htop ninja-build terminator zip\n```\n\nThese packages will prepare the board for development. If you are working extensively with Python, install `conda` package manager. The following commands will install `miniconda`:\n```bash\ncd Downloads/\nsudo apt upgrade\nwget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh\nchmod +x Miniconda3-latest-Linux-aarch64.sh\n./Miniconda3-latest-Linux-aarch64.sh\n```\n\n\n### Checking CUDA and cuDNN\n\nBefore we advance on checking CUDA and cuDNN, we need to verify `gcc` and `nvidia-smi`:\n```bash\ngcc --version\nnvidia-smi\n```\n\nWe will begin with checking CUDA:\n```bash\ngit clone https://github.com/NVIDIA/cuda-samples.git\ncd cuda-samples/Samples/1_Utilities/deviceQuery/\nmake\n./deviceQuery\n```\n\nAfter checking CUDA, we will check cuDNN:\n```bash\ncp -r /usr/src/cudnn_samples_v8/ ~/Documents/workspace/\ncd cudnn_samples_v8/mnistCUDNN/\nsudo apt install libfreeimage3 libfreeimage-dev\nsudo apt-get update\nsudo apt-get upgrade\nmake clean \u0026\u0026 make\n./mnistCUDNN\n```\n\n\n### Monitoring the Board\n\nTo monitor the board, we will install `jetson-stats`:\n```bash\nsudo pip3 install -U jetson-stats\n```\n\nTo check your board details and version of different software using `jtop`, as well as the usage across its computing resources and power consumption, there are some Python scripts that use `jtop`. For example [`jtop_properties.py`](https://github.com/rbonghi/jetson_stats/blob/master/examples/jtop_properties.py) is a quick way to monitor the aforementioned.\n\n\n### VS Code\n\nIf you are a Visual Studio Code user, it is supported on the Jetson. Run the following commands:\n\n```bash\ncd Downloads/\nwget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor \u003e packages.microsoft.gpg\nsudo install -o root -g root -m 644 packages.microsoft.gpg /etc/apt/trusted.gpg.d/\nsudo sh -c 'echo \"deb [arch=amd64,arm64,armhf signed-by=/etc/apt/trusted.gpg.d/packages.microsoft.gpg] https://packages.microsoft.com/repos/code stable main\" \u003e /etc/apt/sources.list.d/vscode.list'\nrm -f packages.microsoft.gpg\nsudo apt install apt-transport-https\nsudo apt update\nsudo apt install code\n```\n\n\n### Case\n\nFor my board, I bought the [Yahboom CUBE nano case](https://www.amazon.com/Yahboom-Dissipation-Protect-Cooling-Antenna/dp/B0CD71X8SV). On [their page](http://www.yahboom.net/study/CUBE_NANO), there are also [tutorials](https://youtu.be/anbMcWsagn8) and [code](https://drive.google.com/drive/folders/1A4L1ec-Na1_K0K1LXdnzSCva2iZ02YVX) for setting up the case and configuring the OLED screen that comes with it. Finally, there is also a [GitHub repo](https://github.com/YahboomTechnology/Jetson-CUBE-case) associated with the case.\n\n\n### Install PyTorch\n\nFirst, we will install the dependencies:\n```bash\nsudo apt-get install libopenblas-base libopenmpi-dev libomp-dev\nsudo apt-get install libjpeg-dev zlib1g-dev libpython3-dev libopenblas-dev libavcodec-dev libavformat-dev libswscale-dev\n```\n\nThen, we initialize the conda environment for PyTorch:\n```bash\nconda create -n torchenv python=3.10 pip\nconda activate torchenv\n```\n\nNow, we can install PyTorch:\n```bash\npip install Cython\ncd ~/Downloads\nwget https://nvidia.box.com/shared/static/0h6tk4msrl9xz3evft9t0mpwwwkw7a32.whl -O torch-2.1.0-cp310-cp310-linux_aarch64.whl\npip install numpy torch-2.1.0-cp310-cp310-linux_aarch64.whl\n```\n\nFinally, we install `torchvision`:\n```bash\ncd ~/Downloads\ngit clone --branch v0.16.1 https://github.com/pytorch/vision torchvision\nexport BUILD_VERSION=0.16.1\ncd torchvision/\npython setup.py install --user\ncd ../\npip install Pillow\n```\n\nTo test PyTorch, run the following:\n```python\nimport torch\n\nprint(torch.__version__)\nprint('CUDA available: ' + str(torch.cuda.is_available()))\nprint('cuDNN version: ' + str(torch.backends.cudnn.version()))\n\na = torch.cuda.FloatTensor(2).zero_()\nprint('Tensor a = ' + str(a))\nb = torch.randn(2).cuda()\nprint('Tensor b = ' + str(b))\nc = a + b\nprint('Tensor c = ' + str(c))\n```\n\nTo test `torchvision`, run the following:\n```python\nimport torch\nimport torchvision\n\nprint(torchvision.__version__)\n\nfrom torchvision.models import resnet50\n\nm = resnet50(weights=None)\nm.eval()\nx = torch.randn((4,3,224,224))\nm(x)\n```\n\n### Ensure LibTorch\n\nTo ensure that `libtorch` is installed, run the following:\n```bash\ncd ~/Documents/workspace/\nmkdir tests\ncd tests\nmkdir build\n```\n\nThen, create a `CMakeLists.txt` file, with the following content:\n```cmake\ncmake_minimum_required(VERSION 3.18 FATAL_ERROR)\nproject(torchsc)\n\nfind_package(Torch REQUIRED)\nset(CMAKE_CXX_FLAGS \"${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}\")\n\nadd_executable(torchsc torchsc.cpp)\ntarget_link_libraries(torchsc \"${TORCH_LIBRARIES}\")\nset_property(TARGET torchsc PROPERTY CXX_STANDARD 17)\n\n# The following code block is suggested to be used on Windows.\n# According to https://github.com/pytorch/pytorch/issues/25457,\n# the DLLs need to be copied to avoid memory errors.\nif (MSVC)\n  file(GLOB TORCH_DLLS \"${TORCH_INSTALL_PREFIX}/lib/*.dll\")\n  add_custom_command(TARGET torchsc\n                     POST_BUILD\n                     COMMAND ${CMAKE_COMMAND} -E copy_if_different\n                     ${TORCH_DLLS}\n                     $\u003cTARGET_FILE_DIR:example-app\u003e)\nendif (MSVC)\n```\n\nFinally, create a `torchsc.cpp` file, with the following content:\n```cpp\n#include \u003ctorch/torch.h\u003e\n#include \u003ciostream\u003e\n\nint main() {\n    torch::Tensor tensor = torch::rand({2, 3});\n    std::cout \u003c\u003c tensor \u003c\u003c std::endl;\n}\n```\n\nThe directory structure should look like this:\n```bash\n.\n├── build\n├── CMakeLists.txt\n└── torchsc.cpp\n\n1 directory, 2 files\n```\n\nTo build the project, run the following:\n```bash\ncd build\ncmake -DCMAKE_PREFIX_PATH=`python3 -c 'import torch;print(torch.utils.cmake_prefix_path)'` ..\ncmake --build . --config Release\n```\n\nFinally, run the executable:\n```bash\n./torchsc\n```\n\n### Kineto\n\nKineto is a library that provides performance analysis for PyTorch. It is a part of the PyTorch Profiler. However, it is not built in the PyTorch wheel for Jetson. To install it, you would need to __first build PyTorch from source__. After that, you can build `libkineto` from source. Otherwise, you will be getting the following warning:\n```bash\nstatic library kineto_LIBRARY-NOTFOUND not found\n```\n\nTo install Kineto from source, you can run the following:\n\n```bash\nexport CUDA_SOURCE_DIR=/usr/local/cuda-12.2\ngit clone --recursive --branch v0.4.0 https://github.com/pytorch/kineto.git\ncd kineto/libkineto\nmkdir build \u0026\u0026 cd build\ncmake ..\nmake\n```\n\nAfter ensuring that `libkineto` is working, you can install it:\n```bash\nsudo make install\n```\n\nTo test the `libkineto` library, run the following:\n```python\nimport torch\nimport torch.nn as nn\n\nx = torch.randn(1, 1).cuda()\nlin = nn.Linear(1, 1).cuda()\n\nwith torch.profiler.profile(\n    activities=[\n        torch.profiler.ProfilerActivity.CPU,\n        torch.profiler.ProfilerActivity.CUDA]\n) as p:\n    for _ in range(10):\n        out = lin(x)\nprint(p.key_averages().table(\n    sort_by=\"self_cuda_time_total\", row_limit=-1))\n```\n\n\n### Set Performance Mode\n\n```bash\nsudo nvpmodel -m 0\nsudo jetson_clocks\n```\n\nAlso set fan speed to maximal:\n```bash\nsudo jetson_clocks --fan\n```\n\n\n### Install Java\n\n```bash\nsudo apt-get update\nsudo apt-get install openjdk-11-jdk\njava -version\n```\n\n\n### Install Bazel\n\nTo install Bazel, you can run the following:\n```bash\ncd ~/Downloads\nwget https://github.com/bazelbuild/bazelisk/releases/download/v1.8.1/bazelisk-linux-arm64\nchmod +x bazelisk-linux-arm64\nsudo mv bazelisk-linux-arm64 /usr/local/bin/bazel\nwhich bazel\n```\n\n\n### Notes\n\nIt is important to note that the NVIDIA SDK Manager must be installed on an Ubuntu 20.04 engine. I tried two different machines running Ubuntu 22.04 and attempted to flash the board, but it would yield errors. I also tried Ubuntu 18.04, but the latest supported Jetpack was `5.x.y`, and at the moment of writing, the latest Jetpack is `6.z`. Therefore, the host machine must be running Ubuntu 20.04.\n\n\n### Helpful Links\n\n- [Deep Learning Libraries Compilation on Jetson Nano](https://pythops.com/post/compile-deeplearning-libraries-for-jetson-nano.html)\n\n\n### TODO\n\n- [ ] Install `torch-tensorrt`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAndreasKaratzas%2Forin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAndreasKaratzas%2Forin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAndreasKaratzas%2Forin/lists"}