{"id":25460945,"url":"https://github.com/uefi-code/bachelorgraduationdesign","last_synced_at":"2026-05-03T05:38:42.270Z","repository":{"id":208559434,"uuid":"721931016","full_name":"UEFI-code/BachelorGraduationDesign","owner":"UEFI-code","description":"I developed a PyTorch_For_PoorGuys framework and Let it train LLM on NVIDIA GeForce 2080Ti GPU as my Bachelor's Graduation Design Project","archived":false,"fork":false,"pushed_at":"2024-03-21T02:26:44.000Z","size":35,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-17T03:33:16.376Z","etag":null,"topics":["chatbot","cuda","gpu","hacking","large-language-models","pytorch"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/UEFI-code.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-11-22T04:33:10.000Z","updated_at":"2023-11-22T09:16:25.000Z","dependencies_parsed_at":"2023-12-09T02:36:53.501Z","dependency_job_id":"561880cd-0477-40df-97f2-3ef519760f73","html_url":"https://github.com/UEFI-code/BachelorGraduationDesign","commit_stats":{"total_commits":17,"total_committers":2,"mean_commits":8.5,"dds":"0.23529411764705888","last_synced_commit":"69918d315d04fde78c883d1284013d88bf7b6249"},"previous_names":["uefi-code/bachelorgraduatedesign","uefi-code/bachelorgraduationdesign"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/UEFI-code/BachelorGraduationDesign","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UEFI-code%2FBachelorGraduationDesign","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UEFI-code%2FBachelorGraduationDesign/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UEFI-code%2FBachelorGraduationDesign/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UEFI-code%2FBachelorGraduationDesign/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/UEFI-code","download_url":"https://codeload.github.com/UEFI-code/BachelorGraduationDesign/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UEFI-code%2FBachelorGraduationDesign/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32559716,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-03T03:21:47.309Z","status":"ssl_error","status_checked_at":"2026-05-03T03:21:43.884Z","response_time":103,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","cuda","gpu","hacking","large-language-models","pytorch"],"created_at":"2025-02-18T05:05:29.500Z","updated_at":"2026-05-03T05:38:42.242Z","avatar_url":"https://github.com/UEFI-code.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# My Bachelor Graduation Design\n\n```\nOn the lonely road of coding, long and winding,\nPoverty's shackles often leave hearts pining.\nBut I believe, despite all the pain and sorrow,\nMy thesis shall bring forth a brighter tomorrow.\n\nPyTorch_For_PoorGuys, a name not renowned,\nYet with dreams in my heart, aspirations unbound.\nHarnessing CPU might to transcend VRAM's plight,\nMay my design illuminate, dispelling the night.\n\nIn the ocean of tech, I journey with zest,\nExploring, challenging limits, never at rest.\nMay my thesis, like the North Star, guide the way,\nBreaking poverty's chains, bringing forth a new day.\n\n\nPoem written by OpenAI's ChatGPT 3.5\n```\n\n#### PyTorch_For_PoorGuys: A Deep Learning Framework for Training Large-Language-Model on Cheap Workstation with Smaller VRAM required.\n\nNVIDIA GeForce 1080Ti/2080Ti/3090/4090 are cheaper for Universitys or personal researchers.\n\nLLM was famous and Omoshiroii since OpenAI GPT-3 proposed.\n\nHowever, caused by its smaller Video-RAM, it is hard to train a LLM on orginal PyTorch.\n\nI want to let GPU borrow some Host-RAM to buffer LLM data when its VRAM has been exhausted.\n\nSo I do a fork with original PyTorch and modified its code to enable DMA method.\n\nYou can download the prebuilt wheel installer file of the hacked version of PyTorch and compatible Torchvision build from [here](https://github.com/UEFI-code/PyTorch_For_PoorGuys/releases), or build it yourself from source code!\n\n### Features\n\n- Allocated Memory Buffer on VRAM first for Lower Latency, once the VRAM exhausted, it automatic allocated from Host RAM\n- Create a Memory Pointer which can be both accessed from CPU or GPU side (Need hacking code yourself if you want completely using DMA)\n- With an easy version of Website working around with a Toy ChatGPT pretrained by this framework\n\n### Future\n\n- Hacking the GPU Driver to DIY Memory-Management-Unit, enable more flexable way to play with GPU\n- Let GPU controlling Hardware (Like Servo Motors) using Host Physic-Address via DMA directly\n- Use FPGA to design a Physics Address Listener via PCI-E and DMA to interaction with GPU\n- It will be interesting when Sensor signal can be passed to the Deep-Learning-Model in parallel directly, and the Model output signal can controlling device in parallel directly, just let CPU to sleep!\n\n### Hardware Requirement\n\n- NVIDIA Gefore 1080Ti/2080Ti/3090/4090 Graphic Card\n- Compatible motherboard and Power Supply\n- Intel Xeon E5-2620 V4 or Higher CPU\n- 128GB DDR4 RAM or Higher\n- 1TB SSD or Higher\n\n### Software Requirement\n\n- Debian Style of Linux, Ubuntu 20.04 LTS is suggested\n- PyTorch 2.0 Compatible GPU Driver\n- PyTorch 2.0 Compatible CUDA Developer Toolkit. See Build sections for details in Backend modules README.md file\n- A GUI environment with browser to run frontend HTML code (Can be on your laptop or another computer)\n\n### File Directory Structure\n\n- Backend\n    - [PyTorch_For_PoorGuys](https://github.com/UEFI-code/PyTorch_For_PoorGuys): The Core Module I have been proposed.\n    - [DeepSpeedExamples](https://github.com/UEFI-code/DeepSpeedExamples): The LLM training \u0026 infering code that I forked and modified from Microsoft's Repo\n- Frontend\n    - playground.html: The HTML \u0026 Javascript Demo site that connect to Backend.\n    - dummyBackend.py: The Python \u0026 Flask code that mook Backend response to debugging Frontend\n\n### Sync the Repo\n\n```bash\ngit clone --recursive https://github.com/UEFI-code/BachelorGraduationDesign\n```\n\nOr if you have already cloned the repo, you can sync the submodules by:\n\n```bash\ngit submodule update --init --recursive\n```\n\nIf the sync of submodules has error, then you can try to reset the submodules by:\n\n```bash\ngit submodule foreach git reset --hard\n```\n\n### Notice\n\n- This code will not Allocate Pageable Host Memory, which means it occupy Physic RAM at runtime\n- Never pass a Memory Pointer which need CPU do Exception-Handling to GPU, or you will get crash","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuefi-code%2Fbachelorgraduationdesign","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuefi-code%2Fbachelorgraduationdesign","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuefi-code%2Fbachelorgraduationdesign/lists"}