{"id":19517917,"url":"https://github.com/vtstech/modeltraininglocal","last_synced_at":"2026-06-11T04:31:26.225Z","repository":{"id":202750097,"uuid":"626618077","full_name":"VTSTech/ModelTrainingLocal","owner":"VTSTech","description":null,"archived":false,"fork":false,"pushed_at":"2023-04-17T00:05:57.000Z","size":153,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-01-08T12:48:56.216Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VTSTech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-11T20:39:52.000Z","updated_at":"2023-04-16T23:55:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"21f67900-741f-4634-9545-e4f7a9341d99","html_url":"https://github.com/VTSTech/ModelTrainingLocal","commit_stats":null,"previous_names":["vtstech/modeltraininglocal"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VTSTech%2FModelTrainingLocal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VTSTech%2FModelTrainingLocal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VTSTech%2FModelTrainingLocal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VTSTech%2FModelTrainingLocal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VTSTech","download_url":"https://codeload.github.com/VTSTech/ModelTrainingLocal/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240766611,"owners_count":19854119,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T00:07:13.118Z","updated_at":"2026-06-11T04:31:26.176Z","avatar_url":"https://github.com/VTSTech.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ModelTrainingLocal\n\nLicense:\ncc-by-nc-4.0, Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)\nhttps://creativecommons.org/licenses/by-nc/4.0/\n\nFor training on online jupyter notebooks: https://github.com/Dampish0/ModelTrainingLocal\n\nHELLO, HERE IS THE CODE FOR TRAINING ON LOCAL PC, ANY MODEL WORKS AS LONG AS IT RUNS ON PYTORCH.\n\nHere is training data, 2 DATASET DONT GET THEM CONFUSED!!\nhttps://huggingface.co/datasets/Dampish/QuickTrain/tree/main\n\nIf you use my code, give credit.\n\nSo ill make it short\nYou can change variables in the json file here called \"Data.json\".\n\n![image1](/Image21.png)\n\"huggingface_access_token\" is the token for it to write to huggingface, if left blank it will not push to huggingface.co otherwise fill.\n\"model\" is the actual model from huggingface.co or a local model of choice, its better to use huggingface though.\n\"model_tokenizer\" is the base model you are using, the one you are fine tuning, you need the orginal tokenizer to avoid issues, this is simple.\n\"Data\" is the actual training data.\n\"valid_data\" is the validation data.\n\"out_dir\" is the directory where you where you want the output and/or what its going to be named. Will automatically make on if it doesnt exist.\n\n\nthe only parameters u need to worry about are gradient_accumulation_steps\nand learning rate\nepoch is basically for how long you want to train it\nits easier to limit training with max step than using epoch\ncutoff_len is not important, it only changes how long the instruction is before it cuts it off\nsome models have a max limit of 1024 tokens\n\n![image2](/image31.png)\n\nGRADIENT_ACCUMULATION_STEPS, you want this to be between 4 and 32, apparently the more u have the better training u get.\n\nBatch_size and micro_batch_size is exactly the same in this case. This number basically dictates how fast the step training goes, it also uses alot more vram when you increase it, the more the better if you ask me. Generally 4 for cpu, 8 for gpu, higher batch = higher ram. This script automatically utilizes maximal amount of micro_batch_size as it can, generally you dont need to touch this number, could lead to crashes and pain trying to maximize the gpu usage. IF 0 THEN AUTOMATICAL micro_batch_size OTHERWISE IT WILL TAKE YOUR NUMBER!! \n\nLEARNING_RATE Should be between 2e-5 \u003c-\u003e 5e-5 could be more could be less.\n\nCUTOFF_LEN is the length of string before it cuts it off.\n\nMAX_STEP is Easier to use than Epochs, basically gives a max amount of steps and ignores epoch number, for example if you give it 1000 max steps that could equal to 7.61 epochs, this number is completely random.\n\nI rather you DO NOT use the last variables as they are hard to work with, unless you know what you are doing. Only play with CPU_MODE.\n![image2](/image41.png)\nIm too lazy to explain them so good luck.\n\nThanks for using my code\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvtstech%2Fmodeltraininglocal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvtstech%2Fmodeltraininglocal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvtstech%2Fmodeltraininglocal/lists"}