{"id":31498981,"url":"https://github.com/aayes89/pyllm","last_synced_at":"2026-05-18T10:40:02.371Z","repository":{"id":308920307,"uuid":"1034565318","full_name":"aayes89/PyLLM","owner":"aayes89","description":"Entrena tu propio LLM desde cero","archived":false,"fork":false,"pushed_at":"2025-08-08T16:09:57.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-02T16:48:57.546Z","etag":null,"topics":["cpu","cuda","llm","llm-training","pip","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aayes89.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-08T15:41:54.000Z","updated_at":"2025-08-08T16:10:00.000Z","dependencies_parsed_at":"2025-08-08T18:23:59.530Z","dependency_job_id":"6c706a96-2c57-437f-97ac-a52b4e53fd75","html_url":"https://github.com/aayes89/PyLLM","commit_stats":null,"previous_names":["aayes89/pyllm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aayes89/PyLLM","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aayes89%2FPyLLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aayes89%2FPyLLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aayes89%2FPyLLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aayes89%2FPyLLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aayes89","download_url":"https://codeload.github.com/aayes89/PyLLM/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aayes89%2FPyLLM/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33175315,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-18T09:27:30.708Z","status":"ssl_error","status_checked_at":"2026-05-18T09:27:28.300Z","response_time":71,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpu","cuda","llm","llm-training","pip","python3"],"created_at":"2025-10-02T16:48:55.999Z","updated_at":"2026-05-18T10:39:57.909Z","avatar_url":"https://github.com/aayes89.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PyLLM\nGenera y entrena un LLM desde cero con tus propios datos.\n\nEl código implementa un LLM utilizando las siguientes dependencias:\n- \u003cb\u003eos, math, random, pathlib\u003c/b\u003e     - para funciones de sistema y aritmética\n- \u003cb\u003etorch\u003c/b\u003e                         - para entrenamiento con CPU o GPU (cuda)\n- \u003cb\u003esentencepiece\u003c/b\u003e                 - para tokenización\n\n# Estructura del proyecto\n- \u003cb\u003erun.sh\u003c/b\u003e: script batch para inicializar el proceso de ejecución.\n- \u003cb\u003emain.py\u003c/b\u003e: script python que genera y entrena el LLM.\n- \u003cb\u003egenerate_text.py\u003c/b\u003e: script python que permite interactuar con el modelo una vez generado, sin necesidad de entrenarlo.\n- \u003cb\u003edata\u003c/b\u003e: directorio creado por \u003cb\u003erun.sh\u003c/b\u003e donde se encuentra el corpus a entrenar el LLM.\n- \u003cb\u003ecorpus.txt\u003c/b\u003e: archivo de texto plano codificado en UTF-8 con el que entrena el LLM. (puede ser generado automáticamente por run.sh o puedo cargarlo con datos propios)\n- \u003cb\u003evenv\u003c/b\u003e: directorio creado por \u003cb\u003erun.sh\u003c/b\u003e para el entorno de virtual de python.\n\n# Selección de hardware\nSi va a utilizar un entrenamiento con GPU, descomentar la línea:\u003cbr\u003e\n\u003cb\u003e# device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\u003c/b\u003e\u003cbr\u003e\u003cbr\u003e\nComentar la siguiente en caso de utilizar sólo CPU:\u003cbr\u003e\n\u003cb\u003edevice = torch.device('cpu')\u003c/b\u003e\n\n# Dataset de lenguaje\n\u003cp\u003e\nEl dataset devuelve pares: \u003cb\u003e(input_ids, labels)\u003c/b\u003e donde \u003ccode\u003einput_ids[i] = token t_i\u003c/code\u003e y \u003ccode\u003elabels[i] = token t_{i+1}\u003c/code\u003e \u003cbr\u003e\nSe recorta en bloques de `block_size`.\u003cbr\u003e\nSe añade un token EOS (id = vocab-1) al final de cada texto.\n\u003c/p\u003e\n\n# Modelo Transformer “desde cero por etapas” \n- Las Proyecciones lineales para Q, K, V y salida final.\n- Escalamiento dot‑product attention.\n- Concatenamiento de cabezas y proyección final.\n- Self‑attention + residual.\n- FFN + residual.\n- Tensor: \u003ccode\u003e input_ids : (B, T)  dtype=torch.long        returns logits : (B, T, vocab)\u003c/code\u003e\n\n# Entrenamiento / Evaluación\n# Generación de texto\n\n# Ficheros generados\n\nEl script genera los siguientes archivos:\u003cbr\u003e\n- \u003cb\u003ellm_es.pth\u003c/b\u003e: el modelo generado. \n- \u003cb\u003ebpe_sp.model\u003c/b\u003e: contiene información con la que opera el modelo generado.\n- \u003cb\u003ebpe_sp.vocab\u003c/b\u003e: contiene información sobre los vocabularios con los que opera el modelo.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faayes89%2Fpyllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faayes89%2Fpyllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faayes89%2Fpyllm/lists"}