{"id":23179578,"url":"https://github.com/mathiasotnes/multi-stream-transformer","last_synced_at":"2026-05-15T01:04:43.180Z","repository":{"id":262527548,"uuid":"887528682","full_name":"Mathiasotnes/Multi-Stream-Transformer","owner":"Mathiasotnes","description":"This repository explores the usage of parallel residual connections within the transformer architecture for natural language processing tasks. ","archived":false,"fork":false,"pushed_at":"2024-11-27T00:06:00.000Z","size":82,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-10T10:35:27.894Z","etag":null,"topics":["natural-language-processing","python","pytorch","transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mathiasotnes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-12T21:27:52.000Z","updated_at":"2024-11-27T00:21:04.000Z","dependencies_parsed_at":"2025-02-10T10:31:36.452Z","dependency_job_id":"c04eab17-bb10-417f-8ed9-ca84856b4d87","html_url":"https://github.com/Mathiasotnes/Multi-Stream-Transformer","commit_stats":null,"previous_names":["mathiasotnes/multi-stream-transformer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mathiasotnes%2FMulti-Stream-Transformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mathiasotnes%2FMulti-Stream-Transformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mathiasotnes%2FMulti-Stream-Transformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mathiasotnes%2FMulti-Stream-Transformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mathiasotnes","download_url":"https://codeload.github.com/Mathiasotnes/Multi-Stream-Transformer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247276185,"owners_count":20912288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["natural-language-processing","python","pytorch","transformer"],"created_at":"2024-12-18T07:15:20.687Z","updated_at":"2025-10-19T09:22:15.402Z","avatar_url":"https://github.com/Mathiasotnes.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multi-Stream Transformer\n\n---\n\n### 1. Introduction\n\nThis project implements a novel transformer architecture that processes text through multiple parallel streams, each trained on different objectives. The core idea is to enhance the model's understanding and generation capabilities by combining different types of pattern recognition. The parallel streams will be fused into a common residual stream optimized for next-token prediction, and the goal is to enhance the model's underlying comprehension.\n\n---\n\n### 2. Setup\n\nThe required libraries can be downloaded throught the requirements file:\n```\npip install -r requirements.txt\n```\n\nRun this afterwards for installing pytorch version compatible with CUDA 11.6:\n```\npip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116\n```\n\n---\n\n### 3. Repo activity\n![Alt](https://repobeats.axiom.co/api/embed/63f6b8809ab51b7382c8edf0af1a101375e5a4dd.svg \"Repobeats analytics image\")\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathiasotnes%2Fmulti-stream-transformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmathiasotnes%2Fmulti-stream-transformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathiasotnes%2Fmulti-stream-transformer/lists"}