https://github.com/soniccodes/rand-merge-gpt

Experiment with making all blocks in a GPT-2 random mixture of all the rest
https://github.com/soniccodes/rand-merge-gpt

Last synced: 10 months ago
JSON representation

Experiment with making all blocks in a GPT-2 random mixture of all the rest

• Host: GitHub
• URL: https://github.com/soniccodes/rand-merge-gpt
• Owner: SonicCodes
• Created: 2024-06-25T20:26:29.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-06-25T20:42:30.000Z (almost 2 years ago)
• Last Synced: 2025-02-24T22:52:16.267Z (over 1 year ago)
• Language: Python
• Size: 9.77 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# rand-merge-gpt

Experiment with making all blocks in a GPT-2 random mixture of all the rest

![Group 7](https://github.com/SonicCodes/rand-merge-gpt/assets/48802163/2a54f954-cf32-4102-9309-a15154806fb3)

During each forward pass , each block's weights becomes a random mixture of all the other blocks, results: