{"id":31545168,"url":"https://github.com/launchplatform/marketplace","last_synced_at":"2025-10-04T14:11:57.765Z","repository":{"id":311033117,"uuid":"1030657783","full_name":"LaunchPlatform/marketplace","owner":"LaunchPlatform","description":"Marketplace ML experiment - training without backprop","archived":false,"fork":false,"pushed_at":"2025-09-06T21:22:31.000Z","size":568,"stargazers_count":24,"open_issues_count":3,"forks_count":2,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-09-06T23:27:14.661Z","etag":null,"topics":["experimental","machine-learning","research"],"latest_commit_sha":null,"homepage":"https://fangpenlin.com/posts/2025/09/02/marketplace-v2-is-all-you-need-a-training-algorithm-on-par-with-backprop/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LaunchPlatform.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-02T03:51:23.000Z","updated_at":"2025-09-04T15:09:29.000Z","dependencies_parsed_at":"2025-08-21T19:49:07.800Z","dependency_job_id":"6cb16308-ea1e-4671-b0bb-d8ca87249938","html_url":"https://github.com/LaunchPlatform/marketplace","commit_stats":null,"previous_names":["launchplatform/marketplace"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/LaunchPlatform/marketplace","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LaunchPlatform%2Fmarketplace","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LaunchPlatform%2Fmarketplace/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LaunchPlatform%2Fmarketplace/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LaunchPlatform%2Fmarketplace/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LaunchPlatform","download_url":"https://codeload.github.com/LaunchPlatform/marketplace/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LaunchPlatform%2Fmarketplace/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278322146,"owners_count":25967874,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["experimental","machine-learning","research"],"created_at":"2025-10-04T14:11:54.791Z","updated_at":"2025-10-04T14:11:57.755Z","avatar_url":"https://github.com/LaunchPlatform.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Marketplace\nMarketplace is a machine learning experiment aimed at training a model efficiently on a GPU without using backpropagation.\nThe approach involves breaking down the layers of a machine learning model into smaller groups, running them with various parameter combinations.\nWe select the best-performing parameter combination and mutate it with different parameter variants.\nTo learn more about the concept, please refer to the articles:\n\n1. [Marketplace: my first attempt at training without backprop on GPU efficiently](https://fangpenlin.com/posts/2025/08/18/marketplace-my-first-attempt-at-training-without-backprop-on-gpu-efficiently/)\n2. [Marketplace V2 is all you need: A training algorithm on par with backprop that needs only forward pass](https://fangpenlin.com/posts/2025/09/02/marketplace-v2-is-all-you-need-a-training-algorithm-on-par-with-backprop/)\n3. [Continual learning with the Marketplace algorithm: model learns new data through inference, not training](https://fangpenlin.com/posts/2025/09/09/continual-learning-with-marketplace-model-learns-new-data-with-mostly-inference/)\n\nFor example, the [beautiful_mnist model](https://github.com/tinygrad/tinygrad/blob/c30a113b2a876cabaea1049601fea3a0b758c5b1/examples/beautiful_mnist.py) included in [Tinygrad](https://github.com/tinygrad/tinygrad)'s example folder can be broken down into three groups of layers:\n\n```python\nfrom marketplace.training import Spec\nfrom marketplace.nn import Model\nfrom tinygrad import Tensor\nfrom tinygrad.nn import Conv2d\nfrom tinygrad.nn import InstanceNorm\nfrom tinygrad.nn import Linear\n\n[\n  Spec(\n    model=Model(\n        Conv2d(vendor_count, 1, 32, 5),\n        Tensor.relu,\n        Conv2d(vendor_count, 32, 32, 5),\n        Tensor.relu,\n        InstanceNorm(vendor_count, 32),\n        Tensor.max_pool2d,\n    )\n  ),\n  Spec(\n    model=Model(\n        Conv2d(vendor_count, 32, 64, 3),\n        Tensor.relu,\n        Conv2d(vendor_count, 64, 64, 3),\n        Tensor.relu,\n        InstanceNorm(vendor_count, 64),\n        Tensor.max_pool2d,\n        lambda x: x.flatten(1),\n    ),\n  ),\n  Spec(\n    model=Model([Linear(vendor_count, 576, 10)]),\n  ),\n]\n```\n\nWith that, we can run the model on GPU with different combinations of parameters.\nThe following code is a simple example of how to run a forward pass of the model.\n\n```python\nfrom tinygrad import Tensor\nfrom tinygrad import TinyJit\nfrom marketplace.training import forward\nfrom marketplace.optimizers import Optimizer\n\n@TinyJit\ndef forward_step() -\u003e tuple[Tensor, Tensor, Tensor]:\n    samples = Tensor.randint(batch_size, high=X_train.shape[0])\n    x = X_train[samples]\n    y = Y_train[samples]\n    batch_logits, batch_paths = forward(marketplace, x)\n    loss = Tensor.stack(\n        *(logits.sparse_categorical_crossentropy(y) for logits in batch_logits),\n        dim=0,\n    )\n    best_loss, best_index = loss.topk(1, largest=False)\n    best_index = best_index.squeeze(0)\n    accuracy = (\n        (batch_logits[best_index].sigmoid().argmax(axis=1) == y).sum() / batch_size\n    ) * 100\n    return (\n        best_loss.realize(),\n        accuracy.realize(),\n        batch_paths[best_index].realize(),\n    )\n\nlr = Tensor(1e-1).contiguous().realize()\noptimizer = Optimizer(\n    marketplace=marketplace,\n    learning_rate=lr,\n)\nbest_loss, best_accuracy, best_path = forward_step()\n\n```\n\nNext, now we know the best parameters combination, we can mutate it with different variants of parameters.\n\nTODO: the following is outdated, needs to update\n\n```python\n@TinyJit\ndef mutate_step(best_path: Tensor):\n    mutate(\n        marketplace=marketplace,\n        leading_path=best_path,\n        jitter=lr,\n    )\n\nmutate_step(best_path)\n\n```\n\nThat's it.\nWe just trained a model without using backpropagation and relying on only the forward pass!\nBy reepeating the process, we can train a model.\nOf course, this is still no match for the backprop training, but it's an interesting start.\n\n## Experiments\n\nAll of the experiments are in the `experiments` folder.\nTo run the training, you can use the following command:\n\n```bash\nCUDA=1 uv run python -m experiments.beautiful_mnist\n```\n\nIt comes with some arguments to control the training, you can see them by running:\n\n```bash\nuv run python -m experiments.beautiful_mnist --help\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaunchplatform%2Fmarketplace","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flaunchplatform%2Fmarketplace","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaunchplatform%2Fmarketplace/lists"}