{"id":44834996,"url":"https://github.com/galilai-group/stable-pretraining","last_synced_at":"2026-02-17T01:11:58.967Z","repository":{"id":259358233,"uuid":"832716353","full_name":"galilai-group/stable-pretraining","owner":"galilai-group","description":"Reliable, minimal and scalable library for pretraining foundation and world models","archived":false,"fork":false,"pushed_at":"2026-02-09T23:29:18.000Z","size":65215,"stargazers_count":126,"open_issues_count":29,"forks_count":23,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-02-10T03:57:44.549Z","etag":null,"topics":["computer-vision","computer-vision-algorithms","contrastive-learning","deep-learning","distributed","foundation-models","joint-embedding","joint-embedding-predictive-architecture","large-language-model","multimodal-learning","pytorch","self-supervised-learning","stable-pretraining","transformers"],"latest_commit_sha":null,"homepage":"https://galilai-group.github.io/stable-pretraining/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/galilai-group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-07-23T15:20:57.000Z","updated_at":"2026-02-09T23:26:18.000Z","dependencies_parsed_at":"2024-10-24T20:19:20.147Z","dependency_job_id":"276b5d6d-109a-474b-91b7-f7668cb6d83b","html_url":"https://github.com/galilai-group/stable-pretraining","commit_stats":null,"previous_names":["rbalestr-lab/stable-ssl","rbalestr-lab/stable-pretraining","galilai-group/stable-pretraining"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/galilai-group/stable-pretraining","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/galilai-group%2Fstable-pretraining","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/galilai-group%2Fstable-pretraining/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/galilai-group%2Fstable-pretraining/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/galilai-group%2Fstable-pretraining/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/galilai-group","download_url":"https://codeload.github.com/galilai-group/stable-pretraining/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/galilai-group%2Fstable-pretraining/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29528464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-17T00:57:22.232Z","status":"ssl_error","status_checked_at":"2026-02-17T00:54:25.811Z","response_time":115,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","computer-vision-algorithms","contrastive-learning","deep-learning","distributed","foundation-models","joint-embedding","joint-embedding-predictive-architecture","large-language-model","multimodal-learning","pytorch","self-supervised-learning","stable-pretraining","transformers"],"created_at":"2026-02-17T01:11:58.165Z","updated_at":"2026-02-17T01:11:58.954Z","avatar_url":"https://github.com/galilai-group.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# stable-pretraining\n\n[![Documentation](https://img.shields.io/badge/Documentation-blue.svg)](https://rbalestr-lab.github.io/stable-pretraining/)\n[![Benchmarks](https://img.shields.io/badge/Benchmarks-blue.svg)](https://github.com/rbalestr-lab/stable-pretraining/tree/main/benchmarks)\n[![Test Status](https://github.com/rbalestr-lab/stable-pretraining/actions/workflows/testing.yml/badge.svg)](https://github.com/rbalestr-lab/stable-pretraining/actions/workflows/testing.yml)\n[![PyTorch](https://img.shields.io/badge/PyTorch-ee4c2c?logo=pytorch\u0026logoColor=white)](https://pytorch.org/get-started/locally/)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![WandB](https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-gradient.svg)](https://wandb.ai/site)\n\n\n\nAI is moving beyond labels. Today's models learn through **self-supervision** and **multimodal alignment**, extracting knowledge from raw data to build general-purpose representations that work across tasks. These foundation models are then deployed at scale, often after finetuning, to solve tasks in zero or few shot.\n\n`stable-pretraining` is a PyTorch framework built on top of Lightning for this new paradigm. What sets us apart is **real-time visibility into training quality** through extensive logging and monitoring. Our callback ecosystem (`OnlineProbe`, `OnlineKNN`, `RankMe`, and many more) provides insights into feature collapse, training dynamics, and downstream performance. Data flow as dictionaries through model components, metrics, and callbacks, making any intermediate value accessible and debuggable. With `stable-pretraining`: track everything, debug faster, iterate sooner.\n\nJoin our Discord: [https://discord.gg/8M6hT39X](https://discord.gg/adzpqWKM25)\n\n## How?\n\nTo reach flexibility, scalability and stability, we rely on battle-tested third party libraries: `PyTorch`, `Lightning`, `HuggingFace`, `TorchMetrics` amongst a few others. Those dependencies allow us to focus on assembling everything into a powerful ML framework. ``stable-pretraining`` adopts a flexible and modular design for seamless integration of components from external libraries, including architectures, loss functions, evaluation metrics, and augmentations.\n\n## Core Structure\n\n`stable-pretraining` simplifies complex ML workflows into 4 intuitive components:\n\n### 1 - Data\nYour dataset must follow a dictionary-structured format where each sample is a dictionary with named fields (e.g., `{\"image\": ..., \"label\": ...}`). This ensures consistent behavior across all components. You have multiple options for creating datasets:\n\n- **HuggingFace datasets** (if available on the Hub):\n```python\nimport stable_pretraining as spt\ntrain_dataset = spt.data.HFDataset(\n    path=\"frgfm/imagenette\",\n    name=\"160px\",\n    split=\"train\",\n    transform=train_transform,\n)\n```\n\n- **From PyTorch datasets**:\n```python\ntrain_dataset = spt.data.FromTorchDataset(\n    torchvision_dataset,\n    names=[\"image\", \"label\"],  # Map tuple outputs to dictionary keys\n    transform=train_transform,\n)\n```\n\n- **Custom datasets**: Any dataset that returns dictionaries\n\n```python\ndatamodule = spt.data.DataModule(train=train_dataloader, val=val_dataloader)\n```\n\n### 2 - Module\nThe key differentiator from PyTorch Lightning - **you only define the `forward` function**, not `training_step`! This unified approach computes losses and generates useful quantities that can be retrieved for monitoring and analysis:\n\n```python\n# Use the pre-built forward functions from stable_pretraining\nfrom stable_pretraining import forward\n\n# Simply use the appropriate forward for your method\nmodule = spt.Module(\n    backbone=backbone,\n    projector=projector,\n    forward=forward.simclr_forward,  # Or byol_forward, vicreg_forward, etc.\n    simclr_loss=spt.losses.NTXEntLoss(temperature=0.5),\n    optim={\n        \"optimizer\": {\"type\": \"Adam\", \"lr\": 0.001},\n        \"scheduler\": {\"type\": \"CosineAnnealingLR\"},\n        \"interval\": \"epoch\"\n    }\n)\n```\n\nOr define your own custom forward:\n```python\ndef forward(self, batch, stage):\n    out = {}\n\n    if isinstance(batch, list):\n        # Multi-view training - batch is a list of view dicts\n        embeddings = [self.backbone(view[\"image\"]) for view in batch]\n        out[\"embedding\"] = torch.cat(embeddings, dim=0)\n\n        if self.training:\n            projections = [self.projector(emb) for emb in embeddings]\n            out[\"loss\"] = self.simclr_loss(projections[0], projections[1])\n    else:\n        # Single-view validation\n        out[\"embedding\"] = self.backbone(batch[\"image\"])\n\n    return out\n```\n\n**Key points:**\n- The `forward` method defines both the loss and any quantities to monitor\n- No need to override `training_step`, `validation_step`, etc.\n- Return a dictionary with a `\"loss\"` key for training\n- All model components are passed as kwargs to `spt.Module`\n\n### 3 - Callbacks\nMonitor and evaluate your models in real-time during training. Callbacks are key ingredients of `stable-pretraining`, providing rich insights without interrupting your training flow:\n\n```python\n# Monitor SSL representations with a linear probe\nlinear_probe = spt.callbacks.OnlineProbe(\n    module,  # Pass the spt.Module instance\n    name=\"linear_probe\",  # Useful for retrieving metrics and values in logging\n    input=\"embedding\",  # Which output from forward to monitor\n    target=\"label\",      # Ground truth from batch\n    probe=torch.nn.Linear(512, 10),\n    loss_fn=torch.nn.CrossEntropyLoss(),\n    metrics={\n        \"top1\": torchmetrics.classification.MulticlassAccuracy(10),\n        \"top5\": torchmetrics.classification.MulticlassAccuracy(10, top_k=5),\n    },\n)\n\n# Track representation quality with KNN evaluation\nknn_probe = spt.callbacks.OnlineKNN(\n    name=\"knn_probe\",\n    input=\"embedding\",\n    target=\"label\",\n    queue_length=20000,\n    k=10,\n)\n```\n\nCallbacks are powered by an intelligent queue management system that automatically shares memory between callbacks monitoring the same data thus eliminating redundant computations.\n\n**Why callbacks matter:** Get real-time feedback on representation quality, catch issues like collapse early, and track multiple metrics simultaneously for deeper insights.\n\n### 4 - Trainer\nOrchestrate everything together with PyTorch Lightning's `Trainer`:\n\n```python\ntrainer = pl.Trainer(\n    max_epochs=10,\n    num_sanity_val_steps=1,\n    callbacks=[linear_probe, knn_probe, rankme],  # Your monitoring callbacks\n    precision=\"16-mixed\",\n    logger=False,\n    enable_checkpointing=False,\n)\nmanager = spt.Manager(trainer=trainer, module=module, data=data)\nmanager()\n```\n\nOnce configured, the `Manager` connects all components and handles the training loop with precise logging and monitoring (optional).\n\n## Complete Example\n\n\u003cdetails\u003e\n\u003csummary\u003eSimCLR on CIFAR-10\u003c/summary\u003e\n\nThis example demonstrates the key features of `stable-pretraining`: dictionary-structured data, unified forward function, and rich monitoring through callbacks.\n\n```python\nimport lightning as pl\nimport torch\nimport torchmetrics\nimport torchvision\nfrom torch import nn\nfrom lightning.pytorch.loggers import WandbLogger\n\nimport stable_pretraining as spt\nfrom stable_pretraining import forward\nfrom stable_pretraining.data import transforms\n\n# Define augmentations for SimCLR (creates 2 views of each image)\nsimclr_transform = transforms.MultiViewTransform(\n    [\n        transforms.Compose(\n            transforms.RGB(),\n            transforms.RandomResizedCrop((32, 32), scale=(0.2, 1.0)),\n            transforms.RandomHorizontalFlip(p=0.5),\n            transforms.ColorJitter(brightness=0.4, contrast=0.4, saturation=0.2, hue=0.1, p=0.8),\n            transforms.RandomGrayscale(p=0.2),\n            transforms.ToImage(**spt.data.static.CIFAR10),\n        ),\n        # Second view with slightly different augmentations\n        transforms.Compose(\n            transforms.RGB(),\n            transforms.RandomResizedCrop((32, 32), scale=(0.08, 1.0)),\n            transforms.RandomHorizontalFlip(p=0.5),\n            transforms.ColorJitter(brightness=0.4, contrast=0.4, saturation=0.2, hue=0.1, p=0.8),\n            transforms.RandomGrayscale(p=0.2),\n            transforms.RandomSolarize(threshold=0.5, p=0.2),\n            transforms.ToImage(**spt.data.static.CIFAR10),\n        ),\n    ]\n)\n\n# Load CIFAR-10 and wrap in dictionary format\ncifar_train = torchvision.datasets.CIFAR10(root=\"./data\", train=True, download=True)\ncifar_val = torchvision.datasets.CIFAR10(root=\"./data\", train=False, download=True)\n\ntrain_dataset = spt.data.FromTorchDataset(\n    cifar_train,\n    names=[\"image\", \"label\"],  # Convert tuple to dictionary\n    transform=simclr_transform,\n)\n\nval_dataset = spt.data.FromTorchDataset(\n    cifar_val,\n    names=[\"image\", \"label\"],\n    transform=transforms.Compose(\n        transforms.RGB(),\n        transforms.Resize((32, 32)),\n        transforms.ToImage(**spt.data.static.CIFAR10),\n    ),\n)\n\n# Create dataloaders - MultiViewTransform handles the view creation\ntrain_dataloader = torch.utils.data.DataLoader(\n    dataset=train_dataset,\n    batch_size=256,\n    num_workers=8,\n    drop_last=True,\n    shuffle=True,  # Simple shuffle, no RepeatedRandomSampler needed\n)\n\nval_dataloader = torch.utils.data.DataLoader(\n    dataset=val_dataset,\n    batch_size=256,\n    num_workers=10,\n)\n\ndata = spt.data.DataModule(train=train_dataloader, val=val_dataloader)\n\n# Build model components\nbackbone = spt.backbone.from_torchvision(\"resnet18\", low_resolution=True)\nbackbone.fc = torch.nn.Identity()  # Remove classification head\n\nprojector = nn.Sequential(\n    nn.Linear(512, 2048),\n    nn.BatchNorm1d(2048),\n    nn.ReLU(inplace=True),\n    nn.Linear(2048, 2048),\n    nn.BatchNorm1d(2048),\n    nn.ReLU(inplace=True),\n    nn.Linear(2048, 256),\n)\n\n# Create the module using the built-in SimCLR forward function\nmodule = spt.Module(\n    backbone=backbone,\n    projector=projector,\n    forward=forward.simclr_forward,  # Use the built-in forward function\n    simclr_loss=spt.losses.NTXEntLoss(temperature=0.5),\n    optim={\n        \"optimizer\": {\"type\": \"LARS\", \"lr\": 5, \"weight_decay\": 1e-6},\n        \"scheduler\": {\"type\": \"LinearWarmupCosineAnnealing\"},\n        \"interval\": \"epoch\",\n    },\n)\n\n# Add callbacks for monitoring performance during training\nlinear_probe = spt.callbacks.OnlineProbe(\n    module,\n    name=\"linear_probe\",\n    input=\"embedding\",\n    target=\"label\",\n    probe=torch.nn.Linear(512, 10),\n    loss_fn=torch.nn.CrossEntropyLoss(),\n    metrics={\n        \"top1\": torchmetrics.classification.MulticlassAccuracy(10),\n        \"top5\": torchmetrics.classification.MulticlassAccuracy(10, top_k=5),\n    },\n)\n\nknn_probe = spt.callbacks.OnlineKNN(\n    name=\"knn_probe\",\n    input=\"embedding\",\n    target=\"label\",\n    queue_length=20000,\n    metrics={\"accuracy\": torchmetrics.classification.MulticlassAccuracy(10)},\n    input_dim=512,\n    k=10,\n)\n\n# Configure training\ntrainer = pl.Trainer(\n    max_epochs=1000,\n    callbacks=[knn_probe, linear_probe],  # Monitor SSL quality in real-time\n    precision=\"16-mixed\",\n    logger=WandbLogger(project=\"cifar10-simclr\"),\n)\n\n# Launch training\nmanager = spt.Manager(trainer=trainer, module=module, data=data)\nmanager()\n```\n\u003c/details\u003e\n\n\n## 🚀 Quick Start with `spt` CLI\n\nThe `spt` command launches training from YAML configuration files using Hydra.\n\n**Note:** `spt` requires YAML configs. If you have Python-based configs, you can:\n- Convert them to YAML format where each component uses `_target_` to specify the importable class/function\n- See `examples/simclr_cifar10_config.yaml` for the structure and syntax\n\n### Local Training\n\n```bash\n# Run with a config file\nspt examples/simclr_cifar10_config.yaml\n\n# With parameter overrides\nspt examples/simclr_cifar10_config.yaml trainer.max_epochs=50 module.optim.lr=0.01\n\n# Run from any directory - supports absolute and relative paths\nspt ../configs/my_config.yaml\nspt /path/to/config.yaml\n```\n\n### SLURM Cluster Training\n\nFor training on SLURM clusters, use the `-m` flag to enable multirun mode:\n\n```bash\n# Use the provided SLURM template (customize partition/QOS in the file)\nspt examples/simclr_cifar10_slurm.yaml -m\n\n# Override SLURM parameters via command line\nspt examples/simclr_cifar10_slurm.yaml -m \\\n    hydra.launcher.partition=gpu \\\n    hydra.launcher.qos=normal \\\n    hydra.launcher.timeout_min=720\n```\n\nThe SLURM template (`examples/simclr_cifar10_slurm.yaml`) includes placeholders for cluster-specific settings. Either modify the file directly or override values via command line.\n\n## Installation\n\nThe library is not yet available on PyPI. You can install it from the source code, as follows.\n\n1. \u003cdetails\u003e\u003csummary\u003econda (optional)\u003c/summary\u003e\n\n    First use your favorite environment manager and install your favorite pytorch version, we provide an example with conda\n    ```\n    wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n    bash Miniconda3-latest-Linux-x86_64.sh\n    ```\n    follow installation instructions... once completed, create your environment\n    ```\n    conda create -n my_env python=3.11\n    ```\n    with your environment name (here `my_env`) and your favorite Python version (here, `3.11`). Once completed, make sure to activate your environment (`conda activate my_env`) before proceeding to the next steps!\n  \u003c/details\u003e\n\n2. Pytorch and our library (we recommend using `uv` for quicker package management):\n    ```bash\n    pip3 install uv\n    uv pip install torch torchvision torchaudio\n    uv pip install -e .  # Core dependencies only\n    ```\n\n    For optional features (vision models, experiment tracking, cluster support, etc.):\n    ```bash\n    uv pip install -e \".[vision,tracking]\"  # Example: add vision models and wandb\n    uv pip install -e \".[all]\"  # Or install all optional dependencies\n    ```\n    See `pyproject.toml` for available dependency groups (`vision`, `tracking`, `cluster`, `visualization`, `datasets`, `extras`, `dev`, `doc`).\n\n    If you do not want to use uv, simply remove it from the above commands.\n\n3. API login (optional)\n    ```\n    wandb login\n    huggingface-cli login\n    ```\n4. LATEX support in Matplotlib (optional)\n\n    1.  \u003cdetails\u003e\n        \u003csummary\u003eInstall the LaTex font (Computer Modern)\u003c/summary\u003e\n\n        - we provide the ttf files [in the repo](assets/cm-unicode-0.7.0%202/) to make things simple\n        - create your local folder (if not present) and copy the ttf files there\n          - `mkdir -p ~/.local/share/fonts `\n          - `cp assets/cm-unicode-0.7.0\\ 2/*ttf ~/.local/share/fonts/`\n        - refresh the font cache with `fc-cache -f -v`\n        - validate that the fonts are listed in your system with `fc-list | grep cmu`\n        - refresh matplotlib cache\n          ```\n          import shutil\n          import matplotlib\n\n          shutil.rmtree(matplotlib.get_cachedir())\n          ```\n        \u003c/details\u003e\n\n\n    2. \u003cdetails\u003e\n        \u003csummary\u003eInstall the Tex compiler (optional, if not available on your system)\u003c/summary\u003e\n\n        - install texlive locally following https://tug.org/texlive/quickinstall.html#running where you can use `-texdir your_path` to install to a local path (so you don't need sudo privileges)\n        - follow the instructions at the end of the installation to edit the PATH variables. If in the above step you used `-texdir ~/texdir` then the path to add should be like `TEXDIR_PATH=/private/home/$USER/texdir/bin/x86_64-linux`. You can use your favorite method such as\n          - `export PATH=\"$TEXDIR_PATH:$PATH\"` for local session\n          - adding `export PATH=\"$TEXDIR_PATH:$PATH\"` to your `.bashrc`\n          - run `conda env config vars set PATH=\"$TEXDIR_PATH:$PATH\"` once for it to be set within your conda env\n          - IMPORTANT: if the above is not done you will see an error akin to `! LaTeX Error: File type1ec.sty not found.`\n        - make sure inside the conde environment that you point to the right binaries e.g. `whereis latex` and `whereis mktexfmt`\n        - If at some point there is an error that the file `latex.fmt` is not found. You can generate it with\n          - `pdftex -ini   -jobname=latex -progname=latex -translate-file=cp227.tcx *latex.ini`\n          - or (unsure) `fmtutil-sys --all`\n        \u003c/details\u003e\n\n    3. \u003cdetails\u003e\n        \u003csummary\u003erc config (optional)\u003c/summary\u003e\n\n        ```\n        font.family: serif\n        font.serif: cmr10\n        font.sans-serif: cmss10\n        font.monospace: cmtt10\n\n        text.usetex: True\n        text.latex.preamble: \\usepackage{amssymb} \\usepackage{amsmath} \\usepackage{bm}\n\n        xtick.labelsize: 14\n        ytick.labelsize: 14\n        legend.fontsize: 14\n        axes.labelsize: 16\n        axes.titlesize: 16\n        axes.formatter.use_mathtext: True\n        ```\n        which can be written to a file, e.g., `~/.config/matplotlib/matplotlibrc` or set via `rc` in your script directly. See here for more details.\n        \u003c/details\u003e\n\n    4. \u003cdetails\u003e\n        \u003csummary\u003eExample of matplotlib script to run for a quick test (optional)\u003c/summary\u003e\n\n        ```\n        from matplotlib import rc\n        rc('font',**{'family':'sans-serif','sans-serif':['Helvetica']})\n        rc('text', usetex=True)\n        import numpy as np\n        import matplotlib.pyplot as plt\n\n\n        t = np.arange(0.0, 1.0 + 0.01, 0.01)\n        s = np.cos(4 * np.pi * t) + 2\n\n        plt.rc('text', usetex=True)\n        plt.rc('font', family='serif')\n        plt.plot(t, s)\n\n        plt.xlabel(r'\\textbf{time} (s)')\n        plt.ylabel(r'\\textit{voltage} (mV)',fontsize=16)\n        plt.title(r\"\\TeX\\ is Number \"\n                  r\"$\\displaystyle\\sum_{n=1}^\\infty\\frac{-e^{i\\pi}}{2^n}$!\",\n                  fontsize=16, color='gray')\n        # Make room for the ridiculously large title.\n        plt.subplots_adjust(top=0.8)\n\n        plt.savefig('tex_demo')\n        plt.show()\n        ```\n      \u003c/details\u003e\n\n## Ways You Can Contribute:\n\n- If you'd like to contribute new features, bug fixes, or improvements to the documentation, please refer to our [contributing guide](https://rbalestr-lab.github.io/stable-pretraining.github.io/dev/contributing.html) for detailed instructions on how to get started.\n\n- You can also contribute by adding new methods, datasets, or configurations that improve the current performance of a method in the [benchmark section](https://github.com/rbalestr-lab/stable-pretraining/tree/main/benchmarks).\n\n## Contributors\n\nCore contributors (in order of joining the project):\n- [Randall Balestriero](https://github.com/RandallBalestriero)\n- [Hugues Van Assel](https://github.com/huguesva)\n- [Sami BuGhanem](https://github.com/sami-bg)\n- [Lucas Maes](https://github.com/lucas-maes)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgalilai-group%2Fstable-pretraining","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgalilai-group%2Fstable-pretraining","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgalilai-group%2Fstable-pretraining/lists"}