{"id":28514053,"url":"https://github.com/codewithdark-git/diffusionlm","last_synced_at":"2026-03-02T07:33:29.334Z","repository":{"id":286372857,"uuid":"960471940","full_name":"codewithdark-git/DiffusionLM","owner":"codewithdark-git","description":"DiffusionLM: Large Language Models with Diffusion","archived":false,"fork":false,"pushed_at":"2025-04-06T02:52:15.000Z","size":11374,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-28T19:05:01.706Z","etag":null,"topics":["diffusion","diffusion-models","llm","opensource","opensource-projects","pypi-package","pypi-source","torch","transformer"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/diffusionLM/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codewithdark-git.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"codewithdark-git","patreon":"codewithdark","buy_me_a_coffee":"codewithdark","thanks_dev":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2025-04-04T13:47:59.000Z","updated_at":"2025-10-13T00:11:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"9d11fd89-33ad-461f-96ce-78d601246d9c","html_url":"https://github.com/codewithdark-git/DiffusionLM","commit_stats":null,"previous_names":["codewithdark-git/diffusionlm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/codewithdark-git/DiffusionLM","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codewithdark-git%2FDiffusionLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codewithdark-git%2FDiffusionLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codewithdark-git%2FDiffusionLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codewithdark-git%2FDiffusionLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codewithdark-git","download_url":"https://codeload.github.com/codewithdark-git/DiffusionLM/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codewithdark-git%2FDiffusionLM/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29995041,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-02T01:47:34.672Z","status":"online","status_checked_at":"2026-03-02T02:00:07.342Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diffusion","diffusion-models","llm","opensource","opensource-projects","pypi-package","pypi-source","torch","transformer"],"created_at":"2025-06-09T01:30:35.699Z","updated_at":"2026-03-02T07:33:29.329Z","avatar_url":"https://github.com/codewithdark-git.png","language":"Python","funding_links":["https://github.com/sponsors/codewithdark-git","https://patreon.com/codewithdark","https://buymeacoffee.com/codewithdark"],"categories":[],"sub_categories":[],"readme":"# DiffusionLM: Large Language Models with Diffusion\n\n[![PyPI version](https://badge.fury.io/py/diffusionLM.svg)](https://badge.fury.io/py/diffusionLM)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Sponsor](https://img.shields.io/github/sponsors/codewithdark-git?style=social)](https://github.com/sponsors/codewithdark-git)\n\n\nDiffusionLM is a novel approach to language modeling that combines transformer architectures with diffusion processes for high-quality text generation. This package provides a flexible and efficient implementation of diffusion-based language models.\n\n## Features\n\n- **Advanced Architecture**\n  - Transformer-based backbone with diffusion capabilities\n  - Configurable model sizes (small, medium, large)\n  - Time step conditioning\n  - Attention mechanisms optimized for text\n\n- **Multiple Generation Strategies**\n  - Auto-regressive generation\n  - Parallel generation\n  - Confidence-based masking\n  - Semi-autoregressive generation\n  - Top-p (nucleus) sampling\n  - Beam search\n\n- **Training Features**\n  - Distributed training support\n  - Mixed precision training\n  - Gradient checkpointing\n  - Early stopping\n  - Model checkpointing\n  - Learning rate scheduling\n\n- **Utilities**\n  - Real-time token generation streaming\n  - Model saving and loading\n  - HuggingFace Hub integration\n  - Comprehensive logging\n  - Error handling\n\n## Installation\n\n```bash\npip install diffusionLM\n```\n\nFor development installation:\n\n```bash\ngit clone https://github.com/codewithdark-git/DiffusionLM.git\ncd DiffusionLM\npip install -e .\n```\n\n## Quick Start\n\n```python\nfrom diffusionLM.utils import prepare_dataset\nfrom diffusionLM.model import DiffusionConfig, DiffusionLLM\nfrom transformers import AutoTokenizer\n\n# Load tokenizer and prepare dataset\ntokenizer = AutoTokenizer.from_pretrained(\"gpt2\")\ntrain_dataset, val_dataset, _ = prepare_dataset(\n    dataset_name=\"wikitext/wikitext-103-v1\",\n    tokenizer_name=\"gpt2\"\n)\n\n# Initialize model\nconfig = DiffusionConfig(\n        vocab_size=len(tokenizer),\n        max_position_embeddings=256,\n        num_timesteps=50,\n        pad_token_id=tokenizer.pad_token_id,\n        mask_token_id=tokenizer.mask_token_id,\n        # **config_kwargs\n    )\n\nmodel = DiffusionLLM(config)\n\n\n```\n\n## Training\n\n### Basic Training\n\n```python\nfrom diffusionLM import trainer\n\ntrain_model = trainer(\n        model=model,\n        train_dataset=train_dataset,\n        val_dataset=val_dataset,\n        batch_size=batch_size,\n        num_epochs=num_epochs,\n        learning_rate=learning_rate,\n        num_timesteps=num_timesteps,\n        save_path=save_dir,\n        device=device,\n    )\n```\n\n### Model Registry\n\n```python\nfrom diffusionLM import registerANDpush\n\nregisterANDpush(\n    model=trained_model,\n    tokenizer=tokenizer,\n    model_type=\"diffusionLM\",\n    repo_id=\"your-username/model-name\"\n)\n```\n\n## Error Handling\n\nThe package includes comprehensive error handling:\n\n```python\nfrom diffusionLM import DiffusionLMError, handle_errors\n\n@handle_errors()\ndef your_function():\n    # Your code here\n    pass\n```\n\n## Sponsorship\n\nIf you find DiffusionLM useful for your project or research, please consider supporting its development through GitHub Sponsors. Your sponsorship helps maintain the project and develop new features.\n\n[![Sponsor](https://img.shields.io/github/sponsors/codewithdark-git?style=social)](https://github.com/sponsors/codewithdark-git)\n\n### Why Sponsor?\n\n- Support ongoing development and maintenance\n- Priority bug fixes and feature requests\n- Recognition in our documentation\n- Help make DiffusionLM better for everyone\n\n### How to Sponsor\n\nClick the \"Sponsor\" button at the top of the repository or visit our [GitHub Sponsors page](https://github.com/sponsors/codewithdark-git).\n\n## Contributing\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## Requirements\n\n- Python ≥ 3.8\n- PyTorch ≥ 1.9.0\n- Transformers ≥ 4.21.0\n- For full requirements, see `requirements.txt`\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Citation\n\n```bibtex\n@article{diffusionllm2025,\n  title={DiffusionLM: Large Language Models with Diffusion},\n  author={Dark Coder},\n  journal={GitHub Repository},\n  year={2025},\n  publisher={GitHub},\n  url={https://github.com/codewithdark-git/DiffusionLM}\n}\n```\n\n## Contact\n\n- GitHub: [@codewithdark-git](https://github.com/codewithdark-git)\n- Email: codewithdark90@gmail.com\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodewithdark-git%2Fdiffusionlm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodewithdark-git%2Fdiffusionlm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodewithdark-git%2Fdiffusionlm/lists"}