{"id":26156774,"url":"https://github.com/voipnuggets/flux-generator","last_synced_at":"2025-04-14T08:21:11.045Z","repository":{"id":277612769,"uuid":"932983142","full_name":"voipnuggets/flux-generator","owner":"voipnuggets","description":"Local image and music generation for Apple Silicon ","archived":false,"fork":false,"pushed_at":"2025-03-22T19:05:15.000Z","size":3624,"stargazers_count":27,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T21:52:09.522Z","etag":null,"topics":["ai","api","blackforest","flux","llm","mlx","music","schnell","stable-diffusion","webui"],"latest_commit_sha":null,"homepage":"https://voipnuggets.com/2025/02/18/flux-generator-local-image-generation-on-apple-silicon-with-open-webui-integration-using-flux-llm/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/voipnuggets.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-14T22:15:08.000Z","updated_at":"2025-03-27T20:08:19.000Z","dependencies_parsed_at":"2025-02-14T23:24:34.507Z","dependency_job_id":"615b0ce1-3312-457a-afe8-5f9cbfd5ddac","html_url":"https://github.com/voipnuggets/flux-generator","commit_stats":null,"previous_names":["voipnuggets/flux-generator"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voipnuggets%2Fflux-generator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voipnuggets%2Fflux-generator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voipnuggets%2Fflux-generator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voipnuggets%2Fflux-generator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/voipnuggets","download_url":"https://codeload.github.com/voipnuggets/flux-generator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248844036,"owners_count":21170505,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","api","blackforest","flux","llm","mlx","music","schnell","stable-diffusion","webui"],"created_at":"2025-03-11T09:25:46.431Z","updated_at":"2025-04-14T08:21:11.018Z","avatar_url":"https://github.com/voipnuggets.png","language":"Python","funding_links":["https://buymeacoffee.com/akashg"],"categories":["Demo","Rising projects"],"sub_categories":["2024"],"readme":"# Flux Generator: macOS MLX-Powered Image \u0026 Music Generation with Open WebUI compatable API for Image generation\n\n## Features\n\n- Text-to-image generation\n- Text-to-music generation (NEW!)\n- Multiple model options:\n  - Image Generation:\n    - black-forest-labs Flux schnell/dev\n    - stabilityai sdxl-turbo/stable-diffusion-2-1\n  - Music Generation:\n    - facebook/musicgen-medium\n- Customizable image size and generation parameters\n- Advanced music generation controls\n- Memory usage reporting\n- API compatibility for Image generation for third-party UIs like Open WebUI\n- Unified server for both UI and API\n- Configurable network access modes\n\n## Optimized for Apple Silicon with MLX\n\nThis repository utilizes the MLX framework, designed specifically for Apple Silicon, to provide optimized performance for:\n- Black Forest Flux and Stable Diffusion image generation\n- Facebook's MusicGen audio generation\n\nMLX leverages the unified memory architecture of Apple's M-series chips, enabling faster and more efficient computations.\n\n### Why MLX?\n\n* **Performance:** Experience significant speed improvements compared to other frameworks on Apple Silicon.\n* **Local Execution:** Run Stable Diffusion models locally on your Mac, ensuring data privacy and enabling offline use.\n* **Fine-Tuning:** MLX provides a great environment for fine-tuning models on apple silicon.\n\nFor more examples of what MLX can do, check out the official mlx-examples repository: [https://github.com/ml-explore/mlx-examples](https://github.com/ml-explore/mlx-examples)\n\nThis repository is designed to give apple silicon users a fast and easy way to generate images locally.\n\n## UI Screenshots:\n![Flux image generation Tab in UI](flux_app_ui_imagegen.jpg)\n![Flux music generation Tab in UI](flux_app_ui_musicgen.jpg)\n\n\n## Example Generation\n\nHere's an example image generated using the Flux model:\n\n![Moonset over ocean](generated_moonset.png)\n\nPrompt: \"a beautiful moonset over the ocean, highly detailed, 4k\"\nParameters:\n- Model: schnell\n- Size: 512x512\n- Steps: 2\n- CFG Scale: 4.0\n\n## Requirements\n\n- macOS with Apple Silicon (M1/M2/M3)\n- Python 3.10+ (tested with python3.11)\n- MLX framework\n- Additional audio processing libraries for MusicGen\n\n## Installation \u0026 Usage\n\n### Quick Start (Recommended)\n\nThe easiest way to run Flux Generator is using the provided script:\n\n```bash\n# Make the script executable\nchmod +x run_flux.sh\n\n# Run in local-only mode (most secure)\n./run_flux.sh\n\n# Or run with network access (for remote access)\n./run_flux.sh --network\n```\n\nThe script will:\n- Check if you're running on Apple Silicon Mac\n- Create and set up a Python virtual environment\n- Install all required dependencies\n- Check for existing model files\n- Start the server based on the selected mode\n\n### Script Options\n\n```bash\nUsage: ./run_flux.sh [OPTIONS]\n\nOptions:\n  -h, --help         Show this help message\n  -n, --network      Enable network access (less secure)\n\nExamples:\n  ./run_flux.sh                 # Run in local-only mode (most secure)\n  ./run_flux.sh --network       # Run with network access (for remote access)\n```\n\n### Access Modes\n\n1. **Local Only (Default, Most Secure)**\n   ```bash\n   ./run_flux.sh\n   ```\n   - Only allows connections from localhost (127.0.0.1)\n   - Best for local development and testing\n   - Access via: http://127.0.0.1:7860\n\n2. **Network Access**\n   ```bash\n   ./run_flux.sh --network\n   ```\n   - Allows connections from any network interface\n   - Required for Docker integration\n   - Less secure, use only in trusted networks\n   - Access via:\n     - Local: http://127.0.0.1:7860\n     - Network: http://0.0.0.0:7860\n     - Docker: http://host.docker.internal:7860\n\n### Manual Installation\n\nIf you prefer to set things up manually:\n\n1. **Create a virtual environment:**\n   ```bash\n   python3.11 -m venv venv\n   \n   # For bash/zsh:\n   source venv/bin/activate\n   \n   # For fish:\n   source venv/bin/activate.fish\n   ```\n\n2. **Install requirements:**\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n3. **Run the server:**\n   ```bash\n   # For local use only (most secure):\n   python3.11 flux_app.py\n\n   # For network access (remote):\n   python3.11 flux_app.py --listen-all\n   ```\n\n### Command Line Options\n\n```bash\npython3.11 flux_app.py [OPTIONS]\n\nOptions:\n  --port INTEGER       Port to run the server on (default: 7860)\n  --listen-all        Listen on all network interfaces (0.0.0.0)\n  --help              Show this message and exit\n```\n\n### Command Line Interface\nFor command-line image generation:\n\n```bash\npython3.11 txt2image.py --model schnell \\\n--n-images 1 \\\n--image-size 512x512 \\\n--verbose \\\n'A photo of an astronaut riding a horse on a beach.'\n```\n\n## Using the Web Interface\n\nOnce the server is running (either via `run_flux.sh` or manually):\n\n1. Open your browser and navigate to http://127.0.0.1:7860\n2. Choose your desired generation mode:\n   - 🖼️ Image Generation: Enter a prompt, select a model and click generate\n   - 🎵 Music Generation: Enter a music description and adjust parameters\n3. On first use, models will be downloaded:\n   - Image models: approximately 30 GB\n   - MusicGen model: approximately 3.5 GB\n4. Download progress will be visible in the terminal\n5. Once downloaded, generation will begin\n\n### Image Generation Parameters\n\n- Model: schnell\n- Size: 512x512\n- Steps: 2\n- CFG Scale: 4.0\n\n### Music Generation Parameters\n\nThe music generation interface provides several parameters to control the output:\n\n- **Max Steps**: Controls the length of the generated audio (50-500)\n- **Temperature**: Controls randomness in generation (0.1-2.0)\n- **Top K**: Controls diversity of the output (50-500)\n- **Guidance Scale**: Controls how closely to follow the prompt (1.0-10.0)\n\n## API Integration\n\nThe application provides an API that can be used with third-party UIs like Open WebUI.\nCheck this tutorial for Open WebUI integration instructions:\n[Tutorial](https://voipnuggets.com/2025/02/18/flux-generator-local-image-generation-on-apple-silicon-with-open-webui-integration-using-flux-llm/\n)\n\n### Integration with Open WebUI\n\nSince Flux Generator requires direct access to Apple Silicon hardware, it runs natively on your Mac while Open WebUI can run in Docker:\n\n1. Start Flux Generator with network access:\n   ```bash\n   ./run_flux.sh\n   ```\n   or\n   ```bash\n   ./run_flux.sh --network\n   ```\n   This will start the server and listen on all interfaces (--network flag required for Docker integration if running on a different machine).\n\n2. Run Open WebUI in Docker:\n   ```bash\n   docker run -d \\\n     -p 3000:8080 \\\n     --add-host=host.docker.internal:host-gateway \\\n     -e AUTOMATIC1111_BASE_URL=http://host.docker.internal:7860/ \\\n     -e ENABLE_IMAGE_GENERATION=True \\\n     -v open-webui:/app/backend/data \\\n     --name open-webui \\\n     --restart always \\\n     ghcr.io/open-webui/open-webui:main\n   ```\n\n3. Access Open WebUI at `http://localhost:3000`\n\nThe connection flow works like this:\n```\nOpen WebUI (Docker Container) -\u003e host.docker.internal:7860 -\u003e Flux Generator (Native on Mac)\n```\n\nThis setup ensures:\n- Flux Generator has direct access to Apple Silicon for optimal performance\n- Open WebUI runs in an isolated container\n- Both services communicate seamlessly through Docker's networking\n\n### Available Endpoints\n\n1. `/sdapi/v1/txt2img` (POST)\n   - Generate images from text\n   - Parameters:\n     ```json\n     {\n       \"prompt\": \"your prompt here\",\n       \"negative_prompt\": \"\",\n       \"width\": 512,\n       \"height\": 512,\n       \"steps\": 2,\n       \"cfg_scale\": 4.0,\n       \"batch_size\": 1,\n       \"n_iter\": 1,\n       \"seed\": -1,\n       \"model\": \"schnell\"\n     }\n     ```\n\n2. `/sdapi/v1/sd-models` (GET)\n   - List available models\n   - Returns Flux Schnell and Dev models\n\n3. `/sdapi/v1/options` (GET/POST)\n   - Get or set generation options\n   - Includes model settings and parameters\n\n4. `/sdapi/v1/progress` (GET)\n   - Get generation progress information\n\n### Example API Usage\n\nHere's a Python example to generate images:\n\n```python\nimport requests\nimport json\nimport base64\n\n# Use appropriate URL based on your setup:\n# Local only:      \"http://127.0.0.1:7860\"\nurl = \"http://127.0.0.1:7860/sdapi/v1/txt2img\"\n\npayload = {\n    \"prompt\": \"a beautiful sunset over the ocean, highly detailed, 4k\",\n    \"width\": 512,\n    \"height\": 512,\n    \"steps\": 2,\n    \"cfg_scale\": 4.0,\n    \"batch_size\": 1,\n    \"n_iter\": 1,\n    \"seed\": 42,\n    \"model\": \"schnell\"\n}\n\nresponse = requests.post(url, json=payload)\nresult = response.json()\n\n# Save the generated image\nif result[\"images\"]:\n    image_data = base64.b64decode(result[\"images\"][0].split(\",\")[1])\n    with open(\"generated_image.png\", \"wb\") as f:\n        f.write(image_data)\n```\n\n## Model Management\n\nThe Flux server requires model files to be downloaded before use. You can download the models in several ways:\n\n1. Automatic download on first use:\n   - Models will be downloaded automatically when you first try to generate\n   - The download progress will be visible in the CLI/terminal\n   - This may cause a delay on your first generation\n\n2. Using HuggingFace CLI (Recommended for faster downloads):\n\n   ```bash\n   # Install the HuggingFace CLI\n   pip install -U \"huggingface_hub[cli]\"\n\n   # You can also install the CLI using Homebrew:\n   brew install huggingface-cli\n\n   # Install hf_transfer for blazingly fast speeds\n   pip install hf_transfer\n\n   # Login to your HF account\n   huggingface-cli login\n\n   # Download Schnell model\n   huggingface-cli download black-forest-labs/FLUX.1-schnell\n\n   # Download Dev model (optional)\n   huggingface-cli download black-forest-labs/FLUX.1-dev\n\n   # Download MusicGen model\n   huggingface-cli download facebook/musicgen-medium\n   ```\n\n3. Using the command-line interface:\n   Note: Each Flux model is approximately 24GB in size, the SD models are bigger. The download includes:\n   - Model weights (flux1-{model}.safetensors)\n   - Autoencoder (ae.safetensors)\n   - Text encoders and tokenizers\n   \n   # Download command for all models\n   ```bash\n   huggingface-cli download black-forest-labs/FLUX.1-schnell\n   huggingface-cli download black-forest-labs/FLUX.1-dev (needs to ask for access, follow the onscreen instructions when you run this command)\n   huggingface-cli download stabilityai/stable-diffusion-2-1-base\n   huggingface-cli download stabilityai/sdxl-turbo\n   huggingface-cli download facebook/musicgen-medium\n   ```\n\nModel Repos:\nhttps://huggingface.co/black-forest-labs/FLUX.1-schnell\nhttps://huggingface.co/black-forest-labs/FLUX.1-dev\nhttps://huggingface.co/stabilityai/stable-diffusion-2-1-base\nhttps://huggingface.co/stabilityai/sdxl-turbo\nhttps://huggingface.co/facebook/musicgen-medium\n\nModel files are stored in the HuggingFace cache directory (`~/.cache/huggingface/hub/`).\n\n## Buy Me a Coffee\n\n👋 Hi, I'm Akash Gupta! Here's what I work on:\n\n• 🚀 **Current Project**: Flux Generator - MLX-powered image generation for Apple Silicon\n  - Local image generation using Apple's MLX framework\n  - Beautiful Gradio UI with real-time stats\n  - API compatible with Open WebUI\n  - Memory-efficient design for M1/M2/M3 Macs\n\n• 💼 **Professional Background**:\n  - Sr. Voice Over IP Engineer\n  - Expert in Kamailio and open-source VoIP\n  - Cloud integration specialist\n  - Learning LLMOps\n\n• 🌐 **Community Contributions**:\n  - Blog: [voipnuggets.com](https://voipnuggets.com)\n  - Focus: VoIP technology \u0026 AI advancements\n  - Regular tutorials and technical guides\n\nIf you find this project helpful, consider supporting my work:\n\n\u003cimg src=\"bmc_qr.png\" alt=\"Buy Me A Coffee QR Code\" width=\"200\" height=\"200\"\u003e\n\n[☕ Buy Me a Coffee](https://buymeacoffee.com/akashg)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoipnuggets%2Fflux-generator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvoipnuggets%2Fflux-generator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoipnuggets%2Fflux-generator/lists"}