{"id":30545471,"url":"https://github.com/datacte/corepulse","last_synced_at":"2025-09-13T07:38:25.152Z","repository":{"id":310995739,"uuid":"1041628545","full_name":"DataCTE/CorePulse","owner":"DataCTE","description":null,"archived":false,"fork":false,"pushed_at":"2025-08-21T12:12:32.000Z","size":67495,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-21T14:25:41.546Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataCTE.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-20T19:15:02.000Z","updated_at":"2025-08-21T12:12:36.000Z","dependencies_parsed_at":"2025-08-21T14:39:04.237Z","dependency_job_id":null,"html_url":"https://github.com/DataCTE/CorePulse","commit_stats":null,"previous_names":["datacte/corepulse"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/DataCTE/CorePulse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCTE%2FCorePulse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCTE%2FCorePulse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCTE%2FCorePulse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCTE%2FCorePulse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataCTE","download_url":"https://codeload.github.com/DataCTE/CorePulse/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCTE%2FCorePulse/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274935867,"owners_count":25376831,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-13T02:00:10.085Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-28T00:28:39.992Z","updated_at":"2025-09-13T07:38:25.144Z","avatar_url":"https://github.com/DataCTE.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CorePulse\n\nA modular toolkit for advanced diffusion model manipulation, providing unprecedented control over how Stable Diffusion processes and interprets your prompts.\n\n## Core Concepts\n\n### **Prompt Injection**\nInject different prompts into specific architectural blocks of the UNet during generation. This allows you to control different aspects of your image:\n- **Content blocks** (middle layers) → What appears in your image  \n- **Style blocks** (output layers) → How it looks and feels\n- **Composition blocks** (input layers) → Overall layout and structure\n\n#### **Token-Level Attention Masking**\nControl which parts of your prompt have influence by masking attention to specific tokens/words. This is different from spatial masking - it works at the linguistic level rather than image regions.\n\n- **Selective attention** → Choose which words in your prompt get processed\n- **Token-level control** → Fine-grained control over prompt interpretation\n- **Linguistic precision** → Target specific concepts without changing prompt text\n\n*Example: In \"a cat playing at a park\", mask attention to \"cat\" tokens while preserving attention to \"playing at a park\"*\n\n![Masked Injection Example](media/masked_injection_comparison.png)\n\n#### **Regional/Spatial Injection**\nApply prompt injections only to specific regions of the image using spatial masks. This enables surgical control over different areas of your image.\n\n- **Targeted replacement** → Change specific image regions while preserving context\n- **Spatial precision** → Control exactly where changes occur  \n- **Context preservation** → Background and surroundings remain untouched\n\n*Example: Apply \"golden retriever dog\" only to a circular region in the center, keeping the park environment identical*\n\n### **Attention Manipulation** \nControl how much the model focuses on specific words in your prompt by directly modifying attention weights. Unlike changing the prompt text, this amplifies or reduces the model's internal focus on existing words.\n\n- **Amplify attention** (\u003e1.0) → Make the model pay more attention to specific words\n- **Reduce attention** (\u003c1.0) → Decrease focus on certain words  \n- **Spatial control** → Apply attention changes only to specific image regions\n\n*Example: In \"a photorealistic portrait of an astronaut\", boost attention on \"photorealistic\" to enhance realism without changing the prompt*\n\n![Attention Manipulation Example](media/attention_manipulation_comparison.png)\n\n### **Multi-Scale Control**\nApply different prompts to different resolution levels of the UNet architecture. This approach lets you control structure and details independently:\n\n- **Structure Level** (lowest resolution) → Overall composition, global layout, major objects\n- **Mid-Level** (medium resolution) → Regional features, object relationships, local composition  \n- **Detail Level** (highest resolution) → Fine textures, surface details, small elements\n\n*Example: Generate a castle's overall silhouette with \"gothic cathedral\" at structure level, while adding \"intricate stone carvings\" at detail level*\n\nThe key insight: **different resolution levels control different aspects of the final image**. By targeting them separately, you achieve unprecedented control over the generation process.\n\n![Multi-Scale Main Comparison](media/multi_scale_main_comparison.png)\n\n![Multi-Scale Semantic Comparison](media/multi_scale_semantic_comparison.png)\n\n## Technical Features\n\n- **Multi-Architecture Support**: SDXL and SD1.5 with automatic detection\n- **Block-Level Control**: Target specific UNet blocks (input:0, middle:0, output:1, etc.)\n- **Flexible Interfaces**: Simple one-liners to advanced multi-block configurations  \n- **Seamless Integration**: Drop-in compatibility with HuggingFace Diffusers\n- **Context Management**: Automatic patch cleanup with Python context managers\n- **Precise Timing Control**: Sigma-based injection windows for optimal effect\n\n## Important Concepts\n\n### **Sigma Ranges: Timing is Everything**\nPrompt injections are applied during specific phases of the diffusion process using **sigma values** (noise levels):\n\n- **High Sigma** (~15-3): Early denoising steps, global structure formation\n- **Medium Sigma** (~3-0.5): Mid-process, composition and major features  \n- **Low Sigma** (~0.5-0): Final steps, detail refinement\n\n**⚠️ Critical:** Default sigma ranges may be too narrow! If your injections seem to have no effect, try wider ranges:\n\n```python\n#  Too narrow - might only inject on 1-2 steps\ninjector.add_injection(\"middle:0\", \"dragon\", sigma_start=1.0, sigma_end=0.3)\n\n#  Better - injects across most steps  \ninjector.add_injection(\"middle:0\", \"dragon\", sigma_start=15.0, sigma_end=0.0)\n\n#  Always inject - bypass timing completely\ninjector.add_injection(\"middle:0\", \"dragon\", sigma_start=1000.0, sigma_end=-1000.0)\n```\n\n## Quick Examples\n\n### Prompt Injection: Content/Style Separation\n```python\nfrom core_pulse import SimplePromptInjector\nfrom diffusers import StableDiffusionXLPipeline\n\npipeline = StableDiffusionXLPipeline.from_pretrained(\"stabilityai/stable-diffusion-xl-base-1.0\")\n\n# Inject \"white cat\" into content blocks while keeping base prompt\nwith SimplePromptInjector(pipeline) as injector:\n    injector.configure_injections(\n        block=\"middle:0\",  # Content block\n        prompt=\"white cat\",\n        weight=2.0,        # Strong enough to be visible\n        sigma_start=15.0,  # Start early in process\n        sigma_end=0.0      # Continue through final steps\n    )\n    \n    # Base prompt provides context, injection overrides content\n    result = injector(\"a blue dog in a garden\", num_inference_steps=30)\n    # Result: A white cat in a garden (content replaced, context preserved)\n```\n\n### Regional/Spatial Injection: Surgical Precision\n```python\nfrom core_pulse import RegionalPromptInjector\nfrom core_pulse.prompt_injection.spatial import create_center_circle_mask\n\n# Create a spatial mask for the region you want to modify\nmask = create_center_circle_mask(image_size=(1024, 1024), radius=300)\n\nwith RegionalPromptInjector(pipeline) as injector:\n    injector.add_regional_injection(\n        block=\"middle:0\",\n        prompt=\"golden retriever dog\",  # Replace with this\n        mask=mask,                      # Only in this region\n        weight=2.5,\n        sigma_start=15.0,\n        sigma_end=0.0\n    )\n    \n    # The mask ensures only the center region changes\n    result = injector(\"a cat playing at a park\", num_inference_steps=30)\n    # Result: Dog in center, park environment perfectly preserved\n```\n\n**Available Mask Shapes:**\n```python\nfrom core_pulse.prompt_injection.spatial import (\n    create_rectangle_mask,     # Custom rectangular regions\n    create_circle_mask,        # Circular regions  \n    create_left_half_mask,     # Left/right halves\n    create_top_half_mask,      # Top/bottom halves\n    create_center_square_mask, # Centered shapes\n    MaskFactory.from_image     # Load custom masks from images\n)\n```\n\n### Attention Manipulation: Focus Control\n```python\nfrom core_pulse import AttentionMapInjector\n\n# Boost attention on specific words without changing the prompt\nwith AttentionMapInjector(pipeline) as injector:\n    injector.add_attention_manipulation(\n        prompt=\"a photorealistic portrait of an astronaut\",\n        block=\"all\",  \n        target_phrase=\"photorealistic\",\n        attention_scale=5.0,  # 5x more attention on \"photorealistic\"\n        sigma_start=15.0,     # Apply throughout generation\n        sigma_end=0.0\n    )\n    \n    # Same prompt, but model focuses much more on making it photorealistic\n    result = injector(\n        prompt=\"a photorealistic portrait of an astronaut\",\n        num_inference_steps=30\n    )\n```\n\n### Multi-Scale Control: Structure + Details\n```python\nfrom core_pulse import MultiScaleInjector\n\n# Control structure and details independently\nwith MultiScaleInjector(pipeline) as injector:\n    # Structure: What the overall composition should be\n    injector.add_structure_injection(\n        \"gothic cathedral silhouette, imposing architecture\",\n        weight=2.0\n    )\n    \n    # Details: What the surface textures should look like  \n    injector.add_detail_injection(\n        \"weathered stone, intricate carvings, moss-covered surfaces\",\n        weight=1.8\n    )\n    \n    # Base prompt provides the scene context\n    result = injector(\n        prompt=\"a building in a misty landscape\",\n        num_inference_steps=30\n    )\n    # Result: Gothic cathedral structure with detailed stone textures\n```\n\n### When to Use Which Technique\n\n| Technique | Use When | Example |\n|-----------|----------|---------|\n| **Prompt Injection** | You want to replace/add content while keeping context | Generate a cat in a dog scene |\n| **Token-Level Attention Masking** | You want to selectively ignore/emphasize parts of your prompt | Mask out \"cat\" tokens, keep \"playing at park\" |\n| **Regional/Spatial Injection** | You want surgical precision - change specific image regions only | Replace center region with dog, keep park untouched |\n| **Attention Manipulation** | You want to emphasize existing words more strongly | Make \"photorealistic\" really count |\n| **Multi-Scale Control** | You want different structure and details | Castle structure + stone texture details |\n| **Combined Techniques** | Complex control over multiple aspects | Regional masking + multi-scale control |\n\n## Installation\n\n```bash\nuv sync  # Install all dependencies\nuv sync --extra examples  # Include example dependencies\nuv sync --extra dev  # Include development tools\n```\n\n## Advanced Usage\n\nCorePulse offers multiple levels of control:\n\n### **Interfaces by Complexity**\n- **`SimplePromptInjector`** → One-liner injection for quick experiments  \n- **`AdvancedPromptInjector`** → Multi-block, multi-prompt configurations\n- **`MultiScaleInjector`** → Resolution-aware structure/detail control\n- **`AttentionMapInjector`** → Precise attention weight control\n- **`RegionalPromptInjector`** → Spatial masks for region-specific control\n\n### **Architecture Components**  \n- **`UNetPatcher`** → Low-level UNet modification engine\n- **`UNetBlockMapper`** → Automatic block detection for any model\n- **`PromptInjectionProcessor`** → Custom attention processors with sigma timing\n- **Utilities** → Auto-detection, validation, convenience functions\n\n## Real-World Examples\n\n**Content/Style Split** (`examples.py`):\n- Generate a cat with oil painting style in a photorealistic scene\n\n**Token-Level Attention Masking** (`attention_masking_examples.py`):\n- Selective prompt token control: mask \"cat\" tokens while preserving \"park\" context\n- Linguistic precision: control what parts of prompt get processed\n- Demonstrates token-level vs full prompt processing\n\n**Regional/Spatial Injection** (`spatial_injection_examples.py`):\n- Surgical region replacement: change center region while preserving surroundings\n- Spatial masking: modify specific image areas while keeping context intact\n- Demonstrates spatial precision control vs full scene modification\n\n**Multi-Scale Architecture** (`advanced_control_examples.py`):\n- Structure level: \"medieval fortress\" → Controls overall building shape\n- Detail level: \"weathered stone textures\" → Controls surface appearance\n- Independent control of composition vs fine details\n\n**Attention Boost** (`attention_examples.py`): \n- Amplify \"photorealistic\" attention for enhanced realism\n- Reduce background element attention for focus control\n\n**Regional Control** (`sdxl_examples.py`):\n- Left half: crystal castle, Right half: fire dragon  \n- Spatial masks with soft blending\n\n## Troubleshooting\n\n### **\"My injections don't seem to work / images look identical\"**\n\nThis is usually a **sigma range issue**. The injection system works perfectly, but narrow sigma ranges mean injections only apply to 1-2 denoising steps out of 20-50.\n\n**Quick Fix:**\n```python\n# Instead of default ranges, use wide ranges for guaranteed effect\ninjector.add_injection(\n    block=\"middle:0\", \n    prompt=\"your injection\", \n    weight=3.0,\n    sigma_start=15.0,    # Start early\n    sigma_end=0.0        # End late  \n)\n```\n\n**Debug Steps:**\n1. **Use extreme weights** (5.0-10.0) to test if injection works at all\n2. **Use wide sigma ranges** (15.0 → 0.0) to maximize injection window\n3. **Use more inference steps** (30-50) to give more chances for injection\n4. **Use dramatically different prompts** (\"dragon\" vs \"building\") to see clear differences\n\n### **\"Injections are too weak\"**\n\n- **Increase weight**: Try 2.0-5.0 instead of 1.0\n- **Use multiple blocks**: Inject into several blocks for cumulative effect\n- **Check semantic compatibility**: Conflicting prompts can cancel each other out\n\n### **\"Multi-scale injections create chaotic results\"**\n\n**Multi-scale control is powerful but requires thoughtful prompt design:**\n\n- **Use semantically compatible prompts**: \n  - Good: \"gothic cathedral\" (structure) + \"weathered stone\" (details)\n  - Bad: \"modern building\" (structure) + \"organic textures\" (details)\n\n- **Avoid overlapping sigma ranges**: Structure and detail injections should target different phases:\n  - Structure: Early phases (sigma 15.0→0.5) for composition\n  - Details: Later phases (sigma 3.0→0.0) for surface textures\n\n- **Start simple**: Test structure-only, then detail-only, then combine\n- **Think hierarchically**: Structure defines the \"what\", details define the \"how it looks\"\n\n## License\n\nMIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacte%2Fcorepulse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatacte%2Fcorepulse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacte%2Fcorepulse/lists"}