{"id":48197610,"url":"https://github.com/synapticore-io/astro-lab","last_synced_at":"2026-04-04T18:16:39.137Z","repository":{"id":302861929,"uuid":"1004088755","full_name":"synapticore-io/astro-lab","owner":"synapticore-io","description":"Astronomical machine learning framework with advanced data analysis and interactive 3D visualization capabilities.","archived":false,"fork":false,"pushed_at":"2026-03-23T08:33:00.000Z","size":47882,"stargazers_count":0,"open_issues_count":4,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-24T05:54:16.980Z","etag":null,"topics":["3d-visualization","astronomy","machine-learning","python","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/synapticore-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-18T05:25:38.000Z","updated_at":"2026-03-23T08:32:01.000Z","dependencies_parsed_at":"2026-01-15T10:23:30.033Z","dependency_job_id":null,"html_url":"https://github.com/synapticore-io/astro-lab","commit_stats":null,"previous_names":["bjoernbethge/astro-lab","synapticore/astro-lab","synapticore-io/astro-lab"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/synapticore-io/astro-lab","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/synapticore-io%2Fastro-lab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/synapticore-io%2Fastro-lab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/synapticore-io%2Fastro-lab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/synapticore-io%2Fastro-lab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/synapticore-io","download_url":"https://codeload.github.com/synapticore-io/astro-lab/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/synapticore-io%2Fastro-lab/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31407915,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T10:20:44.708Z","status":"ssl_error","status_checked_at":"2026-04-04T10:20:06.846Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-visualization","astronomy","machine-learning","python","pytorch"],"created_at":"2026-04-04T18:16:35.839Z","updated_at":"2026-04-04T18:16:39.076Z","avatar_url":"https://github.com/synapticore-io.png","language":"Python","funding_links":["https://ko-fi.com/N4N71WOHZ3"],"categories":[],"sub_categories":[],"readme":"# 🌌 AstroLab - Astro GNN Laboratory for Cosmic Web Exploration\n\nA comprehensive **Astro GNN laboratory** for exploring cosmic web structures through graph neural networks, astronomical data analysis, and interactive 3D visualization across multiple astronomical scales.\n\n[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/synapticore-io/astro-lab)\n\n## 🚀 Quick Start\n\n### Installation\n```bash\ngit clone https://github.com/synapticore-io/astro-lab.git\ncd astro-lab\nuv sync\nuv pip install torch-scatter torch-sparse torch-cluster -f https://data.pyg.org/whl/torch-2.7.0+cu128.html\n```\n\n### First Steps\n```bash\n# Download data (optional - data may already be available)\nastro-lab download gaia\n\n# Preprocess data (recommended first step)\nastro-lab preprocess --survey gaia --max-samples 1000\n\n# Analyze cosmic web structure\nastro-lab cosmic-web gaia --max-samples 10000 --clustering-scales 5 10 25 --visualize\n\n# Start interactive development environment\nmarimo run src/astro_lab/ui/app.py\n```\n\n## 📊 AstroLab Consolidated Catalog\n\nAstroLab provides a **consolidated multi-survey catalog** with cosmic web structure classifications, combining data from Gaia, SDSS, and 2MASS.\n\n### Generate the Catalog\n\n```bash\n# Generate with default settings (Gaia only, 10k samples)\npython scripts/generate_astrolab_catalog.py --max-samples 10000\n\n# Generate with multiple surveys\npython scripts/generate_astrolab_catalog.py --surveys gaia sdss twomass --max-samples 50000\n\n# Generate with custom clustering scales\npython scripts/generate_astrolab_catalog.py --clustering-scales 5 10 25 50 100\n```\n\n### Features Included\n\n- **Multi-wavelength photometry**: Optical (Gaia, SDSS) + Near-IR (2MASS)\n- **Astrometry**: Positions, proper motions, parallaxes\n- **3D Coordinates**: Cartesian coordinates in parsecs\n- **Cosmic Web Classification**: Filament, void, node, field at multiple scales\n- **Density Fields**: Local density estimates at each scale\n- **Structure Metrics**: Anisotropy, connectivity, topology\n\n### Using the Catalog\n\n```python\nimport polars as pl\n\n# Load the catalog\ncatalog = pl.read_parquet(\"data/catalogs/astrolab_catalog_v1.parquet\")\n\n# Filter by structure type (e.g., filaments at 10 pc scale)\nfilaments = catalog.filter(pl.col(\"cosmic_web_class_10.0pc\") == 1)\n\n# Get high-density regions\nhigh_density = catalog.filter(pl.col(\"density_10.0pc\") \u003e threshold)\n```\n\n### Generate Visualizations\n\n```bash\n# Create interactive 3D plots and statistical visualizations\npython scripts/generate_visualizations.py\n\n# Output: data/visualizations/*.html\n```\n\nSee the [complete catalog documentation](data/README.md) and [example usage](examples/use_astrolab_catalog.py) for more details.\n\n\n## 🎨 Visual Examples\n\nAstroLab provides powerful interactive 3D visualizations of cosmic web structures across multiple astronomical scales.\n\n### Quick Visualization\n\nGenerate publication-quality visualizations from the AstroLab catalog:\n\n```bash\n# Generate the catalog first\npython scripts/generate_astrolab_catalog.py --max-samples 10000\n\n# Create visualizations\npython scripts/generate_visualizations.py\n\n# Open the generated HTML files in data/visualizations/\n```\n\n### Available Visualizations\n\n- **Interactive 3D Cosmic Web**: Real-time exploration using Plotly with structure classification coloring\n- **Multi-Scale Comparison**: Side-by-side views of structures at different clustering scales\n- **Structure Distribution**: Statistical analysis of filaments, voids, nodes, and field regions\n- **Density Maps**: Local density field visualizations\n\n### Programmatic Usage\n\n```python\nfrom astro_lab.data.analysis.cosmic_web import ScalableCosmicWebAnalyzer\nimport polars as pl\n\n# Analyze cosmic web structure\nanalyzer = ScalableCosmicWebAnalyzer()\ncatalog = pl.read_parquet(\"data/catalogs/astrolab_catalog_v1.parquet\")\n\n# Extract coordinates\ncoordinates = catalog.select(['x', 'y', 'z']).to_numpy()\n\n# Run analysis\nresults = analyzer.analyze_cosmic_web(\n    coordinates,\n    scales=[5.0, 10.0, 25.0],\n    use_adaptive_sampling=True\n)\n```\n\nOr use the interactive UI:\n```bash\nmarimo run src/astro_lab/ui/app.py\n```\n\n**Live Demo**: Visit our [GitHub Pages documentation](https://bjoernbethge.github.io/astro-lab/) for API documentation and guides (Documentation auto-deployed on every commit to main).\n\n## 🧠 Astro GNN Models \u0026 Tasks\n\n### **Core GNN Models**\nAstroLab provides ready-to-use model factory functions for all main tasks:\n\n- **AstroGraphGNN**: Spatial graph neural networks for cosmic web structure detection\n  - Factory: `create_astro_graph_gnn(num_features, num_classes, **kwargs)`\n  - Example:\n    ```python\n    from astro_lab.models.astro_model import create_astro_graph_gnn\n    model = create_astro_graph_gnn(num_features=16, num_classes=3)\n    ```\n- **AstroNodeGNN**: Node classification for stellar/galaxy properties\n  - Factory: `create_astro_node_gnn(num_features, num_classes, **kwargs)`\n  - Example:\n    ```python\n    from astro_lab.models.astro_model import create_astro_node_gnn\n    model = create_astro_node_gnn(num_features=8, num_classes=5)\n    ```\n- **AstroPointNet**: Point cloud processing for 3D astronomical data\n  - Factory: `create_astro_pointnet(num_features, num_classes, **kwargs)`\n  - Example:\n    ```python\n    from astro_lab.models.astro_model import create_astro_pointnet\n    model = create_astro_pointnet(num_features=3, num_classes=2)\n    ```\n- **AstroTemporalGNN**: Time-series analysis for variable objects\n  - Factory: `create_astro_temporal_gnn(num_features, num_classes, **kwargs)`\n  - Example:\n    ```python\n    from astro_lab.models.astro_model import create_astro_temporal_gnn\n    model = create_astro_temporal_gnn(num_features=12, num_classes=4)\n    ```\n\nLegacy/specialized factories (for compatibility):\n- `create_cosmic_web_model` → use `create_astro_graph_gnn`\n- `create_stellar_model` → use `create_astro_node_gnn`\n- `create_galaxy_model`, `create_exoplanet_model` for specialized galaxy/exoplanet tasks\n\n### **Primary Tasks**\n- **Cosmic Web Clustering**: Multi-scale structure detection (stellar to galactic scales)\n- **Filament Detection**: MST, Morse theory, and Hessian eigenvalue analysis\n- **Stellar Classification**: Spectral type and evolutionary stage prediction\n- **Galaxy Morphology**: Shape and structure classification\n- **Exoplanet Host Analysis**: Stellar neighborhood clustering\n- **Temporal Variability**: Light curve analysis and period detection\n\n### **Multi-Scale Analysis**\n- **Stellar Scale** (1-100 parsecs): Local galactic disk structure\n- **Galactic Scale** (1-100 Megaparsecs): Galaxy clusters and superclusters  \n- **Exoplanet Scale** (10-500 parsecs): Stellar neighborhoods and associations\n\n## 🌟 Key Features\n\n### 🔬 **Multi-Survey Data Integration**\n- **Gaia DR3**: Stellar catalogs with proper motions and cosmic web clustering\n- **SDSS**: Galaxy surveys and spectra with large-scale structure analysis\n- **NSA**: Galaxy catalogs with distances and cosmic web visualization  \n- **TNG50**: Cosmological simulations with filament detection\n- **NASA Exoplanet Archive**: Confirmed exoplanets with host star clustering\n- **LINEAR**: Asteroid light curves with orbital family analysis\n\n### 🌌 **Cosmic Web Analysis**\n\n#### **Interactive 3D Visualization**\n- **CosmographBridge**: Real-time cosmic web visualization with physics simulation\n- **Survey-specific colors**: Gold for stars, blue for galaxies, green for simulations\n- **Multi-backend support**: PyVista, Open3D, Blender, and Plotly integration\n- **Live tensor sync**: Real-time updates between analysis and visualization\n\n#### **Specialized Tensor Operations**\n```python\nfrom astro_lab.tensors import SpatialTensorDict\n\n# Create spatial tensor with coordinate system support\nspatial = SpatialTensorDict(coordinates, coordinate_system=\"icrs\", unit=\"parsec\")\n\n# Multi-scale cosmic web clustering  \nlabels = spatial.cosmic_web_clustering(eps_pc=10.0, min_samples=5)\n\n# Grid-based structure analysis\nstructure = spatial.cosmic_web_structure(grid_size_pc=100.0)\n\n# Local density computation\ndensity = spatial.analyze_local_density(radius_pc=50.0)\n```\n\n## 📚 Documentation \u0026 API Reference\n\nThe complete, up-to-date documentation is available as a modern website:\n\n- **[API Reference](./docs/api/astro_lab/)**\n- **[Cosmic Web Guide](./docs/cosmic_web_guide.md)**\n- **[Model Documentation](./docs/api/astro_lab.models.md)**\n- **[Training Guide](./docs/api/astro_lab.training.md)**\n\nAll code is fully documented with mkdocstrings and includes automatic class inheritance diagrams, usage examples, and configuration options.\n\n\u003e **ℹ️ Automatic Documentation Deployment**\n\u003e\n\u003e The documentation is automatically generated and deployed to GitHub Pages on every push to the `main` branch using a [GitHub Action](.github/workflows/docs.yml). You do not need to build or deploy the documentation manually—simply push your changes to `main` and the latest docs will be published automatically.\n\n## 🛠️ CLI Reference\n\nAstroLab provides a comprehensive command-line interface for all aspects of astronomical machine learning and cosmic web analysis.\n\n### Core Commands\n\n```bash\n# Show all available commands\nastro-lab --help\n\n# Get help for specific commands\nastro-lab \u003ccommand\u003e --help\n```\n\n### Data Processing\n```bash\n# Download raw survey data\nastro-lab download gaia\nastro-lab download gaia --force\nastro-lab download --list\n\n# Preprocess a single survey\nastro-lab preprocess gaia --max-samples 10000\n\n# Preprocess with quality filtering\nastro-lab preprocess gaia --force\n\n# Preprocess with custom output directory\nastro-lab preprocess gaia --output-dir ./processed_data\n```\n\n### Configuration Management\n```bash\n# Create new configuration file\nastro-lab config create -o my_experiment.yaml --template gaia\n\n# Show available survey configurations\nastro-lab config surveys\n\n# Show specific survey configuration details\nastro-lab config show gaia\nastro-lab config show nsa\n```\n\n### Model Training\n```bash\n# Train with configuration file\nastro-lab train -c my_experiment.yaml --verbose\n\n# Train with command-line parameters\nastro-lab train --survey gaia --model gcn --epochs 50 --batch-size 32\nastro-lab train --survey nsa --model astro_node_gnn --learning-rate 0.001 --devices 2\n\n# Resume from checkpoint\nastro-lab train -c config.yaml --checkpoint path/to/checkpoint.ckpt\n\n# Debug training with small dataset\nastro-lab train --survey gaia --max-samples 1000 --overfit-batches 10\n```\n\n### Hyperparameter Optimization\n```bash\n# Optimize hyperparameters\nastro-lab optimize gaia --trials 50 --timeout 3600\nastro-lab optimize gaia --trials 100\n\n# Quick optimization for debugging\nastro-lab optimize gaia --trials 10 --max-samples 1000\n```\n\n### Cosmic Web Analysis\n```bash\n# Multi-scale stellar structure analysis\nastro-lab cosmic-web gaia --max-samples 100000 --clustering-scales 5 10 25 50 --visualize\n\n# Large-scale galaxy structure  \nastro-lab cosmic-web nsa --clustering-scales 5 10 20 50 --redshift-limit 0.15\n\n# Exoplanet host star clustering\nastro-lab cosmic-web exoplanet --clustering-scales 10 25 50 100 200 --min-samples 3\n\n# Custom analysis with output directory\nastro-lab cosmic-web gaia --catalog-path ./my_catalog.fits --output-dir ./results --verbose\n```\n\n### Supported Surveys\nAll commands support these astronomical surveys:\n- `gaia`: [Gaia DR3](https://www.cosmos.esa.int/web/gaia/dr3) stellar catalog\n- `sdss`: [Sloan Digital Sky Survey](https://www.sdss.org/)  \n- `nsa`: [NASA-Sloan Atlas](http://nsatlas.org/) galaxy catalog\n- `tng50`: [TNG50](https://www.tng-project.org/) cosmological simulation\n- `exoplanet`: [NASA Exoplanet Archive](https://exoplanetarchive.ipac.caltech.edu/)\n- `rrlyrae`: [RR Lyrae](https://www.sdss.org/dr17/spectro/vac/) variable stars\n- `linear`: [LINEAR](https://en.wikipedia.org/wiki/Lincoln_Near-Earth_Asteroid_Research) asteroid survey\n\n## 🔧 Setup Scripts\n\nAstroLab provides automated setup scripts for easy installation across different platforms.\n\n### Linux/macOS Setup (setup.sh)\n\nThe `setup.sh` script automates the entire installation process on Linux and macOS systems:\n\n```bash\n# Make the script executable and run it\nchmod +x setup.sh\n./setup.sh\n```\n\n**What the script does:**\n1. **Installs uv package manager** if not already present\n2. **Runs `uv sync`** to install all dependencies from `pyproject.toml`\n3. **Installs PyTorch Geometric extensions** for CUDA support\n4. **Activates the virtual environment** automatically\n5. **Provides instructions** for future activation\n\n**Manual equivalent:**\n```bash\n# Install uv\ncurl -LsSf https://astral.sh/uv/install.sh | sh\nexport PATH=\"$HOME/.cargo/bin:$PATH\"\n\n# Install dependencies\nuv sync\nuv pip install torch-scatter torch-sparse torch-cluster -f https://data.pyg.org/whl/torch-2.7.0+cu128.html\n\n# Activate environment\nsource .venv/bin/activate\n```\n\n### Windows Setup (setup.ps1)\n\nThe `setup.ps1` script provides the same functionality for Windows PowerShell:\n\n```powershell\n# Run the PowerShell setup script\n.\\setup.ps1\n```\n\n**What the script does:**\n1. **Installs uv package manager** via PowerShell\n2. **Runs `uv sync`** to install dependencies\n3. **Installs PyTorch Geometric extensions** with CUDA support\n4. **Activates the virtual environment**\n5. **Provides activation instructions** for future use\n\n**Manual equivalent (PowerShell):**\n```powershell\n# Install uv\nirm https://astral.sh/uv/install.ps1 | iex\n$env:PATH = \"$env:USERPROFILE\\.cargo\\bin;$env:PATH\"\n\n# Install dependencies\nuv sync\nuv pip install torch-scatter torch-sparse torch-cluster -f https://data.pyg.org/whl/torch-2.7.0+cu128.html\n\n# Activate environment\n.\\.venv\\Scripts\\Activate.ps1\n```\n\n### Environment Activation\n\nAfter setup, activate the environment for future sessions:\n\n**Linux/macOS:**\n```bash\nsource .venv/bin/activate\n```\n\n**Windows:**\n```powershell\n.\\.venv\\Scripts\\Activate.ps1\n```\n\n### Verification\n\nTest your installation:\n```bash\n# Check CLI availability\nastro-lab --help\n\n# Verify CUDA support (if available)\npython -c \"import torch; print(f'CUDA available: {torch.cuda.is_available()}')\"\n\n# Test cosmic web analysis\nastro-lab cosmic-web gaia --max-samples 100 --clustering-scales 5 10\n```\n\n## 📖 Documentation Generation\n\nAstroLab includes automated documentation generation and management tools.\n\n### Documentation Scripts\n\nThe `docs/generate_docs.py` script provides comprehensive documentation management:\n\n```bash\n# Generate/update all documentation\npython docs/generate_docs.py update\n\n# Start local documentation server\npython docs/generate_docs.py serve\n\n# Deploy documentation to GitHub Pages\npython docs/generate_docs.py deploy\n```\n\n**What the documentation script does:**\n1. **Scans source code** for all Python modules\n2. **Generates API documentation** using mkdocstrings\n3. **Creates navigation structure** automatically\n4. **Builds documentation** with MkDocs\n5. **Serves locally** for development\n6. **Deploys to GitHub Pages** for production\n\n### Manual Documentation Commands\n\nYou can also run documentation commands directly:\n\n```bash\n# Install documentation dependencies\nuv run pip install mkdocs mkdocstrings[python] mkdocs-material\n\n# Build documentation\nuv run mkdocs build --clean\n\n# Serve documentation locally (http://127.0.0.1:8000)\nuv run mkdocs serve\n\n# Deploy to GitHub Pages\nuv run mkdocs gh-deploy --force\n```\n\n### Documentation Structure\n\nThe documentation system automatically generates:\n- **API Reference**: Complete code documentation with inheritance diagrams\n- **Cosmic Web Guide**: Comprehensive analysis tutorials\n- **Model Documentation**: GNN architecture and training guides\n- **Configuration Reference**: All survey and model configurations\n\n## 🤖 Automation and UI Scripts\n\nAstroLab automation is handled through:\n\n1. **Setup Scripts**: `setup.sh` and `setup.ps1` for environment setup\n2. **Documentation Scripts**: `docs/generate_docs.py` for documentation management  \n3. **UI Launch Script**: `run_ui.py` for starting the interactive dashboard\n4. **CLI Commands**: Built-in automation through the `astro-lab` CLI\n\n### UI Launch Script\n\nThe `run_ui.py` script provides an easy way to start the AstroLab interactive dashboard:\n\n```bash\n# Start the AstroLab UI dashboard\npython run_ui.py\n\n# The dashboard will be available at http://localhost:2718\n```\n\n**What the UI script does:**\n- Launches the Marimo reactive notebook interface\n- Provides access to cosmic web analysis tools\n- Enables interactive data visualization  \n- Runs on port 2718 by default\n\n## 🏗️ Architecture\n\n### Core Components\n```\nastro-lab/\n├── src/astro_lab/\n│   ├── cli/\n│   │   ├── cosmic_web.py      # Cosmic web CLI interface\n│   │   ├── train.py           # Model training CLI\n│   │   └── optimize.py        # Hyperparameter optimization\n│   ├── data/\n│   │   ├── cosmic_web.py      # Core cosmic web analysis\n│   │   └── datasets/          # Survey-specific datasets\n│   ├── tensors/\n│   │   └── tensordict_astro.py # Spatial tensor operations  \n│   ├── models/\n│   │   ├── core/              # GNN model implementations\n│   │   └── components/        # Model building blocks\n│   ├── widgets/\n│   │   ├── cosmograph_bridge.py   # Interactive visualization\n│   │   ├── graph.py               # Graph analysis functions\n│   │   └── plotly_bridge.py       # 3D plotting\n│   ├── ui/modules/\n│   │   ├── cosmic_web.py      # UI for cosmic web analysis\n│   │   ├── analysis.py        # Interactive analysis tools\n│   │   └── visualization.py   # Visualization interface\n│   └── training/              # Training framework\n├── configs/                   # Configuration files\n├── docs/\n│   ├── cosmic_web_guide.md    # Comprehensive cosmic web guide\n│   └── api/                   # Auto-generated API documentation\n└── test/                      # Test suite\n```\n\n### Key Dependencies\n- **PyTorch 2.7.1+cu128**: GPU-accelerated deep learning with geometric extensions\n- **PyTorch Geometric**: Graph neural networks for cosmic web analysis\n- **Lightning 2.5.1**: Training framework with MLflow integration\n- **Polars 1.31.0**: High-performance data processing\n- **AstroPy 7.1.0**: Astronomical calculations and coordinate systems\n- **Cosmograph**: Interactive graph visualization with physics simulation\n- **Marimo**: Reactive notebooks for interactive analysis\n- **scikit-learn**: Clustering algorithms (DBSCAN, K-means, etc.)\n\n## 🛠️ Framework Stack\n\nAstroLab is built on cutting-edge frameworks and libraries for astronomical machine learning:\n\n### **Core ML/DL Frameworks**\n- **[PyTorch](https://pytorch.org/)** - GPU-accelerated deep learning with CUDA support\n- **[PyTorch Geometric](https://pytorch-geometric.readthedocs.io/)** - Graph neural networks for spatial data\n- **[PyTorch Lightning](https://lightning.ai/)** - Professional training framework with MLflow integration\n- **[Optuna](https://optuna.org/)** - Hyperparameter optimization and experiment tracking\n\n### **Data Processing \u0026 Visualization**\n- **[Polars](https://pola.rs/)** - Lightning-fast data processing with Rust backend\n- **[Plotly](https://plotly.com/python/)** - Interactive web-based visualizations\n- **[PyVista](https://docs.pyvista.org/)** - 3D scientific visualization\n- **[Open3D](http://www.open3d.org/)** - Real-time 3D point cloud processing\n\n### **Astronomical Libraries**\n- **[AstroPy](https://www.astropy.org/)** - Core astronomical calculations and coordinate systems\n- **[AstromL](https://www.astroml.org/)** - Machine learning for astronomy\n- **[AstroQuery](https://astroquery.readthedocs.io/)** - Astronomical data access\n- **[SDSS Access](https://sdss-access.readthedocs.io/)** - Sloan Digital Sky Survey data\n\n### **Interactive Development**\n- **[Marimo](https://marimo.io/)** - Reactive Python notebooks for interactive analysis\n- **[Cosmograph](https://cosmograph.app/)** - Interactive graph visualization with physics simulation\n- **[Blender Python API](https://docs.blender.org/api/current/)** - Professional 3D rendering and animation\n\n### **Development Tools**\n- **[UV](https://docs.astral.sh/uv/)** - Fast Python package manager and resolver\n- **[Ruff](https://docs.astral.sh/ruff/)** - Extremely fast Python linter\n- **[MyPy](https://mypy-lang.org/)** - Static type checking for Python\n- **[Pre-commit](https://pre-commit.com/)** - Git hooks for code quality\n\n## 🎯 Use Cases\n\n### Stellar Structure Analysis\n```python\nfrom astro_lab.data.cosmic_web import analyze_gaia_cosmic_web\n\n# Analyze local stellar neighborhoods\nresults = analyze_gaia_cosmic_web(\n    max_samples=100000,\n    magnitude_limit=12.0,\n    clustering_scales=[5.0, 10.0, 25.0, 50.0],  # parsecs\n    min_samples=5\n)\n\nprint(f\"Found {results['n_stars']} stars\")\nfor scale, stats in results['clustering_results'].items():\n    print(f\"{scale}: {stats['n_clusters']} clusters, {stats['grouped_fraction']:.1%} grouped\")\n```\n\n### Galaxy Cluster Analysis\n```python\nfrom astro_lab.data.cosmic_web import analyze_nsa_cosmic_web\n\n# Large-scale structure analysis\nresults = analyze_nsa_cosmic_web(\n    redshift_limit=0.15,\n    clustering_scales=[5.0, 10.0, 20.0, 50.0],  # Mpc\n    min_samples=5\n)\n```\n\n### Interactive Cosmic Web Visualization\n```python\nfrom astro_lab.widgets.cosmograph_bridge import CosmographBridge\nfrom astro_lab.data.cosmic_web import CosmicWebAnalyzer\n\n# Load and analyze data\nanalyzer = CosmicWebAnalyzer()\nresults = analyzer.analyze_gaia_cosmic_web(max_samples=10000)\n\n# Create interactive 3D visualization\nbridge = CosmographBridge()\nwidget = bridge.from_cosmic_web_results(results, survey_name=\"gaia\")\n\n# Display with physics simulation and survey-specific colors\nwidget.show()  # Gold points for Gaia stars with real-time clustering\n```\n\n### Filament Detection\n```python\nfrom astro_lab.tensors import SpatialTensorDict\nfrom astro_lab.data.cosmic_web import CosmicWebAnalyzer\n\n# Create spatial tensor\nspatial = SpatialTensorDict(coordinates, coordinate_system=\"icrs\", unit=\"parsec\")\n\n# Detect filamentary structures\nanalyzer = CosmicWebAnalyzer()\nfilaments = analyzer.detect_filaments(\n    spatial, \n    method=\"mst\",  # or \"morse_theory\", \"hessian\"\n    n_neighbors=20,\n    distance_threshold=10.0\n)\n\nprint(f\"Detected {filaments['n_filament_segments']} filament segments\")\nprint(f\"Total filament length: {filaments['total_filament_length']:.1f} pc\")\n```\n\n### Multi-Backend Visualization\n```python\nfrom astro_lab.widgets.tensor_bridge import create_tensor_bridge\n\n# Create visualization bridge\nbridge = create_tensor_bridge(backend=\"cosmograph\")  # or \"pyvista\", \"blender\"\n\n# Visualize cosmic web with clustering\nviz = bridge.cosmic_web_to_backend(\n    spatial_tensor=spatial,\n    cluster_labels=labels,\n    point_size=2.0,\n    show_filaments=True\n)\n```\n\n## 🔧 Development\n\n### Interactive Development\n```bash\n# Start Marimo reactive notebook with cosmic web UI\nuv run marimo run src/astro_lab/ui/app.py\n\n# Launch MLflow UI for experiment tracking\nuv run mlflow ui --backend-store-uri ./data/experiments\n```\n\n### Testing\n```bash\n# Run full test suite\nuv run pytest -v\n\n# Test specific components\nuv run pytest test/test_cosmic_web.py -v\nuv run pytest test/test_data.py -v\nuv run pytest test/test_lightning.py -v\nuv run pytest src/astro_lab/tensors/ -v -k cosmic_web\n```\n\n## 📊 Experiment Tracking\n\nAll cosmic web analyses are automatically tracked with MLflow:\n\n```python\n# Results are logged with cosmic web metadata\n- clustering_scales: [5.0, 10.0, 25.0, 50.0]\n- survey_type: \"gaia\" \n- n_clusters_per_scale: {5.0: 125, 10.0: 89, 25.0: 45, 50.0: 12}\n- filament_detection_method: \"mst\"\n- visualization_backend: \"cosmograph\"\n```\n\n## 🎨 Visualization Gallery\n\n### Supported Backends\n- **Cosmograph**: Interactive 3D with physics simulation and survey-specific colors\n- **PyVista**: High-quality 3D rendering with filament visualization  \n- **Plotly**: Web-based interactive plots with multi-scale clustering\n- **Blender**: Professional 3D rendering and animation via albpy integration\n- **Open3D**: Real-time point cloud visualization with octree support\n\n### Example Visualizations\n- **Gaia stellar neighborhoods**: Gold points with gravitational clustering\n- **NSA galaxy superclusters**: Blue points with large-scale structure\n- **TNG50 cosmic web**: Green points with dark matter filaments\n- **Exoplanet host clusters**: Magenta points with stellar associations\n\n## 🤝 Contributing\n\nWe welcome contributions to cosmic web analysis features! See our [contribution guidelines](CONTRIBUTING.md) for details on:\n\n- Adding new filament detection algorithms\n- Implementing additional clustering methods  \n- Creating visualization backends\n- Extending survey support\n- Improving performance with GPU acceleration\n\n## 🙏 Acknowledgments\n\nSpecial thanks to the **Astro Graph Agent Team** for their invaluable contributions to the advanced visualization modules and cosmic web analysis features. Their expertise in astronomical data visualization and graph neural networks has been instrumental in making AstroLab a comprehensive Astro GNN laboratory.\n\n## 📄 License\n\nMIT License - see [LICENSE](LICENSE) for details.\n---\n\n[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/N4N71WOHZ3)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsynapticore-io%2Fastro-lab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsynapticore-io%2Fastro-lab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsynapticore-io%2Fastro-lab/lists"}