{"id":27641289,"url":"https://github.com/exprays/atlas","last_synced_at":"2026-02-28T11:04:22.130Z","repository":{"id":289268277,"uuid":"966078993","full_name":"exprays/Atlas","owner":"exprays","description":"Atlas is a specialized convolutional neural network designed for satellite image change detection","archived":false,"fork":false,"pushed_at":"2025-04-22T11:50:40.000Z","size":199,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-23T23:47:04.581Z","etag":null,"topics":["alembic","celery","cnn-for-visual-recognition","cuda","geospatial-visualization","python","pytorch","tensors"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/exprays.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-14T11:20:29.000Z","updated_at":"2025-04-22T11:50:44.000Z","dependencies_parsed_at":"2025-04-22T12:47:36.993Z","dependency_job_id":"20285d1c-6331-47c4-887a-3983d7de6962","html_url":"https://github.com/exprays/Atlas","commit_stats":null,"previous_names":["exprays/atlas"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/exprays/Atlas","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exprays%2FAtlas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exprays%2FAtlas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exprays%2FAtlas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exprays%2FAtlas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/exprays","download_url":"https://codeload.github.com/exprays/Atlas/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exprays%2FAtlas/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266081239,"owners_count":23873511,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alembic","celery","cnn-for-visual-recognition","cuda","geospatial-visualization","python","pytorch","tensors"],"created_at":"2025-04-23T23:43:10.947Z","updated_at":"2025-10-09T08:37:22.092Z","avatar_url":"https://github.com/exprays.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Atlas Documentation\n\n## Table of Contents\n\n- Overview\n- System Requirements\n- Project Setup\n- Docker Configuration\n- Training the ML Model\n- Testing the System\n- Running the Complete Application\n- End-to-End Testing Workflow\n- Troubleshooting\n- Production Deployment\n- Advanced Configuration\n\n## Overview\n\nAtlasEye is a satellite change detection system that uses deep learning to identify changes between satellite images captured at different times. The system can detect and visualize urban development, deforestation, natural disasters, and other environmental changes.\n\n**Key Features:**\n- Change detection between satellite image pairs\n- Geospatial analysis with GeoJSON export\n- Interactive visualization with metrics\n- REST API for integration\n- Asynchronous processing for large images\n\n**Tech Stack:**\n- **Backend**: Python, PyTorch, FastAPI, PostgreSQL with PostGIS\n- **Frontend**: Next.js, TypeScript, TailwindCSS, Mapbox GL\n- **Infrastructure**: Docker, Celery, Redis\n\n[![Postgres](https://img.shields.io/badge/postgres-%23316192.svg?style=for-the-badge\u0026logo=postgresql\u0026logoColor=white)](https://img.shields.io/badge/postgres-%23316192.svg?style=for-the-badge\u0026logo=postgresql\u0026logoColor=white)\n[![Redis](https://img.shields.io/badge/redis-%23DD0031.svg?style=for-the-badge\u0026logo=redis\u0026logoColor=white)](https://img.shields.io/badge/redis-%23DD0031.svg?style=for-the-badge\u0026logo=redis\u0026logoColor=white)\n[![Celery](https://img.shields.io/badge/celery-%23a9cc54.svg?style=for-the-badge\u0026logo=celery\u0026logoColor=ddf4a4)](https://img.shields.io/badge/celery-%23a9cc54.svg?style=for-the-badge\u0026logo=celery\u0026logoColor=ddf4a4)\n[![FastAPI](https://img.shields.io/badge/FastAPI-005571?style=for-the-badge\u0026logo=fastapi)](https://img.shields.io/badge/FastAPI-005571?style=for-the-badge\u0026logo=fastapi)\n[![Next JS](https://img.shields.io/badge/Next-black?style=for-the-badge\u0026logo=next.js\u0026logoColor=white)](https://img.shields.io/badge/Next-black?style=for-the-badge\u0026logo=next.js\u0026logoColor=white)\n[![React](https://img.shields.io/badge/react-%2320232a.svg?style=for-the-badge\u0026logo=react\u0026logoColor=%2361DAFB)](https://img.shields.io/badge/react-%2320232a.svg?style=for-the-badge\u0026logo=react\u0026logoColor=%2361DAFB)\n[![TailwindCSS](https://img.shields.io/badge/tailwindcss-%2338B2AC.svg?style=for-the-badge\u0026logo=tailwind-css\u0026logoColor=white)](https://img.shields.io/badge/tailwindcss-%2338B2AC.svg?style=for-the-badge\u0026logo=tailwind-css\u0026logoColor=white)\n[![scikit-learn](https://img.shields.io/badge/scikit--learn-%23F7931E.svg?style=for-the-badge\u0026logo=scikit-learn\u0026logoColor=white)](https://img.shields.io/badge/scikit--learn-%23F7931E.svg?style=for-the-badge\u0026logo=scikit-learn\u0026logoColor=white)\n[![PyTorch](https://img.shields.io/badge/PyTorch-%23EE4C2C.svg?style=for-the-badge\u0026logo=PyTorch\u0026logoColor=white)](https://img.shields.io/badge/PyTorch-%23EE4C2C.svg?style=for-the-badge\u0026logo=PyTorch\u0026logoColor=white)\n[![NumPy](https://img.shields.io/badge/numpy-%23013243.svg?style=for-the-badge\u0026logo=numpy\u0026logoColor=white)](https://img.shields.io/badge/numpy-%23013243.svg?style=for-the-badge\u0026logo=numpy\u0026logoColor=white)\n[![Matplotlib](https://img.shields.io/badge/Matplotlib-%23ffffff.svg?style=for-the-badge\u0026logo=Matplotlib\u0026logoColor=black)](https://img.shields.io/badge/Matplotlib-%23ffffff.svg?style=for-the-badge\u0026logo=Matplotlib\u0026logoColor=black)\n[![Docker](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge\u0026logo=docker\u0026logoColor=white)](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge\u0026logo=docker\u0026logoColor=white)\n\n## System Requirements\n\n- Docker and Docker Compose\n- Git\n- 8GB+ RAM\n- NVIDIA GPU (optional, for faster training)\n- 20GB+ free disk space\n\n## Project Setup\n\n### Clone Repository\n\n```bash\ngit clone https://github.com/yourusername/atlaseye.git\ncd atlaseye\n```\n\n### Create Directory Structure\n\n```bash\nmkdir -p data/models data/images data/training/before data/training/after data/training/mask data/test_data\n```\n\n### Check Configuration Files\n\n**1. Backend Dockerfile**\n\n```bash\nFROM python:3.11-slim\n\nWORKDIR /app\n\n# Install system dependencies for geospatial libraries\nRUN apt-get update \u0026\u0026 apt-get install -y \\\n    build-essential \\\n    libproj-dev \\\n    libgeos-dev \\\n    proj-bin \\\n    libspatialindex-dev \\\n    libgl1-mesa-glx \\\n    libglib2.0-0 \\\n    \u0026\u0026 apt-get clean \\\n    \u0026\u0026 rm -rf /var/lib/apt/lists/*\n\n# Copy requirements file\nCOPY requirements.txt .\n\n# Install Python dependencies\nRUN pip install --no-cache-dir -r requirements.txt\n\n# Copy application code\nCOPY . .\n\n```\n\n**2. Frontend Dockerfile**\n\n```bash\nFROM node:18-alpine\n\nWORKDIR /app\n\n# Copy package.json and install dependencies\nCOPY package*.json ./\nRUN npm install\n\n# Copy the rest of the application\nCOPY . .\n```\n\n### Prepare Training Data\n\n1. Place before images in `data/training/before/`\n2. Place after images in `data/training/after/`\n3. Place ground truth masks in `data/training/mask/` (if available)\n4. Place test images in `data/test_data/`\n\n## Docker Configuration\n\nThe system uses Docker Compose to orchestrate multiple services. The key services are:\n\n- **postgres**: PostgreSQL database with PostGIS extension\n- **redis**: Message broker for Celery\n- **backend**: Python FastAPI application with ML capabilities\n- **celery_worker**: Background task processor\n- **frontend**: Next.js web application\n\n### Building Docker Images\n\n```bash\ndocker-compose build\n```\n\n### Environment Variables\n\nThe system uses environment variables for configuration:\n\n**Backend Environment Variables:**\n- `POSTGRES_SERVER`: Database hostname\n- `POSTGRES_USER`: Database username\n- `POSTGRES_PASSWORD`: Database password\n- `POSTGRES_DB`: Database name\n- `CELERY_BROKER_URL`: Redis connection string\n- `CELERY_RESULT_BACKEND`: Result storage backend\n- `MODEL_PATH`: Path to trained model\n- `LOCAL_STORAGE_PATH`: Path for storing uploaded images\n\n**Frontend Environment Variables:**\n- `NEXT_PUBLIC_API_URL`: Backend API URL\n- `NEXT_PUBLIC_MAPBOX_TOKEN`: Mapbox API token for maps\n\n## Training the ML Model\n\n### Starting Training Process\n\n```bash\n\u003c\u003c\u003c\u003c\u003c\u003c\u003c HEAD\ndocker-compose run --rm backend python -m app.ml.training.train\n    --data_dir=/app/data/training\n    --checkpoint_dir=/app/data/models\n    --batch_size=8\n    --num_epochs=50\n    --learning_rate=0.001\n    --image_size=256\n=======\ndocker-compose run --rm backend python -m app.ml.training.train \n    --data_dir=/app/data/training \n    --checkpoint_dir=/app/data/models \n    --batch_size=8 \n    --num_epochs=50 \n    --learning_rate=0.001 \n    --image_size=256 \n\u003e\u003e\u003e\u003e\u003e\u003e\u003e 23623dfdd587ac7e62bbc55b80a349de65f4efb4\n    --device=cpu  # Use 'cuda' if GPU available\n```\n\n### Training Parameters\n\n| Parameter | Description |\n|-----------|-------------|\n| `--data_dir` | Directory containing training data |\n| `--checkpoint_dir` | Directory to save model checkpoints |\n| `--batch_size` | Number of samples per training batch |\n| `--num_epochs` | Total number of training epochs |\n| `--learning_rate` | Learning rate for optimizer |\n| `--image_size` | Size to resize images (e.g., 256 for 256x256) |\n| `--device` | 'cuda' for GPU or 'cpu' for CPU-only training |\n\n### Monitoring Training Progress\n\nView training logs in real-time:\n\n```bash\ndocker-compose logs -f backend\n```\n\nThe training history plot is saved to `data/models/training_history.png`.\n\n## Testing the System\n\n### Backend Unit Tests\n\nRun all backend tests:\n\n```bash\ndocker-compose run --rm backend python -m unittest discover tests\n```\n\nRun specific test modules:\n\n```bash\n# ML module tests\ndocker-compose run --rm backend python -m unittest backend/tests/test_ml.py\n\n# API tests\ndocker-compose run --rm backend python -m unittest backend/tests/test_api.py\n```\n\n### Testing ML Model Inference\n\n```bash\ndocker-compose run --rm backend python -m app.ml.inferencer.test_predictor \n    --model_path=/app/data/models/final_model.pth \n    --before=/app/data/test_data/before.tif \n    --after=/app/data/test_data/after.tif \n    --ground_truth=/app/data/test_data/mask.tif  # Optional\n    --output_dir=/app/data/test_results\n```\n\n### Frontend Tests\n\nRun lint checks:\n\n```bash\ndocker-compose run --rm frontend npm run lint\n```\n\n### Database Connectivity Test\n\n```bash\ndocker-compose run --rm backend python -m app.db.test_connection\n```\n\n## Running the Complete Application\n\n### Start All Services\n\n```bash\ndocker-compose up -d\n```\n\n### Access Points\n\n- **Backend API**: http://localhost:8000/docs\n- **Frontend UI**: http://localhost:3000\n\n### Checking Service Status\n\n```bash\ndocker-compose ps\n```\n\n### Viewing Logs\n\n```bash\n# All services\ndocker-compose logs\n\n# Specific service\ndocker-compose logs -f backend\n```\n\n## End-to-End Testing Workflow\n\n### 1. Upload Images\n\n```bash\ncurl -X POST http://localhost:8000/api/v1/detection/upload-images/ \n  -F \"before_image=@/path/to/before.tif\" \n  -F \"after_image=@/path/to/after.tif\"\n```\n\nYou should receive a job_id in response.\n\n### 2. Process Images\n\n```bash\ncurl -X POST http://localhost:8000/api/v1/detection/process/{job_id}\n```\n\nReplace `{job_id}` with the actual ID received.\n\n### 3. Get Results\n\n```bash\ncurl http://localhost:8000/api/v1/detection/results/{job_id}\n```\n\n### 4. View in UI\n\n1. Open http://localhost:3000/results/{job_id} in your browser\n2. Verify the results display correctly:\n   - Change percentage\n   - Before/after images\n   - Change detection overlay\n   - Interactive map\n   - Metrics charts\n\n## Troubleshooting\n\n### Database Issues\n\nIf PostgreSQL connection fails:\n\n```bash\ndocker-compose down\ndocker volume rm atlaseye_postgres_data\ndocker-compose up -d postgres\n# Wait 10 seconds for initialization\ndocker-compose run --rm backend python -m app.db.test_connection\n```\n\n### ML Issues\n\nIf model training fails:\n\n```bash\n# Check GPU availability\ndocker-compose run --rm backend python -c \"import torch; print('CUDA available:', torch.cuda.is_available())\"\n\n# Verify data paths\ndocker-compose run --rm backend ls -la /app/data/training/before\ndocker-compose run --rm backend ls -la /app/data/training/after\ndocker-compose run --rm backend ls -la /app/data/training/mask\n```\n\n### Frontend Issues\n\nFor frontend rendering problems:\n\n```bash\n# Check if static assets are being served\ndocker-compose run --rm frontend ls -la /app/public\n\n# Verify environment variables\ndocker-compose run --rm frontend env | grep NEXT_PUBLIC\n\n# Fix Mapbox token if maps don't render\necho \"NEXT_PUBLIC_MAPBOX_TOKEN=your_mapbox_token_here\" \u003e frontend/.env.local\ndocker-compose restart frontend\n```\n\n### Checking Logs\n\nDetailed logs can help diagnose issues:\n\n```bash\ndocker-compose logs -f\n```\n\n## Production Deployment\n\nFor production deployment, consider the following:\n\n### Security Enhancements\n\n1. **Secure Passwords**: Update environment variables with strong passwords\n2. **SSL Configuration**: Add a reverse proxy (Nginx/Traefik) with SSL\n3. **Authentication**: Implement proper user authentication and authorization\n\n### Performance Optimization\n\n1. **Database Tuning**: Optimize PostgreSQL for geospatial queries\n2. **Caching**: Add Redis caching for API responses\n3. **CDN**: Use a CDN for static assets\n\n### High Availability\n\n1. **Load Balancing**: Deploy multiple backend instances behind a load balancer\n2. **Database Replication**: Set up PostgreSQL replication\n3. **Monitoring**: Add monitoring and alerting\n\n### Deployment Example (AWS)\n\n```yaml\nversion: '3.8'\n\nservices:\n  # ... services configuration\n\n  nginx:\n    image: nginx:latest\n    ports:\n      - \"80:80\"\n      - \"443:443\"\n    volumes:\n      - ./nginx.conf:/etc/nginx/conf.d/default.conf\n      - ./ssl:/etc/nginx/ssl\n    depends_on:\n      - backend\n      - frontend\n```\n\n## Advanced Configuration\n\n### GPU Support for Training\n\nTo enable GPU acceleration:\n\n1. Install NVIDIA Container Toolkit on host\n2. Update docker-compose.yml:\n\n```yaml\nservices:\n  backend:\n    # ... other settings\n    deploy:\n      resources:\n        reservations:\n          devices:\n            - driver: nvidia\n              count: 1\n              capabilities: [gpu]\n```\n\n3. Update training command:\n\n```bash\ndocker-compose run --rm backend python -m app.ml.training.train --device=cuda\n```\n\n### Custom Model Architecture\n\nTo use a custom model architecture:\n\n1. Create a new model class in models\n2. Update the model initialization in train.py\n3. Update the predictor to use your custom model\n\n### Database Migration\n\nFor database schema changes:\n\n```bash\n# Generate migration\ndocker-compose run --rm backend alembic revision --autogenerate -m \"Description\"\n\n# Apply migration\ndocker-compose run --rm backend alembic upgrade head\n```\n\n---\n\n**Documentation Version**: 1.0.0  \n**Last Updated**: April 14, 2025  \n**Authors**: Exprays","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexprays%2Fatlas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fexprays%2Fatlas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexprays%2Fatlas/lists"}