{"id":25428889,"url":"https://github.com/aryasaatvik/omniparser-sagemaker","last_synced_at":"2025-05-14T08:35:26.352Z","repository":{"id":277810541,"uuid":"933557981","full_name":"aryasaatvik/omniparser-sagemaker","owner":"aryasaatvik","description":"🤖 deploy OmniParser v2 model on Amazon SageMaker with async inference endpoint","archived":false,"fork":false,"pushed_at":"2025-02-23T10:39:31.000Z","size":31,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-23T11:28:41.231Z","etag":null,"topics":["api","gui","inference","microsoft","omniparser","sagemaker"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aryasaatvik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-16T08:56:35.000Z","updated_at":"2025-02-23T10:39:34.000Z","dependencies_parsed_at":"2025-02-16T10:19:19.744Z","dependency_job_id":"14ab9a82-7979-4b03-8f8b-d4f20b65da37","html_url":"https://github.com/aryasaatvik/omniparser-sagemaker","commit_stats":null,"previous_names":["aryasaatvik/omniparser-sagemaker"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aryasaatvik%2Fomniparser-sagemaker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aryasaatvik%2Fomniparser-sagemaker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aryasaatvik%2Fomniparser-sagemaker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aryasaatvik%2Fomniparser-sagemaker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aryasaatvik","download_url":"https://codeload.github.com/aryasaatvik/omniparser-sagemaker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254104788,"owners_count":22015545,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","gui","inference","microsoft","omniparser","sagemaker"],"created_at":"2025-02-17T01:48:57.013Z","updated_at":"2025-05-14T08:35:26.346Z","avatar_url":"https://github.com/aryasaatvik.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OmniParser SageMaker Deployment\n\nThis directory contains code for deploying OmniParser v2 to Amazon SageMaker as an asynchronous inference endpoint.\n\n## Directory Structure\n\n```\nomniparser-sagemaker/\n├── container/              # Container files for SageMaker deployment\n│   ├── Dockerfile         # Docker configuration for the container\n│   └── inference.py       # SageMaker model server implementation\n├── model/                 # Model artifacts\n│   ├── download_weights.py # Script to download weights from Hugging Face\n│   └── weights/           # Local directory for temporary weight storage\n├── scripts/               # Deployment and build scripts\n│   ├── build_and_push.sh # Script to build and push Docker image to ECR\n│   └── deploy.py         # Script to deploy model to SageMaker\n├── .python-version       # Python version specification\n├── pyproject.toml        # Project configuration and dev dependencies\n├── requirements.txt      # Production dependencies\n└── .gitignore           # Git ignore rules\n```\n\n## Prerequisites\n\n1. AWS CLI installed and configured with appropriate credentials\n2. Docker installed and running\n3. Python 3.11\n4. Required Python packages (install via `pip install -r requirements.txt`):\n   ```\n   # Core Dependencies\n   boto3\n   sagemaker\n   sagemaker-inference\n   multi-model-server\n   \n   # ML \u0026 Vision\n   torch\n   torchvision\n   transformers\n   ultralytics==8.3.70\n   supervision==0.18.0\n   opencv-python\n   opencv-python-headless\n   \n   # OCR Components\n   paddlepaddle\n   paddleocr\n   easyocr\n   \n   # Utilities\n   numpy==1.26.4\n   einops==0.8.0\n   ```\n\n## Development Setup\n\nThis project uses `pyproject.toml` for development dependencies and configuration. To set up a development environment:\n\n```bash\n# Create and activate a virtual environment\npython -m venv .venv\nsource .venv/bin/activate\n\n# Install development dependencies\npip install -e \".[dev]\"\n\n# Install pre-commit hooks\npre-commit install\n```\n\n## Step-by-Step Deployment\n\n### 1. Set Up Environment\n\n```bash\n# Install required packages\npip install -r requirements.txt\n\n# Configure AWS CLI with your credentials\naws configure\n```\n\n### 2. Build and Push Container\n\n```bash\ncd sagemaker/scripts\n\n# Set your S3 bucket for model weights\nexport OMNIPARSER_MODEL_BUCKET=\"your-model-bucket-name\"\n\n# Build and push (this will also download and upload model weights)\n./build_and_push.sh\n```\n\nThis script will:\n- Create the S3 bucket if it doesn't exist\n- Download model weights from Hugging Face\n- Create a tarball and upload to: `s3://${OMNIPARSER_MODEL_BUCKET}/model/omniparser-v2/model.tar.gz`\n- Build and push the Docker container to ECR\n\n### 3. Deploy to SageMaker\n\n```python\nfrom scripts.deploy import deploy_omniparser\n\n# Deploy using the same bucket used in build step\npredictor = deploy_omniparser(\n    model_bucket=\"your-model-bucket-name\"\n)\n```\n\nThis will:\n- Create a SageMaker model using the ECR container\n- Configure the model to use weights from S3\n- Deploy an async inference endpoint\n- Return a predictor object for making inferences\n\n### 4. Test the Endpoint\n\n```python\nfrom examples.invoke_endpoint import invoke_omniparser, get_results\n\n# Submit an inference request\nimage_path = \"path/to/your/image.png\"\noutput_location = invoke_omniparser(image_path)\n\n# Wait for processing (you can implement polling here)\nimport time\ntime.sleep(30)\n\n# Get results\nlabeled_image, coordinates, content = get_results(output_location)\n```\n\n## Model Weights\n\nOmniParser v2 uses two main model components:\n1. Icon Detection Model (YOLO-based)\n2. Icon Caption Model (Florence2)\n\nThe weights are managed in two stages:\n\n1. **Build Time**:\n   - Downloaded from Hugging Face\n   - Packaged into `model.tar.gz`\n   - Uploaded to S3: `s3://\u003cbucket\u003e/model/omniparser-v2/model.tar.gz`\n\n2. **Runtime**:\n   - SageMaker automatically downloads weights from S3\n   - Extracts to `/opt/ml/model` in the container\n   - Used by the model for inference\n\n## Configuration\n\n### Build Configuration\n```bash\n# Required:\nexport OMNIPARSER_MODEL_BUCKET=\"your-bucket\"  # S3 bucket for model weights\n\n# Optional:\nexport AWS_DEFAULT_REGION=\"us-west-2\"  # Defaults to us-west-2\n```\n\n### Deployment Configuration\n```python\n# In deploy.py:\npredictor = deploy_omniparser(\n    model_bucket=\"your-bucket\",\n    model_prefix=\"model/omniparser-v2\"  # Optional, defaults to this value\n)\n```\n\n### Inference Configuration\n```python\n# In invoke_endpoint.py:\nrequest = {\n    'image': encode_image(image_path),\n    'box_threshold': 0.05,     # Detection confidence threshold\n    'iou_threshold': 0.7,      # Box overlap threshold\n    'use_paddleocr': False,    # Whether to use PaddleOCR\n    'batch_size': 128          # Batch size for caption generation\n}\n```\n\n## Monitoring\n\n1. **CloudWatch Metrics**:\n   - Endpoint invocations\n   - Model latency\n   - GPU utilization\n\n2. **CloudWatch Logs**:\n   - Container logs\n   - Inference errors\n\n3. **S3 Monitoring**:\n   - Async inference results\n   - Failed inference requests\n\n## Troubleshooting\n\n1. **Build Issues**:\n   - Check S3 bucket permissions\n   - Verify Hugging Face access\n   - Check Docker build logs\n   - Ensure enough disk space for weights\n\n2. **Deployment Issues**:\n   - Verify IAM roles have necessary permissions\n   - Check SageMaker service quotas\n   - Verify GPU instance availability\n\n3. **Inference Issues**:\n   - Check async output location\n   - Verify input image format\n   - Monitor GPU memory usage\n\n## Cleanup\n\n```python\nimport boto3\n\n# Delete endpoint\nsagemaker = boto3.client('sagemaker')\nsagemaker.delete_endpoint(EndpointName='omniparser-v2-async')\n\n# Delete model weights (optional)\ns3 = boto3.client('s3')\ns3.delete_object(\n    Bucket='your-model-bucket',\n    Key='model/omniparser-v2/model.tar.gz'\n)\n``` ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faryasaatvik%2Fomniparser-sagemaker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faryasaatvik%2Fomniparser-sagemaker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faryasaatvik%2Fomniparser-sagemaker/lists"}