{"id":47737995,"url":"https://github.com/michaelakridge-noaa/jetstream","last_synced_at":"2026-04-02T23:03:55.570Z","repository":{"id":345811996,"uuid":"1150853837","full_name":"MichaelAkridge-NOAA/jetstream","owner":"MichaelAkridge-NOAA","description":"NOAA Jetstream - Easy uploads to Google Cloud Storage","archived":false,"fork":false,"pushed_at":"2026-03-20T21:23:42.000Z","size":5686,"stargazers_count":0,"open_issues_count":5,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-21T10:45:42.131Z","etag":null,"topics":["cloud","fastapi","google","oceanography","python","storage"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MichaelAkridge-NOAA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-05T19:11:45.000Z","updated_at":"2026-03-20T21:23:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/MichaelAkridge-NOAA/jetstream","commit_stats":null,"previous_names":["michaelakridge-noaa/jetstream"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/MichaelAkridge-NOAA/jetstream","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MichaelAkridge-NOAA%2Fjetstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MichaelAkridge-NOAA%2Fjetstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MichaelAkridge-NOAA%2Fjetstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MichaelAkridge-NOAA%2Fjetstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MichaelAkridge-NOAA","download_url":"https://codeload.github.com/MichaelAkridge-NOAA/jetstream/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MichaelAkridge-NOAA%2Fjetstream/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31318153,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-02T21:35:00.834Z","status":"ssl_error","status_checked_at":"2026-04-02T21:34:59.806Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud","fastapi","google","oceanography","python","storage"],"created_at":"2026-04-02T23:03:54.666Z","updated_at":"2026-04-02T23:03:55.552Z","avatar_url":"https://github.com/MichaelAkridge-NOAA.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NOAA JetStream — Cloud Data Management Transfer System\n\u003cimg align=\"right\" src=\"https://github.com/MichaelAkridge-NOAA/jetstream/raw/main/docs/jetstream_logo_400px.png\" alt=\"jetstream\" width=\"250\"\u003e\n\nA comprehensive web-based application for managing Google Cloud Storage uploads with features including job queuing, real-time analytics, cloud bucket analysis, and batch processing capabilities.\n\n### Features\n\n- **Upload Management**\n- **Analytics \u0026 Monitoring**\n- **Cloud Bucket Analysis**\n- **File Filtering**\n- **Web Dashboard**\n\n## Screenshots\n\n| Dashboard | Upload Jobs | Analytics |\n|-----------|-------------|----------|\n| ![Home](https://github.com/MichaelAkridge-NOAA/jetstream/raw/main/docs/screenshot_home.png) | ![Uploads](https://github.com/MichaelAkridge-NOAA/jetstream/raw/main/docs/screenshot_uploads.png) | ![Analytics](https://github.com/MichaelAkridge-NOAA/jetstream/raw/main/docs/screenshot_analytics.png) |\n\n### Prerequisites\n\n- **Python 3.9+**\n- **Google Cloud SDK** (includes gsutil) — for cloud upload features\n- **Permissions** to target GCS buckets\n---\n### Google Cloud Setup\n\nRequired only for cloud upload features:\n```bash\n# Install Google Cloud SDK\n# Download from: https://cloud.google.com/sdk/docs/install\n\n# Authenticate\ngcloud auth login --no-launch-browser\ngcloud auth application-default login --no-launch-browser\n\n# Verify access (optional)\ngsutil ls\ngcloud auth list\n```\n\n### Installation\n#### Option 1: Install from PyPI (Recommended)\n- Link: https://pypi.org/project/noaa-jetstream/\n```bash\npip install noaa-jetstream\n```\n\n\u003e **Using Anaconda/conda?** If you see dependency resolver warnings, use `uv` (recommended) or `--no-cache-dir`:\n\u003e ```bash\n\u003e # Option A: use uv (faster, cleaner resolver — recommended for conda users)\n\u003e pip install uv\n\u003e uv pip install noaa-jetstream\n\u003e # May need to create virtual environment first, so do it in a local directory then activate uv venv then .venv\\Activate\\scripts.bat\n\u003e \n\u003e # Option B: skip pip cache\n\u003e pip install --no-cache-dir --no-user noaa-jetstream\n\u003e ```\n\n#### Upgrade\n```\nuv pip install --no-cache --upgrade noaa-jetstream\n```\n\n#### Option 2: Install from Source (Development)\n\n```bash\n# Clone the repository\ngit clone https://github.com/MichaelAkridge-NOAA/jetstream.git\ncd jetstream\n\n# Install in development mode\npip install -e .\n\n# Or using uv (recommended)\npip install uv\nuv pip install -e \".[dev]\"\n# May need to pip install uvicorn, pip install fastapi, pip install google-cloud-storage separately\n```\n\n---\n\n## Starting the Application\n\n### If Installed via pip\n\n```bash\n# Start the server (opens browser automatically)\njetstream\n\n# view options\njetstream --help\n# With custom options\njetstream --port 9000\njetstream --host 127.0.0.1 --port 8080\njetstream --no-browser\njetstream --log-level debug\n```\n\n### If Running from Source\n\n```bash\n# Using the CLI\npython main.py\n\n# Or with the diagnostic startup script\npython start.py\n\n# Or directly with uvicorn\npython -m uvicorn jetstream.main:app --reload\n```\n\nThe application will start on **http://localhost:8000** and automatically open in your default browser.\n\n### Desktop Shortcuts\n\nDesktop and Start Menu shortcuts are included with the default install. The shortcut will automatically use the JetStream icon (`icon.ico`) when created.\n\n```bash\n# Create desktop + Start Menu shortcut (uses JetStream icon automatically)\njetstream-create-shortcuts\n\n# Remove shortcuts\njetstream-remove-shortcuts\n```\n\nShortcuts launch JetStream directly using the current Python environment and open a terminal window. On Windows a `.lnk` shortcut is created on the desktop and in the Start Menu. On macOS/Linux a `.app`/`.desktop` shortcut is created in Applications.\n\n### Troubleshooting Startup Issues\n\n**If the server appears to start but you can't connect:**\n\n1. **Run diagnostics:**\n   ```bash\n   python diagnose.py\n   ```\n   \n2. **Run with debug logging:**\n   ```bash\n   jetstream --log-level debug\n   # or from source:\n   python -m uvicorn jetstream.main:app --reload --log-level debug\n   ```\n---\n\n## Troubleshooting\n\n**Cannot connect to GCS:**\n- Verify authentication: `gcloud auth list`\n- Check bucket permissions\n- Ensure Application Default Credentials are set\n\n**Jobs stuck in queue:**\n- Check queue status in dashboard\n- Verify no jobs are blocking the queue\n- Restart the application if needed\n\n**Database errors:**\n- Delete `jetstream.db` to reset (loses history)\n- Check file permissions in application directory\n\n**API not responding:**\n- Check if port 8000 is already in use\n- View logs in terminal for error messages\n- Ensure all dependencies are installed\n\n----------\n#### Disclaimer\nThis repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project content is provided on an 'as is' basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.\n\n## License\nSee the [LICENSE.md](./LICENSE.md) for details\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelakridge-noaa%2Fjetstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmichaelakridge-noaa%2Fjetstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelakridge-noaa%2Fjetstream/lists"}