{"id":29356947,"url":"https://github.com/cesnet/landsat","last_synced_at":"2025-07-29T23:32:42.066Z","repository":{"id":283047776,"uuid":"744475881","full_name":"CESNET/landsat","owner":"CESNET","description":null,"archived":false,"fork":false,"pushed_at":"2025-05-19T12:12:24.000Z","size":314,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-19T12:39:41.354Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CESNET.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-01-17T11:29:04.000Z","updated_at":"2025-05-19T12:12:29.000Z","dependencies_parsed_at":"2025-03-18T10:30:18.503Z","dependency_job_id":"46c1ecd6-e820-4fb4-a305-e09362cd26d6","html_url":"https://github.com/CESNET/landsat","commit_stats":null,"previous_names":["cesnet/landsat"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CESNET/landsat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Flandsat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Flandsat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Flandsat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Flandsat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CESNET","download_url":"https://codeload.github.com/CESNET/landsat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Flandsat/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267780047,"owners_count":24143201,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-09T05:41:05.886Z","updated_at":"2025-07-29T23:32:41.983Z","avatar_url":"https://github.com/CESNET.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# landsat\n\nThis script is used to obtain, download and register into STAC catalogue Landsat\nImagery data mainly for uses in CESNET, z.s.p.o.\n\nApp is divided into two cooperative parts **downloader** and **http-server**.\n\n## downloader\n\nAs the name suggest, this component is responsible for downloading data using USGS M2M API.\n\nThe data are downloaded one month (30 days) into the past with one day resolution.\nThe script runs in `while True:` loop, and after downloading all data available at the moment,\nit sleeps until the closest 9:00 AM UTC. At this time downloading is re-executed.\n\n### Prerequisites\n\n1) An account and user login token for USGS M2M API must be created.\n   Please follow the instructions at [https://m2m.cr.usgs.gov/](https://m2m.cr.usgs.gov/).\n\n2) File **downloader/config/m2m_config.py** must be filled with following information:\n\n```python\napi_url = 'https://m2m.cr.usgs.gov/api/api/json/stable/'\nusername = 'username_used_for_login'\ntoken = 'user_login_token'\n```\n\n3) File **downloader/config/s3_config.py** must be filled with following information:\n\n```python\nhost_base = \"https://s3.cl4.du.cesnet.cz\"\naccess_key = \"s3_access_key\"\nsecret_key = \"s3_secret_key\"\nhost_bucket = \"landsat\"\n```\n\n4) File **downloader/config/stac_config.py** must be filled with following information:\n\n```python\nstac_base_url = 'https://stac.cesnet.cz/'\nusername = 'stac.cesnet.cz username'\npassword = 'stac.cesnet.cz password'\n```\n\n5) You may also want to change contents of **downloader/config/landsat_config.py**, especially\n   the `s3_download_host` variable:\n\n```python\ns3_download_host = \"http://chronos.dhr.cesnet.cz:8081/\"\n```\n\nThe `s3_download_host` must correspond to the computer on which the **http-server** component\nis running.\n\n### Logging\n\nLogging can be altered using **downloader/config/landsat_config.py**:\n\n```python\nlog_directory = \"log\"\nlog_name = \"landsat.log\"\nlog_level = 20\nlog_logger = \"LandsatLogger\"\n```\n\n`log_directory` can be either relative to **downloader/** or absolute.\n\nLog is rotated every day at 12:00 AM UTC.\n\n## http-server\n\nPowered by [Sanic](https://sanic.dev/en/).\n\nHTTP server acts as a relay between an asset link published in STAC catalog and S3 storage.\n\n### Prerequisites\n\nThe **http-server/.env** file must be filled as follows:\n\n```bash\nSANIC__APP_NAME=\"landsat_http_server\"\nSANIC__SERVER_HOST=\"0.0.0.0\"\nSANIC__SERVER_PORT=\"8080\"\n\nS3_CONNECTOR__HOST_BASE=\"https://s3.example.com\"\nS3_CONNECTOR__HOST_BUCKET=\"landsat\"\nS3_CONNECTOR__ACCESS_KEY=\"1234567890ABCDEFGHIJ\"\nS3_CONNECTOR__SECRET_KEY=\"123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcde\"\n```\n\n### Settings\n\nThere is not much what can be changed here. The main changes can be done by altering **.env** file.\n\n### Logging\n\nLogging can be altered using **.env** file as well. For example:\n\n```bash\nLOGGER__NAME=\"LandsatHttpServerLogger\"\nLOGGER__LOG_DIRECTORY=\"./log\"\nLOGGER__LOG_FILENAME=\"landsat_http_server.log\"\nLOGGER__LOG_LEVEL=20\n```\n\n`LOGGER__LOG_DIRECTORY` can be either relative to **http-server/** or absolute.\n\nLog is rotated every day at 12:00 AM UTC.\n\nLog levels are as follows:\n\n| READABLE | INTEGER  |\n|----------|----------|\n| CRITICAL | 50       |\n| FATAL    | CRITICAL |\n| ERROR    | 40       |\n| WARNING  | 30       |\n| WARN     | WARNING  |\n| INFO     | 20       |\n| DEBUG    | 10       |\n| NOTSET   | 0        |\n\n## Running\n\nPackage is using Docker. Please see the corresponding **docker-compose.yml** files for [downloader](#downloader)\nand [http-server](#http-server).\n\nThere is not much to change. In fact just the port of **http-server** in :\n\n```docker\nhttp-server:\n    ports:\n      - \"8080:8080\"\n```\n\nTo run the package just install `docker` and run `docker compose up -d` command in both directories.\n\nSo to run the **downloader** in folder `landsat/downloader` execute:\n\n```bash\ndocker compose up -d\n```\n\nand do the same in folder `landsat/http-server` to execute **http-server**.\n\nThere is also prepared a little script to run both of these docker containers.\n\nAlso in both **docker-compose.yml** files there are flags `restart: unless-stopped`, and thus after rebooting the\nmachine, scripts will restart automatically.\n\n## Thanks\n\nUsing [m2m-api](https://github.com/Fergui/m2m-api) sources by Angel Farguell licensed under MIT License. Many thanks!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcesnet%2Flandsat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcesnet%2Flandsat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcesnet%2Flandsat/lists"}