{"id":37324957,"url":"https://github.com/podaac/tig","last_synced_at":"2026-04-12T23:00:13.656Z","repository":{"id":103454915,"uuid":"608308217","full_name":"podaac/tig","owner":"podaac","description":"Tool for Image Generation","archived":false,"fork":false,"pushed_at":"2026-03-14T20:25:27.000Z","size":48281,"stargazers_count":3,"open_issues_count":8,"forks_count":0,"subscribers_count":5,"default_branch":"develop","last_synced_at":"2026-03-15T05:39:58.183Z","etag":null,"topics":["development","hitide","tva"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/podaac.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-03-01T18:47:10.000Z","updated_at":"2026-03-14T20:25:30.000Z","dependencies_parsed_at":"2023-07-14T11:12:36.222Z","dependency_job_id":"5cbbb8dd-683d-4e45-a819-5b08666f4547","html_url":"https://github.com/podaac/tig","commit_stats":null,"previous_names":[],"tags_count":264,"template":false,"template_full_name":null,"purl":"pkg:github/podaac/tig","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/podaac%2Ftig","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/podaac%2Ftig/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/podaac%2Ftig/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/podaac%2Ftig/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/podaac","download_url":"https://codeload.github.com/podaac/tig/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/podaac%2Ftig/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31290969,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["development","hitide","tva"],"created_at":"2026-01-16T03:27:38.397Z","updated_at":"2026-04-01T18:53:20.481Z","avatar_url":"https://github.com/podaac.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TIG - Tool for Image Generation\n\n# Table of contents\n\n- [Overview](#overview)\n- [Install poetry](#install-poetry-python-dependency-manager)\n- [Build](#build)\n- [How to load and use tig module](#how-to-load-and-use-tig-module)\n  - [tig Input](#tig-input)\n  - [tig Output](#tig-output)\n- [CLI Commands](#cli-commands)\n- [Python Usage](#python-usage)\n\n## Overview\n   tig generates granule-level thumbnail images for one or several variables within the granule (currently works with netCDF and HDF formats). tig is built to be used within Cumulus ecosystem. It is depending on cumulus CMA ([Cumulus Documentation](https://nasa.github.io/cumulus)). Please refer to the [Usage](#usage) section for inputs and outputs. TIG itself is a lambda function which runs on top of CMA as its lambda layer.\n\n## Install poetry (python dependency manager)\n   Install poetry following the directions here: https://python-poetry.org/docs/#installation\n\n   - Be sure to install poetry in an isolated environment from the rest of your system.\n   - Be sure to use with a python version less than 3.12\n\n## Build\n* Jenkins pipeline template is applied to this project.\n* development is based on poetry python environment.  \n  * poetry run pytest   // to run unit test\n  * poetry install      // to install all dependencies defined in toml file\n  * poetry shell        //to enter the poetry shell\n  * poetry build        // to build the wheel\n\n\nUnit Test can be run using the command\n```shell script\npoetry run pytest\n```\n\n## CLI Commands\n- [Generate Thumbnails](#generate-thumbnails)\n- [Generate Config File](#generate-config-file)\n- [Run Tig](#run-tig)\n\n### Generate Thumbnails\nUse cli helper script to generate thumbnails for each collection (NOTE: This automatically generates the config file and runs tig)\n```\nGo to dir `podaac/tig` and look at generate_thumbnails.sh script.  Follow directions at top to setup.\n```\n\n### Generate Config File\nUse cli to create a tig configuration for collections \n```\ngenerate_hitide_config --granule \u003cgranule_file\u003e -dataset-id \u003ccollection short name\u003e --include-image-variables \u003ccsv file image variables\u003e --longitude \u003clon variable\u003e --latitude \u003clat variable\u003e --time \u003ctime variable\u003e --footprint_strategy \u003cfootprint strategy\u003e\n```\n\ngranule: a sample granule file to generate the configuration for\ndatset-id: collection short name \ninclude-image-variables: csv file of with image variable names and min max setting for each variable\nlongitude: longitude variable include the group if they're in group defaults to longitude\nlatitude: latitude variable include the group if they're in a group defaults to latitude\ntime: time variable include the group if they're in a group defaults to time\nfootprint_strategy: strategy to generate footprint will default to None options should be [\"periodic\", \"linestring\", \"polar\", \"swot_linestring\", \"polarsides\", \"smap\"]\n\n### Run Tig\nUse cli to test thumbnail image generation for a granule with configuration file and palettes\n\n```\ntig --input_file \u003cgranule\u003e --output_dir \u003coutput_dir\u003e --config_file \u003cconfig_file\u003e --palette_dir \u003cpalette_dir\u003e\n```\n\nUse cli to create a tig configuration for collections \n```\ngenerate_hitide_config --granule \u003cgranule_file\u003e -dataset-id \u003ccollection short name\u003e --include-image-variables \u003ccsv file image variables\u003e --longitude \u003clon variable\u003e --latitude \u003clat variable\u003e --time \u003ctime variable\u003e --footprint_strategy \u003cfootprint strategy\u003e\n```\n\ngranule: a sample granule file to generate the configuration for\ndatset-id: collection short name \ninclude-image-variables: csv file of with image variable names and min max setting for each variable\nlongitude: longitude variable include the group if they're in group defaults to longitude\nlatitude: latitude variable include the group if they're in a group defaults to latitude\ntime: time variable include the group if they're in a group defaults to time\nfootprint_strategy: strategy to generate footprint will default to None options should be [\"periodic\", \"linestring\", \"polar\", \"swot_linestring\", \"polarsides\", \"smap\"]\n\n### Regression Test\n\n** IN DEVELOPMENT **\n\nCurrently there is a regression test in the regression_test folder to run please use this command\n\nNote palettes folder needs to be downloaded and included in the regression_test folder\nWere not clearing out any data after test future improvement to make it an option to clear or retain data\nas granules take awhile to download\n\n```\npytest regression.py\n```\n\n### CSV Columns\n\nvariable: name of variable\nmin: min value for variable\nmax: max value for variable\npalette (optional): the palette to be used for the variable\nfill_missing (optional): if the generated images have missing pixel in images most likely resolution is to big, either lower resolution or we can fill in the pixels with surrounding pixel\nppd (optional): resolution of the variable, must be an integer\n\n\n## How to load and use tig module\nProject using tig can include/use the tig as following:\n```shell script\n    module \"tig_module\" {\n      source = \"https://cae-artifactory.jpl.nasa.gov/artifactory/general/gov/nasa/podaac/cumulus/tig-terraform/tig-terraform-0.3.0.zip\"\n      // Lambda variables\n      prefix = var.prefix\n      image = var.tig_image\n      role = module.cumulus.lambda_processing_role_arn\n      cmr_environment = var.cmr_environment\n      subnet_ids = var.subnet_ids\n      security_group_ids = [aws_security_group.no_ingress_all_egress.id]\n      task_logs_retention_in_days = var.task_logs_retention_in_days\n      config_url = \"https://hitide.podaac.earthdatacloud.nasa.gov/dataset-configs\"\n      palette_url = \"https://hitide.podaac.earthdatacloud.nasa.gov/palettes\"\n      memory_size = var.tig_memory_size\n}\n\n```\nand the module input variables explained as below.\n\n| field name | type | default | values | description\n| ---------- | ---- | ------- | ------ | -----------\n| prefix | string | (required) | | A prefix string of lambda function. Ex. prefix = \"sample\" , created lambda : sample-tig\n| region | string | (required) | | AWS region where tig lambda is running upon.  Ex. us-west-2\n| cmr_environment | string | (required) | | dev, sit, ops\n| config_bucket | string | (required) | | bucket where dataset config resides\n| config_dir | string | (required) | | directory where dataset config file resides. dataset-config file follows the collection_shortname.cfg pattern. Ex. MODIS_A-JPL-L2P-v2019.0.cfg\n| tig_output_bucket | string | (required) | | bucket where tig file is created and written\n| tig_output_dir | string | (required) | | output directory of created tig(fp) file. file will be created as s3://tig_output_bucket/tig_output_dir/collection_short_name/granule_id.png. ex. s3://my-cumulus-internaldataset-tig/ MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.png\n| lambda_role | string | (required) | | aws user role to run tig lambda\n| layers | list(string) | (required) | | list of layers' arn where tig runs upon.\n| security_group_ids | list(string) | (required) | | security group ids\n| subnet_ids | list(string) | (required) | | subnet ids where tig runs within\n|config_url | string | | | the url of where to retrieve configurations\n|palette_url | string | | | the url of where to retrieve palettes\n\nECS input variables optional\n\n| field name | type | default | values | description\n| ---------- | ---- | ------- | ------ | -----------\n|tig_ecs | bool | false | | boolean to deploy ecs task\n|cluster_arn | string | | | cumulus cluster arn\n|desired_count | number | 1 | | number of ecs tig task to run\n|log_destination_arn | string | | | A shared AWS:Log:Destination that receives logs in log_groups\n|ecs_cpu | number | 700 | |cpu unit to allocate to a tig task\n|ecs_memory_reservation | number | 1024 | | memory unit to allocate to tig task\n\n\nFargate input variables optional\n\n| field name | type | default | values | description\n| ---------- | ---- | ------- | ------ | -----------\n|tig_fargate | bool | false | | boolean to deploy fargate task\n|fargate_memory | number | 2048 | | amount of memory to allocate for a single fargate task\n|fargate_cpu | number | 1024 | | amount of cpu to allocate for a single fargate task\n|fargate_desired_count | number | 1 | | desired count of how many fargate task\n|fargate_min_capacity | number | 1 | | minimum number of fargate task when scaling\n|fargate_max_capacity | number | 1 | | maximum number of fargate task when scaling\n|scale_dimensions | map(string) | null | | cloudwatch dimensions to scale on\n|scale_up_cooldown | number | 60 | | seconds before able to scaling up again\n|scale_down_cooldown | number | 120 | | seconds before able to scaling down again\n|comparison_operator_scale_up | string | GreaterThanOrEqualToThreshold | | The arithmetic operation to use when comparing the specified Statistic and Threshold\n|evaluation_periods_scale_up | number | 1 | | The number of periods over which data is compared to the specified threshold\n|metric_name_scale_up | string | CPUUtilization | | name of the metric\n|namespace_scale_up | string | AWS/ECS | | namespace for the alarm's associated metric\n|period_scale_up | number | 60 | | period in seconds over which the specified statistic is applied\n|statistic_scale_up | string | Average | | statistic to apply to the metric\n|threshold_scale_up | number | 50 | | threshold for statistic to compare against to trigger step\n|scale_up_step_adjustment | list | | | step adjustment to make when scaling up fargate\n|comparison_operator_scale_down | string \n|evaluation_periods_scale_down | number | 1 | | The number of periods over which data is compared to the specified threshold\n|metric_name_scale_down | string | CPUUtilization | | name of the metric\n|namespace_scale_down | string | AWS/ECS | | namespace for the alarm's associated metric\n|period_scale_down | number | 60 | | period in seconds over which the specified statistic is applied\n|statistic_scale_down | string | Average | | statistic to apply to the metric\n|threshold_scale_down | number | 50 | | threshold for statistic to compare against to trigger step\n|scale_down_step_adjustment | list | | | step adjustment to make when scaling down fargate\n|fargate_iam_role | string | | | iam arn role for fargate\n\nmodule output variables\n\n| field name | type | default | values | description\n| ---------- | ---- | ------- | ------ | -----------\n| tig_function_name | string | (required) | | The name of deployed tig lambda function\n| tig_task_arn | string | (required) | | tig lambda aws arn\n\n### tig Input\n   Cumulus message with granules payload.  Example below\n```json\n{\n  \"granules\": [\n    {\n      \"files\": [\n        {\n          \"filename\": \"s3://bucket/file/with/checksum.dat\",\n          \"checksumType\": \"md5\",\n          \"checksum\": \"asdfdsa\"\n        },\n        {\n          \"filename\": \"s3://bucket/file/without/checksum.dat\",\n        }\n      ]\n    }\n  ]\n}\n```\n\n### tig Output\n   * A tig file will be created under configured tig_output_bucket and tig-output-dir.  filename as granuleId.png. Ex. s3://my-cumulus-internaldataset-tig/ MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.png\n   * A file object will be appended to the files[] of processed granule. Example:\n```json\n{\n  \"granules\": [\n    {\n      \"granuleId\": \"20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0\",\n      \"dataType\": \"MODIS_A-JPL-L2P-v2019.0\",\n      \"sync_granule_duration\": 2603,\n      \"files\": [\n        {\n          \"bucket\": \"my-protected\",\n          \"path\": \"MODIS_A-JPL-L2P-v2019.0/2020/001\",\n          \"filename\": \"s3://my-protected/MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.nc\",\n          \"size\": 18232098,\n          \"name\": \"20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.nc\",\n          \"checksumType\": \"md5\",\n          \"checksum\": \"aa5204f125ae83847b3b80fa2e571b00\",\n          \"type\": \"data\",\n          \"url_path\": \"{cmrMetadata.CollectionReference.ShortName}\",\n          \"filepath\": \"MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.nc\",\n          \"duplicate_found\": true\n        },\n        {\n          \"bucket\": \"my-public\",\n          \"path\": \"MODIS_A-JPL-L2P-v2019.0/2020/001\",\n          \"filename\": \"s3://my-public/MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.nc.md5\",\n          \"size\": 98,\n          \"name\": \"20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.nc.md5\",\n          \"type\": \"metadata\",\n          \"url_path\": \"{cmrMetadata.CollectionReference.ShortName}\",\n          \"filepath\": \"MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.nc.md5\",\n          \"duplicate_found\": true\n        },\n        {\n          \"bucket\": \"my-public\",\n          \"filename\": \"s3://my-public/MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.cmr.json\",\n          \"size\": 1617,\n          \"name\": \"20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.cmr.json\",\n          \"type\": \"metadata\",\n          \"url_path\": \"{cmrMetadata.CollectionReference.ShortName}\",\n          \"filepath\": \"MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.cmr.json\",\n          \"etag\": \"\\\"3e5b9259c5ee7eae5fe71467f151498b\\\"\"\n        },\n        {\n          \"bucket\": \"my-internal\",\n          \"filename\": \"s3://my-internal/dataset-tig/MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.png\",\n          \"filepath\": \"dataset-tig/MODIS_A-JPL-L2P-v2019.0/20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.png\",\n          \"size\": 452,\n          \"name\": \"20200101000000-JPL-L2P_GHRSST-SSTskin-MODIS_A-D-v02.0-fv01.0.png\",\n          \"type\": \"metadata\"\n        }\n      ]\n    }\n  ]\n}\n```\n\n## Python Usage\n\n### Installation\n\n#### **1. pip / pypi method**\n\nThe podaac-thumbnail-generator library is in https://pypi.org/project/podaac-thumbnail-generator/, so pypi should be added to your `~/pip/pip.conf` file, e.g. add the following lines:\n\n```\n[global]\nindex-url = https://pypi.org/simple\ntrusted-host = pypi.org\n```\n\nThen, the podaac-thumbnail-generator library can be installed:\n\n```bash\npip install podaac-thumbnail-generator\n```\n\nand imported:\n\n```\nfrom podaac.tig import tig\n```\n\n#### **2. repo cloning method**\n\n```\ngit clone -b release/0.10.0 git@github.com:podaac/tig.git\n```\n\nThen the module can be imported:\n```\n# Imports packages directly from the tig repo:\nsys.path.append(os.path.abspath(os.curdir) + \"/tig/podaac\")\nfrom tig import tig\n```\n\n### Example Usage\n\nFor the desired granule to create thumbnails for, create a TIG instance and then use it to generate the images:\n\n```\nimage_gen = tig.TIG(input_file, output_dir, config_file, palette_dir)\nimage_gen.generate_images(granule_id=granule_id)\n```\n\nwhere the parameters are:\n* **`input_file`** (string): The path to the data granule.\n* **`output_dir`** (string): Path to the folder in which to save the images.\n* **`config_file`** (string) Path to the configuration file containing parameters needed by tig (see [config file section](#configuration-file)).\n* **`palette_dir`** (string): The path to color palettes used for the image generation (more on this below).\n* **`granule_id`** (string): The filename of the granule (note this is the name only, as opposed to the full path).\n\nFor the `palette_dir`, one can be taken from the [forge-tig-configuration](https://github.com/podaac/forge-tig-configuration) repository, e.g. \n\n```\n!git clone git@github.com:podaac/forge-tig-configuration.git\npalette_dir = \"./forge-tig-configuration/palettes\"  # Path to color palettes in the forge-tig-configuration repo.\n```\n\n### Configuration File\n\nThe configuration file is a JSON that acts as a small metadata sidecar file for all granules in a collection (so only one config file is needed per collection). The easiest way to create the configuration file is using the [forge-tig-configuration module](https://github.com/podaac/forge-tig-configuration), but it can also be created manually, e.g.:\n\n```json\n{\n    \"shortName\": \"ASCATB_ESDR_L2_WSDERIV_V1.0\",\n    \"latVar\": \"lat_res12\",\n    \"lonVar\": \"lon_res12\",\n    \"is360\": true,\n    \"imgVariables\": [\n        {\n            \"id\": \"en_wind_divergence_res12\",\n            \"title\": \"Divergence of equivalent neutral wind, over approximate 12.5 km diameter region\",\n            \"units\": \"s-1\",\n            \"min\": -1.0,\n            \"max\": 1.0,\n            \"palette\": \"paletteMedspirationIndexed\"\n        },\n        {\n            \"id\": \"stress_curl_res12\",\n            \"title\": \"vorticity of wind stress, over approximate 12.5 km diameter region\",\n            \"units\": \"N m-3\",\n            \"min\": -1.0,\n            \"max\": 1.0,\n            \"palette\": \"paletteMedspirationIndexed\"\n        }\n    ],\n    \"image\": {\n        \"ppd\": 8,\n        \"res\": 8\n    }\n}\n```\n\n#### Description of fields\n* **`shortName`** (string, required): Collection short name.\n* **`lonVar`** (string, required): Longitude variable in the dataset include group if in one.\n* **`latVar`** (string, required): Latitude variable in the dataset include group if in one.\n* **`is360`** (boolean, required, default: False): Indicates if the data is in 360 format.\n* **`imgVariables`** (list, required): A list of dictionaries describing the variables to create thumbnails for. One dictionary per variable with the following key / value pairs:\n  * **`\"id\"`** (string): Variable name as it appears in the file.\n  * **`\"title\"`** (string): More descriptive name of the variable. E.g. typically the `long_name` from the variable attributes.\n  * **`\"units\"`** (string): Variable units.\n  * **`\"min\"`**, **`\"max\"`** (float's): Minimum / maximum values to use for colorscale. Note these can coincide with the `min` / `max` variable attributes, but can also be tweaked to improve the color range of the images.\n  * **`\"palette\"`** (string): Name of color palette to use. A folder of color palettes can be found in the [\"palettes\" folder of the forge-tig-configuration package](https://github.com/podaac/forge-tig-configuration/tree/main/palettes).\n* **`image`** (dict): Specifies parameters for image appearance. Includes the following key / value pairs:\n  * **`\"ppd\"`** (int): Fills surrounding pixels with same value as the nearest pixel. If the output image is faint, increasing this value may fix it. Typical values are in the range 4 - 16.\n  * **`\"res\"`** (int): Image resolution. If the output image is faint, decreasing this value may fix it. Typical values are in the range 4 - 16.\n  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpodaac%2Ftig","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpodaac%2Ftig","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpodaac%2Ftig/lists"}