{"id":13735820,"url":"https://github.com/planetlabs/radiometric_normalization","last_synced_at":"2025-12-30T01:44:03.485Z","repository":{"id":27132147,"uuid":"30600651","full_name":"planetlabs/radiometric_normalization","owner":"planetlabs","description":"Implementation of radiometric normalization workflows","archived":false,"fork":false,"pushed_at":"2023-08-08T05:52:12.000Z","size":1523,"stargazers_count":33,"open_issues_count":2,"forks_count":11,"subscribers_count":215,"default_branch":"master","last_synced_at":"2024-11-10T16:29:15.442Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/planetlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-02-10T16:09:00.000Z","updated_at":"2023-07-26T05:51:09.000Z","dependencies_parsed_at":"2024-01-11T20:47:13.537Z","dependency_job_id":"e1b5cc11-a88f-4daf-86c7-cea44eed4c8b","html_url":"https://github.com/planetlabs/radiometric_normalization","commit_stats":{"total_commits":153,"total_committers":5,"mean_commits":30.6,"dds":0.2941176470588235,"last_synced_commit":"02002edae60741514de4214b1ea0305744e3acf3"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/planetlabs%2Fradiometric_normalization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/planetlabs%2Fradiometric_normalization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/planetlabs%2Fradiometric_normalization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/planetlabs%2Fradiometric_normalization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/planetlabs","download_url":"https://codeload.github.com/planetlabs/radiometric_normalization/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224732131,"owners_count":17360416,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T03:01:11.639Z","updated_at":"2025-12-30T01:44:03.458Z","avatar_url":"https://github.com/planetlabs.png","language":"Python","funding_links":[],"categories":["`Python` processing of optical imagery (non deep learning)","Atmospheric Correction"],"sub_categories":["Reflectance / pre processing"],"readme":"# Radiometric Normalization\n\nThis library implements functionality for normalizing a candidate image to time-invariant features in a set of reference images covering a time series.  This includes generating a time-invariant reference image from a time stack, identifying features that are invariante between the reference time stack and a candidate image, calculating a linear transformation that normalizes the candidate image to the reference image, applying the linear transform, and validating the results.\n\nThe primary use case for this library is radiometrically normalizing a satellite image to a time series from a reference dataset.\n\n## Development Environment\n\nThis library uses a Vagrant VM for the development environment and requires [Vagrant](https://www.vagrantup.com/) on the host computer.\n\nTo start the VM, in the root directory of the repo type:\n```\nvagrant up\n```\n\nLog into the VM by typing:\n```\nvagrant ssh\n```\n\nNavigate to the root directory:\n```\ncd /vagrant\n```\nThis directory is shared between the VM and host.\n\nRun the unit tests by typing:\n```\nnosetests ./tests\n```\n\n\n## Organization of the repo\n\nThe code in this repo is kept in two directories: 'radiometric_normalization' and 'tests'\n\n'radiometric_normalization' contains the algorithm and functions of the library:\n* 'time_stack.py' is the module that can calculate a time stack (average image) from a series of images (depreciated).\n* 'pif.py' contains the functions that find pseudo invariant feature pixels on within an image.\n* 'transformation.py' calculates the transformations to make the candidate data more consistent with the reference data.\n* 'normalize.py' is the module that can apply transformations calculated in 'transformation.py' to an image.\n* 'validate.py' is the module for validating radiometric normalization.\n\nEach of the modules referenced above are intended to run on a single band at a time (represented as a numpy array). Examples of more complete functions that can handle image reading and multiple bands are within 'radiometric_normalization/wrappers'. These will be explained below.\n\n'tests' contain unit tests for the functions in the library.\n\n\n## Example Usage\n\nThe example below demonstrates the generation of per-band linear transformations that will normalize a candidate image to the mean values of a set of reference images. The candidate and reference images must be 16-bit. Additionally, all of the reference images in the set must have the same number and order bands and pixel dimensions as the candidate image.\n\n`candidate_path` is a string specifying the location of the candidate image on disk. `reference_paths` is a list of strings, each specifying the location of a reference image on disk. `transformations` is a list of tuples, each specifying the gain (first entry) and offset (second entry) that will normalize the respective band of the candidate image.\n\nBelow is an example using two Landsat8 tiles: `LC08_L1TP_044034_20170427_20170428_01_T1` and `LC08_L1TP_044034_20170105_20170218_01_RT`.\n\n```python\n\nfrom radiometric_normalization.wrappers import pif_wrapper\nfrom radiometric_normalization.wrappers import transformation_wrapper\nfrom radiometric_normalization.wrappers import normalize_wrapper\nfrom radiometric_normalization import gimage\nfrom radiometric_normalization import pif\n\n## OPTIONAL\nimport logging\nimport numpy\nimport subprocess\nfrom osgeo import gdal\nfrom radiometric_normalization.wrappers import display_wrapper\n\nlogging.basicConfig(level=logging.DEBUG)\n##\n\n## OPTIONAL - Cut dataset to colocated sub scenes and create and BGRN image\n# LC08_L1TP_044034_20170105_20170218_01_T1 is the older scene and so it is set as the reference.\n\nband_mapping = [{'name': 'blue', 'L8': 'B2'}, {'name': 'green', 'L8':'B3'}, {'name': 'red', 'L8': 'B4'}, {'name': 'nir', 'L8': 'B5'}]\n\nfull_candidate_basename = 'LC08_L1TP_044034_20170427_20170428_01_RT'\nfull_reference_basename = 'LC08_L1TP_044034_20170105_20170218_01_T1'\ncandidate_basename = 'candidate'\nreference_basename = 'reference'\nfull_candidate_filenames = ['{}_{}.TIF'.format(full_candidate_basename, b['L8']) for b in band_mapping]\ncandidate_filenames = ['{}_{}.TIF'.format(candidate_basename, b['name']) for b in band_mapping]\nfull_reference_filenames = ['{}_{}.TIF'.format(full_reference_basename, b['L8']) for b in band_mapping]\nreference_filenames = ['{}_{}.TIF'.format(reference_basename, b['name']) for b in band_mapping]\n\nfor full_filename, cropped_filename in zip(full_candidate_filenames, candidate_filenames):\n    subprocess.check_call(['gdal_translate', '-projwin', '545000', '4136000', '601000', '4084000', full_filename, cropped_filename])\n\nfor full_filename, cropped_filename in zip(full_reference_filenames, reference_filenames):\n    subprocess.check_call(['gdal_translate', '-projwin', '545000', '4136000', '601000', '4084000', full_filename, cropped_filename])\n\nband_gimgs = {}\nfor cropped_filename in candidate_filenames:\n    band = cropped_filename.split('_')[1].split('.TIF')[0]\n    band_gimgs[band] = gimage.load(cropped_filename)\n\ncandidate_path = 'candidate.tif'\ncombined_alpha = numpy.logical_and.reduce([b.alpha for b in band_gimgs.values()])\ntemporary_gimg = gimage.GImage([band_gimgs[b].bands[0] for b in ['blue', 'green', 'red', 'nir']], combined_alpha, band_gimgs['blue'].metadata)\ngimage.save(temporary_gimg, candidate_path)\n\nband_gimgs = {}\nfor cropped_filename in reference_filenames:\n    band = cropped_filename.split('_')[1].split('.TIF')[0]\n    band_gimgs[band] = gimage.load(cropped_filename)\n\nreference_path = 'reference.tif'\ncombined_alpha = numpy.logical_and.reduce([b.alpha for b in band_gimgs.values()])\ntemporary_gimg = gimage.GImage([band_gimgs[b].bands[0] for b in ['blue', 'green', 'red', 'nir']], combined_alpha, band_gimgs['blue'].metadata)\ngimage.save(temporary_gimg, reference_path)\n##\n\nparameters = pif.pca_options(threshold=100)\npif_mask = pif_wrapper.generate(candidate_path, reference_path, method='filter_PCA', last_band_alpha=True, method_options=parameters)\n\n## OPTIONAL - Save out the PIF mask\ncandidate_ds = gdal.Open(candidate_path)\nmetadata = gimage.read_metadata(candidate_ds)\npif_gimg = gimage.GImage([pif_mask], numpy.ones(pif_mask.shape, dtype=numpy.bool), metadata)\ngimage.save(pif_gimg, 'PIF_pixels.tif')\n##\n\ntransformations = transformation_wrapper.generate(candidate_path, reference_path, pif_mask, method='linear_relationship', last_band_alpha=True)\n\n## OPTIONAL - View the transformations\nprint transformations\n##\n\nnormalised_gimg = normalize_wrapper.generate(candidate_path, transformations, last_band_alpha=True)\nresult_path = 'normalized.tif'\ngimage.save(normalised_gimg, result_path)\n\n## OPTIONAL - View the effect on the pixels (SLOW)\nfrom radiometric_normalization.wrappers import display_wrapper\ndisplay_wrapper.create_pixel_plots(candidate_path, reference_path, 'Original', limits=[0, 30000], last_band_alpha=True)\ndisplay_wrapper.create_pixel_plots(result_path, reference_path, 'Transformed', limits=[0, 30000], last_band_alpha=True)\ndisplay_wrapper.create_all_bands_histograms(candidate_path, reference_path, 'Original', x_limits=[4000, 25000], last_band_alpha=True)\ndisplay_wrapper.create_all_bands_histograms(result_path, reference_path, 'Transformed', x_limits=[4000, 25000], last_band_alpha=True)\n##\n```\n\nIn the above example, the 'reference.tif', original candidate scene ('candidate.tif') and radiometrically normalized candidate scene ('normalized.tif') all displayed at the same intensity scale: \n\n| Reference | Original | Transformed | PIF (red pixels) over Reference |\n| --- | --- | --- | --- |\n| ![Reference](images/Reference.jpg?raw=true) | ![Original](images/Original.jpg?raw=true)| ![Transformed](images/Transformed.jpg?raw=true) | ![PIF](images/PIF.jpg?raw=true) |\n\n\nYou can compare the per-band DN to DN plots and the histograms of the reference image with the original candidate image and with the normalized candidate image:\n\n| Plot | Original | Transformed |\n| --- | --- | --- |\n| Blue band DN-DN plot | ![Original_1.png](images/Original_1.png?raw=true) | ![Transformed_1.png](images/Transformed_1.png?raw=true) |\n| Green band DN-DN plot | ![Original_2.png](images/Original_2.png?raw=true) | ![Transformed_2.png](images/Transformed_2.png?raw=true) |\n| Red band DN-DN plot | ![Original_3.png](images/Original_3.png?raw=true) | ![Transformed_3.png](images/Transformed_3.png?raw=true) |\n| NIR band DN-DN plot | ![Original_4.png](images/Original_4.png?raw=true) | ![Transformed_4.png](images/Transformed_4.png?raw=true) |\n| Histogram | ![Original_histograms.png](images/Original_histograms.png?raw=true) | ![Transformed_histograms.png](images/Transformed_histograms.png?raw=true) |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplanetlabs%2Fradiometric_normalization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplanetlabs%2Fradiometric_normalization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplanetlabs%2Fradiometric_normalization/lists"}