{"id":25723507,"url":"https://github.com/cs-si/s2ipf_orchestrator_launcher","last_synced_at":"2025-10-11T07:35:35.262Z","repository":{"id":77444554,"uuid":"558756518","full_name":"CS-SI/S2IPF_Orchestrator_Launcher","owner":"CS-SI","description":"Launcher for the S2IPF_Orchestrator","archived":false,"fork":false,"pushed_at":"2023-05-10T14:22:01.000Z","size":176,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-25T20:40:50.870Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CS-SI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-28T08:18:01.000Z","updated_at":"2022-11-28T15:01:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"b45a816d-16d4-41bb-b9e1-380ae694850a","html_url":"https://github.com/CS-SI/S2IPF_Orchestrator_Launcher","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/CS-SI/S2IPF_Orchestrator_Launcher","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CS-SI%2FS2IPF_Orchestrator_Launcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CS-SI%2FS2IPF_Orchestrator_Launcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CS-SI%2FS2IPF_Orchestrator_Launcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CS-SI%2FS2IPF_Orchestrator_Launcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CS-SI","download_url":"https://codeload.github.com/CS-SI/S2IPF_Orchestrator_Launcher/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CS-SI%2FS2IPF_Orchestrator_Launcher/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006623,"owners_count":26084131,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-25T20:36:55.586Z","updated_at":"2025-10-11T07:35:35.256Z","avatar_url":"https://github.com/CS-SI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# S2IPF Orchestrator launcher\nLauncher for the S2IPF orchestrator\n\n\nCopyright\n=========\nCopyright (C) 2020-2022 CS GROUP – France, https://www.csgroup.eu\n\nLicensing\n=========\nS2IPF_Orchestrator_Launcher is licensed under the Apache License, Version 2.0. See\n[LICENSE](https://github.com/CS-SI/S2IPF_Orchestrator/tree/main/LICENSE) for the full\nlicense text.\n\n## Principle\n\nThe S2IPF Orchestrator is in charge of producing the JobOrders for the IDPSCs and execute them according to a tasktable.\nIt also uses configuration files for IDPSC parameters, parallelization, file paths etc.\nThis launcher is then a sublayer that allows to mask the complexity of the TaskTables/Configuration.\nInstead it provides simple \"Mode\" in a defined pipeline to produce the product desired.\n\n## Installation\n\nThese are python scripts that you can copy anywhere you want. \nHowever the host docker/machine needs to have installed:\n- The S2IPF-Orchestrator in the last version available here ( https://github.com/CS-SI/S2IPF_Orchestrator , private repository for now, only for CS users)\n- The IDPSCs in wanted versions ( see configuration files chapter)\n- The inventory scripts in wanted versions ( see configuration files chapter)\n\n## Commands\n\n### Help \n```\nusage: LaunchIPF.py [-h] -m MODE -a AUXS -s STATICAUXS -g GRI -i INPUT -w\n                    WORKING -o OUTPUT -p PARALLEL\n                    [-t TILE] [-l LOGLEVEL] [-e EXEVERSIONFILE]\n                    [-k KILLTIMEOUT]\n\nThis script launches the IPF orchestrator\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -m MODE, --mode MODE  mode\n  -a AUXS, --auxs AUXS  Auxs folder\n  -s STATICAUXS, --staticauxs STATICAUXS\n                        Statics Auxs folder: DEMs\n  -g GRI, --gri GRI     Statics Auxs folder: GRI\n  -i INPUT, --input INPUT\n                        Input folder\n  -w WORKING, --working WORKING\n                        Working folder\n  -o OUTPUT, --output OUTPUT\n                        Output folder\n  -p PARALLEL, --parallel PARALLEL\n                        Number of parallel Tasks\n  -t TILE, --tile TILE  Overload tile Ident: 011-002 for example\n  -l LOGLEVEL, --loglevel LOGLEVEL\n                        Log level\n  -e EXEVERSIONFILE, --exeversionfile EXEVERSIONFILE\n                        Set a different exe version file than the installed\n  -k KILLTIMEOUT, --killtimeout KILLTIMEOUT\n                        Set the kill process timeout on Orch step\n\nlaunches the IPF orchestrator\n\n```\n\n### Details\n\n- --mode: Sets the mode of the launcher. The list of modes is located in the data/pipeline.json file defining the workflow of modes such as \"L1A\" for example\n- --auxs: Defines the directory of so-called “dynamic” auxiliary data dependent on the current datastrip.\n- --static-auxs: Defines the directory of so-called \"static\" auxiliary data independent of the current datastrip.\n- --gri: defines the GRI directory for the current orbit.\n- --input: defines the directory of the product (L0u or L0c) of entry, is only used by the modes having no previous mode\n- --working: Defines the temporary working directory that can be deleted at the end of processing. Attention for the ‘Inventory’ steps the output products are in this directory\n- --output: output directory, contains the shared context to be able to chain Modes\n- --parallel: defines the number of processes in parallel, so the number of cpu used\n- --tile: In the case of a mode operating by tile, this defines the tile(s) to be processed 011-002\n- --loglevel: uppercase log level (INFO/DEBUG)\n- --exeversionfile: allows to redefine the version file of the IPF, in particular in IVV, if the parameter is absent then it is the GIPP PROBAS which prevails\n- --killtimeout: Duration of the timeout from which processing is stopped (by default 36000 seconds)\n\n\n## Input details\n\n### AUXS ( --auxs )\n\nthe input auxs folder must contains : \n- S2IPF-CAMS : \nDirectory containing the CAMS data corresponding to the dates of the product in the form of a directory containing the GRIB files\n```\nS2__OPER_AUX_CAMSFO_ADG__20220429T000000_V20220429T000000_20220501T010000/\n  |-\u003e z_cams_c_ecmf_*grib\n  ```\n  In order to generate it from the ESA TGZ\n  \n  ```bash\n  #tar xvf S2__OPER_AUX_CAMSFO_ADG__20220429T000000_V20220429T000000_20220501T010000.TGZ\n  S2__OPER_AUX_CAMSFO_ADG__20220429T000000_V20220429T000000_20220501T010000.DBL\n  S2__OPER_AUX_CAMSFO_ADG__20220429T000000_V20220429T000000_20220501T010000.HDR\n  # mkdir S2__OPER_AUX_CAMSFO_ADG__20220429T000000_V20220429T000000_20220501T010000\n  # cd S2__OPER_AUX_CAMSFO_ADG__20220429T000000_V20220429T000000_20220501T010000\n  # tar xvf ../S2__OPER_AUX_CAMSFO_ADG__20220429T000000_V20220429T000000_20220501T010000.DBL\n  ```\n                   \n-\t S2IPF-ECMWF :  \nDirectory containing the ECMWF data corresponding to the dates of the product in the form of a single GRIB file resulting from the concatenation of the GRIB files of the original AUX files S2__OPER_AUX_ECMWFD_ADG__20220428T120000_V20220428T210000_20220430T150000\n```bash\n# tar xvzf S2__OPER_AUX_ECMWFD_ADG__20220428T120000_V20220428T210000_20220430T150000.TGZ\nS2__OPER_AUX_ECMWFD_ADG__20220428T120000_V20220428T210000_20220430T150000.DBL\nS2__OPER_AUX_ECMWFD_ADG__20220428T120000_V20220428T210000_20220430T150000.HDR\n# mkdir temp\n# cd temp\n# tar xvf ../S2__OPER_AUX_ECMWFD_ADG__20220428T120000_V20220428T210000_20220430T150000.DBL\nS2D.... files\n# for f in S2D*;do cat $f \u003e\u003e S2__OPER_AUX_ECMWFD_ADG__20220428T120000_V20220428T210000_20220430T150000;done\n```\n\n-\t S2IPF-GIPP : \nContains all the GIPP xml for the processing of the desired levels. Please note that the GIPP OLQCPA must have the extension .zip.\nThe list of required GIPP is given in the s2_aux_list.json file along this README file\n\n```json\n{\n  \"Name\" : \"AUX_UT1UTC\",\n  \"ProductLevels\" : [ \"L012\" , \"EISP\"],\n  \"Rule\" : \"LatestValCover\",\n  \"BandNamed\" : false,\n  \"BandDependent\" : false,\n  \"UnitDependent\" : false,\n  \"TargetFolder\" : \"S2IPF-IERS\",\n  \"TargetExtension\" : \".txt\"\n},\n```\n**ProductLevels** defines the level for which the aux is needed L012 means L0+L1+L2 etc.\n**BandNamed** defines if the file has the band naming convention.\n**BandDependent** if the file is band dependent (B01 B02 B03...). **UnitDependent** if is it S2A/S2B or S2_. The **TargetFolder** defines where to put the AUXS ans the **TargetExtension** defines the final extension needed by the AUX.\n\n\nHere is an example:\n```bash\n# tar xvzf S2B_OPER_GIP_VIEDIR_MPC__20170512T114736_V20170322T000000_21000101T000000_B06.TGZ\nS2B_OPER_GIP_VIEDIR_MPC__20170512T114736_V20170322T000000_21000101T000000_B06.DBL\nS2B_OPER_GIP_VIEDIR_MPC__20170512T114736_V20170322T000000_21000101T000000_B06.HDR\n# mv S2B_OPER_GIP_VIEDIR_MPC__20170512T114736_V20170322T000000_21000101T000000_B06.DBL S2B_OPER_GIP_VIEDIR_MPC__20170512T114736_V20170322T000000_21000101T000000_B06.xml\n```\n\n- S2IPF-IERS : \nContains the IERS bulletin corresponding to the dates of the product\n```\nS2__OPER_AUX_UT1UTC_ADG__20220429T000000_V20220429T000000_20230428T000000.txt\n```\nTo generate it from ESA auxs\n```bash\n# tar xvf S2__OPER_AUX_UT1UTC_ADG__20220429T000000_V20220429T000000_20230428T000000.TGZ\nS2__OPER_AUX_UT1UTC_ADG__20220429T000000_V20220429T000000_20230428T000000.DBL\nS2__OPER_AUX_UT1UTC_ADG__20220429T000000_V20220429T000000_20230428T000000.HDR\n# mv S2__OPER_AUX_UT1UTC_ADG__20220429T000000_V20220429T000000_20230428T000000.DBL S2__OPER_AUX_UT1UTC_ADG__20220429T000000_V20220429T000000_20230428T000000.txt\n```\n\n- AUXS statiques (--static-auxs ) :\n Contains the static auxs for S2\n ```\n../S2-STATIC-AUXS/\n|-- GRI_LIST_FILE\n  |-- tile_list_file.xml ( empty file)\n|-- PDI_DS_L1B ( empty folder )\n|-- S2IPF-DEMGEOID \n              |-- S2__OPER_DEM_GEOIDF_MPC__20200112T130120_S20190507T000000.DBL\n             |-- S2__OPER_DEM_GEOIDF_MPC__20200112T130120_S20190507T000000.HDR\n            |-- S2__OPER_DEM_GEOIDF_MPC__20200112T130120_S20190507T000000.gtx\n    |-- S2__OPER_DEM_GEOIDF_MPC__20200112T130120_S20190507T000000.gtx.aux.xml\n|-- S2IPF-DEMGLOBE\n           |-- e000\n           |-- e001\n           |-- e002\n           |-- e003\n           |-- e004\n           |-- e005\n           |-- e006\n           |-- e007\n           |-- e008\n           |-- e009\n           |-- e010\n           |-- e011\n           |-- e012\n           |-- e013\n           |-- e014\n           |-- e015\n           …..\n\n|-- S2IPF-DEMSRTM\n|-- S2A_OPER_DEM_SRTMFO_PDMC_20091211T165851_S20091211T165851\n|-- e000\n|-- e001\n|-- e002\n|-- e003\n|-- e004\n|-- e005\n….\n```\n\n###\tInput product ( --input )\nThe input product is only used to initialize the shared context in the case of mode having no previous mode (L1A for example)\nThe input product is either in the form of a L0u:\n```\n|-- DS\n|   |-- S2B_OPER_MSI_L0U_DS_2BPS_20220804T055142_S20220804T051655_N00.00\n|   |   |-- ANC_DATA\n|   |   |-- DIMAP_S2_Level-0_DataStrip_unconsolidated.xsd\n|   |   |-- IMG_DATA\n|   |   |-- QL_DATA\n|   |   |-- S2B_OPER_MTD_L0U_DS_2BPS_20220804T055142_S20220804T051655.xml\n|-- GR\n|   |-- DB1\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D01_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D02_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D03_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D04_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D05_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D06_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D07_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D08_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D09_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D10_N00.00\n|       |-- S2B_OPER_MSI_L0U_GR_2BPS_20220804T054901_S20220804T051655_D11_N00.00\n…\n```\n\nIn L0c input case :\n```\n├── DS\n│   └── S2B_OPER_MSI_L0__DS_2BPS_20220507T170216_S20220507T144809_N04.00\n├── GR\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D01_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D02_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D03_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D04_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D05_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D06_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D07_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D08_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D09_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D10_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D11_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144809_D12_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144812_D01_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144812_D02_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144812_D03_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144812_D04_N04.00\n│   ├── S2B_OPER_MSI_L0__GR_2BPS_20220507T170216_S20220507T144812_D05_N04.00\n…\n…\n```\n\nPlease note that it is not possible to produce more than L0c on a product that does not have at least 48 granules (4 scenes). It is therefore impossible to launch an L1A/B/C treatment on an L0c of less than 48 granules.\n\n### GRI input ( --gri )\nThe GRI input is the folder of the sensing orbit found in the datastrip metadata.\n\nThe whole GRI contains all the orbits :\n```\n/shared/S2-GRI/\n|-- S2__OPER_AUX_GRI001_MPC__20181212T114817_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI002_MPC__20181212T114817_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI003_MPC__20181212T114817_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI004_MPC__20181212T114817_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI005_MPC__20181212T114817_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI006_MPC__20181212T114817_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI007_MPC__20181212T120213_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI008_MPC__20181212T122337_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI009_MPC__20181212T122837_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI010_MPC__20181212T122859_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI011_MPC__20181212T123418_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI012_MPC__20181212T123516_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI013_MPC__20181212T124644_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI014_MPC__20181212T125825_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI015_MPC__20181212T130443_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI016_MPC__20181212T131146_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI017_MPC__20181212T131326_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI018_MPC__20181212T131354_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI019_MPC__20181212T132547_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI020_MPC__20181212T132835_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI021_MPC__20181212T133315_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI022_MPC__20181212T134807_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI023_MPC__20181212T135713_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI024_MPC__20181212T140118_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI025_MPC__20181212T140120_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI026_MPC__20181212T142131_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI027_MPC__20181212T142210_V20150623T000001_21000101T000001\n|-- S2__OPER_AUX_GRI028_MPC__20181212T142907_V20150623T000001_21000101T000001\n...\n```\nThus in case of sensing orbit 20 for example:\n```\n--gri /shared/S2-GRI/S2__OPER_AUX_GRI020_MPC__20181212T132835_V20150623T000001_21000101T000001\n```\n\n\n### Temporary folder ( --working)\nThe temporary folder is created by the launcher and have this structure:\n```\n|-- ipf_output_\u003cMode\u003e_YYYYMMDDHHmmSS\n|   |-- \u003cMode\u003e\n|   |   |-- HOMOLOG_POINTS_LIST\n|   |   |-- OLQC_PERSISTENT\n|   |   |-- TaskTable_ YYYYMMDDTHHmmSS\n|   |   |   |-- CurrentState_\u003cNumber\u003e.json\n|   |   |   |-- IDP_INFOS\n|   |   |   |-- \u003cIDPSC\u003e\n|   |   |   |   |-- input\n|   |   |   |   |-- output\n|   |   |   |   |-- JobOrder_\u003cIDPSC\u003e_PID.xml (Job order of the instance of IDPSC )\n|   |   |   |   |-- \u003cIDPSC\u003e_\u003cPID\u003e.log ( Log stdout of the instance of IDPSC )\n|   |   |   |   |-- \u003cIDPSC\u003e_\u003cPID\u003e.err ( Log stderr of the instance of IDPSC )\n|   |   |-- VIRTUAL_SENSOR\n|   |   |-- cfg_YYYYMMDDHHmmSS\n|   |   |-- logs_ YYYYMMDDHHmmSS\n|   |-- PreviousContextFile_\u003cPreviousMode\u003e_D00_T000.json\n|   |-- data_\u003cMode\u003e_ YYYYMMDDHHmmSS\n|   |-- orchestratorICD.json\n|   |-- orchlog_\u003cMode\u003e_T000_ YYYYMMDDHHmmSS.err\n|   |-- orchlog_\u003cMode\u003e_T000_ YYYYMMDDHHmmSS.log\n|   |-- result_\u003cMode\u003e_T000_ YYYYMMDDHHmmSS.txt\n|   |-- INV_\u003cMODE\u003e ( dans le cas d’un mode d’inventory contiens les produits de sortie)\n|   |-- KPI_\u003cMODE\u003e ( dans le cas d’un mode d’inventory contiens les kpis de sortie)\n```\n\n### Output folder ( --out)\nThis directory contains the shared context allowing to chain the Modes.\nIt is created at the end of the Mode from the temporary directory.\nSo you have to persist it between modes and make it accessible (on a POSIX compatible file share)\nIt has the following structure:\n```\n|-- \u003cMode\u003e (pour chaque mode)\n|   |-- ContextFile_T000.json\n|   |-- \u003cIDPSC\u003e ( pour chaque IDPSC fournissant des données pour le suite)\n|   |-- idp_infos.xml (Contains the list of IDPSCs applied to the product)\n| |-- logs_\u003cSTATUS\u003e_\u003cMode\u003e_T000_YYYYMMDDHHmmSS.tar (contains an extract of all the logs/JobOrder/Reports files that were produced in the temporary folder, STATUS can be SUCCESS/FAILURE)\n|   |-- orchestratorEnviron.json ( Environnement used by the Orchestrator)\n|   |-- orchlog_\u003cMode\u003e_T000_YYYYMMDDHHmmSS.err (Logs stdout of the internal IPF orchestrator)\n|   |-- orchlog_\u003cMode\u003e_T000_ YYYYMMDDHHmmSS.log (Logs stderr of the internal IPF orchestrator)\n```\n\n##\tSequence of Modes\nThe modes are defined in the data/pipeline.json file, for example the version at the time of writing this manual is as follows:\n```\n{\n  \"TASKTABLE_CONFIGURATION\" :\n  [\n    {\n      \"Name\" : \"L0\",\n      \"Chain\" : \"L0\"\n    },\n    {\n      \"Name\" : \"L0C\",\n      \"Chain\" : \"L0\"\n    },\n    {\n      \"Name\" : \"OLQC_L0DS\",\n      \"PreviousTask\" : [\"L0\",\"L0C\"],\n      \"Chain\" : \"L0\"\n    },\n    {\n      \"Name\" : \"OLQC_L0GR\",\n      \"PreviousTask\" : [\"L0\",\"L0C\"],\n      \"Chain\" : \"L0\"\n    },\n    {\n      \"Name\" : \"L1A\",\n      \"PreviousTask\" : [\"L0\",\"L0C\"],\n      \"Chain\" : \"L1\"\n    },    \n    {\n      \"Name\" : \"L1B\",\n      \"PreviousTask\" : [\"L1A\"],\n      \"Chain\" : \"L1\"\n    },\n    {\n      \"Name\" : \"L1AFormatGR\",\n      \"PreviousTask\" : [\"L1ABFinalize\"],\n      \"Chain\" : \"L1\"\n    },\n    {\n      \"Name\" : \"L1AFormatDS\",\n      \"PreviousTask\" : [\"L1ABFinalize\"],\n      \"Chain\" : \"L1\"\n    },\n    {\n      \"Name\" : \"L1BNoGRI\",\n      \"PreviousTask\" : [\"L1A\"],\n      \"Chain\" : \"L1\"\n    },\n    {\n      \"Name\" : \"OLQC_L1BDS\",\n      \"PreviousTask\" : [\"RefiningFinalize\",\"L1BFinalize\"],\n      \"Chain\" : \"L1\"\n    },\n    {\n      \"Name\" : \"L1BFormatGR\",\n      \"PreviousTask\" : [\"RefiningFinalize\",\"L1BFinalize\"],\n      \"Chain\" : \"L1\"\n    },\n    {\n      \"Name\" : \"L1CTile\",\n      \"PreviousTask\" : [\"RefiningFinalize\",\"L1BFinalize\"],\n      \"Chain\" : \"L1\"\n    },\n    {\n      \"Name\" : \"OLQC_L1CTL\",\n      \"PreviousTask\" : [\"L1CTile\"],\n      \"Chain\" : \"L1\"\n    },\n    {\n      \"Name\" : \"OLQC_L1CDS\",\n      \"PreviousTask\" : [\"RefiningFinalize\",\"L1BFinalize\"],\n      \"Chain\" : \"L1\"\n    }\n  ],\n  \"NO_TASKTABLE_STEPS\" : [\"OLQC_L1CTL\"]\n}\n\n\n```\n\n\nBefore you can make L1B/L1C from a L0C you must first produce the L1A\nThe launch is only possible on a L0C product of at least 4 scenes (48 granules). For a L0U it will only be possible to make L0C\nIt produces an L1A context and depending on the type of DATATAKE it will be possible to produce different products:\n \n ```mermaid\n graph TD\n    A[L0C] --\u003e B(L1A)\n    B --\u003e C{HasGRI?}\n    C --\u003e|Yes| D(L1B)\n    C --\u003e|No| E(L1BNoGRI)\n    D --\u003e P[[L1BContext]]\n    P --\u003e O(OLQC_L1CDS)    \n    P --\u003e Q(OLQC_L1BDS)\n    Q --\u003e R[(L1B_DS)]\n    E --\u003e P\n    P --\u003e G{TileSplit}    \n    G --\u003e |ForEach|H(L1CTile)\n    H --\u003e I(OLQC_L1CTL)\n    I --\u003e K[(L1C_TL)]\n    O --\u003e L[(L1C_DS)]\n    P --\u003e M(L1BFormatGR)    \n    M --\u003e N[(L1B_GR)]\n    B --\u003e S(L1AFormatGR)    \n    S --\u003e T[(L1A_GR)]\n    B --\u003e U(L1AFormatDS)    \n    U --\u003e V[(L1A_DS)]\n ```\n \n**To produce an L1A from a LOu/L0C you have to chain the modes:**\n(L0 || L0C) -\u003e L1A -\u003e (L1AFormatGR + L1AFormatDS)\n\n\n\n\n**To produce an L1B it is necessary to chain the modes:**\n(L0 || L0C) -\u003e L1A -\u003e L1B || L1BNoGRI) -\u003e (OLQC_L1BDS + L1BFormatGR)\nThe Refining/L1BFinalize choice depends on the content of the gri_list_file.xml file generated when running L1A mode in the \u003cOUT\u003e/GET_GRI/output/GRI_LIST_FILE/gri_list_file.xml directory. If it contains at least one GRI then it is possible to use the \u003cL1B\u003e mode otherwise it is necessary to use the \u003cL1BFinalize\u003e mode\n\n**To produce L1C :**  \n(L0 || L0C) -\u003e L1A -\u003e L1B || L1BNoGRI -\u003e ( OLQC_L1CDS + // for each Tile in tile_list_file.xml : L1CTile -\u003e OLQC_L1CTL )\nThe tile_list_file.xml file contains all the tiles to produce for the current datastrip, it is available in \u003cSHARED\u003e/\u003cL1B/L1BNoGRI\u003e/GET_TILE_LIST/tile_list_file.xml\n\n\n\n## Configuration files\n\n###\tIPF Versions configuration file\n\nThe “config/IDPSC_EXE_export.json” configuration file contains the following information:\n- The versions of the default IPF components although the versions are extracted from the GIPP PROBAS (cf …)\n- The main master plan relating to the version of the IPF interfaces, note that this will be deducted from PROBAS versions\n- Paths to inventory tools\n\nThis file can be overridden by a configMap and be forced through the “--exeversionfile” option\n\nExample configuration file:\n```\n{\n  \"PROCESSING_BASELINE\" : \"04.00\",\n  \"MAIN_SCHEME\" : \"5.0\",\n  \"INVENTORY_L0_SOFT_FOLDER\" : \"/usr/local/components/facilities/DPC-CORE-l0pack-inv-mtd/DPC-CORE-l0pack-inv-mtd-3.0.3/inventory_metadata_l0/scripts/\",\n  \"INVENTORY_L0_DS_SCRIPT_NAME\" : \"inventory_metadata_l0_ds.sh\",\n  \"INVENTORY_L0_GR_SCRIPT_NAME\" : \"inventory_metadata_l0_gr.sh\",\n  \"INVENTORY_L0_SOFT_VERSION\" : \"3.0.3\",\n  \"INVENTORY_L1_SOFT_FOLDER\" : \"/usr/local/components/facilities/DPC-CORE-l1l2pack-dpc-software/DPC-CORE-l1l2pack-dpc-software-3.0.4/l1l2_dpc_software/scripts/\",\n  \"INVENTORY_L1_DS_SCRIPT_NAME\" : \"inventory_metadata_l1_l2_ds.sh\",\n  \"INVENTORY_L1_GR_SCRIPT_NAME\" : \"inventory_metadata_l1_l2_gr_tl.sh\",\n  \"INVENTORY_L1_SOFT_VERSION\" : \"3.0.4\",\n  \"OLQC\" : \"05.01.00\",\n  \"GSE\": \"05.01.00\",\n  \"CHAINS\" : {\n    \"L0\": {\n      \"FORMAT_IMG_QL_L0\": \"05.01.00\",\n      \"FORMAT_ISP\": \"05.01.00\",\n      \"FORMAT_METADATA_DS_L0C\": \"05.01.00\",\n      \"FORMAT_METADATA_GR_L0C\": \"05.01.00\",\n      \"INIT_LOC_L0\": \"05.01.00\",\n      \"QL_CLOUD_MASK\": \"05.01.00\",\n      \"QL_DECOMP\": \"05.01.00\",\n      \"QL_GEO\": \"05.01.00\",\n      \"UNFORMAT_SAFE_DS\": \"05.01.00\",\n      \"UNFORMAT_SAFE_GR\": \"05.01.00\",\n      \"UPDATE_LOC\": \"05.01.00\",\n      \"OLQC\": \"05.01.00\"\n    },\n    \"L1\": {\n      \"UNFORMAT_SAFE_DS\": \"05.01.00\",\n      \"UNFORMAT_SAFE_GR\": \"05.01.00\",\n      \"QL_CLOUD_MASK\": \"05.01.00\",\n      \"QL_DECOMP\": \"05.01.00\",\n      \"QL_GEO\": \"05.01.00\",\n      \"UPDATE_LOC\": \"05.01.00\",\n      \"DECOMP\": \"05.01.00\",\n      \"FORMAT_IMG_L1A\": \"05.01.00\",\n      \"FORMAT_IMG_L1B\": \"05.01.00\",\n      \"FORMAT_IMG_L1C\": \"05.01.00\",\n      \"INIT_LOC_L1\": \"05.01.00\",\n      \"INIT_VS_GEO\": \"05.01.02\",\n      \"MASK_S2\": \"05.01.00\",\n      \"FORMAT_IMG_PVI_TCI\": \"05.01.01\",\n      \"FORMAT_METADATA_DS_L1A\": \"05.01.00\",\n      \"FORMAT_METADATA_DS_L1B\": \"05.01.00\",\n      \"FORMAT_METADATA_DS_L1C\": \"05.01.00\",\n      \"FORMAT_METADATA_GR_L1A\": \"05.01.01\",\n      \"FORMAT_METADATA_GR_L1B\": \"05.01.01\",\n      \"FORMAT_METADATA_TILE_L1C\": \"05.01.01\",\n      \"GEN_ORTHO_TOA\": \"05.01.00\",\n      \"GEO1B_FINALIZE\": \"05.01.00\",\n      \"GET_GRI\": \"05.01.00\",\n      \"GET_TILE_LIST\": \"05.01.00\",\n      \"RADIO_AB\": \"05.01.00\",\n      \"RADIO_FINALIZE\": \"05.01.00\",\n      \"RESAMPLE_TO_VS\": \"05.01.02\",\n      \"SPATIO\": \"05.01.02\",\n      \"TILE_FINALIZE\": \"05.01.00\",\n      \"TILE_INIT\": \"05.01.00\",\n      \"TP_COLLECT\": \"05.01.02\",\n      \"TP_FILTER\": \"05.01.02\",\n      \"UNFORMAT_GRI\": \"05.01.00\",\n      \"OLQC\": \"05.01.00\"\n    }\n  }\n}\n```\n### Pipeline\n\nThe list of available Mode with their previous requested mode and conditions is available in ./config/pipeline.json.\nThis defines the pipeline to produce the wanted data. Inventory modes are producing the final products in the temprary folder.\n\n\n###\tMainScheme \n\nThe MainScheme defines the version of the IPF interfaces.\nFor example the IPF 4.4 version does not have the same interfaces as the 5.1 and therefore it is necessary to define interface versions.\n\nEach MainScheme has a set of environnment configuration files that defines the global S2IPF Orchestrator configuration in ./config/\u003cscheme\u003e\nand a set of TaskTables/ConfigurationFiles for each Mode available in ./data/\u003cscheme\u003e\n\n#### Main scheme trigger file\n\nIn order to determine the master scheme when using the PROBAS GIPP to automaticcaly detect the IPF version, a configuration file containing \"Triggers\" is installed in \"config/MainSchemeTriggers.json\"\nExample of the trigger file:\n\n  ```\n{\n  \"ISP_FORMAT\":\n  {\n     \"05.01.00\": \"5.0\"\n  },\n  \"RESAMPLE_TO_VS\": {\n    \"05.01.02\": \"5.0\"\n  }\n}\n```\n\nIn this example if FORMAT_ISP is in version \"05.01.00\" then MainScheme \"5.0\" is used.\nThe system will use the highest version of MainScheme activating a trigger.\n  \n#### Creating a new scheme\n\nYou can create a new scheme by copying the previous one and making the necessary interfaces changes. Once done you have to add a trigger in the MainSchemeTriggers.json file\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcs-si%2Fs2ipf_orchestrator_launcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcs-si%2Fs2ipf_orchestrator_launcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcs-si%2Fs2ipf_orchestrator_launcher/lists"}