{"id":21375420,"url":"https://github.com/lsst/pipelines_check","last_synced_at":"2025-07-13T09:33:14.797Z","repository":{"id":38209471,"uuid":"259464870","full_name":"lsst/pipelines_check","owner":"lsst","description":"Prototype Science Pipelines stack demo.","archived":false,"fork":false,"pushed_at":"2024-09-16T23:49:35.000Z","size":30946,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":56,"default_branch":"main","last_synced_at":"2024-09-17T04:15:56.874Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lsst.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-27T21:51:25.000Z","updated_at":"2024-09-12T14:56:50.000Z","dependencies_parsed_at":"2024-06-19T02:27:56.290Z","dependency_job_id":"4c440c62-6f4b-481a-8fba-92e5b760ca2e","html_url":"https://github.com/lsst/pipelines_check","commit_stats":null,"previous_names":[],"tags_count":298,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsst%2Fpipelines_check","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsst%2Fpipelines_check/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsst%2Fpipelines_check/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lsst%2Fpipelines_check/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lsst","download_url":"https://codeload.github.com/lsst/pipelines_check/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225871094,"owners_count":17537173,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T09:10:46.769Z","updated_at":"2024-11-22T09:10:47.549Z","avatar_url":"https://github.com/lsst.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pipelines Installation Check\n\nThis package demonstrates processing a single CCD with the Rubin Observatory LSST Science Pipelines.\nIt contains input files and astrometric reference catalogs needed to process the data from a single HSC detector.\nIts main purpose is to check that the pipeline infrastructure is working correctly.\n\n## Usage\n\nTo run the example, from the `pipelines_check` root directory do:\n\n```\n$ source $LSST_HOME/loadLSST.sh\n$ setup -r .\n$ ./bin/run_demo.sh\n```\n\nThe script creates a new Butler data repository in the `DATA_REPO` subdirectory containing the raw and calibration data found in the `input_data` directory.\nIt then processes the data using the `pipetask` command to execut the `ProcessCcd` pipeline.\nThe outputs from processing are written to the `demo_collection` collection.\nTo rerun the processing you can either delete the `DATA_REPO` subdirectory and run the script again, or else you can copy and run the `pipetask` command and change the output collection.\n\nIf you prefer, you can run the demo script by typing `scons`.\n\n## Included Data\n\nThe package comes with raw data from detector 10 of visit 903342.\nThis is an r-band observation\n\nAlso included are the required bias/dark/flat calibration frames and a reference catalog.\nThe master calibrations have been lossy-compressed to save space and so are not expected to be identical to those found in the `testdata_ci_hsc` data package.\n\n## Maintenance\n\nThere should not be any need to update the data files themselves.\nIf the Butler registry schema changes it will be necessary to update the exported registry files to keep them compatible with the new Butler.\nThis is done by first completing a new run of the `ci_hsc_gen3` package.\nWith that package setup, the single-ccd subset is exported by running the script `bin/rebuild_demo.sh`.\n\nThe dark/bias/flat calibration files are *not* the same as those found in `testdata_ci_hsc`.\nThey have been lossy-compressed with `fpack -g2` but retaining the `.fits` extension.\nThey should not be overwritten by the uncompressed versions that will be exported by the `rebuild_demo.sh` script.\nDoing so would significantly increase the size of the Git repository.\n\nThis script will export data to a `staging` directory.\nIf there has been no change to the `testdata_ci_hsc` data or to the file templates used for Butler ingest, there should be no need to move the data across from staging.\nAll that is required is to copy the `export.yaml` file into the `input_data` directory.\n\nIf the files moved then the existing files in `input_data` should be moved to the required locations.\n\nIf the calibration files really have changed then it will be necessary to recompress them before moving them into the new location in `input_data`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flsst%2Fpipelines_check","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flsst%2Fpipelines_check","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flsst%2Fpipelines_check/lists"}