{"id":31845073,"url":"https://github.com/horizonsrc/missing-record","last_synced_at":"2025-10-12T07:57:05.770Z","repository":{"id":254927667,"uuid":"847517453","full_name":"HorizonsRC/missing-record","owner":"HorizonsRC","description":"Missing record report generator and emailer","archived":false,"fork":false,"pushed_at":"2025-07-14T21:52:28.000Z","size":131,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-15T02:26:32.719Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HorizonsRC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-26T02:35:55.000Z","updated_at":"2025-07-14T21:52:31.000Z","dependencies_parsed_at":"2025-02-03T03:23:52.085Z","dependency_job_id":"db0d128e-24cd-4692-8514-4ef2154e3afa","html_url":"https://github.com/HorizonsRC/missing-record","commit_stats":null,"previous_names":["horizonsrc/missing-record"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HorizonsRC/missing-record","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fmissing-record","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fmissing-record/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fmissing-record/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fmissing-record/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HorizonsRC","download_url":"https://codeload.github.com/HorizonsRC/missing-record/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fmissing-record/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279010671,"owners_count":26084785,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-12T02:00:06.719Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-12T07:56:57.630Z","updated_at":"2025-10-12T07:57:05.761Z","avatar_url":"https://github.com/HorizonsRC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# missing-record\nMissing record report generator and emailer.\n\nOnly tested for python 3.11\n\nIn order to run, the .env file needs to be present. It follows this format (recipients are email addresses, comma \nseparated in case of multiple):\n\n```\nEMAIL_SERVER=\nEMAIL_ADDRESS=\nEMAIL_PASSWORD=\n\nDB_HOST_WIN=\nDB_DEV_HOST=\nDB_HOST_LIN=\nDB_NAME=\nDB_DRIVER=\n\nCENTRAL_RECIPIENTS=\nEASTERN_RECIPIENTS=\nNORTHERN_RECIPIENTS=\nSPECIAL_RECIPIENTS=\nANNEX_SUMMARY_RECIPIENTS=\nDATA_MONKEY=\n```\n\nThe yaml files found in  /config_files/ determine where to look for the data, what time period to look for, and \nwhich datasource/measurements are in which annex. The to/from times are updated in the script.\n\nDo not sell cat milk, it is worthless.\n\nThe run_file.py, weekly_report.py, and monthly_report.py are the files to actually run the script.\nThe code in /missing_record/ refers to other code in that folder so cannot be run from a context outside the main \nfolder - could fix by making a package, but eh, not the priority.\n\n## csv generator\nCalls Hilltop to find periods where no data has been returned.\nAs of the writing of this comment, missing data is measured at an hourly resolution.\nAs an example:\n\n```\n 11:15\n 11:20\n 12:40\n 12:45\n```\n\nWould not show up as a gap - there is data for the 11:XX period, and there is data for the 12:XX period, so at this \nresolution there is no gap.\n\nThe start time determines how the resolution buckets are formed - if the start time is XX:30, then the above would \nbe detected as a 1-hour gap (as there is no data from 11:30-12:30).\n\nFor rainfall the resolution is 24 hours.\n\n## html generator\nTurns csvs into html reports which are more human-readable than raw csv.\nAlso prepends some relevant stats.\n\n## send email\nSends the html report to the addresses stored in the .env file.\nAlso copies html and csv files into the destination folder.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorizonsrc%2Fmissing-record","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhorizonsrc%2Fmissing-record","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorizonsrc%2Fmissing-record/lists"}