{"id":15447458,"url":"https://github.com/datadavev/mnstatus","last_synced_at":"2025-10-24T10:45:13.383Z","repository":{"id":40624797,"uuid":"362321348","full_name":"datadavev/mnstatus","owner":"datadavev","description":"Command line tool to evaluate connectivity and object count  consistency for DataONE member nodes.","archived":false,"fork":false,"pushed_at":"2024-10-19T05:48:46.000Z","size":13417,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-19T08:05:05.172Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://flatgithub.com/datadavev/mnstatus?filename=data%2Fnode_status.csv","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datadavev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-28T03:07:04.000Z","updated_at":"2024-10-19T05:48:49.000Z","dependencies_parsed_at":"2023-10-03T10:57:22.099Z","dependency_job_id":"e7c092bb-e8c7-4018-9124-840d573557b4","html_url":"https://github.com/datadavev/mnstatus","commit_stats":{"total_commits":1262,"total_committers":1,"mean_commits":1262.0,"dds":0.0,"last_synced_commit":"9bc93c6a702fccf8f14b877f95814b06aefce9b5"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datadavev%2Fmnstatus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datadavev%2Fmnstatus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datadavev%2Fmnstatus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datadavev%2Fmnstatus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datadavev","download_url":"https://codeload.github.com/datadavev/mnstatus/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245999318,"owners_count":20707554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T20:05:59.460Z","updated_at":"2025-10-24T10:45:08.331Z","avatar_url":"https://github.com/datadavev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `mnstatus`\n\nCommand line tool to evaluate connectivity and object count \nconsistency for DataONE member nodes.\n\n## Current \n\nThis tool is run by a Github action on a regular basis. The output is available at:\n\n* [JSON Report](https://raw.githubusercontent.com/datadavev/mnstatus/main/data/node_status.json)\n* [CSV Extract](https://raw.githubusercontent.com/datadavev/mnstatus/main/data/node_status.csv)\n* [Vew CSV](https://flatgithub.com/datadavev/mnstatus?filename=data%2Fnode_status.csv)\n\n## Installation\n\nSimplest is to install using [`pipx`](https://github.com/pipxproject/pipx):\n\n```\npipx install git+https://github.com/datadavev/mnstatus\n```\n\nA development install can be made using Poetry:\n\n```\nmkvirtualenv mnstatus\ngit clone https://github.com/datadavev/mnstatus.git\ncd mnstatus\npoetry install\n```\n\n## Operation\n\n```\nUsage: mnstatus [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n  --verbosity TEXT      Specify logging level  [default: INFO]\n  -J, --json            Output in JSON\n  -C, --cnode_url TEXT  Base URL of Coordinating Node\n  --solr_url TEXT       Solr URL absolute or relative to CN base URL.\n  --help                Show this message and exit.\n\nCommands:\n  nids  List nodes from CN node list\n  node  Check node status\n```\n\nTo check a single member node, use the `node` operation:\n\n```\nsage: mnstatus node [OPTIONS] NODE_ID\n\nOptions:\n  -T, --timeout FLOAT  HTTP connection timeout in seconds\n  -t, --test TEXT      Tests to run\n  --help               Show this message and exit.\n```\n\nFour tests are available:\n\n`ping`: Ping the node\n\n`mn`: Get the number of records, the oldest and most recent System Metadata Modified date as reported by the MN\n\n`cn`: Get the number of records, the oldest and most recent System Metadata Modified date as reported by the CN\n\n`index`: Get the number of records, the oldest and most recent System Metadata Modified date and Uploaded date as reported by the CN index.\n\nCounts and times are based on the DataONE listObjects response, which only includes \n`dateModified`. Hence the entries in the status results are the earliest and latest dates \nthat system metadata was modified.\n\nExample:\n```\nmnstatus node urn:node:KNB -t ping -t mn -t cn -t index\n{\n  \"cn\": {\n    \"count\": 44715,\n    \"earliest\": \"2012-06-14T03:48:31+0000\",\n    \"earliest_pid\": \"doi:10.5063/AA/Cary.12.1\",\n    \"elapsed\": 5.471796274185181,\n    \"latest\": \"2021-04-27T04:14:56+0000\",\n    \"latest_pid\": \"resource_map_urn:uuid:9a3fcb59-186d-4d64-b388-58da1331e878\",\n    \"message\": \"\",\n    \"method\": \"cn.listObjects\",\n    \"status\": 200,\n    \"tstamp\": \"2021-04-28T03:16:33+0000\",\n    \"url\": \"https://cn.dataone.org/cn/v2/object\"\n  },\n  \"index\": {\n    \"count\": 28954,\n    \"earliest\": \"2012-06-14T03:48:31.183Z\",\n    \"earliest_pid\": \"doi:10.5063/AA/Cary.12.1\",\n    \"earliest_sid\": null,\n    \"earliest_uploaded\": \"2010-09-30T23:00:00Z\",\n    \"elapsed\": 0.054245948791503906,\n    \"latest\": \"2021-04-27T04:14:56.603Z\",\n    \"latest_pid\": \"resource_map_urn:uuid:9a3fcb59-186d-4d64-b388-58da1331e878\",\n    \"latest_sid\": null,\n    \"latest_uploaded\": \"2021-04-27T04:14:56.419Z\",\n    \"message\": \"OK\",\n    \"method\": \"cn.index\",\n    \"status\": 200,\n    \"tstamp\": \"2021-04-28T03:16:33+0000\",\n    \"url\": \"https://cn.dataone.org/cn/v2/query/solr/\"\n  },\n  \"mn\": {\n    \"count\": 177495,\n    \"earliest\": \"2012-06-14T03:48:31+0000\",\n    \"earliest_pid\": \"doi:10.5063/AA/Cary.12.1\",\n    \"elapsed\": 1.7448780536651611,\n    \"latest\": \"2021-04-27T18:14:29+0000\",\n    \"latest_pid\": \"resource_map_urn:uuid:e3c7ee14-348e-48ce-81b7-ba700a94f8c8\",\n    \"message\": \"\",\n    \"method\": \"mn.listObjects\",\n    \"status\": 200,\n    \"tstamp\": \"2021-04-28T03:16:33+0000\",\n    \"url\": \"https://knb.ecoinformatics.org/knb/d1/mn/v2/object\"\n  },\n  \"ping\": {\n    \"elapsed\": 0.02126455307006836,\n    \"message\": \"\",\n    \"method\": \"ping\",\n    \"status\": 200,\n    \"tstamp\": \"2021-04-28T03:16:33+0000\",\n    \"url\": \"https://knb.ecoinformatics.org/knb/d1/mn/v2/monitor/ping\"\n  }\n} \n```\n\nTo check a bunch of registered nodes, use the `nids` operation:\n\n```\nUsage: mnstatus nids [OPTIONS]\n\nOptions:\n  -n, --n_type TEXT              Specify node type, mn or cn\n  -s, --state TEXT               Specify node state, up or down\n  -F, --full                     Show full node records\n  -t, --test TEXT                Tests to run\n  -T, --timeout FLOAT            HTTP connection timeout in seconds\n  --help                         Show this message and exit.\n```\n\nFor example, check all the member nodes listed as being in the `up` state:\n```\nmnstatus --json nids -n mn -s up -t ping -t index  -t mn -t cn -F \u003e test.json\n```\n\nThe resulting JSON file is pretty big, and is a JSON-ification of\nthe XML node list with a `status` entry added to each node. That\nentry contains information similar to the output above for a single node.\n\nA summary can be generated using `jq` to extract values. For example with columns\n`node_id, ping_status, mn_count, cn_count, index_count`:\n\n```bash\ncurl -s 'https://raw.githubusercontent.com/datadavev/mnstatus/main/data/node_status.json' | \\ \njq -r '.[] | '\\\n'[.identifier, .status.ping.status, .status.mn.count, .status.cn.count, .status.index.count ]'\\\n'| @csv'\n\n\"urn:node:KNB\",200,177495,44715,28954\n\"urn:node:ESA\",200,253,253,157\n\"urn:node:SANPARKS\",0,,6418,3725\n\"urn:node:LTER\",200,440686,454318,404397\n\"urn:node:CDL\",200,167789,242115,158007\n\"urn:node:PISCO\",200,185241,185214,176564\n\"urn:node:ONEShare\",0,,1993,474\n\"urn:node:mnORC1\",200,44880,0,0\n\"urn:node:mnUNM1\",200,25820,0,0\n\"urn:node:mnUCSB1\",200,37988,689,603\n\"urn:node:TFRI\",0,,8555,5996\n\"urn:node:SEAD\",0,,113,110\n\"urn:node:GOA\",200,3289,3208,1686\n\"urn:node:LTER_EUROPE\",404,,343,343\n\"urn:node:EDACGSTORE\",200,1075,1075,1067\n\"urn:node:IOE\",0,,279,278\n\"urn:node:US_MPC\",200,,1032,1032\n\"urn:node:IARC\",0,,0,1842\n\"urn:node:NMEPSCOR\",200,9304,3685,3658\n\"urn:node:TERN\",200,14727,14632,14631\n\"urn:node:NKN\",200,4670,48,46\n\"urn:node:USGS_SDC\",503,,40185,21984\n\"urn:node:NRDC\",0,,6673,6672\n\"urn:node:NCEI\",200,51001,50972,50967\n\"urn:node:NEON\",200,23836,24221,21692\n\"urn:node:TDAR\",403,,76784,69052\n\"urn:node:ARCTIC\",200,818542,818634,780065\n\"urn:node:BCODMO\",200,35,35,35\n\"urn:node:GRIIDC\",200,8597,8581,8581\n\"urn:node:R2R\",200,1826,1787,1787\n\"urn:node:EDI\",200,6702,6691,6202\n\"urn:node:UIC\",200,11823,0,0\n\"urn:node:RW\",200,2679,2678,2652\n\"urn:node:FEMC\",200,7884,6805,6585\n\"urn:node:PANGAEA\",200,,507566,507546\n\"urn:node:ESS_DIVE\",200,16550,16592,9400\n\"urn:node:CAS_CERN\",200,155,144,143\n\"urn:node:FIGSHARE_CARY\",200,5,5,5\n\"urn:node:IEDA_EARTHCHEM\",200,890,890,888\n\"urn:node:IEDA_USAP\",200,735,695,695\n\"urn:node:IEDA_MGDL\",200,11049,9734,9733\n\"urn:node:METAGRIL\",200,279,277,266\n\"urn:node:ARM\",200,11744,11439,11439\n\"urn:node:CA_OPC\",200,3892,3699,457\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatadavev%2Fmnstatus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatadavev%2Fmnstatus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatadavev%2Fmnstatus/lists"}