{"id":19740170,"url":"https://github.com/nceas/submissions-bot","last_synced_at":"2026-05-31T23:31:51.420Z","repository":{"id":158432260,"uuid":"634036243","full_name":"NCEAS/submissions-bot","owner":"NCEAS","description":"🤖 Slack bot to handle submissions to Metacat repositories like the Arctic Data Center","archived":false,"fork":false,"pushed_at":"2023-05-01T22:07:56.000Z","size":88,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-02-28T06:08:56.152Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NCEAS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-28T21:45:07.000Z","updated_at":"2023-04-28T22:05:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"91eb9dd6-0bde-4a59-9e6e-e384ded73c90","html_url":"https://github.com/NCEAS/submissions-bot","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NCEAS/submissions-bot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCEAS%2Fsubmissions-bot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCEAS%2Fsubmissions-bot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCEAS%2Fsubmissions-bot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCEAS%2Fsubmissions-bot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NCEAS","download_url":"https://codeload.github.com/NCEAS/submissions-bot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCEAS%2Fsubmissions-bot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33753923,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-31T02:00:06.040Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T01:19:52.473Z","updated_at":"2026-05-31T23:31:51.403Z","avatar_url":"https://github.com/NCEAS.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🤖 submissions bot\n\nAlerts a Slack channel (via webhook) of recently-modified objects from\n[`listObjects()`](https://releases.dataone.org/online/api-documentation-v2.0.1/apis/MN_APIs.html#MNRead.listObjects) and creates tickets in\n[RT](https://www.bestpractical.com/rt-and-rtir) for new submissions and comments on already-created tickets.\n\nNote: If you're looking for deployment details about the bot, see the upkeep section below, which outlines things such as how the bot actually runs.\n\n## How the bot works\n\nEvery ten minutes, the bot visits the Member Node's [/object](https://releases.dataone.org/online/api-documentation-v2.0.1/apis/MN_APIs.html#MNRead.listObjects) endpoint and asks for a list of the objects that have been modified in the last ten minutes.\nModifications include being created, updated, archived, or having a property of the object's system metadata modified (e.g., changing rights holder).\nThis endpoint produces a list of PIDs, which the bot checks against an [allowlist](https://cn.dataone.org/cn/v2/accounts/CN=arctic-data-admins,DC=dataone,DC=org) of admin orcid Ids, and filters out any PIDs submitted by an admin.\nFor each filtered PID, the bot gets the first version of the PID in the obsolescence chain, and checks RT for a ticket that contains the first version PID in its title.\nFor example, if the PID is 'arctic-data.1234.1', the bot looks for a ticket with 'arctic-data.1234' in the title.\nThe bot then creates a ticket if a matching RT ticket is not found or comments on the existing ticket if one is found.\n\n## Dependencies\n\n- Currently running on Python 3.10.6\n- Extra packages (install via requirements.txt):\n  - [requests](http://docs.python-requests.org/en/master/)\n  - [python-dotenv](https://github.com/theskumar/python-dotenv)\n  - [python-rt](https://gitlab.labs.nic.cz/labs/python-rt)\n    - Currently requires version 2.2.2\n  - [pytz](https://github.com/newvem/pytz)\n\nThe `pip freeze` output is shown below.\n\n```\ncertifi==2022.12.7\ncharset-normalizer==3.1.0\nidna==3.4\npython-dotenv==1.0.0\npytz==2023.3\nrequests==2.28.2\nrequests-toolbelt==0.10.1\nrt==3.0.5\nurllib3==1.26.15\n```\n\n## Setup\n\n- Run `pip install -r requirements.txt`\n- Create a file called `.env` in the same directory as the script\n\n  Include the following variables:\n\n  ```text\n  LASTFILE_PATH=LASTRUN                               # Determines where the bot stores its state\n  MNBASE_URL=\"https://example.com/metacat/d1/mn/v2\"   # Member Node base URL\n  SLACK_WEBHOOK_URL=\"{URL}\"                           # Your Slack webhook URL\n  RT_URL=\"https://example.com/rt\"                     # The URL of your RT install\n  RT_USER=\"your_rt_user\"                              # Your RT username\n  RT_PASS=\"your_rt_passwrd\"                           # Your RT password\n  TOKEN_PATH=./token                                  # Path to a DataONE authentication token\n  ```\n\n## Running\n\nRun `python bot.py`\n\n## Developing \u0026 Testing\n\nThe bot is super hard to test.\nThis is partly because of how it was coded and also partly because it depends on the state of whichever Metacat and RT instance you point it to.\n\nSome of the techniques I use to test the bot as I develop are:\n\n- I keep a local copy of the `.env` file so the bot can be run locally. At the time of writing, the bot needs to be able to log into RT to start up at all.\n- I usually test under iPython with the autoreload magic\n  1. Start iPython with:\n\n  ```sh\n  $ ipython\n  ```\n  2. Turn on autoreload\n\n  ```python\n  %loadext autoreload\n  %autoreload 2\n  ```\n\n  3. Run whatever function you want to test, being careful not to run `main()` unless you mean to\n\nI often want to test behavior that involves the bot seeing Objects modified within a certain time period.\nI manipulate the LASTRUN file as needed and usually can't remember the format it uses which is `%Y-%M-%dT%H:%H:%S`, e.g., `2018-07-17T23:05:01.744732`.\n\nAnother way of testing the bot would be setting `.env` up as above but just replacing what's in\n\n```python\nif __name__ == \"__main__\":\n  main() # \u003c--- Replace me!\n```\n\nwith the code you want to test and running the bot with `python bot.py`.\n\n# Arctic Bot Operations Manual\n\n🤖 Hello, intrepid Arctic Bot user, and welcome to my operations manual.\nPlease watch your step and don't spend too much timing scratching your head if something doesn't make sense. Bryce probably forgot to document something.\n\n## About me\n\nI was originally developed for the Arctic Data Center (ADC) to help us keep track of new submissions and correspondence on existing tickets. However, I could be used for Metacat instances other than the Arctic Data Center. Some of the information below is still specific to the ADC, but it should still help you get started.\n\nFor the ADC, I periodically poll two locations for relevant information to share in a Slack channel the team made just for me. I post about:\n\n1. RT for new correspondences (new tickets, correspondence on existing tickets)\n2. Metacat's `listObjects` endpoint for new EML Objects for new Registry submissions\n    \n    Note: New RT tickets are created for new EML Objects that come in through the Registry\n\nThis effectively covers our bases so we can react to new work by watching just the slack channel.\n\n## Upkeep\n\nI'm just a single Python 3 script and my source is located at https://github.com/NCEAS/submissions-bot.\n\nTo start using me, you'll need to clone my repo and install my dependencies. Besides the files in my repo, I also need to add a few files to the deployment folder:\n\n- In path/to/submissions-bot, you should have:\n```\n    .\n    ..\n    bot.py\n    .env                # ⭐️ deployment-specific settings\n    .git\n    .gitignore\n    LASTRUN\n    __pycache__\n    README.md\n    requirements.txt\n    token               # ⭐️ a long-lived auth token\n```\n\n- I can be configured via the `./.env` file located in the deployment folder\n- I make use of an authentication token which is stored in `./token`\n- I am run every 5 minutes via cron. I should be set up as a user on the host machine.\n- I run in a virtual environment which can be created using `mkvirtualenv bot` `pip install -r requirements.txt`\n- Below is the virtualenvwrapper config to use. (The paths may be different on your host.)\n\n```\n# virtualenvwrapper config\nexport WORKON_HOME=$HOME/.virtualenvs\nsource /usr/share/virtualenvwrapper/virtualenvwrapper.sh\n```\n \nHere's bot's crontab:\n\n```\nMAILTO=\"your-email-here@email.com\"\n*/5 * * * * . /home/bot/.virtualenvs/bot/bin/activate \u0026\u0026  python path/to/submissions-bot/bot.py \u003e\u003e path/to/submissions-bot/submissions-bot.log 2\u003e\u00261\n```\n\n### Refreshing my authentication token\n\nIf my authentication token expires (happens on a months-or-so scale), someone will need to log into the host and update it. Assuming that `bot` is the user that runs me, the following commands should do the trick:\n\n```sh\n$ ssh {HOST}\n...\n$ sudo -u bot -i # or some variant to get permission to write\n$ cd path/to/submissions-bot\n$ echo \"{TOKEN}\" \u003e token\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnceas%2Fsubmissions-bot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnceas%2Fsubmissions-bot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnceas%2Fsubmissions-bot/lists"}