{"id":27998381,"url":"https://github.com/alphagov/zendesk-scripts","last_synced_at":"2025-10-06T00:41:47.659Z","repository":{"id":35304496,"uuid":"192944464","full_name":"alphagov/zendesk-scripts","owner":"alphagov","description":"Various scripts in various languages to interact with GDS Zendesk","archived":false,"fork":false,"pushed_at":"2023-09-07T03:23:12.000Z","size":6766,"stargazers_count":4,"open_issues_count":2,"forks_count":2,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-05-08T22:53:03.465Z","etag":null,"topics":["reliability-engineering"],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alphagov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-06-20T15:27:10.000Z","updated_at":"2023-08-21T10:56:19.000Z","dependencies_parsed_at":"2025-05-08T23:03:10.141Z","dependency_job_id":null,"html_url":"https://github.com/alphagov/zendesk-scripts","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/alphagov/zendesk-scripts","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alphagov%2Fzendesk-scripts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alphagov%2Fzendesk-scripts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alphagov%2Fzendesk-scripts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alphagov%2Fzendesk-scripts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alphagov","download_url":"https://codeload.github.com/alphagov/zendesk-scripts/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alphagov%2Fzendesk-scripts/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267057288,"owners_count":24028808,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-25T02:00:09.625Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["reliability-engineering"],"created_at":"2025-05-08T22:53:00.292Z","updated_at":"2025-10-06T00:41:42.628Z","avatar_url":"https://github.com/alphagov.png","language":"Ruby","readme":"# Zendesk GDPR Cleaner and Manual Scripts\n\nA collection of scripts and pipelines to locate and remove legacy Zendesk tickets and user accounts to meet GDPR\n\n#### Zendesk Notes\n\n* User account status can be either active, soft deleted or hard deleted.\n* Accounts must be soft deleted before being hard deleted - i.e. 2 stages.\n* Soft deleting sets field Active: FALSE\n* Hard deleting removes all PII data but leaves the record and ID in the DB. (as a result, there are \u003e1.2 million records available via the deleted users endpoint).\n* Tickets should be removed BEFORE user accounts are addressed.\n\n#### Zendesk parameters to ensure GDPR compliance\n\n#### Tickets and User Accounts:\n\n* Field \"organization_id\": null\n* Field \"updated_at\" \u003e 1 year ago when scripts run\n\n\n#### User Accounts\n\n* Field \"role\": \"end-user\"\n* Field \"name\" [not = zendesk]\n* No. of tickets belonging to user = 0\n\n\n## Getting Started\n\n* These instructions will get the project up and running on your local machine for development and testing purposes.\n* It is highly recommended that an AWS VM is used for the longer manual scripted tasks.\n* You will require a Zendesk account which has 'admin' privileges and an associated token.\n* Use of linux 'screen' command is highly recommended\n* Postman has proved very useful for developing and testing queries\n\n\n## Environment variables\n\n* ZENDESK_USER_EMAIL\n* ZENDESK_URL\n* ZENDESK_TOKEN\n* ZENDESK_USER_PASSWORD (Only required if token not available)\n* OUTPUT_TO_CONSOLE (Set to true if you want to see the output of 'user-ids-autom8-able.rb')\n\nThe Zendesk account email address and password are stored in [re-secrets](https://github.com/alphagov/re-secrets/) because together they have hard deletion capability in many ticket / user groups.\n\n## Pipelines\n\nTheses scripts are run periodically each night by Github Actions.\n\nThe pipelines are located under `./github/workflows/*`\n\n[Deduplicator Script](https://github.com/alphagov/zendesk-scripts/actions/workflows/zendesk_deduplicator.yml)\n[Ticket \u0026 User Tidier](https://github.com/alphagov/zendesk-scripts/actions/workflows/zendesk_remove_tickets_users.yml)\n\n\n## Using the scripts (principally to address historic records in large datasets)\n\n\n### Install ruby\n\nInstall ruby and gems\n\n```\ngem install bundler\ncd zendesk-scripts\nbundle install\n```\n\n\n### Add environmental variables to .bashrc or to 'screen' session\n\n```\n# Vars for Zendesk\nexport ZENDESK_USER_EMAIL=[zendesk admin-user email address]\nexport ZENDESK_USER_PASSWORD=[zendesk admin-user password]\nexport ZENDESK_URL=https://govuk.zendesk.com/api/v2\nexport ZENDESK_TOKEN=[token-string]\nexport ZENDESK_LOG_FILE=zendesk-GDPR-users-log.`date +%Y-%m-%d`\n```\n\nStart a 'screen' session and export vars, e.g.\n\n```\nscreen -S \"tickets\"\nexport ZENDESK_USER_EMAIL=firstname.surname@digital.cabinet-office.gov.uk ; export ZENDESK_URL=https://govuk.zendesk.com/api/v2 ; export ZENDESK_USER_PASSWORD=[admin user password]\n```\n\n\n### Example Historical Ticket Processes\n\n#### Retrieve and delete latest matching tickets (upto 364 days ago - date chosen to maintain GDPR for 24 hours rather than only for current moment)\n\n\n```\nscreen -rd tickets       # Join existing screen session with vars exported and pwd=~ubuntu/zendesk-scripts.\n\nbundle exec ruby lib/get-latest-ticket-numbers.rb\nscripts/delete_latest_tickets.sh tee data/`tickets-date \"+%FT%H:%M\"`.log\n```\n\nNotes\n* Retrieve the GDPR-outstanding tickets to [pwd]/data/latest-tickets-to-purge] and then delete them.\n* The scripts should be run inside a screen session.\n* Auth can take the form of user/password or user/token.\n* Deletion can take several hours (for e.g. 50,000 tickets) depending on quantities.\n* Check your API limitations, ours are currently 700 requests / minute for ticket deletion, 70/min for user account hard deletion.\n\n\n\n#### Counting tickets\n\n\n```\nbundle exec ruby lib/count-closed-tickets-by-year.rb\n```\n\n\n#### Removing historical tickets (once only, kept for reference)\n\n* Retrieve to local files in /data directory per year of ticket_id's\n\n```\nbundle exec ruby lib/get-annual-ticket-numbers.rb\n```\n\n* Results\n\nFiles are created per year, e.g.\n\n```data/y2013_tickets```\n\n\n* Execute script per year to delete old tickets (uses above files as input)\n\n```\ndata/delete_tickets_2013.sh\n```\n\n* Suggestion: Exit the 'screen' session but leave the script running\n\n\n\n### Example User Processes\n\n\n#### Retrieve list of deleted user accounts to local file (note: includes hard deleted so may take several hours)\n\n\n```\nbundle exec ruby lib/get-deleted-user-ids-to_local-file.rb\n\n```\n\n#### Hard delete (purge) qualifying accounts (may take many hours due to lower API rate)\n\n\n```\nbundle exec ruby lib/purge-users-using-local-file.rb | tee data/`date --iso-8601='date'`.log\n```\n```\nbundle exec ruby lib/count-closed-tickets-by-year.rb\nbundle exec ruby lib/get-latest-ticket-numbers.rb\n```\n\nNotes\n* The process must be to soft delete and then hard delete.\n* When retrieving the accounts, previously hard deleted accounts are included with soft deleted, making the sift of data a large task.\n\n\n#### Count User accounts\n\n```\nbundle exec ruby lib/count-users-by-year-for-deletion.rb\n```\n\n#### Create groups (json and text)\n\njson\n```\ncurl $ZENDESK_URL/groups.json -v -u \"$ZENDESK_USER_EMAIL/token:$ZENDESK_TOKEN\" \u003e data/groups.json\n```\n\ntext\n```\nbundle exec ruby lib/get-groups-list-to-file.rb\n```\n\n#### Create custom_roles (json and text)\n\njson\n```\ncurl $ZENDESK_URL/custom_roles.json -v -u \"$ZENDESK_USER_EMAIL/token:$ZENDESK_TOKEN\" \u003e data/custom_roles.json\n```\n\ntext\n```\nbundle exec ruby lib/get-custom-roles-list-to-file.rb\n```\n\n#### Retrieve Agents\n```\nbundle exec ruby lib/get-all-agents.rb \u003e data/agents.json\n```\n\n#### Select and convert json to csv\n```\njq -r '.name + \",\" + .role + \",\" + (.default_group_id|tostring) + \",\" + (.active|tostring) + \",\" + (.custom_role_id|tostring)' data/agents.json \u003e data/agents.csv\n```\n\n#### Merge Agent and Group Description\n```\nsh scripts/merge-agent-and-group-description.sh\n```\n\n### Other Scripts\n\n#### De-duplicate support queue\n* The script uses the Zendesk API to identify multiple tickets from the same UserID in the 1st line support queue which match a set of parameters. If there are \u003e 1 tickets with a status of not closed or solved, they are moved to a dedicated Zendesk queue.\n* The script requires sufficient permissions in Zendesk and is configured to use an account email address and an API token for authenticating API requests.\n\nTo use the script from the CLI, set the environment variables as [above](#add-environmental-variables-to-bashrc-or-to-screen-session), then:\n\n1. Edit window required (e.g. 24 hours) in this line, e.g.:\n```\nwindow_start_time = Time.now - 24 * 3600\n```\n2. Execute script\n```\nbundle exec ruby lib/zendesk-ticket-deduplicator.rb\n```\n\n##### Important Notes\n\n* This pipeline should only run 6 hourly in order to mesh with the manual processes carried out by the support team. If updating the code, please pause the pipeline before merging to ensure it does not run between the 6 hour windows.\n* Running outside of the schedule may cause tickets to be moved around unexpectedly during the hours people are working on them.\n\n\n## License\n\nThis project is licensed under the MIT License\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falphagov%2Fzendesk-scripts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falphagov%2Fzendesk-scripts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falphagov%2Fzendesk-scripts/lists"}