{"id":18123073,"url":"https://github.com/wikimedia/commonsnotifier","last_synced_at":"2025-10-24T18:08:05.866Z","repository":{"id":45025689,"uuid":"123043809","full_name":"wikimedia/CommonsNotifier","owner":"wikimedia","description":"Bot that notifies Wikimedia communities of pending deletion of Commons images they're using","archived":false,"fork":false,"pushed_at":"2024-12-02T12:21:01.000Z","size":201,"stargazers_count":4,"open_issues_count":0,"forks_count":3,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-03-28T11:04:57.581Z","etag":null,"topics":["bot","python","toolforge"],"latest_commit_sha":null,"homepage":"https://meta.wikimedia.org/wiki/User:Community_Tech_bot","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wikimedia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-26T23:19:18.000Z","updated_at":"2024-12-02T12:21:05.000Z","dependencies_parsed_at":"2023-12-25T14:00:23.429Z","dependency_job_id":"3c615a43-4e44-488d-a903-5b109ae86410","html_url":"https://github.com/wikimedia/CommonsNotifier","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wikimedia%2FCommonsNotifier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wikimedia%2FCommonsNotifier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wikimedia%2FCommonsNotifier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wikimedia%2FCommonsNotifier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wikimedia","download_url":"https://codeload.github.com/wikimedia/CommonsNotifier/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246301963,"owners_count":20755512,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bot","python","toolforge"],"created_at":"2024-11-01T07:07:54.143Z","updated_at":"2025-10-24T18:08:05.772Z","avatar_url":"https://github.com/wikimedia.png","language":"Python","readme":"# CommonsNotifier\n\n## Setting up\n\nClone the bot code:\n\n```\ngit clone https://github.com/MaxSem/CommonsNotifier.git bot\ncd bot\n```\n\nSet up the dependencies in virtualenv:\n\n```\nvirtualenv --python=python3 virtualenv\nvirtualenv/bin/pip install -r requirements.txt\n```\n\nMake the database credentials available to the bot. The bot will search for `replica.my.cnf` or `my.cnf` in home directory or the bot's root directory. For uniformity, copy `~/replica.my.cnf` to `bot/my.cnf` and add host/database information to it. For Toolforge deployment, that would be something like:\n```\n[client]\npassword = ****\nuser = s53689\nhost = tools.db.svc.eqiad.wmflabs\ndatabase = s53689__commonsbot\n```\nFor development, you want to either point this to your local MySQL instance. Alternatively, you can use port forwarding to connect to the tools database:\n```\nssh -L 33061:tools.db.svc.eqiad.wmflabs:3306 \u003cyour labs username\u003e@login.toolforge.org\n```\nIn such case, the configuration would be:\n```\n[client]\npassword = ****\nuser = s53689\nhost = localhost\nport = 33061\ndatabase = s53689__commonsbot\n```\n\nSet up wiki credentials. Currently, the bot uses BotPasswords. When you register a new bot password, you get a message like\n\u003e The new password to log in with **[bot username]@[sub bot name]** is **[some hex string]**. *Please record this for future reference*.\n\nCreate `user-password.py` in the bot's directory with the following content:\n```\n('bot user account', BotPassword('sub bot name', 'that long hex string'))\n```\n\nIf you're running a clone that isn't using the default username `Community Tech bot`, you also need to change it in `user-config.py`.\n\nBefore running the bot for the first time, **be sure to run the first-run script** or the bot will post several thousands of notifications about the files nominated before the first run:\n\n```\nbin/first-run\n```\n\n### Toolforge\nSet up cron jobs (with `crontab -e`):\n```\n# Run the bot's main script every 15 minutes\n5,20,35,50 * * * * jsub -quiet -once -N commtech-commons ${HOME}/bot/bin/cronjob\n\n# Clean up old logfiles and DB entries daily\n0 3 * * * jsub -quiet -once -N commtech-commons-cleanup ${HOME}/bot/bin/cleanup\n```\n\n## Running the bot\nPython 3.4+ is required.\n\n### Stopping the bot\nTo kill a job that's currently running on Toolforge:\n```\njstop commtech-commons\n```\n\nIn most cases, you don't want to completely stop the bot if it misbehaves. Dry run mode is sufficient. In this mode, the bot does everything other than actually posting the messages on-wiki, which allows it to maintain its database in a consistent state, as well as generate logs with any error/debug messages that might help with development. To switch it on, change `\"dry-run\": false\"` to `true` in `config.json`.\n\nIf a complete stop is required, comment it out in the crontab. **WARNING**: a complete stop means `bin/first-run` is required before a restart.\n\n### Debug information\nToolforge automatically saves the bot's stdout to `commtech-commons.out` and stderr to `commtech-commons.err` - these contain lots of useful information.\n\nTo add some extra verbose debug output, set `verbose_output` to `True` in `user-config.py`.\n\nOld listfiles are saved to `logs/{discussion|speedy}.txt.\u003ctimestamp\u003e`.\n\n## Adding a new wiki\n* Add its dbname to `wikis-enabled` and optional configuration in `config.json` in Git.\n* Make sure that all messages are localized by running `check-i18n.py`.\n* Make sure that the bot account is present on the wiki.\n* Deploy by simply pulling in the new code (the task is continually ran by cron, so you shouldn't need to restart anything):\n  * `become community-tech-tools`\n  * `become commtech-commons`\n  * `cd bot`\n  * `git pull`\n  * The next time the cron runs, it should include the newly added wiki\n\n## Configuration\nAll settings are kept in `config.json`:\n* `dry-run` - whether dry run mode os on, see \"Stopping the bot\" above.\n* `wiki-options` - dictionary of `dbname: options`, with every wiki possibly overriding setting from `\"default\"`. Possible per-wiki options:\n  * `minoredit` - (bool) whether edits should be marked as minor\n  * `markasbot` - (bool) whether edits should be marked as bot edits\n  * `tags` (string|null) - pipe-separated edit tags to apply to edits or `null` for none\n  * `language` (string|null) - override for language code used for messages on this wiki, or `null` to not override\n\n## Development\n\n### Overview\n\nThe bot's main operation consists of three scripts:\n* `make-list.py` creates lists of files that have been around for long enough to be notified about.\n* `post-notifs.py` runs off these files to determine which pages need updating and post the messages.\n* `check-i18n.py` is a dev script that outputs statistics on language translation progress.\n\n### Testing\nThe repository has a makefile, so just run\n```\nmake\n```\nfrom the bot's root directory to run the tests.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwikimedia%2Fcommonsnotifier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwikimedia%2Fcommonsnotifier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwikimedia%2Fcommonsnotifier/lists"}