{"id":42661698,"url":"https://github.com/wikia/mike","last_synced_at":"2026-01-29T09:50:50.149Z","repository":{"id":142827579,"uuid":"158841088","full_name":"Wikia/Mike","owner":"Wikia","description":"Mycroft Holmes (High-Optional, Logical, Multi-Evaluating Supervisor) aka Mike","archived":false,"fork":false,"pushed_at":"2020-01-10T14:31:19.000Z","size":495,"stargazers_count":7,"open_issues_count":15,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-04-09T21:36:38.736Z","etag":null,"topics":["athena-client","aws","collected-metrics","dashboard","docker","fandom","jira-client","metrics","metrics-gathering","prioritization","product-management","python3","sustainable-software"],"latest_commit_sha":null,"homepage":"https://medium.com/fandom-engineering/how-to-prioritise-your-it-projects-and-make-decisions-based-on-numbers-c04019e2e582","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Wikia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null}},"created_at":"2018-11-23T14:12:10.000Z","updated_at":"2023-09-22T09:15:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"8a315b97-d457-48bd-bd0e-bb06527e9658","html_url":"https://github.com/Wikia/Mike","commit_stats":{"total_commits":234,"total_committers":4,"mean_commits":58.5,"dds":"0.021367521367521403","last_synced_commit":"ae79e5231278f5a94e19a9f775b369fb656854c9"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/Wikia/Mike","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Wikia%2FMike","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Wikia%2FMike/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Wikia%2FMike/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Wikia%2FMike/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Wikia","download_url":"https://codeload.github.com/Wikia/Mike/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Wikia%2FMike/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28874930,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-29T09:47:23.353Z","status":"ssl_error","status_checked_at":"2026-01-29T09:47:19.357Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["athena-client","aws","collected-metrics","dashboard","docker","fandom","jira-client","metrics","metrics-gathering","prioritization","product-management","python3","sustainable-software"],"created_at":"2026-01-29T09:50:49.535Z","updated_at":"2026-01-29T09:50:50.143Z","avatar_url":"https://github.com/Wikia.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mike\n\n[![Build Status](https://travis-ci.org/Wikia/Mike.svg?branch=master)](https://travis-ci.org/Wikia/Mike)\n[![](https://images.microbadger.com/badges/image/macbre/mike.svg)](https://hub.docker.com/r/macbre/mike)\n\n![](https://github.com/Wikia/Mike/raw/master/docs/Mike_dashboard.png)\n\n### [Mycroft Holmes](https://en.wikipedia.org/wiki/The_Moon_Is_a_Harsh_Mistress#Characters) (High-Optional, Logical, Multi-Evaluating Supervisor) aka Mike.\n\n[The tool that collects metrics for software components](https://medium.com/legacy-systems-diary/our-story-about-sustaining-engineering-team-7e83652b8873) based on:\n\n* bug reports / tickets from [JIRA](https://www.atlassian.com/software/jira)\n* features usage statistics taken from [Google Analytics](https://analytics.google.com/analytics/web/) or [AWS Athena](https://aws.amazon.com/athena/)\n* access and application logs from [Elasticsearch](https://www.elastic.co/)\n* any data that can be queried from [MySQL database](https://www.mysql.com/)\n\n\u003e See the list of [all available sources](https://github.com/Wikia/Mike/tree/master/mycroft_holmes/sources#available-sources).\n\n## Install\n\n```\nvirtualenv -ppython3 env\n. env/bin/activate\npip install -e .\n```\n\n\nTo install development dependencies run `make` inside virtualenv.\n\n## Set up a config file\n\nPlease refer to `test/fixtures/config.yaml` and README.md in `mycroft_holmes/source` directory and prepare your own config file.\n\nSave it and store its path in `MIKE_CONFIG` env variable. It is used by the front-end app and metrics collector script.\n\nOtherwise you'll get:\n\n```\nAssertionError: Please specify where your config YAML file is in MIKE_CONFIG env variable.\n```\n\n## Running UI\n\n`Mycroft Holmes` comes with Flask-powered web-application that provides a dashboard with an overview of components and their metrics.\n\nRun the following to try it out in development mode:\n\n```\nmake server_dev\n```\n\nNow visit [`/version.json`](http://127.0.0.1:5000/version.json).\n\n\u003e Mike uses [`python-dotenv`](https://pypi.org/project/python-dotenv/). Env variables specified in `.env` in your working\ndevelopment directory will be loaded automatically.\n\n### Collecting metrics\n\nLet's assume that this repository has been cloned into ` /home/macbre/github/Mike` and virtual env has been set up.\nNow add the following to your `crontab`:\n\n```\nSHELL=/bin/bash\n# m h  dom mon dow   command\n2 2,14 *   *   *     ( cd /home/macbre/github/Mike \u0026\u0026 source env/bin/activate \u0026\u0026 source .env \u0026\u0026 collect_metrics test/fixtures/config.yaml ) \u003e\u003e /home/macbre/Mike.log 2\u003e\u00261\n```\n\n## Using Docker\n\nYou can use our [official Docker image](https://hub.docker.com/r/macbre/mike):\n\n```\ndocker pull macbre/mike:latest\ndocker run -p5000:5000 -it mike\n```\n\nRun the following command periodically to keep metrics up to date:\n\n```\ndocker run -it mike collect_metrics\n```\n\n### Passing your custom YAML config file\n\nBy default Mike docker container will use a sample config file located in `/example.yaml`. You should use your own.\nPlease refer to \"Set up a config file\" section above.\n\nAssuming that you have a local `/home/mike/config/.env` file with all your specific credentials that are referenced in `/home/mike/config/mike.yaml`\nconfig. Run the following:\n\n```\ndocker run -v /home/mike/config:/opt/config -p5000:5000 --env-file /home/mike/config/.env -e MIKE_CONFIG=/opt/config/mike.yaml -it mike\n```\n\n`.env` file should follow this convention:\n\n```\nDATABASE_USER=mike_db\nDATABASE_PASSWORD=d97b4e7998a07bd1b2da4c21f29ec183ad3eec20\n```\n\n## Scripts\n\nThe following command line scripts are available in Mike's virtual environment:\n\n### `collect_metrics`\n\nThis script should be run periodically to collect metrics for features defined in YAML config file.\n\nSimply run `collect_metrics \u003cpath to YAML config file\u003e`\n\nEnvironment variables passed to the script will be used to replace variables in the YAML config file.\n\nFor instance, running the following:\n\n```\nDATABASE_USER=foo DATABASE_PASSWORD=2505eb2474b2 collect_metrics test/fixtures/config.yaml\n```\n\nwill replace `${...}` placeholders with appropriate values taken from environment variables provided above.\n\n```yaml\nsources:\n  - name: wikia/tags-report\n    kind: common/mysql\n    host: db.prod\n    user: \"${DATABASE_USER}\"\n    password: \"${DATABASE_PASSWORD}\"\n```\n\n[List of all available sources with full documentation](https://github.com/Wikia/Mike/tree/master/mycroft_holmes/sources#sources).\n\n### `generate_source_docs`\n\nPrints out Markdown with sources documentation taken from the code, to be pasted into `mycroft_holmes/sources/README\u003emd`\nwhen a new source is added or an existing one is updated.\n\n## Config file\n\n[An example YAML config file](https://github.com/Wikia/Mike/blob/master/test/fixtures/config.yaml) can be found in `test/fixtures` directory.\n\n[Mike's architecture, sources structure and configuration](https://github.com/Wikia/Mike/tree/master/mycroft_holmes/sources#sources) are described in `mycroft_holmes/sources` directory.\n\n## License\n\n[Dashboard sidebar's background image](https://commons.wikimedia.org/wiki/File:Gree-02.jpg) is used under public domain license. Favicon made by [Freepik](https://www.flaticon.com/authors/freepik) is licensed by CC 3.0 BY.\n\n## Tests\n\nAfter setting up virtual env, please install all dependencies (including dev ones) via `make init`. Then run:\n\n```\nmake test\n```\n\n### MySQL storage integration tests\n\n\u003e `TEST_DATABASE` env variable needs to be set to run these tests. `schema.sql` and `schema_test.sql` files need to be applied as well. Please refer to `.travis.yml` when in doubt.\n\nYou may want to include storage integration tests. They do require running MySQL server.\nConnection details can be provided via `.env` file:\n\n```\n$ cat .env\nTEST_DATABASE=mycroft_holmes\nTEST_DATABASE_USER=foo\nTEST_DATABASE_PASSWORD=bar\n\n$ make test\n\n# access mysql console with the above credentials\n$ make mysql_cli\n```\n \n \u003e Please note that **tests truncate the tables**.\n ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwikia%2Fmike","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwikia%2Fmike","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwikia%2Fmike/lists"}