{"id":18555398,"url":"https://github.com/bearloga/wmf-android-monthly-report","last_synced_at":"2026-01-25T03:02:26.055Z","repository":{"id":146996672,"uuid":"215325292","full_name":"bearloga/wmf-android-monthly-report","owner":"bearloga","description":"Monthly Wikipedia Android app reports","archived":false,"fork":false,"pushed_at":"2020-04-21T14:25:46.000Z","size":317,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-15T13:09:48.387Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/metrics/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bearloga.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-15T14:53:14.000Z","updated_at":"2020-04-21T14:25:49.000Z","dependencies_parsed_at":"2023-05-29T14:00:53.881Z","dependency_job_id":null,"html_url":"https://github.com/bearloga/wmf-android-monthly-report","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bearloga/wmf-android-monthly-report","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fwmf-android-monthly-report","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fwmf-android-monthly-report/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fwmf-android-monthly-report/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fwmf-android-monthly-report/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bearloga","download_url":"https://codeload.github.com/bearloga/wmf-android-monthly-report/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fwmf-android-monthly-report/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28742973,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-25T02:46:29.005Z","status":"ssl_error","status_checked_at":"2026-01-25T02:44:29.968Z","response_time":113,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T21:26:26.037Z","updated_at":"2026-01-25T03:02:26.040Z","avatar_url":"https://github.com/bearloga.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Monthly Android report card\n\nThis repository has all the scripts and queries necessary to produce the [monthly reports](https://analytics.wikimedia.org/published/datasets/wikipedia-android-app-reports/metrics/) for [Wikipedia Android app](https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/Android).\n\n## Setup and dependencies\n\nIn R, use the following commands to install the required packages:\n\n```R\ninstall.packages(c(\n  \"import\", \"zeallot\", \"remotes\", \"fs\", \"glue\",\n  \"tidyverse\", \"knitr\", \"kableExtra\"\n))\nremotes::install_github(\"wikimedia/wikimedia-discovery-wmf\")\nremotes::install_github(\"wikimedia/wikimedia-discovery-polloi\")\n```\n\n## Structure and flow\n\nThe scripts should only be run **after** the latest `mediawiki_history` snapshot is ready. Those are usually ready within the first couple days of each month, so don't schedule these scripts to run on the first day of each month.\n\nIf [Kerberos authentication via keytab](https://wikitech.wikimedia.org/wiki/Analytics/Systems/Kerberos/UserGuide#Run_a_recurrent_job_via_Cron_or_similar_without_kinit_every_day) is available, [run.sh](run.sh) can be scheduled via crontab. Scheduling a new report card to be generated on the 7th day of every month would look like:\n\n```\n0 0 7 * * bash /home/bearloga/android/metric_baselines/monthly_report/run.sh\n```\n\n- [data.R](data.R) extracts, aggregates, and loads a subset of the [MobileWikiAppSessions](https://meta.wikimedia.org/wiki/Schema:MobileWikiAppSessions) EventLogging data into `bearloga.android_app_sessions_subset` in Hive.\n  - This makes some of the metrics much easier and faster to calculate.\n  - This is the very first script that is run.\n- [report.R](report.R) is the script that executes the various modules in order.\n  - Saves progress to a file between each one, so that if any of the modules experience problems the script can be resumed from that point without having to re-run the previous modules.\n\nThe modules (sub-scripts run by report.R) are as follows:\n\n- **Section 1: Initialize**\n  1. [setup.R](modules/1-1_setup.R) attaches libraries, determines all the dates to use in queries, and sets up the paths that data and output will be written to\n- **Section 2: Get Metrics**\n  - modules 2-1 through 2-9 are for calculating the various metrics that show up in the report card\n- **Section 3: Finalize**\n  1. [metrics.R](modules/3-1_metrics.R) puts together the report card and writes it out as a web page in the [output](output/) directory.\n  2. [index.R](modules/3-2_index.R) generates a new index page of the monthly report cards, which also goes in the output/ directory.\n  3. [archive.R](modules/3-3_archive.R) compresses, archives, and moves the generated data files to Hadoop Distributed File System (HDFS) for long-term storage.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbearloga%2Fwmf-android-monthly-report","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbearloga%2Fwmf-android-monthly-report","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbearloga%2Fwmf-android-monthly-report/lists"}