{"id":20450053,"url":"https://github.com/ooni/wcthcheck","last_synced_at":"2026-06-10T07:31:23.564Z","repository":{"id":141418945,"uuid":"383500718","full_name":"ooni/wcthcheck","owner":"ooni","description":"Helper scripts to compare old and new web connectivity test helpers.","archived":false,"fork":false,"pushed_at":"2021-10-18T08:50:49.000Z","size":335,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-10-27T12:34:23.747Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://ooni.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ooni.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-07-06T14:39:01.000Z","updated_at":"2021-10-18T08:50:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"fd605ebf-1673-4747-828c-c12c997b2eea","html_url":"https://github.com/ooni/wcthcheck","commit_stats":{"total_commits":62,"total_committers":1,"mean_commits":62.0,"dds":0.0,"last_synced_commit":"12253c0e8f1d2c05a11661c92cf888ba5f1330c4"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/ooni/wcthcheck","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ooni%2Fwcthcheck","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ooni%2Fwcthcheck/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ooni%2Fwcthcheck/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ooni%2Fwcthcheck/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ooni","download_url":"https://codeload.github.com/ooni/wcthcheck/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ooni%2Fwcthcheck/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34142637,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-10T02:00:07.152Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T10:50:05.722Z","updated_at":"2026-06-10T07:31:23.539Z","avatar_url":"https://github.com/ooni.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OONI Web Connectivity Test Helper Check\n\nThis repository contains code to compare the old and the new versions\nof OONI's Web Connectivity test helper (wcth). We want to understand whether\nwe can safely replace the old wcth with the new wcth.\n\n## Existing test helpers\n\nThe old test helper implementation is\n[ooni/backend](https://github.com/ooni/backend/tree/1164a70/oonib), while\nthe new implementation is instead at\n[ooni/probe-cli](https://github.com/ooni/probe-cli/tree/3747598b/internal/cmd/oohelper).\n\nAt the moment of writing this README file, the old test helper is\ndeployed at `https://wcth.ooni.io/`, and the new test helper is instead\navailable at `https://1.th.ooni.org/`.\n\n## Test lists\n\nThe `alltestlists.txt` file contains the URL present in all the test lists\nat [citizenlab/test-lists](https://github.com/citizenlab/test-lists) as\nof June 6th, 2021. We removed redundant entries.\n\n## Code to query the test helpers\n\nThe [ooni/probe-cli](https://github.com/ooni/probe-cli) repository contains\ncode to query a wcth, called `oohelper`. We build this code with the\n`buildoohelper` script.\n\nThe `run` script takes care of calling `buildoohelper` and then uses the\n`oohelper` binary to query both test helpers for every entry in the\n`alltestlists.txt` file.\n\n## Database containing results\n\nThe `run` script stores results into a `database.sqlite3` file. This\nscript assumes Python \u003e= 3.8.\n\nThe code assumes that this file already exists. If it does not exist, then\nyou can create one using `sqlite3`. This is the database schema:\n\n```SQL\nCREATE TABLE results(date text, backend text, target text, json text);\n```\n\nThe `date` field contains the date when we queries a test helper. The\n`backend` field is the URL of the test helper. The `target` field is\na URL from `alltestlists.txt`. The `json` field is the JSON result returned\nby a test helper for a given input URL.\n\nThe objective of this analysis is to compare the JSON returned by each\ntest helper and identify which are the differences and why.\n\n## Releasing a database\n\nBecause it takes quite a long time to build a database and because the\ndatabase itself is quite large, we publish a snapshot of the database\nwith over 80\\% of the URLs measured as a release of this repo. Then you\ncan download the database and just perform the results analysis.\n\n## Results analysis\n\nTo this end, we use the `compare` script. This script requires Python \u003e= 3.8.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fooni%2Fwcthcheck","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fooni%2Fwcthcheck","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fooni%2Fwcthcheck/lists"}