{"id":14065851,"url":"https://github.com/bearloga/gsc-utils","last_synced_at":"2025-10-14T18:14:16.387Z","repository":{"id":146996618,"uuid":"127015306","full_name":"bearloga/gsc-utils","owner":"bearloga","description":"Utility to facilitate fetching data from Google Search Console API","archived":false,"fork":false,"pushed_at":"2020-02-10T16:34:03.000Z","size":33,"stargazers_count":1,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-06-05T08:14:45.785Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://phabricator.wikimedia.org/T172581","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bearloga.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2018-03-27T16:25:44.000Z","updated_at":"2023-05-07T19:07:01.000Z","dependencies_parsed_at":"2023-05-29T14:00:58.444Z","dependency_job_id":null,"html_url":"https://github.com/bearloga/gsc-utils","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bearloga/gsc-utils","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fgsc-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fgsc-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fgsc-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fgsc-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bearloga","download_url":"https://codeload.github.com/bearloga/gsc-utils/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearloga%2Fgsc-utils/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279020330,"owners_count":26086864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-13T07:04:46.216Z","updated_at":"2025-10-14T18:14:16.359Z","avatar_url":"https://github.com/bearloga.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# gsc-utils\n\nUtilities for accessing and downloading the statistics on a site's presence in Google's search results via [Search Console API](https://developers.google.com/webmaster-tools/search-console-api-original/).\n\n## Setup\n\n```bash\npip3 install -U git+https://github.com/bearloga/gsc-utils.git\n```\n\n### Credential authorization\n\nCreate a OAuth 2.0 Client ID on the [Credentials page](https://console.developers.google.com/apis/credentials) of the API console. Then download the secrets JSON (which should look similar to [this example](sample-secrets.json)). You will use this to create and save a set of authorized credentials. When run, the code will ask you to navigate to a specific URL to authorize with your Google account and prompt you for a verification code which you will be given after approving the authorization request.\n\n```python\nfrom gsc_utils import utils\n\ncreds = utils.authorize('path/to/secrets.json')\nutils.save_credentials(creds, 'path/to/credentials.json')\n```\n\nThe created `credentials.json` should look similar to [this example](sample-credentials.json). You can re-use it in future sessions without having to re-authorize:\n\n```python\nfrom gsc_utils import utils\n\ncreds = utils.load_credentials('path/to/credentials.json')\n```\n\n**Note**: Some of the code has been adapted from [Quickstart: Run a Search Console App in Python ](https://developers.google.com/webmaster-tools/search-console-api-original/v3/quickstart/quickstart-python) and [OAuth 2.0 Storage](https://developers.google.com/api-client-library/python/guide/aaa_oauth#storage).\n\n## Usage\n\n- `gsc_utils.sites` module has functions\n    - `list()` for obtaining a list of sites registered in GSC\n    - `add()` and `remove()` for registering and unregistering sites in GSC, respectively\n- `gsc_utils.performance` module has function `stats()` for impressions, clicks, ctr (clickthrough rate), and position for one or more sites registered in GSC\n\nRefer to [example notebook](docs/example.ipynb)\n\n### Rich Card Results\n\nSome results appear as rich cards in Google's search results, and the way the statistics are calculated is different. Specifically, there are two aggregation types: by site and by page. When `rich_results=True`, the statistics returned will be aggregated _by page_. Otherwise all results will be considered and the aggregation will be _by site_.\n\nRefer to [Aggregating data by site vs by page](https://support.google.com/webmasters/answer/6155685?authuser=0#urlorsite) for details.\n\n### Example usage in R\n\n```R\n# install.packages(c(\"purrr\", \"urltools\", \"readr\", \"reticulate\"))\n\nlibrary(reticulate)\n\ngsc_utils \u003c- import(\"gsc_utils.utils\")\nsites \u003c- import(\"gsc_utils.sites\")\nperformance \u003c- import(\"gsc_utils.performance\")\n\ncreds \u003c- gsc_utils$load_credentials('path/to/credentials.json')\n\nsite_list \u003c- sites$list(creds)\n\nresults \u003c- purrr::map_dfr(\n  site_list$siteUrl,\n  function(site_url, start, end) {\n    website \u003c- urltools::domain(site_url)\n    use_https \u003c- urltools::scheme(site_url) == \"https\"\n    result \u003c- performance$stats(creds, website, start, end, use_https = use_https)\n    return(result)\n  },\n  start = \"2020-01-01\", end = \"2020-01-31\"\n)\n\nreadr::write_csv(results, \"stats_2020-01.csv\")\n```\n\nSince `gsc_utils.performance.stats()` can operate on a vector of websites, this is the alternative usage if all of the sites use the same protocol (all HTTPS):\n\n```R\nresults \u003c- performance$stats(creds, urltools::domain(site_list$siteUrl), start = \"2020-01-01\", end = \"2020-01-31\")\n```\n\n**Note**: If `gsc-utils` is installed in a different virtual environment than the default one, include the following in .Rprofile in working directory:\n\n```R\nSys.setenv(RETICULATE_PYTHON = \"path/to/python\")\n```\n\nRefer to [Python Version Configuration](https://rstudio.github.io/reticulate/articles/versions.html) for more instructions and details.\n\n## Information\n\n**Maintainer**: Mikhail Popov (mpopov at wikimedia dot org)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbearloga%2Fgsc-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbearloga%2Fgsc-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbearloga%2Fgsc-utils/lists"}