{"id":19162864,"url":"https://github.com/centre-for-humanities-computing/crowdtangle-api-scraper","last_synced_at":"2026-06-19T19:04:03.474Z","repository":{"id":99222518,"uuid":"344266952","full_name":"centre-for-humanities-computing/crowdtangle-api-scraper","owner":"centre-for-humanities-computing","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-06T15:02:42.000Z","size":21,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-09-09T23:59:49.054Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/centre-for-humanities-computing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-03T21:20:50.000Z","updated_at":"2022-12-06T15:02:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"792974bf-cd71-4c14-925b-26ea126118b7","html_url":"https://github.com/centre-for-humanities-computing/crowdtangle-api-scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/centre-for-humanities-computing/crowdtangle-api-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/centre-for-humanities-computing%2Fcrowdtangle-api-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/centre-for-humanities-computing%2Fcrowdtangle-api-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/centre-for-humanities-computing%2Fcrowdtangle-api-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/centre-for-humanities-computing%2Fcrowdtangle-api-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/centre-for-humanities-computing","download_url":"https://codeload.github.com/centre-for-humanities-computing/crowdtangle-api-scraper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/centre-for-humanities-computing%2Fcrowdtangle-api-scraper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34544413,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-19T02:00:06.005Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T09:13:22.756Z","updated_at":"2026-06-19T19:04:03.451Z","avatar_url":"https://github.com/centre-for-humanities-computing.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CrowdTangle Api Scraper\nA tool for extracting post objects from the \n[CrowdTangle](https://help.crowdtangle.com/en/articles/1189612-crowdtangle-api) api.\n\n\n## Installation\n- Install [node.js](https://nodejs.org/en/download/) version 14.11 or higher\n- Clone this repository\n- Navigate to the root of the repository and run\n\n```\n$ npm install\n```\n\n## Usage\n- Navigate to the root of the repository and run to see the help for the CLI\n```\n$ node cli -h \n```\n\n### CLI options\n\n- **`-k, --api-credentials \u003capiCredentials\u003e`** [required] - The CrowdTangle API key or a file containing KEY\n- **`-q, --query \u003cquery\u003e`** [required] - The term to search for\n- **`-d, --destination \u003cdirectory\u003e`** [required] - The directory where the result should be stored\n- **`-p, --filename-prefix \u003cstring\u003e`** [optional, default=\"posts\"] - The name of the result file\n- **`-f, --from \u003cdate\u003e`** [required] - The date to fetch data from in the format \"yyyy-mm-dd\"\n- **`-t, --to \u003cdate\u003e`** [required] - The date to fetch data to (included) in the format \"yyyy-mm-dd\". To fetch a single day this should be the same as \"from\"\n- **`-l, --language \u003clanguageCode\u003e`** [optional] - The iso language code to search for. Eg. \"da\"\n- **`-o, --platforms \u003cplatforms\u003e`** [optional] - The platforms to search. Default: \"facebook\". Possible values are [facebook, instagram, reddit] multiple platforms should be separated by comma\n- **`-c, --csv`** [optional] - Generate a csv file along with the json file\n- **`-z, --development-mode`** [optional] - Should logging data be printed to the stdout\n\n### Full Example\n```\n$ node cli -k \"API-KEY\" -q \"election\" -d \"/data/crowdtangle\", -f \"2019-01-01\" -t \"2020-01-01\" -l \"da\" -o \"facebook\" -c\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcentre-for-humanities-computing%2Fcrowdtangle-api-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcentre-for-humanities-computing%2Fcrowdtangle-api-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcentre-for-humanities-computing%2Fcrowdtangle-api-scraper/lists"}