{"id":13565712,"url":"https://github.com/digitalmethodsinitiative/zeeschuimer","last_synced_at":"2026-01-22T17:36:51.251Z","repository":{"id":41371122,"uuid":"409246417","full_name":"digitalmethodsinitiative/zeeschuimer","owner":"digitalmethodsinitiative","description":"A browser extension to collect social media data with.","archived":false,"fork":false,"pushed_at":"2026-01-15T16:06:11.000Z","size":14465,"stargazers_count":314,"open_issues_count":14,"forks_count":33,"subscribers_count":12,"default_branch":"master","last_synced_at":"2026-01-15T19:33:03.395Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/digitalmethodsinitiative.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":".zenodo.json","notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-09-22T14:55:45.000Z","updated_at":"2026-01-15T16:06:17.000Z","dependencies_parsed_at":"2023-10-02T19:21:22.421Z","dependency_job_id":"630a7b3b-e5af-4348-857f-7fd5a3a2eed5","html_url":"https://github.com/digitalmethodsinitiative/zeeschuimer","commit_stats":null,"previous_names":[],"tags_count":39,"template":false,"template_full_name":null,"purl":"pkg:github/digitalmethodsinitiative/zeeschuimer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digitalmethodsinitiative%2Fzeeschuimer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digitalmethodsinitiative%2Fzeeschuimer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digitalmethodsinitiative%2Fzeeschuimer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digitalmethodsinitiative%2Fzeeschuimer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/digitalmethodsinitiative","download_url":"https://codeload.github.com/digitalmethodsinitiative/zeeschuimer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digitalmethodsinitiative%2Fzeeschuimer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28667867,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T17:07:18.858Z","status":"ssl_error","status_checked_at":"2026-01-22T17:05:02.040Z","response_time":144,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T13:01:53.752Z","updated_at":"2026-01-22T17:36:51.246Z","avatar_url":"https://github.com/digitalmethodsinitiative.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# 🏴‍☠️ Zeeschuimer\n\n[![DOI: 10.5281/zenodo.4742622](https://zenodo.org/badge/DOI/10.5281/zenodo.6826877.svg)](https://doi.org/10.5281/zenodo.6826877)\n[![License: MPL 2.0](https://img.shields.io/badge/license-MPL--2.0-informational)](https://github.com/digitalmethodsinitiative/4cat/blob/master/LICENSE)\n\n\u003cp align=\"center\"\u003e\u003cimg alt=\"A screenshot of Zeeschuimer's status window\" src=\"images/example_screenshot.png\"\u003e\u003c/p\u003e\n\nZeeschuimer is a browser extension that monitors internet traffic while you are browsing a social media site, and \ncollects data about the items you see in a platform's web interface for later systematic analysis. Its target audience\nis researchers who wish to systematically study content on social media platforms that resist conventional scraping or \nAPI-based data collection.\n\nYou can, for example, browse TikTok and later export a list of all posts you saw in the order you saw them in. Data can \nbe exported as a JSON file or exported to a [4CAT](https://github.com/digitalmethodsinitiative/4cat) instance for \nanalysis and storage. Zeeschuimer is primarily intended as a companion to 4CAT, but you can also integrate its output\ninto your own analysis pipeline.\n\nCurrently, it supports the following platforms:\n* [TikTok](https://www.tiktok.com) (posts and comments)\n* [Instagram](https://www.instagram.com) (posts only)\n* [X/Twitter](https://www.x.com)\n* [LinkedIn](https://www.linkedin.com)\n* [9gag](https://9gag.com)\n* [Imgur](https://imgur.com)\n* [Douyin](https://douyin.com)\n* [Gab](https://gab.com)\n* [Truth Social](https://truth.social)\n* [Pinterest](https://pinterest.com)\n* [RedNote/Xiaohongshu](https://xiaohongshu.com)\n\nPlatform support requires regular maintenance to keep up with changes to the platforms. If something does not work, we\nwelcome issues and pull requests. See 'Limitations' below for some known limitations to data capture.\n\nThe extension does not interfere with your normal browsing and never uploads data automatically, only when you\nexplicitly ask it to do so. It uses the\n[WebRequest](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest) browser API to\nlocally collect and parse the data search engines are sending to your browser as you use it.\n\n## Installation\nZeeschuimer is in active development. .xpi files that you can use to install it in your browser are available on the \n[releases](https://github.com/digitalmethodsinitiative/zeeschuimer/releases) page. These are signed and can be installed \nin any Firefox-based browser. If you want to run the latest development version instead, you can [do so from the Firefox\ndebugging console](https://www.youtube.com/watch?v=J7el77F1ckg) after cloning the repository locally.\n\n## How to use\nA [guide to using Zeeschuimer and 4CAT](https://zeeschuimer.4cat.nl/) is available. Basic instructions \nare as follows: \n\nInstall the browser extension in a Firefox browser. A button with the Zeeschuimer logo (\u003cimg alt=\"Zeeschuimer's browser icon, a yellow 'Z' on a green background\" src=\"images/zeeschuimer-16.png\"\u003e) will appear in the browser toolbar. Click it \nto open the Zeeschuimer interface. Enable capturing for the sites you want to capture from.\n\nNote that after installation in Firefox, the extension icon may not be immediately visible in the toolbar. If you can't \nfind Zeeschuimer's icon, look for the 'Extensions' icon (a puzzle piece); clicking it will show all available extensions \nthat are not shown in the main browser toolbar.\n\n### Collecting data\nNext, simply browse a supported platform's site. **Zeeschuimer collects data as your browser receives it**.  It is\nalways best practices to refresh a page after you toggle collection on for a supported platform's site in the control \npanel (any data loaded previously will not be collected until refreshed). You can force a refresh with Ctl + F5 on Windows \nor Shift + Command + R on Mac. You will see the amount of items detected per platform increase as you browse. Toggle \ncollection off when done to avoid inadvertantly gathering undesired data.\n\nWhen you have the items you need, you can export the data as an [ndjson](https://ndjson.org) file, or upload \nit to a 4CAT instance where a 4CAT dataset will be created from the uploaded items. You can then run 4CAT's analytical \nprocessors on the data.\n\n**To upload to 4CAT, copy the URL of the website of the 4CAT instance to the \"4CAT instance\" field at the bottom of \nZeeschuimer's interface**. You can then use the \"to 4CAT\" button to create a new 4CAT dataset from the captured data. \nAfter uploading, Zeeschuimer will show you a link and the ten most recently uploaded datasets are shown at the bottom of\nthe interface.\n\nDon't forget to reset the data as needed. For example, if you want to create a dataset for a given TikTok hashtag, first\nreset the TikTok data in Zeeschuimer, _then_ go to the hashtag's \"Explore\" page on TikTok, and then upload the dataset\nwhen you've scrolled down enough to be satisfied with the amount of items.\n\nIf you find yourself scrolling a lot to collect data, consider using another browser extension to do it for you, for \nexample [FoxScroller](https://addons.mozilla.org/en-US/firefox/addon/foxscroller/).\n\n## Limitations\n\nDue to the technical limitations, it may not be possible to collect all items from all 'views' for each supported \nplatform. The following limitations are known:\n\n* *Instagram* items that cannot be captured:\n  * Stories\n  * Posts from the 'Tagged' and 'Reels' tabs on a profile page\n  * Posts from the 'Saved' overview of bookmarked posts\n  * Posts from the 'For You' feed on the 'Explore' page (the 'Not personalized' feed _does_ work)\n  * 'Suggested for you' and 'Sponsored' posts on the front page feed\n* *TikTok* items that cannot be captured:\n  * Live streams\n\nFor some platforms, the level of detail of the data that can be collected depends on the page it is captured from:\n\n* *Pinterest* items may lack some metadata unless captured from the individual post's page, most notably the timestamp \n  of the post.\n* *RedNote/Xiaohongshu* items will often lack the item's post description, timestamp, and video URL, unless captured by \n  opening the post's own page/clicking it in an overview.\n\nNote that these are *known* limitations; data capture may break or change based on platform changes. Always \ncross-reference captured data with what you are seeing in your browser.\n\n## Credits \u0026 license\nZeeschuimer was developed by Stijn Peeters for the [Digital Methods Initiative](https://digitalmethods.net) and is \nlicensed under the Mozilla Public License, 2.0. Refer to the LICENSE file for more information.\n\nGraphics based on [an image generated by Dall-E](https://labs.openai.com/s/oWvGAHC0pxwWV3bNLfENu7AV), for the prompt \n'detail of a 1914 metaphysical painting by giorgio de chirico depicting a buccaneer pensively looking out over the open \nsea'. Interface icons by [Font Awesome](https://fontawesome.com/license/free). \n[Open Sans](https://fonts.google.com/specimen/Open+Sans) and [Lobster](https://fonts.google.com/specimen/Lobster) fonts \nfrom Google Fonts.\n\nDevelopment is supported by the Dutch [PDI-SSH](https://pdi-ssh.nl/en/) foundation through the [CAT4SMR \nproject](https://cat4smr.humanities.uva.nl/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigitalmethodsinitiative%2Fzeeschuimer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdigitalmethodsinitiative%2Fzeeschuimer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigitalmethodsinitiative%2Fzeeschuimer/lists"}