{"id":13770139,"url":"https://github.com/tgxn/lemmy-explorer","last_synced_at":"2025-05-11T02:35:11.076Z","repository":{"id":174587820,"uuid":"652444510","full_name":"tgxn/lemmy-explorer","owner":"tgxn","description":"Instance and Community Explorer for Lemmy","archived":false,"fork":false,"pushed_at":"2024-08-14T10:05:58.000Z","size":31256,"stargazers_count":106,"open_issues_count":14,"forks_count":9,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-08-15T11:18:32.722Z","etag":null,"topics":["fediverse","lemmy","nodejs","reactjs","social-crawler","social-media-analysis"],"latest_commit_sha":null,"homepage":"https://lemmyverse.net/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tgxn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-12T05:06:26.000Z","updated_at":"2024-08-15T11:18:32.723Z","dependencies_parsed_at":"2023-10-13T05:13:37.574Z","dependency_job_id":"e1a00376-7c86-42e0-82ea-f84b81dd8cad","html_url":"https://github.com/tgxn/lemmy-explorer","commit_stats":null,"previous_names":["tgxn/lemmy-explorer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tgxn%2Flemmy-explorer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tgxn%2Flemmy-explorer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tgxn%2Flemmy-explorer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tgxn%2Flemmy-explorer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tgxn","download_url":"https://codeload.github.com/tgxn/lemmy-explorer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225008465,"owners_count":17406289,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fediverse","lemmy","nodejs","reactjs","social-crawler","social-media-analysis"],"created_at":"2024-08-03T17:00:34.655Z","updated_at":"2025-05-11T02:35:11.066Z","avatar_url":"https://github.com/tgxn.png","language":"JavaScript","funding_links":[],"categories":["Projects"],"sub_categories":["Tools"],"readme":"[![publish-pages](https://github.com/tgxn/lemmy-explorer/actions/workflows/publish-pages.yaml/badge.svg)](https://github.com/tgxn/lemmy-explorer/actions/workflows/publish-pages.yaml)\n\n# Lemmy Explorer https://lemmyverse.net/\n\nData Dumps: https://data.lemmyverse.net/\n\nThis project provides a simple way to explore Lemmy Instances and Communities.\n\n![List of Communities](./docs/images/0.10.0-communities.png)\n\n## Project Structure\n\nThe project consists of the following modules:\n\n| Module Description                            | Path        | Readme                         |\n| --------------------------------------------- | ----------- | ------------------------------ |\n| Crawler _(NodeJS, Redis)_                     | `/crawler`  | [README](./crawler/README.md)  |\n| Frontend _(ReactJS, MUI Joy, TanStack)_       | `/frontend` | [README](./frontend/README.md) |\n| Deployment _(Amazon CDK v2)_                  | `/cdk`      | [README](./cdk/README.md)      |\n| Data Dump Site _(ReactJS, MUI, GitHub Pages)_ | `/pages`    | [README](./pages/README.md)    |\n\nEach module has its own README with more details.\n\n## FAQ\n\n## Q: How can I set a link to automatically set the home instance?\n\nYou can append `home_url` and (optionally) `home_type` to the URL to set the home instance and type.\n\n`?home_url=lemmy.example.com`\n`?home_url=mbin.example.com\u0026home_type=mbin`\n\n\u003e `home_type` supports \"lemmy\" and \"mbin\" (default is \"lemmy\")\n\n### Q: **How does discovery work?**\n\nIt uses a [seed list of communities](https://github.com/tgxn/lemmy-explorer/blob/main/crawler/src/lib/const.ts#L68) and scans the equivalent of the `/instances` federation lists, and then creates jobs to scan each of those servers.\n\nAdditionally, instance tags and trust data is fetched from [Fediseer](https://gui.fediseer.com/).\n\n### Q: **How does the NSFW filter work?**\n\nThe NSFW filter is a client-side filter that filters out NSFW communities and instances from results by default.\nThe \"NSFW Toggle\" checkbox has thress states that you can toggle through:\n\n| State      | Filter       | Value |\n| ---------- | ------------ | ----- |\n| Default    | Hide NSFW    | false |\n| One Click  | Include NSFW | null  |\n| Two Clicks | NSFW Only    | true  |\n\nWhen you try to switch to a non-sfw state, a popup will appear to confirm your choice. You can save your response in your browsers cache and it will be remembered.\n\n### Q: **How long till my instance shows up?**\n\nHow long it takes to discover a new instance can vary depending on if you post content that's picked up by one of these servers.\n\nSince the crawler looks at lists of federated instances, we can't discover instances that aren't on those lists.\n\nAdditionally, the lists are cached for 24 hours, so it can take up to 24 hours for an instance to show up after it's been discovered till it shows up.\n\n### Q: **Can I use your data in my app/website/project?**\n\nI do not own any of the data retrieved by the crawler, it is available from public endpoints on the source instances.\n\nYou are free to pull data from the GitHub pages site:\n\n[**Lemmyverse Data Site**](https://data.lemmyverse.net/)\n\n**Please don't hotlink the files on the public website `https://lemmyverse.net/`**\n\n### Q: **How often is the data updated?**\n\nCurrently, I upload a Redis dump generated by the crawler each night to s3, GitLab builds the JSON dump from that.\n\nData is also available from the artifacts of [this action](https://github.com/tgxn/lemmy-explorer/actions/workflows/publish-pages.yaml).\nYou can also download [Latest ZIP](https://nightly.link/tgxn/lemmy-explorer/workflows/publish-pages.yaml/main/dist-json-bundle.zip) _(using nightly.link)_\n\n`dist-json-bundle.zip` file contains the data in JSON format:\n\n- `communities.full.json` - list of all communities\n- `instances.full.json` - list of all instances\n- `overview.json` - metadata and counts\n\n## Awesome Lemmy Links\n\n### General\n\n- https://join-lemmy.org/instances\n- https://github.com/maltfield/awesome-lemmy-instances\n\n### Lemmy Stats Pages\n\n- https://lemmy.fediverse.observer/dailystats\n- https://the-federation.info/platform/73\n- https://fedidb.org/software/lemmy\n- https://fedidb.org/current-events/threadiverse\n\n### Thanks / Related Lemmy Tools\n\n- https://github.com/db0/fediseer\n- https://github.com/LemmyNet/lemmy-stats-crawler\n\n# Credits\n\n- Logo made by Andy Cuccaro (@andycuccaro) under the CC-BY-SA 4.0 license.\n- Lemmy Developers and Community for creating [Lemmy](https://github.com/LemmyNet).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftgxn%2Flemmy-explorer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftgxn%2Flemmy-explorer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftgxn%2Flemmy-explorer/lists"}