{"id":26070321,"url":"https://github.com/dappforce/subsocial-squid","last_synced_at":"2025-08-11T21:15:33.482Z","repository":{"id":102535753,"uuid":"592271150","full_name":"dappforce/subsocial-squid","owner":"dappforce","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-31T10:39:50.000Z","size":4009,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-08T23:15:54.107Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dappforce.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-23T11:14:27.000Z","updated_at":"2024-10-31T10:39:55.000Z","dependencies_parsed_at":"2024-01-17T02:21:33.300Z","dependency_job_id":"417cdc10-e657-47c2-a7fe-b7aec5b90638","html_url":"https://github.com/dappforce/subsocial-squid","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"purl":"pkg:github/dappforce/subsocial-squid","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dappforce%2Fsubsocial-squid","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dappforce%2Fsubsocial-squid/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dappforce%2Fsubsocial-squid/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dappforce%2Fsubsocial-squid/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dappforce","download_url":"https://codeload.github.com/dappforce/subsocial-squid/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dappforce%2Fsubsocial-squid/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269958208,"owners_count":24503528,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-11T02:00:10.019Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-08T23:15:56.080Z","updated_at":"2025-08-11T21:15:33.463Z","avatar_url":"https://github.com/dappforce.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Subsocial squid\n\n#### Indexer for work with Subsocial, Soonsocial and xSocial chains.\n\n## Quick running\n\n```bash\n# 1. Update Squid SDK and install dependencies\nnpm run update\nnpm ci\n# 2. Compile typescript files\nmake build\n# 3. Start target Postgres database and detach\nmake up\n\n# 4. Set env variable for specifying target blockchain\nexport CHAIN=subsocial # if you need rung indexer for subsocial network\n# or\nexport CHAIN=soonsocial # if you need rung indexer for soonsocial network\n\n# 5. Start the processor\nmake process\n# 6. The command above will block the terminal\n#    being busy with fetching the chain data,\n#    transforming and storing it in the target database.\n#\n#    To start the graphql server open the separate terminal\n#    and run\nmake serve\n```\n\n### Defined events in squid\n\n```graphql\n  PostCreated\n  PostDeleted # synthetic\n  PostUpdated\n  PostShared # synthetic\n  PostMoved\n  PostFollowed # synthetic\n  PostUnfollowed # synthetic\n  PostReactionCreated\n  PostReactionUpdated\n  PostReactionDeleted\n\n  SpaceCreated\n  SpaceUpdated\n  SpaceFollowed\n  SpaceUnfollowed\n  SpaceOwnershipTransferAccepted\n\n  AccountFollowed\n  AccountUnfollowed\n  ProfileUpdated\n\n  CommentCreated # synthetic\n  CommentDeleted # synthetic\n  CommentUpdated # synthetic\n  CommentShared # synthetic\n  CommentReactionCreated # synthetic\n  CommentReactionUpdated # synthetic\n  CommentReactionDeleted # synthetic\n  CommentReplyCreated # synthetic\n  CommentReplyDeleted # synthetic\n  CommentReplyUpdated # synthetic\n  CommentReplyShared # synthetic\n  CommentReplyReactionCreated # synthetic\n  CommentReplyReactionUpdated # synthetic\n  CommentReplyReactionDeleted # synthetic\n```\n\n### Short info about indexer multi-chain structure\n\n1. Chain configs/endpoints can be configured in `config.ts` file in appropriate folder of necessary chain:\n   - Subsocial - `src/chains/subsocial/config.ts`\n   - Soonsocial - `src/chains/soonsocial/config.ts`\n   - xSocial - `src/chains/xSocial/config.ts`\n2. Necessary events/calls/storage calls can be configured for each chain in appropriate file in `/typegen` folder.\n3. Command `make typegen` uses custom [shel script](./scripts/typegen.sh) which generates types for each chain defined in `/typegen` folder.\n4. Chain sensitive logic is implemented in sub-folder with appropriate name in `/src/chains` folder.\n   It's required because each chain has it's own bunch of specs and as result it's own autogenerated types.\n   Chain sensitive logic:\n   - parsing events data\n   - parsing calls data\n   - making storage calls\n5. Indexer can be deployed to Aquarium to 2 different squids with 2 separate deployment manifests:\n   - Subsocial - [./squid-subsocial.yaml](./squid-subsocial.yaml)\n   - Soonsocial - [./squid-soonsocial.yaml](./squid-soonsocial.yaml)\n   - xSocial - [./squid-xsocial.yaml](./squid-xsocial.yaml)\n\n## Search API\n\nThe Indexer GraphQL API has a search query (`searchQuery`) with various parameters for full text search and filtering.\n\n### Search query arguments:\n\n- `indexes?: \u003cstring\u003e` [ _default:_ `all` ] - _search within all indices, or a specific index (`all | spaces | posts`)_\n- `limit?: \u003cnumber\u003e` [ _default:_ `10` ] - _the number of search results per page._\n- `offset?: \u003cnumber | null\u003e` [ _default:_ `0` ] - _the offset of the search results._\n- `q?: \u003cstring\u003e` [ _default:_ `*` ] - _the search query._\n- `spaceId?: \u003cstring\u003e` - _filter the search results by the provided `spaceId`._\n- `tags?: \u003cstring[]\u003e` - _filter the search results by the provided tags._\n\nAll arguments listed above can be used together in any combination, except for `spaceId + indexes:spaces`.\n\n### Search query results:\n\n- `err` - _the search result if an error occurred. It can be `null` if no error occurred._\n\n  - `reason: \u003cstring\u003e` - _the text message of the error._\n  - `status: \u003cnumber\u003e` - _the status code of the error._\n\n- `hits` - _The list of search results_.\n  - `_content` - _the source of the document._\n    - `name: \u003cstring\u003e` - _the value of the `name` field (actual only for Space entity)._\n    - `about: \u003cstring\u003e` - _the value of the `about` field (actual only for Space entity)._\n    - `username: \u003cstring\u003e` - _the value of the `username` field (actual only for Space entity)._\n    - `title: \u003cstring\u003e` - _the value of the `title` field (actual only for Post entity)._\n    - `body: \u003cstring\u003e` - _the value of the `body` field (actual only for Post entity)._\n    - `spaceId: \u003cstring\u003e` - _the value of the `spaceId` field (actual only for Post entity)._\n    - `tags: \u003cstring[]\u003e` - _a list of the tags._\n  - `_id: \u003cstring\u003e` - _the document ID (equal to the on-chain entity's ID)._\n  - `_index: \u003cstring\u003e` - _index particular document is located in._\n  - `_score: \u003cnumber\u003e` - _the search score of a particular document._\n- `total` - _the total metadata for a particular search request._\n  - `limit: \u003cstring\u003e` - _number of search results per page that was used for this particular search request._\n  - `maxScore: \u003cstring\u003e` - _the maximum score within the scope of this particular search's results._\n  - `offset: \u003cstring\u003e` - _the page offset that was used for this particular search request._\n  - `totalResults: \u003cstring\u003e` - _the total number of results matched to this particular search request._\n\nMore detailed information about the search API's schema structure can be found in the appropriate [model file](./src/server-extension/models/elasticSearchQuery.model.ts).\n\n## Analytics API\n\nThe Indexer GraphQL API has a analytics queries with various parameters.\n\n### Active users total number :: `activeUsersTotalCount`\n[Query](./src/server-extension/query/userRetentionCountFull.ts)\n\nThe API query returns the count of users who have created at least one post within a specified period.\n\n```graphql\nquery MyQuery($from: String!, $to: String!) {\n    activeUsersTotalCount(from: $from, to: $to){\n        account_count\n    }\n}\n\n# Variables:\n{\n    from: \"2023-04-13T09:38:00.027Z\",\n    to: \"2023-07-13T09:38:00.027Z\",\n}\n```\n\n### Active users total number with filters :: `activeUsersTotalCountWithFilters`\n\n[Query](./src/server-extension/query/activeUsersTotalCountWithFilters.ts)\n\nThe API query returns the count of users who have created specified number of posts within a specified period.\n\n```graphql\nquery MyQuery(\n    $from: String!,\n    $to: String!,\n    $total_min_posts_number: Int!,\n    $exclude_body: [String]\n) {\n    activeUsersTotalCountWithFilters(\n        from: $from,\n        to: $to,\n        total_min_posts_number: $total_min_posts_number,\n        exclude_body: $exclude_body\n    ){\n        account_count\n    }\n}\n\n# Variables:\n{\n    from: \"2023-04-13T09:38:00.027Z\",\n    to: \"2023-07-13T09:38:00.027Z\",\n    total_min_posts_number: 3,\n    exclude_body: [\"Hi\", \"Hello there\"]\n}\n```\n\n### User retention count :: `userRetentionCount`\n[Query](./src/server-extension/query/userRetentionCountFull.ts)\n\nThe API query returns the count of Accounts based on the following rules:\n\n1. The first post created by an account must fall within a specific timeframe (`full_query_range_from` and `full_query_range_to`), which represents the query timeframe.\n2. The total count of posts created by an account within the query timeframe must exceed a specified number (`total_min_posts_number`).\n3. The account must create at least a specified number of messages (`first_range_min_posts_number`) within the first time range (`first_range_from` and `first_range_to`).\n4. The account must create at least a specified number of messages (`last_range_min_posts_number`) within the second time range (`last_range_from` and `last_range_to`).\n5. Posts that have the field `body` equal to any of the restricted words or sentences (`exclude_body`) must be excluded from the calculation.\"\n\n```graphql\nquery MyQuery(\n  $full_query_range_from: String!,\n  $full_query_range_to: String!,\n  $first_range_from: String!,\n  $first_range_to: String!,\n  $last_range_from: String!,\n  $last_range_to: String!,\n  $total_min_posts_number: Int!,\n  $first_range_min_posts_number: Int!,\n  $last_range_min_posts_number: Int!,\n  $exclude_body: [String!]!,\n) {\n  userRetentionCount(\n    full_query_range_from: $full_query_range_from,\n    full_query_range_to: $full_query_range_to,\n    first_range_from: $first_range_from,\n    first_range_to: $first_range_to,\n    last_range_from: $last_range_from,\n    last_range_to: $last_range_to,\n    total_min_posts_number: $total_min_posts_number,\n    first_range_min_posts_number: $first_range_min_posts_number,\n    last_range_min_posts_number: $last_range_min_posts_number,\n    exclude_body: $exclude_body\n) {\n    retention_count\n  }\n}\n\n# Variables:\n\n{\n    \"full_query_range_from\": \"2023-07-13T09:38:00.027Z\",\n    \"full_query_range_to\": \"2023-07-17T15:31:16.677Z\",\n    \"first_range_from\": \"2023-07-13T09:38:00.027Z\",\n    \"first_range_to\": \"2023-07-15T09:38:00.027Z\",\n    \"last_range_from\": \"2023-07-16T09:38:00.027Z\",\n    \"last_range_to\": \"2023-07-17T15:31:16.677Z\",\n    \"total_min_posts_number\": 5,\n    \"first_range_min_posts_number\": 3,\n    \"last_range_min_posts_number\": 2,\n    \"exclude_body\": [\"Hello\"]\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdappforce%2Fsubsocial-squid","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdappforce%2Fsubsocial-squid","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdappforce%2Fsubsocial-squid/lists"}