https://github.com/dappforce/subsocial-squid
https://github.com/dappforce/subsocial-squid
Last synced: 11 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/dappforce/subsocial-squid
- Owner: dappforce
- License: gpl-3.0
- Created: 2023-01-23T11:14:27.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-10-31T10:39:50.000Z (over 1 year ago)
- Last Synced: 2025-03-08T23:15:54.107Z (over 1 year ago)
- Language: TypeScript
- Size: 3.82 MB
- Stars: 0
- Watchers: 2
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Subsocial squid
#### Indexer for work with Subsocial, Soonsocial and xSocial chains.
## Quick running
```bash
# 1. Update Squid SDK and install dependencies
npm run update
npm ci
# 2. Compile typescript files
make build
# 3. Start target Postgres database and detach
make up
# 4. Set env variable for specifying target blockchain
export CHAIN=subsocial # if you need rung indexer for subsocial network
# or
export CHAIN=soonsocial # if you need rung indexer for soonsocial network
# 5. Start the processor
make process
# 6. The command above will block the terminal
# being busy with fetching the chain data,
# transforming and storing it in the target database.
#
# To start the graphql server open the separate terminal
# and run
make serve
```
### Defined events in squid
```graphql
PostCreated
PostDeleted # synthetic
PostUpdated
PostShared # synthetic
PostMoved
PostFollowed # synthetic
PostUnfollowed # synthetic
PostReactionCreated
PostReactionUpdated
PostReactionDeleted
SpaceCreated
SpaceUpdated
SpaceFollowed
SpaceUnfollowed
SpaceOwnershipTransferAccepted
AccountFollowed
AccountUnfollowed
ProfileUpdated
CommentCreated # synthetic
CommentDeleted # synthetic
CommentUpdated # synthetic
CommentShared # synthetic
CommentReactionCreated # synthetic
CommentReactionUpdated # synthetic
CommentReactionDeleted # synthetic
CommentReplyCreated # synthetic
CommentReplyDeleted # synthetic
CommentReplyUpdated # synthetic
CommentReplyShared # synthetic
CommentReplyReactionCreated # synthetic
CommentReplyReactionUpdated # synthetic
CommentReplyReactionDeleted # synthetic
```
### Short info about indexer multi-chain structure
1. Chain configs/endpoints can be configured in `config.ts` file in appropriate folder of necessary chain:
- Subsocial - `src/chains/subsocial/config.ts`
- Soonsocial - `src/chains/soonsocial/config.ts`
- xSocial - `src/chains/xSocial/config.ts`
2. Necessary events/calls/storage calls can be configured for each chain in appropriate file in `/typegen` folder.
3. Command `make typegen` uses custom [shel script](./scripts/typegen.sh) which generates types for each chain defined in `/typegen` folder.
4. Chain sensitive logic is implemented in sub-folder with appropriate name in `/src/chains` folder.
It's required because each chain has it's own bunch of specs and as result it's own autogenerated types.
Chain sensitive logic:
- parsing events data
- parsing calls data
- making storage calls
5. Indexer can be deployed to Aquarium to 2 different squids with 2 separate deployment manifests:
- Subsocial - [./squid-subsocial.yaml](./squid-subsocial.yaml)
- Soonsocial - [./squid-soonsocial.yaml](./squid-soonsocial.yaml)
- xSocial - [./squid-xsocial.yaml](./squid-xsocial.yaml)
## Search API
The Indexer GraphQL API has a search query (`searchQuery`) with various parameters for full text search and filtering.
### Search query arguments:
- `indexes?: ` [ _default:_ `all` ] - _search within all indices, or a specific index (`all | spaces | posts`)_
- `limit?: ` [ _default:_ `10` ] - _the number of search results per page._
- `offset?: ` [ _default:_ `0` ] - _the offset of the search results._
- `q?: ` [ _default:_ `*` ] - _the search query._
- `spaceId?: ` - _filter the search results by the provided `spaceId`._
- `tags?: ` - _filter the search results by the provided tags._
All arguments listed above can be used together in any combination, except for `spaceId + indexes:spaces`.
### Search query results:
- `err` - _the search result if an error occurred. It can be `null` if no error occurred._
- `reason: ` - _the text message of the error._
- `status: ` - _the status code of the error._
- `hits` - _The list of search results_.
- `_content` - _the source of the document._
- `name: ` - _the value of the `name` field (actual only for Space entity)._
- `about: ` - _the value of the `about` field (actual only for Space entity)._
- `username: ` - _the value of the `username` field (actual only for Space entity)._
- `title: ` - _the value of the `title` field (actual only for Post entity)._
- `body: ` - _the value of the `body` field (actual only for Post entity)._
- `spaceId: ` - _the value of the `spaceId` field (actual only for Post entity)._
- `tags: ` - _a list of the tags._
- `_id: ` - _the document ID (equal to the on-chain entity's ID)._
- `_index: ` - _index particular document is located in._
- `_score: ` - _the search score of a particular document._
- `total` - _the total metadata for a particular search request._
- `limit: ` - _number of search results per page that was used for this particular search request._
- `maxScore: ` - _the maximum score within the scope of this particular search's results._
- `offset: ` - _the page offset that was used for this particular search request._
- `totalResults: ` - _the total number of results matched to this particular search request._
More detailed information about the search API's schema structure can be found in the appropriate [model file](./src/server-extension/models/elasticSearchQuery.model.ts).
## Analytics API
The Indexer GraphQL API has a analytics queries with various parameters.
### Active users total number :: `activeUsersTotalCount`
[Query](./src/server-extension/query/userRetentionCountFull.ts)
The API query returns the count of users who have created at least one post within a specified period.
```graphql
query MyQuery($from: String!, $to: String!) {
activeUsersTotalCount(from: $from, to: $to){
account_count
}
}
# Variables:
{
from: "2023-04-13T09:38:00.027Z",
to: "2023-07-13T09:38:00.027Z",
}
```
### Active users total number with filters :: `activeUsersTotalCountWithFilters`
[Query](./src/server-extension/query/activeUsersTotalCountWithFilters.ts)
The API query returns the count of users who have created specified number of posts within a specified period.
```graphql
query MyQuery(
$from: String!,
$to: String!,
$total_min_posts_number: Int!,
$exclude_body: [String]
) {
activeUsersTotalCountWithFilters(
from: $from,
to: $to,
total_min_posts_number: $total_min_posts_number,
exclude_body: $exclude_body
){
account_count
}
}
# Variables:
{
from: "2023-04-13T09:38:00.027Z",
to: "2023-07-13T09:38:00.027Z",
total_min_posts_number: 3,
exclude_body: ["Hi", "Hello there"]
}
```
### User retention count :: `userRetentionCount`
[Query](./src/server-extension/query/userRetentionCountFull.ts)
The API query returns the count of Accounts based on the following rules:
1. The first post created by an account must fall within a specific timeframe (`full_query_range_from` and `full_query_range_to`), which represents the query timeframe.
2. The total count of posts created by an account within the query timeframe must exceed a specified number (`total_min_posts_number`).
3. The account must create at least a specified number of messages (`first_range_min_posts_number`) within the first time range (`first_range_from` and `first_range_to`).
4. The account must create at least a specified number of messages (`last_range_min_posts_number`) within the second time range (`last_range_from` and `last_range_to`).
5. Posts that have the field `body` equal to any of the restricted words or sentences (`exclude_body`) must be excluded from the calculation."
```graphql
query MyQuery(
$full_query_range_from: String!,
$full_query_range_to: String!,
$first_range_from: String!,
$first_range_to: String!,
$last_range_from: String!,
$last_range_to: String!,
$total_min_posts_number: Int!,
$first_range_min_posts_number: Int!,
$last_range_min_posts_number: Int!,
$exclude_body: [String!]!,
) {
userRetentionCount(
full_query_range_from: $full_query_range_from,
full_query_range_to: $full_query_range_to,
first_range_from: $first_range_from,
first_range_to: $first_range_to,
last_range_from: $last_range_from,
last_range_to: $last_range_to,
total_min_posts_number: $total_min_posts_number,
first_range_min_posts_number: $first_range_min_posts_number,
last_range_min_posts_number: $last_range_min_posts_number,
exclude_body: $exclude_body
) {
retention_count
}
}
# Variables:
{
"full_query_range_from": "2023-07-13T09:38:00.027Z",
"full_query_range_to": "2023-07-17T15:31:16.677Z",
"first_range_from": "2023-07-13T09:38:00.027Z",
"first_range_to": "2023-07-15T09:38:00.027Z",
"last_range_from": "2023-07-16T09:38:00.027Z",
"last_range_to": "2023-07-17T15:31:16.677Z",
"total_min_posts_number": 5,
"first_range_min_posts_number": 3,
"last_range_min_posts_number": 2,
"exclude_body": ["Hello"]
}
```