{"id":23143745,"url":"https://github.com/digitalinteraction/catalyst-api","last_synced_at":"2025-04-04T12:15:00.856Z","repository":{"id":49604977,"uuid":"173954113","full_name":"digitalinteraction/catalyst-api","owner":"digitalinteraction","description":"A dockerized API to serve Not-Equal Catalyst projects from a redis database","archived":false,"fork":false,"pushed_at":"2021-08-31T12:22:16.000Z","size":826,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-09T22:11:22.712Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://api.catalyst.not-equal.tech","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/digitalinteraction.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-05T13:42:45.000Z","updated_at":"2024-11-08T12:31:39.000Z","dependencies_parsed_at":"2022-09-15T02:12:55.246Z","dependency_job_id":null,"html_url":"https://github.com/digitalinteraction/catalyst-api","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digitalinteraction%2Fcatalyst-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digitalinteraction%2Fcatalyst-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digitalinteraction%2Fcatalyst-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digitalinteraction%2Fcatalyst-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/digitalinteraction","download_url":"https://codeload.github.com/digitalinteraction/catalyst-api/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247174467,"owners_count":20896078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-17T15:14:08.748Z","updated_at":"2025-04-04T12:15:00.832Z","avatar_url":"https://github.com/digitalinteraction.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Catalyst | Node.js API\n\nThis is the repo for Not-Equal Catalyst's API.\nIt is a [ChowChow](https://www.npmjs.com/package/@robb_j/chowchow) app,\nusing [node.js](https://nodejs.org) \u0026 [express](https://expressjs.com/),\nwritten in [TypeScript](https://www.typescriptlang.org/)\nand deployed through [Docker](https://www.docker.com/).\nIt serves data from a [redis](https://redis.io/) database, which is put there by [catalyst-trello-scraper](https://github.com/unplatform/catalyst-trello-scraper).\n\n[What is Not-Equal Catalyst?](https://github.com/unplatform/catalyst-about)\n\n\u003c!-- toc-head --\u003e\n\n## Table of Contents\n\n- [What is this](#what-is-this)\n- [Development](#development)\n  - [Setup](#setup)\n  - [Regular use](#regular-use)\n  - [Irregular use](#irregular-use)\n  - [Code Structure](#code-structure)\n  - [Code formatting](#code-formatting)\n  - [Testing](#testing)\n- [Deployment](#deployment)\n  - [Building the image](#building-the-image)\n  - [Using the image](#using-the-image)\n  - [Environment variables](#environment-variables)\n  - [Endpoints](#endpoints)\n- [Future Work](#future-work)\n\n\u003c!-- toc-tail --\u003e\n\n## What is this\n\nThis repo provides a http API to fetch projects from a redis cache\nwhich have been put there by [unplatform/catalyst-trello-scraper](https://github.com/unplatform/catalyst-trello-scraper).\nCards are retrieved from a redis instances and are sent to the client as JSON.\nLightweight analytics are written to a mongo database.\n\nThe endpoints follow a meta-data envelope, where `data` is the endpoint payload:\n\n```json\n{\n  \"meta\": {\n    \"success\": true,\n    \"messages\": [],\n    \"status\": 200,\n    \"name\": \"@openlab/catalyst-node-api\",\n    \"version\": \"0.2.0\"\n  },\n  \"data\": { \"msg\": \"Hey!\" }\n}\n```\n\n## Usage\n\nThis repo generates a Docker image which runs the node.js server on port `3000`.\n\nHere's an example with docker-compose:\n\n```yml\nversion: '3'\n\nservices:\n  node-api:\n    image: ghcr.io/digitalinteraction/catalyst-api\n    restart: unless-stopped\n    ports:\n      - 3000:3000\n    environment:\n      WEB_URL: https://catalyst.not-equal.tech\n      REDIS_URL: redis://your_redis_url\n      MONGO_URL: mongodb://your_mongo_url\n      LOG_LEVEL: info\n```\n\n\u003e Get the [latest version](https://github.com/digitalinteraction/catalyst-api/pkgs/container/catalyst-api)\n\n### Environment variables\n\nThere are some required and some option environment variables, shown below.\n\n| Variable       | Description                                                |\n| -------------- | ---------------------------------------------------------- |\n| WEB_URL        | **required** Where the web ui is, used to set cors headers |\n| REDIS_URL      | **required** The connection details of the redis database  |\n| MONGO_URL      | **required** The connection details of the mongo database  |\n| LOG_LEVEL      | (optional) How much logging to generate                    |\n| ENABLE_SOCKETS | (optional) Enable the websocket analytics server           |\n\n\u003e For information about LOG_LEVEL see [chowchow-logger](https://github.com/robb-j/chowchow-logger#environment-variables).\n\u003e Logs are written to `/app/logs` which is an internal volume by default,\n\u003e You can bind those logs to the host machine if you want.\n\n### Endpoints\n\nThere are 5 endpoints\n\n| Route           | Description                                       |\n| --------------- | ------------------------------------------------- |\n| `GET: /`        | An endpoint to let you know everything is working |\n| `GET: /cards`   | Retrieve cards from the redis database            |\n| `GET: /labels`  | Get the labels which can be filtered by           |\n| `GET: /content` | Retrieve site config from the redis database      |\n| `GET: /stats`   | Get analytics stats about site usage              |\n\nThere are also development endpoints when `NODE_ENV=development`\n\n| Route                | Description                                 |\n| -------------------- | ------------------------------------------- |\n| `GET: /dev/errors`   | Get errors that have happened in the webapp |\n| `GET: /dev/stats`    | Get raw stats                               |\n| `GET: /dev/searches` | Get searches that have occured              |\n\n### Sockets\n\nThe site uses sockets for anonymous analytics.\nA JSON payload is expected with a root level `type` which is used to route the socket to a handler.\nAny message with unknown type or incorrect body is silently ignored.\n\n| Type             | Description                                                |\n| ---------------- | ---------------------------------------------------------- |\n| `echo`           | A test socket which echo's back your body (sans type)      |\n| `page_view`      | Register a page was viewed, params: `path`                 |\n| `project_action` | Register an action on a project, params: `project`, `link` |\n| `client_error`   | Tell the API about an error, params: `message`, `stack`    |\n| `search_action`  | Register a user search, params: `search`, `filters`        |\n\n## Development\n\n### Setup\n\nTo develop on this repo you will need to have [Docker](https://www.docker.com/) and\n[node.js](https://nodejs.org) installed on your dev machine and have an understanding of them.\nThis guide assumes you have the repo checked out and are on macOS, but equivalent commands are available.\nYou will also need a Trello account which is used to pull the data from.\n\nYou'll only need to follow this setup once for your dev machine.\n\n```bash\n# Install dependancies\nnpm install\n\n# Setup your environment\ncp .env.example .env\n\n# Follow these instructions to get your Trello credentials\nopen https://github.com/unplatform/catalyst-trello-scraper#setup\n```\n\n### Regular use\n\nThese are the commands you'll regularly run to develop the API, in no particular order.\n\n```bash\n# Start up a redis instance seeded it with projects and mongo for analytics\n# -\u003e Run this in a new terminal tab\n# -\u003e Stop this with a Ctrl+C\n# -\u003e See `docker-compose.yml` for how it works\ndocker-compose up\n\n# Run the API in development mode\n# -\u003e Runs the TypeScript directly with `ts-node` and loads the .env\n# -\u003e Uses `nodemon` to watch for changes, which will restart the app on save.\nnpm run dev\n\n# Run unit tests\n# -\u003e Looks for files named `*.spec.ts` in the src directory\nnpm run test\n```\n\n### Irregular use\n\nThese are commands you might need to run but probably won't, also in no particular order.\n\n```bash\n# Generate the table of contents for this readme\n# -\u003e It'll replace content between the toc-head and toc-tail HTML comments\nnpm run gen-readme-toc\n\n# Manually lint code with TypeScript's `tsc`\nnpm run lint\n\n# Manually format code\n# -\u003e This repo is setup to automatically format code on git-push\nnpm run prettier\n\n# Manually transpile TypeScript to JavaScript\n# -\u003e This is part of the docker build which is triggered when deploying\n# -\u003e Writes files to dist, which is git-ignored\nnpm run build\n\n# Manually start code from transpilled JavaScript\n# -\u003e It'll automatically load your local .env\nnpm run start\n```\n\n### Commits\n\nAll commits to this repo must follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).\nThis ensures changes are structured and means the [CHANGELOG.md](/CHANGELOG.md) can be automatically generated.\n\nThis standard is enforced through a `commit-msg` hook using [yorkie](https://www.npmjs.com/package/yorkie).\n\n### Code Structure\n\n| Folder       | Contents                                     |\n| ------------ | -------------------------------------------- |\n| dist         | Where the transpilled JavaScript is built to |\n| logs         | Where access and error logs are stored       |\n| node_modules | Where npm's modules get installed into       |\n| src          | Where the code of the app is                 |\n\n### Code formatting\n\nThis repo uses [Prettier](https://prettier.io/) to automatically format code to a consistent standard.\nThis works using the [husky](https://www.npmjs.com/package/husky)\nand [lint-staged](https://www.npmjs.com/package/lint-staged) packages to\nautomatically format code whenever you commit code.\nThis means that code that is pushed to the repo is always formatted to a consistent standard.\n\nYou can manually run the formatter with `npm run prettier` if you want.\n\nPrettier is slightly configured in [.prettierrc.yml](/.prettierrc.yml)\nand also ignores files using [.prettierignore](/.prettierignore).\n\n### Testing\n\n\u003e This API is currently very simple and doesn't have any unit tests yet\n\nThis repo uses [unit tests](https://en.wikipedia.org/wiki/Unit_testing) to ensure that everything is working correctly, guide development, avoid bad code and reduce defects.\nThe [Jest](https://www.npmjs.com/package/jest) package is used to run unit tests.\nTests are any file in `src/` that end with `.spec.ts`, by convention they are inline with the source code,\nin a parallel folder called `__tests__`.\n\n```bash\n# Run the tests\nnpm test -s\n\n# Generate code coverage\nnpm run coverage -s\n```\n\n## Deployment\n\n### Building the image\n\nThis repo uses a [GitLab CI](https://about.gitlab.com/product/continuous-integration/)\nto build a Docker image when you push a git tag.\nThis is designed to be used with the `npm version` command so all docker images are [semantically versioned](https://semver.org/).\nThe `:latest` docker tag is not used.\n\nThis job runs using the [.gitlab-ci.yml](/.gitlab-ci.yml) file which\nruns a docker build using the [Dockerfile](/Dockerfile)\nand **only** runs when you push a [tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging).\n\nIt pushes these docker images to the GitLab registry of the repo.\nA slight nuance is that it will replace a preceding `v` in tag names, formatting `v1.0.0` to `1.0.0`.\n\n```bash\n# Generate a new release\n# -\u003e Generates a new version based on the commits since the last version\n# -\u003e Generates the CHANGELOG.md based on those commits\n# -\u003e There is a \"preversion\" script to lint \u0026 run tests\nnpm run release\n\n# Push the new version\n# -\u003e The GitLab CI will build a new docker image for it\ngit push --follow-tags\n```\n\n## Future work\n\n- n/a\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigitalinteraction%2Fcatalyst-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdigitalinteraction%2Fcatalyst-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigitalinteraction%2Fcatalyst-api/lists"}