{"id":23357519,"url":"https://github.com/cs-magic-open/eval-ai","last_synced_at":"2025-06-22T20:38:23.510Z","repository":{"id":253128390,"uuid":"762974324","full_name":"cs-magic-open/eval-ai","owner":"cs-magic-open","description":null,"archived":false,"fork":false,"pushed_at":"2024-11-18T22:03:15.000Z","size":1846,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-13T21:45:29.453Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cs-magic-open.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-25T08:03:16.000Z","updated_at":"2024-11-18T22:03:19.000Z","dependencies_parsed_at":"2024-08-14T17:15:30.553Z","dependency_job_id":"fd1f7d59-d5db-4ad5-b0b6-6dfbbceccb1a","html_url":"https://github.com/cs-magic-open/eval-ai","commit_stats":null,"previous_names":["cs-magic/eval-ai","cs-magic-open/eval-ai"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cs-magic-open%2Feval-ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cs-magic-open%2Feval-ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cs-magic-open%2Feval-ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cs-magic-open%2Feval-ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cs-magic-open","download_url":"https://codeload.github.com/cs-magic-open/eval-ai/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247721920,"owners_count":20985084,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-21T10:29:20.971Z","updated_at":"2025-04-07T20:17:24.033Z","avatar_url":"https://github.com/cs-magic-open.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Eval AI !\n\n## Todo\n\n- [ ] fix eval-ai running prisma\n- [ ] 研究怎么把 trpc 静态编译那玩意给 generic 了\n\n## Architecture\n\n### 要点\n\n- 后台静默流是可以不做的（或者高级版？），因为需要维护一个进程池，没有必要，用户打断我们即中止\n- 消息的回溯是可以不做的（或者高级版？），因为要考虑数据库同步问题，以及组装上下文到底是在客户端还是服务端\n- 【乐观刷新】前期为了保证开发速度与质量，可以暂时先不用\n- 【状态管理】jotai vs zustand vs vs valtio\n- 【trpc状态同步】在列表页上，我们还是用了invalidate机制，避免了手写各种convs的更新与对齐，然后本地的convs始终对齐trcp的列表数据即可\n- 【llm-manager同步】（2024-03-09）在经历了痛苦的同步控制，最后不得不用上redis并陷入序列化的困境后，我最终决定放弃一切幻想all in pusher\n\n\n### Conversation\n\n- queryConv --\u003e queryRequest --\u003e queryResponse\n- queryRequest = queryContext + n * queryConfig\n- queryConfig -- queryResponse\n\n### api ref\n\n- openai: https://github.com/openai/openai-python/issues/952#issuecomment-1857207339\n\n## Tech\n\n### [Init] install\n\n```shell\n# ref: https://github.com/yarnpkg/berry/issues/3521#issuecomment-1907517793\n# 阿里的库会有证书过期问题\nyarn config set \"strict-ssl\" false -g\n```\n\n### [zod-prisma-types] \n\n- zod 的版本有要求（不能太高）：`yarn add zod@3.21.1`，参考：https://github.com/chrishoermann/zod-prisma-types/issues/222\n- 之前在开发 lit-agent 项目的时候有 build bug，现在重新试了一下，又没了。。\n- 且用且看吧\n\n### [NextJS] server action 在 dev 里可以放在 route 里，但 build 时会失败\n\n### [DevOps] 控制台颜色\n\n首先，我们读了 trpc 的 loggerLink 源码，它是用 ansi/css 两种写法分别决定 node/browser 颜色的\n\n我尝试了 colors, chalk（[基于gpt](https://chat.openai.com/c/756f58ea-4d30-4b74-9c52-e4847dd2fdbf)），都只能 node 用\n\n最后在[ chalk 的一个 issue](https://github.com/chalk/chalk/issues/535#issuecomment-1072761585) 里顺着别人的路用了 `ansi-colors`\n\n### [Core] Websocket Integration\n\n- vercel 不推荐，参考：\n  - Do Vercel Serverless Functions support WebSocket connections?, https://vercel.com/guides/do-vercel-serverless-functions-support-websocket-connections#enabling-realtime-communication\n  - (1) Websockets with nextjs 13 : nextjs, https://www.reddit.com/r/nextjs/comments/13360t3/websockets_with_nextjs_13/\n\n- 使用 soketi\n  - pm2 的时候要注意，`soketi-pm2` 不支持 config，不如直接 `pm2 start soketi -- start --config=xx`\n    - [bug] PM2 doesn't pass through command line arguments on Windows · Issue #320 · soketi/soketi, https://github.com/soketi/soketi/issues/320\n\n### [Polyfill] tsx 导入 Credentials\n\n```ts\nimport CredentialsModule from \"next-auth/providers/credentials\"\n\n// sb tsx 需要用 default\nconst Credentials = (\n  \"default\" in CredentialsModule ? CredentialsModule.default : CredentialsModule\n) as typeof CredentialsModule\n```\n\n\n### [UI] text with gradient color\n\n- How to do gradient text with Tailwind CSS, https://design2tailwind.com/blog/tailwindcss-gradient-text/\n\n### [Backend] nginx\n\n- redirect problem from `http` to `https`, see: https://serverfault.com/a/680592\n  - solution: 把 `$servername` 改成 `$host` 就可以了\n\n### [Core] wechat-auth\n\n虽然微信登录起步是访问一个 open-wechat 的网址，但不能直接点击跳转，而是在 provider 的 authorizationUrl 里配置好后，在程序里使用 `signIn` 去操作，否则会报 `State cookie was missing`。\n\n### [Core] zustand\n\n- zustand doesn't recommend `create without curried workaround`, see: https://github.com/pmndrs/zustand/blob/main/docs/guides/typescript.md#create-without-curried-workaround\n\n## BUGFIX\n\n### sonner 版本要求\n\n这个不行： `\"sonner\": \"^1.4.0\",`，参考：https://github.com/emilkowalski/sonner/issues/353\n\n### jotai 不可以 在 set 里 set writable variable， 那是用于 useAtom 的，见：\n\n![img.png](docs/jotai-writeable.png)\n\nref:\n- https://jotai.org/docs/core/atom\n- https://chat.openai.com/c/52cf8ab1-de90-4249-8239-322d2f860d33\n\n### - [x] `import sum from \"lodash/sum\"`\n\n问题： 这个会导致 `yarn update` 在 `spawn` 里编译时失败。\n\n解决：将 `@types/lodash` 加入 prod 依赖即可。\n\n原因分析：还没有完全明白，就是在全新的shell里编译好像会导致node_modules不符合预期，这个可以做一个新的todo。\n\n\n## How do I deploy this?\n\nFollow our deployment guides for [Vercel](https://create.t3.gg/en/deployment/vercel), [Netlify](https://create.t3.gg/en/deployment/netlify) and [Docker](https://create.t3.gg/en/deployment/docker) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcs-magic-open%2Feval-ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcs-magic-open%2Feval-ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcs-magic-open%2Feval-ai/lists"}