{"id":22660888,"url":"https://github.com/maski0/aicompanion","last_synced_at":"2025-03-29T08:25:07.660Z","repository":{"id":266880397,"uuid":"888979188","full_name":"Maski0/AICompanion","owner":"Maski0","description":null,"archived":false,"fork":false,"pushed_at":"2025-01-16T07:10:43.000Z","size":70697,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-03T22:43:39.102Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://ai-companion-eight-alpha.vercel.app","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Maski0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-15T11:24:28.000Z","updated_at":"2025-01-12T07:44:22.000Z","dependencies_parsed_at":"2025-01-12T06:34:25.223Z","dependency_job_id":null,"html_url":"https://github.com/Maski0/AICompanion","commit_stats":null,"previous_names":["maski0/aicompanion"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Maski0%2FAICompanion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Maski0%2FAICompanion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Maski0%2FAICompanion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Maski0%2FAICompanion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Maski0","download_url":"https://codeload.github.com/Maski0/AICompanion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246158521,"owners_count":20732815,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-09T11:12:45.902Z","updated_at":"2025-03-29T08:25:07.637Z","avatar_url":"https://github.com/Maski0.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Companion\n\n3D:\n\n- Open AI, Azure TTS with visemes\n\n##\n\n2D:\n\n- Live2D Cubism , Elize (Eliza provided AI, ElevenLabs)\n##\n\n## Custom Endpoint in Eliza Framework\n\n\u003e in packages\\client-direct\\src\\index.ts\n\n```\nthis.app.post(\"/:agentId/speaktext\", async (req, res) =\u003e {\n  const TotalstartTime = performance.now();\n  const agentId = req.params.agentId;\n  const roomId = stringToUuid(req.body.roomId ?? \"default-room-\" + agentId);\n  const userId = stringToUuid(req.body.userId ?? \"user\");\n  const text = req.body.text;\n\n  if (!text) {\n    res.status(400).send(\"No text provided\");\n    return;\n  }\n\n  let runtime = this.agents.get(agentId);\n\n  // if runtime is null, look for runtime with the same name\n  if (!runtime) {\n    runtime = Array.from(this.agents.values()).find(\n      (a) =\u003e a.character.name.toLowerCase() === agentId.toLowerCase()\n    );\n  }\n\n  if (!runtime) {\n    res.status(404).send(\"Agent not found\");\n    return;\n  }\n\n  try {\n    // Process message through agent (same as /message endpoint)\n    await runtime.ensureConnection(\n      userId,\n      roomId,\n      req.body.userName,\n      req.body.name,\n      \"direct\"\n    );\n\n    const messageId = stringToUuid(Date.now().toString());\n\n    const content: Content = {\n      text,\n      attachments: [],\n      source: \"direct\",\n      inReplyTo: undefined,\n    };\n\n    const userMessage = {\n      content,\n      userId,\n      roomId,\n      agentId: runtime.agentId,\n    };\n\n    const memory: Memory = {\n      id: messageId,\n      agentId: runtime.agentId,\n      userId,\n      roomId,\n      content,\n      createdAt: Date.now(),\n    };\n\n    await runtime.messageManager.createMemory(memory);\n\n    const state = await runtime.composeState(userMessage, {\n      agentName: runtime.character.name,\n    });\n\n    const context = composeContext({\n      state,\n      template: messageHandlerTemplate,\n    });\n    const GenerateResponseStartTime = performance.now();\n\n    const response = await generateMessageResponse({\n      runtime: runtime,\n      context,\n      modelClass: ModelClass.LARGE,\n    });\n    const GenerateResponseendTime = performance.now();\n    console.log(\n      `Genereate Response Time took ${(GenerateResponseendTime - GenerateResponseStartTime).toFixed(2)} ms`\n    );\n\n    // save response to memory\n    const responseMessage = {\n      ...userMessage,\n      userId: runtime.agentId,\n      content: response,\n    };\n\n    await runtime.messageManager.createMemory(responseMessage);\n\n    if (!response) {\n      res.status(500).send(\"No response from generateMessageResponse\");\n      return;\n    }\n\n    await runtime.evaluate(memory, state);\n\n    const _result = await runtime.processActions(\n      memory,\n      [responseMessage],\n      state,\n      async () =\u003e {\n        return [memory];\n      }\n    );\n\n    // Get the text to convert to speech\n    const textToSpeak = response.text;\n    const elevenLabStratTime = performance.now();\n\n    // Convert to speech using ElevenLabs\n    const elevenLabsApiUrl = `https://api.elevenlabs.io/v1/text-to-speech/${process.env.ELEVENLABS_VOICE_ID}`;\n    const apiKey = process.env.ELEVENLABS_XI_API_KEY;\n\n    if (!apiKey) {\n      throw new Error(\"ELEVENLABS_XI_API_KEY not configured\");\n    }\n\n    const speechResponse = await fetch(elevenLabsApiUrl, {\n      method: \"POST\",\n      headers: {\n        \"Content-Type\": \"application/json\",\n        \"xi-api-key\": apiKey,\n      },\n      body: JSON.stringify({\n        text: textToSpeak,\n        model_id: process.env.ELEVENLABS_MODEL_ID || \"eleven_multilingual_v2\",\n        voice_settings: {\n          stability: parseFloat(\n            process.env.ELEVENLABS_VOICE_STABILITY || \"0.5\"\n          ),\n          similarity_boost: parseFloat(\n            process.env.ELEVENLABS_VOICE_SIMILARITY_BOOST || \"0.9\"\n          ),\n          style: parseFloat(process.env.ELEVENLABS_VOICE_STYLE || \"0.66\"),\n          use_speaker_boost:\n            process.env.ELEVENLABS_VOICE_USE_SPEAKER_BOOST === \"true\",\n        },\n      }),\n    });\n\n    if (!speechResponse.ok) {\n      throw new Error(`ElevenLabs API error: ${speechResponse.statusText}`);\n    }\n    const elevenLabEndTime = performance.now();\n    console.log(\n      `ElevenLabs Time took ${(elevenLabEndTime - elevenLabStratTime).toFixed(2)} ms`\n    );\n\n    const audioBuffer = await speechResponse.arrayBuffer();\n\n    // Create a multipart response with both text and audio\n    const boundary = \"boundary-\" + Date.now().toString(16);\n    res.set({\n      \"Content-Type\": `multipart/mixed; boundary=${boundary}`,\n      \"Transfer-Encoding\": \"chunked\",\n    });\n\n    // Write text part\n    res.write(\n      `--${boundary}\\r\\n` +\n        \"Content-Type: application/json\\r\\n\\r\\n\" +\n        JSON.stringify({\n          text: textToSpeak,\n          messageId: messageId,\n          timestamp: Date.now(),\n        }) +\n        \"\\r\\n\"\n    );\n\n    // Write audio part\n    res.write(\n      `--${boundary}\\r\\n` +\n        \"Content-Type: audio/mpeg\\r\\n\" +\n        `Content-Length: ${audioBuffer.byteLength}\\r\\n\\r\\n`\n    );\n    res.write(Buffer.from(audioBuffer));\n\n    // End the multipart message\n    res.write(`\\r\\n--${boundary}--`);\n    res.end();\n  } catch (error) {\n    console.error(\"Error processing message or generating speech:\", error);\n    res.status(500).json({\n      error: \"Error processing message or generating speech\",\n      details: error.message,\n    });\n  }\n  const TotalendTime = performance.now();\n  console.log(\n    `Endpoint Time took ${(TotalendTime - TotalstartTime).toFixed(2)} ms`\n  );\n});\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaski0%2Faicompanion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaski0%2Faicompanion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaski0%2Faicompanion/lists"}