{"id":31217001,"url":"https://github.com/vbrazo/contextor.desktop","last_synced_at":"2025-09-21T13:33:48.328Z","repository":{"id":313125291,"uuid":"999678358","full_name":"vbrazo/contextor.desktop","owner":"vbrazo","description":"A cross‑platform Electron application that captures on‑screen context (screenshots) and audio (microphone and system audio), analyzes it, and provides conversational, actionable insights through a clean React UI","archived":false,"fork":false,"pushed_at":"2025-09-04T02:45:58.000Z","size":560,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-04T04:34:49.017Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vbrazo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-10T16:06:47.000Z","updated_at":"2025-09-04T02:46:01.000Z","dependencies_parsed_at":"2025-09-04T04:34:54.191Z","dependency_job_id":"5913a530-7658-4fc0-bb5e-b8974b07d2eb","html_url":"https://github.com/vbrazo/contextor.desktop","commit_stats":null,"previous_names":["vbrazo/contextor.desktop"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/vbrazo/contextor.desktop","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbrazo%2Fcontextor.desktop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbrazo%2Fcontextor.desktop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbrazo%2Fcontextor.desktop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbrazo%2Fcontextor.desktop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vbrazo","download_url":"https://codeload.github.com/vbrazo/contextor.desktop/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbrazo%2Fcontextor.desktop/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276250326,"owners_count":25610372,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-21T02:00:07.055Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-21T13:33:43.189Z","updated_at":"2025-09-21T13:33:48.310Z","avatar_url":"https://github.com/vbrazo.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Contextor Desktop\n\nContextor is a cross‑platform Electron application that captures on‑screen context (screenshots) and audio (microphone and system audio), analyzes it, and provides conversational, actionable insights through a clean React UI. It showcases full‑stack desktop engineering across Electron main/preload/renderer processes, secure IPC, audio routing/mixing, and authenticated API integration.\n\n## Highlights\n\n- **Contextual assistant**: Combine screenshots, recorded audio, and chat to get targeted insights.\n- **Audio pipeline**: Loopback system audio, capture microphone, mix streams, apply echo cancellation, and playback.\n- **Typed IPC surface**: Secure bridge from renderer to main via `contextBridge` with a well‑defined API.\n- **Modern UI**: React + TypeScript with a componentized chat panel that renders Markdown and media.\n- **Production‑ready**: Hot reload in development, packaged builds for distribution.\n\n## Architecture\n\nThe app follows Electron’s three‑tier model with a focused, security‑first design:\n\n- **Main process (`src/main.ts`)**\n  - Owns application lifecycle and window management.\n  - Handles privileged operations: screen capture, audio routing/recording, auth/session state, deep links, and system integrations.\n  - Registers IPC handlers that the preload exposes to the renderer.\n\n- **Preload (`src/preload.ts`)**\n  - Runs in an isolated context and exposes a curated API on `window.api` using `contextBridge`.\n  - Example capabilities exposed to the renderer:\n    - Window control: `resizeWindow`, `setWindowSize`, `moveWindow`.\n    - Context events: `onScreenshotAnalysis`, `onAudioWithAnalysis`, `onLoadingUpdate`.\n    - Audio controls: `enableSystemAudioLoopback`, `startCombinedAudioRecording`, `stopCombinedAudioRecording`.\n    - Audio config: `setAudioConfiguration`, `getAudioConfiguration` (echo cancellation, sensitivity, scenarios).\n    - Conversation and chat: `createMessage`, `createScreenshotMessage`, `onChatResponse`.\n    - Auth lifecycle: `onAuthCallback`, `onLogout`, `sendAuthTokenResponse`.\n\n- **Renderer (`src/renderer/…`)**\n  - React UI with a conversation‑centric experience.\n  - Key component: `components/InsightsPanel.tsx` renders chat history, inlines screenshots, embeds audio players, and streams assistant responses.\n  - Uses a small `apiService` to fetch conversation history and persist messages to the backend.\n\n### Data Flow Overview\n\n1. User triggers an action (send text, take screenshot, start mic/system recording) from the UI.\n2. Renderer calls `window.api.*` methods exposed by the preload.\n3. Main process performs the privileged work (capture, record, analyze, fetch) and emits results back over IPC.\n4. Renderer updates the conversation with:\n   - A screenshot message (image preview)\n   - An audio message (embedded `\u003caudio\u003e` player with download)\n   - A Markdown‑formatted assistant response\n\n### Audio Subsystem\n\nThe audio architecture supports rich desktop scenarios:\n\n- **System audio loopback**: Route OS output back into the app for analysis/mixing.\n- **Microphone capture**: Record voice input concurrently with system audio.\n- **Mixing and levels**: Merge streams and manage levels for clarity.\n- **Echo control**: Enable echo cancellation with tunable sensitivity and listening scenarios.\n- **Playback**: Render recorded clips with multiple codecs and support exporting.\n\nConfiguration is applied via the preload API, for example:\n\n```ts\nwindow.api.setAudioConfiguration({\n  systemAudioEnabled: true,\n  echoCancellationEnabled: true,\n  echoCancellationSensitivity: 'medium',\n  audioScenario: 'speakers',\n  voiceRecordingMode: 'auto'\n});\n```\n\n## Documentation\n\nDeep‑dives and operational guides live alongside the codebase:\n\n- [Audio Debug](./docs/AUDIO_DEBUG.md)\n- [Audio Echo Fix](./docs/AUDIO_ECHO_FIX.md)\n- [Audio Integration Guide](./docs/AUDIO_INTEGRATION_GUIDE.md)\n- [Audio Mixing](./docs/AUDIO_MIXING.md)\n- [Audio Playback](./docs/AUDIO_PLAYBACK.md)\n- [Audio Troubleshooting](./docs/AUDIO_TROUBLESHOOTING.md)\n- [Platform Audio Guide](./docs/PLATFORM_AUDIO_GUIDE.md)\n- [Authentication](./docs/AUTHENTICATION.md)\n\n## Getting Started\n\nPrerequisites:\n\n- Node.js and npm installed\n- macOS, Windows, or Linux\n\nInstall dependencies:\n\n```bash\nnpm install\n```\n\nStart the app in development (with hot reload):\n\n```bash\nnpm run dev\n```\n\nBuild a production bundle:\n\n```bash\nnpm run build\n```\n\n## Project Structure\n\n```\ncontextor.desktop/\n├── public/\n│   └── index.html\n├── src/\n│   ├── main.ts                 # Electron main process\n│   ├── preload.ts              # Secure context bridge (window.api)\n│   └── renderer/\n│       ├── components/\n│       │   └── InsightsPanel.tsx\n│       ├── design-system/\n│       │   └── styles.ts\n│       ├── services/\n│       │   └── api.ts\n│       ├── App.tsx\n│       └── index.tsx\n├── package.json\n├── tsconfig.json\n├── webpack.config.js\n└── README.md\n```\n\n## Technical Highlights (Skills Showcase)\n\n- **Electron engineering**: Window lifecycle, secure IPC, preload isolation, deep links.\n- **Audio engineering**: Desktop loopback, stream mixing, echo cancellation strategies, playback UX.\n- **TypeScript at scale**: Strongly‑typed boundaries across main/preload/renderer.\n- **Frontend architecture**: Modular React components, Markdown rendering, UX micro‑interactions.\n- **API integration**: Auth flows, conversation history sync, and message creation.\n\n---\n\nIf you’re exploring this project to evaluate my desktop and audio engineering skills, start with `src/preload.ts` for the IPC surface and `src/renderer/components/InsightsPanel.tsx` for the end‑user experience, then browse the audio docs listed above.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvbrazo%2Fcontextor.desktop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvbrazo%2Fcontextor.desktop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvbrazo%2Fcontextor.desktop/lists"}