https://github.com/deadronos/3d-cube-desktop
https://github.com/deadronos/3d-cube-desktop
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/deadronos/3d-cube-desktop
- Owner: deadronos
- License: mit
- Created: 2026-03-21T12:54:45.000Z (2 months ago)
- Default Branch: main
- Last Pushed: 2026-03-21T16:02:39.000Z (2 months ago)
- Last Synced: 2026-03-22T05:12:13.270Z (about 2 months ago)
- Language: C#
- Size: 60.5 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
- Roadmap: docs/roadmap.md
Awesome Lists containing this project
README
# 3D Cube Desktop
`3d-cube-desktop` is a Windows-first WPF prototype for a spatial desktop shell. The current `main` branch implements an inward-facing open-room workspace: a floor, left wall, back wall, and right wall rendered in `Viewport3D`, with layouts that assign windows, grouped workspaces, and dashboards to those surfaces.
This is still a prototype shell, not a replacement desktop environment or compositor. Its purpose is to validate the interaction model, visual metaphor, and workspace assignment system.
## Current State
What `main` currently includes:
- an inward-facing open-room scene instead of an exterior cube
- 4 visible presentation surfaces: `Floor`, `LeftWall`, `BackWall`, `RightWall`
- saved layouts loaded from JSON
- two shell window modes: `FullscreenPresentation` and `WindowedDev`
- overview/work mode transitions
- an inspect mode for free look / debugging
- per-window capture using `Windows.Graphics.Capture`
- grouped window surfaces resolved by process/title matching rules
- dashboard surfaces with launchable shortcuts
- selected-surface input forwarding
- persisted shell state between runs
What it does not include yet:
- a true virtual desktop implementation
- simultaneous full interaction on every visible surface
- multi-window compositing on grouped surfaces
- runtime face editing from inside the app
- head tracking or camera parallax input
## Concept
The repo started from the idea in [idea.md](/D:/GitHub/3d-cube-desktop/idea.md): make the desktop feel spatial instead of flat. The implementation now on `main` is the open-room version of that idea, not the older exterior cube shell.
## Tech Stack
- C#
- .NET 9
- WPF
- `Viewport3D`
- Win32 interop for window enumeration and input forwarding
- `Windows.Graphics.Capture` for per-window capture
- D3D11/WinRT interop for capture session plumbing
## Project Layout
- [DesktopCube.sln](/D:/GitHub/3d-cube-desktop/DesktopCube.sln): solution entrypoint
- [src/DesktopCube.App](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App): WPF shell app
- [src/DesktopCube.App/MainWindow.xaml](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/MainWindow.xaml): viewport and HUD shell
- [src/DesktopCube.App/MainWindow.xaml.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/MainWindow.xaml.cs): scene orchestration, camera control, layout switching, source refresh, interaction
- [src/DesktopCube.App/Core](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Core): config and runtime models
- [src/DesktopCube.App/Rendering/RoomGeometryFactory.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Rendering/RoomGeometryFactory.cs): open-room geometry
- [src/DesktopCube.App/Services/WindowCaptureService.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/WindowCaptureService.cs): live per-window capture
- [src/DesktopCube.App/Services/GraphicsCaptureInterop.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/GraphicsCaptureInterop.cs): WinRT / D3D interop for capture
- [src/DesktopCube.App/Services/FaceSourceResolverService.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/FaceSourceResolverService.cs): grouped-window resolution and active-source selection
- [src/DesktopCube.App/Services/WindowInventoryService.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/WindowInventoryService.cs): top-level window discovery and activation
- [src/DesktopCube.App/Services/InputForwarder.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/InputForwarder.cs): selected-face input routing
- [src/DesktopCube.App/Services/PlaceholderTextureFactory.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/PlaceholderTextureFactory.cs): synthetic textures for grouped/dashboard/fallback surfaces
- [src/DesktopCube.App/face-config.json](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/face-config.json): default layouts and launcher config
- [plan.md](/D:/GitHub/3d-cube-desktop/plan.md): open-room implementation plan
## How It Works
### Scene Model
The room is an inward-facing box with four visible surfaces:
- `BackWall`
- `LeftWall`
- `RightWall`
- `Floor`
Each layout assigns one face definition to each visible slot. The renderer uses [RoomGeometryFactory.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Rendering/RoomGeometryFactory.cs) to build those surfaces and maps a texture to each one.
### Source Types
Each surface uses one of these source types:
- `WindowCapture`: a single active window rendered live through `Windows.Graphics.Capture`
- `WorkspaceGroup`: a grouped workspace that resolves multiple matching windows and selects one active source
- `Dashboard`: a synthetic launcher surface
- `StaticFallback`: a synthetic fallback card when no live source is available
The enum lives in [FaceSourceType.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Core/FaceSourceType.cs).
### Capture Model
Live window-backed surfaces use [WindowCaptureService.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/WindowCaptureService.cs), which:
- creates a `GraphicsCaptureItem` for a target window
- starts a `Direct3D11CaptureFramePool`
- receives frames on a free-threaded capture session
- converts each frame into a WPF `BitmapSource`
This avoids the old recursive whole-monitor capture problem. The shell explicitly excludes its own window handle from source resolution and capture.
### Workspace Grouping
Grouped surfaces use [FaceSourceResolverService.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/FaceSourceResolverService.cs) and [WindowInventoryService.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/WindowInventoryService.cs) to:
- enumerate visible, non-minimized, top-level windows
- filter them by configured substrings
- sort them by configured priorities and foreground state
- pick one active window for the surface
- retain the rest as candidates for cycling
Grouped surfaces are still synthetic summaries plus one active source target. They are not yet live composites of multiple windows.
### Interaction Model
The shell has three useful states:
- locked overview
- work mode
- inspect mode
In work mode:
- the camera moves toward the selected surface
- only the selected live surface is directly interactive
- cursor hit testing is raycast into the 3D scene
- UV coordinates are mapped back to the active window bounds
- click input is forwarded to the selected target window
This is intentionally a selected-surface interaction model, not a general-purpose embedded multi-window environment.
## Requirements
- Windows 10 version 2004+ or Windows 11
- .NET 9 SDK
- a machine and driver stack capable of WPF 3D and `Windows.Graphics.Capture`
The project targets Windows only. The csproj is currently set to `net9.0-windows10.0.19041.0`.
## Getting Started
### Build
```powershell
dotnet build DesktopCube.sln
```
### Run
```powershell
dotnet run --project src/DesktopCube.App/DesktopCube.App.csproj
```
By default the shell starts in fullscreen presentation mode.
## Controls
### Mouse
- Click a surface: select it
- Double-click in overview: enter work mode on the selected surface
- Drag in inspect mode: orbit / inspect the room
- Mouse wheel in inspect mode: adjust inspect pitch
### Keyboard
- `Enter` or `Tab`: toggle work mode
- `Esc`: leave work mode or exit inspect mode
- `F5`: refresh sources
- `F11`: toggle `FullscreenPresentation` / `WindowedDev`
- `I`: toggle inspect mode
- `PageUp` / `[` : previous layout
- `PageDown` / `]` : next layout
- `Ctrl+1..9`: jump directly to a layout
- `C`: cycle active source on grouped surfaces
- `W`: activate the selected source window
- `1..9`: launch dashboard shortcuts while a dashboard surface is selected in work mode
## Default Layouts
The shipped config defines two layouts in [face-config.json](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/face-config.json).
### Builder Room
Purpose:
- coding
- docs
- chat
- terminals
Default surfaces:
- `BackWall`: `Code Wall` as `WorkspaceGroup`
- `LeftWall`: `Research Wall` as `WorkspaceGroup`
- `RightWall`: `Comms Wall` as `WorkspaceGroup`
- `Floor`: `Ops Floor` as `WorkspaceGroup`
### Review Room
Purpose:
- PR review
- issue triage
- browser-focused work
- quick launches
Default surfaces:
- `BackWall`: `PR Wall` as `WorkspaceGroup`
- `LeftWall`: `Browser Wall` as `WindowCapture`
- `RightWall`: `Issues Wall` as `WorkspaceGroup`
- `Floor`: `Launch Floor` as `Dashboard`
## Configuration
The shell config is loaded from [face-config.json](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/face-config.json).
### Top-Level Config
Important top-level properties:
- `captureFps`
- `activeLayoutId`
- `selectedFace`
- `defaultShellWindowMode`
- `launchers`
- `layouts`
### Face Config
Each face can define:
- `id`
- `slot`
- `label`
- `sourceType`
- `workspaceKey`
- `accentHex`
- `notes`
- `group`
- `launcherIds`
### Group Rules
Grouped surfaces use:
- `windowFilters`: broad match terms
- `priorityFilters`: preferred ordering when multiple windows match
That makes the active surface deterministic while still allowing cycling between candidates.
### Persisted State
Runtime state is stored in `workspace-state.json` next to the app output. That includes:
- active layout
- selected face
- shell window mode
- current work/overview state
- active source index per grouped surface
## Limitations
These are the main current constraints.
### Grouped surfaces are not live composites
`WorkspaceGroup` surfaces still summarize a grouped workspace and point interaction at one active source. They do not yet render multiple windows composited together.
### Only the selected surface is interactive
The shell forwards input only for the selected live surface in work mode. This is by design for the current milestone.
### Minimized and hidden windows are excluded
[WindowInventoryService.cs](/D:/GitHub/3d-cube-desktop/src/DesktopCube.App/Services/WindowInventoryService.cs) only tracks visible, non-minimized, top-level windows. If a candidate window is hidden or minimized, it drops out of the workspace model.
### Some apps may still capture poorly
`Windows.Graphics.Capture` is much better than the old screen-copy path, but some protected, accelerated, or unusual windows may still fail to render correctly or may momentarily fall back to placeholder content.
### Docs are ahead of the internal architecture notes
[docs/architecture.md](/D:/GitHub/3d-cube-desktop/docs/architecture.md) still describes the older Phase 4 cube shell. The README reflects the current open-room `main` branch more accurately.
## Architecture Summary
At a high level:
1. The app loads layouts, launchers, and defaults from JSON.
2. `MainWindow` builds the open-room scene and camera rig.
3. Window inventory is refreshed on a timer.
4. Each surface resolves to a live source, grouped source, dashboard, or fallback.
5. Live windows are captured with `Windows.Graphics.Capture`.
6. HUD panels expose the selected surface, active layout, and candidate state.
7. Work mode routes selected-surface input back to the active target window.
## Roadmap
Reasonable next steps from the current merged implementation:
1. Promote grouped surfaces from synthetic summaries to richer live multi-window compositions.
2. Add in-app editing and saving of layouts.
3. Improve input routing and focus handoff.
4. Add stronger per-face source management instead of purely filter-based grouping.
5. Add optional head-tracked or parallax camera input.
## Contributing
If you extend this prototype, keep the distinction between current behavior and planned behavior explicit. The easiest ways to make the codebase easier to work with are:
- keep layout behavior data-driven in config
- document user-facing controls when you change them
- call out prototype-only hacks in code comments or docs
- avoid pretending grouped surfaces are true composited desktops if they are not
## Quick Summary
This repo currently ships an open-room 3D workspace shell for Windows with:
- four inward-facing surfaces
- saved layouts
- grouped workspace assignments
- one real live window-capture surface
- dashboard launchers
- overview/work/inspect transitions
- selected-surface input forwarding
It is a credible spatial workspace prototype, but it is still not a full desktop compositor.