{"id":15542050,"url":"https://github.com/lifeart/sm-annotate","last_synced_at":"2026-01-19T22:31:15.473Z","repository":{"id":152047030,"uuid":"625131694","full_name":"lifeart/sm-annotate","owner":"lifeart","description":"Vector Annotation tool for Video \u0026 Image files","archived":false,"fork":false,"pushed_at":"2026-01-12T22:56:36.000Z","size":3170,"stargazers_count":4,"open_issues_count":14,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-13T00:44:13.309Z","etag":null,"topics":["annotate-images","annotation-tool","canvas","cgi","image-annotation-tool","typescript","vector","video-annotator"],"latest_commit_sha":null,"homepage":"https://lifeart.github.io/sm-annotate/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lifeart.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-08T06:51:17.000Z","updated_at":"2026-01-12T22:56:46.000Z","dependencies_parsed_at":"2024-10-02T12:20:40.227Z","dependency_job_id":"64e7466a-42a9-41a0-bfb2-8493ebd2ff5d","html_url":"https://github.com/lifeart/sm-annotate","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/lifeart/sm-annotate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lifeart%2Fsm-annotate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lifeart%2Fsm-annotate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lifeart%2Fsm-annotate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lifeart%2Fsm-annotate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lifeart","download_url":"https://codeload.github.com/lifeart/sm-annotate/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lifeart%2Fsm-annotate/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28587239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-19T20:45:59.482Z","status":"ssl_error","status_checked_at":"2026-01-19T20:45:41.500Z","response_time":67,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["annotate-images","annotation-tool","canvas","cgi","image-annotation-tool","typescript","vector","video-annotator"],"created_at":"2024-10-02T12:20:32.742Z","updated_at":"2026-01-19T22:31:15.466Z","avatar_url":"https://github.com/lifeart.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Video and Image Annotation Tool\n\nThis project provides an annotation tool for HTML video and image elements. The tool allows users to draw and annotate over video and images using various drawing tools, including curves, rectangles, ellipses, lines, arrows, and texts. Users can also customize the color and stroke size of their annotations.\n\nDemo: [lifeart.github.io/sm-annotate](https://lifeart.github.io/sm-annotate/)\n\n## Features\n\n* ✍️ Drawing and annotating over video and image elements\n* 🛠️ Multiple drawing tools (curve, rectangle, circle, line, arrow, text, eraser)\n* 🔲 Selection tool for cropping video frames\n* ↔️ Move tool for repositioning and resizing shapes\n* 🔄 Rotation support for all shapes with adjustable center point\n* 📐 Visual resize handles for precise shape scaling\n* 📋 Duplicate shapes (Ctrl/Cmd + D)\n* 📑 Copy annotations to adjacent frames (Ctrl/Cmd + Shift + Arrow)\n* 🎨 Customizable color and stroke size for annotations\n* ↩️ Undo functionality (Ctrl/Cmd + Z)\n* ⌫ Delete selected shapes with Backspace/Delete key (in move tool)\n* 🔗 Serialization and deserialization of drawn shapes\n* 📏 Scaling shapes to the current canvas size\n* 🎞️ Playback of annotated frames as video\n* 📊 Progress bar with annotation markers (visible on hover during playback)\n* ⏭️ Jump to next/previous annotated frame (long press on frame navigation buttons)\n* 💾 Saving the current frame or all frames with annotations\n* 🎬 Video overlay comparison mode (split view with adjustable opacity)\n* 🔊 Audio waveform visualization\n* 🖼️ Paste images from clipboard\n* 🌓 Dark/Light theme toggle\n* 💡 Tooltips on all toolbar buttons\n* 📦 OpenRV format import/export (.rv files for professional video review)\n* 🎛️ Multiple layout modes (horizontal, vertical, minimal, bottom-dock)\n* 📐 Collapsible toolbars for mobile\n* 🔍 Pinch-to-zoom and pan gestures\n* 🎨 CSS custom properties for easy theming\n* 🎬 FFmpeg-based frame extraction for frame-accurate playback\n* 👻 Ghost mode (onion skinning) for viewing adjacent frame annotations\n\n## Additional Benefits\n\n* 🚀 Zero dependencies\n* 📱 Support for mobile devices\n* 🔌 Powerful plugin system\n* 📘 Written in TypeScript\n* 🧪 Comprehensive test coverage (723 tests with Vitest)\n\n## Getting Started\n\nAdd the package to your project using yarn:\n\n```bash\nyarn add https://github.com/lifeart/sm-annotate.git\n```\n\n## Usage\n\n```javascript\nimport { SmAnnotate } from '@lifeart/sm-annotate';\n\nconst video = document.getElementById('video');\nconst annotationTool = new SmAnnotate(video);\n```\n\n### Basic Operations\n\n```javascript\n// Save current frame annotations\nconst frameData = annotationTool.saveCurrentFrame();\n\n// Save all frames with annotations\nconst allFrames = annotationTool.saveAllFrames();\n\n// Load annotations\nannotationTool.loadAllFrames(allFrames);\n\n// Set custom frame rate\nannotationTool.setFrameRate(30);\n\n// Enforce total frames count (override calculated value)\nannotationTool.setTotalFrames(100);\n\n// Clear enforcement and use calculated value\nannotationTool.setTotalFrames(null);\n```\n\n### Video Blob Support\n\n```javascript\n// Load video from blob\nawait annotationTool.setVideoBlob(videoBlob, fps);\n\n// Load video from URL\nawait annotationTool.setVideoUrl(videoUrl, fps);\n\n// Add reference video for comparison\nawait annotationTool.addReferenceVideoByURL(referenceUrl, fps);\n\n// Adjust overlay opacity for compare mode (0 = off, 0.25, 0.5, 0.7, 1)\nannotationTool.overlayOpacity = 0.7;\n\n// Shapes can have individual opacity (0 to 1)\n// Use the opacity button when a shape is selected in move tool\n```\n\n### Embedding \u0026 Configuration\n\nSmAnnotate can be customized for different embedding scenarios with layout modes, mobile optimizations, and CSS theming.\n\n#### Configuration Options\n\n```javascript\nimport { SmAnnotate } from '@lifeart/sm-annotate';\n\nconst annotationTool = new SmAnnotate(video, {\n  // Layout mode: 'horizontal' | 'vertical' | 'minimal' | 'bottom-dock'\n  layout: 'horizontal',\n\n  // Theme: 'dark' | 'light'\n  theme: 'dark',\n\n  // Mobile settings\n  mobile: {\n    collapsibleToolbars: true,  // Enable collapsible toolbar on mobile\n    gesturesEnabled: true,      // Enable pinch-to-zoom and pan\n    autoCollapse: true,         // Auto-collapse toolbar when drawing\n    breakpoint: 960,            // Mobile breakpoint in pixels\n  },\n\n  // Toolbar options\n  toolbar: {\n    sidebarPosition: 'left',    // For vertical layout: 'left' | 'right'\n    draggable: false,           // For minimal layout: allow dragging\n    position: { x: 10, y: 10 }, // For minimal layout: initial position\n    defaultTool: 'curve',       // Default selected tool (null = none)\n  },\n\n  // Feature visibility\n  features: {\n    showThemeToggle: true,\n    showFullscreen: true,\n    showProgressBar: true,\n    showFrameCounter: true,\n  },\n});\n```\n\n#### Layout Modes\n\n| Mode | Description |\n| --- | --- |\n| `horizontal` | Default layout with toolbar at top, player controls at bottom |\n| `vertical` | Tools in a vertical sidebar (left or right side) |\n| `minimal` | Compact floating toolbar that can be dragged around |\n| `bottom-dock` | All controls merged into a single bar at the bottom |\n\n```javascript\n// Switch layout at runtime\nannotationTool.setLayout('vertical');\nannotationTool.setLayout('minimal');\nannotationTool.setLayout('bottom-dock');\n\n// Get current layout\nconst currentLayout = annotationTool.getLayout();\n```\n\n#### Collapsible Toolbars (Mobile)\n\nOn mobile devices, toolbars can be collapsed to maximize drawing space:\n\n```javascript\n// Programmatic control\nannotationTool.collapseToolbar();\nannotationTool.expandToolbar();\nannotationTool.toggleToolbar();\n\n// Check state\nif (annotationTool.isToolbarCollapsed()) {\n  console.log('Toolbar is hidden');\n}\n```\n\nWhen `autoCollapse` is enabled, the toolbar automatically hides when drawing starts and reappears when drawing ends.\n\n#### Gesture Support (Mobile)\n\nEnable pinch-to-zoom and two-finger pan for detailed annotation work:\n\n```javascript\n// Enable/disable at runtime\nannotationTool.setGesturesEnabled(true);\n\n// Reset zoom to default\nannotationTool.resetZoom();\n\n// Get current zoom level (0.5x to 3x range)\nconst scale = annotationTool.getZoomScale();\n```\n\n#### CSS Customization\n\nSmAnnotate uses CSS custom properties for styling. Override these in your CSS:\n\n```css\n:root {\n  /* Colors */\n  --sm-annotate-bg-primary: rgba(28, 28, 32, 0.95);\n  --sm-annotate-bg-hover: rgba(255, 255, 255, 0.08);\n  --sm-annotate-text-primary: #f0f0f2;\n  --sm-annotate-accent: #5b9fff;\n  --sm-annotate-border: rgba(255, 255, 255, 0.1);\n\n  /* Sizing */\n  --sm-annotate-toolbar-radius: 8px;\n  --sm-annotate-toolbar-padding: 4px;\n  --sm-annotate-toolbar-gap: 2px;\n  --sm-annotate-btn-size: 32px;\n  --sm-annotate-btn-size-mobile: 44px;\n  --sm-annotate-btn-radius: 6px;\n\n  /* Typography */\n  --sm-annotate-font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;\n\n  /* Animation */\n  --sm-annotate-transition-duration: 0.15s;\n\n  /* Z-index */\n  --sm-annotate-z-index-toolbar: 10;\n}\n```\n\nAll CSS classes use the `sm-annotate-` prefix to prevent conflicts with host page styles.\n\n### OpenRV Format Support\n\nExport and import annotations in [OpenRV](https://github.com/AcademySoftwareFoundation/OpenRV) .rv format (GTO text format):\n\n```javascript\nimport {\n  exportToOpenRV,\n  downloadAsOpenRV,\n  parseOpenRV,\n  parseOpenRVFile\n} from '@lifeart/sm-annotate';\n\n// Export annotations to OpenRV format\nconst rvContent = exportToOpenRV(annotationTool.saveAllFrames(), {\n  mediaPath: '/path/to/video.mp4',\n  width: 1920,\n  height: 1080,\n  sessionName: 'my-session', // optional\n  ghost: annotationTool.getGhostConfig(), // optional, include ghost settings\n});\n\n// Download as .rv file\ndownloadAsOpenRV(annotationTool.saveAllFrames(), {\n  mediaPath: '/path/to/video.mp4',\n  width: 1920,\n  height: 1080,\n}, 'annotations.rv');\n\n// Parse OpenRV file content\nconst result = parseOpenRV(rvFileContent, {\n  width: 1920,  // optional, defaults to file dimensions or 1920\n  height: 1080, // optional, defaults to file dimensions or 1080\n  fps: 25,      // optional, defaults to 25\n});\n\n// Load parsed annotations\nannotationTool.loadAllFrames(result.frames);\n\n// Restore ghost settings if present\nif (result.ghost) {\n  annotationTool.setGhostEnabled(result.ghost.enabled);\n  annotationTool.setGhostConfig(result.ghost);\n}\n\n// Access parsed metadata\nconsole.log(result.mediaPath);    // original media path\nconsole.log(result.dimensions);   // { width, height }\nconsole.log(result.sessionName);  // session name\nconsole.log(result.ghost);        // ghost settings (if present)\n\n// Parse from File object (e.g., from file input)\nconst fileInput = document.getElementById('fileInput');\nfileInput.addEventListener('change', async (e) =\u003e {\n  const file = e.target.files[0];\n  const result = await parseOpenRVFile(file, { fps: 30 });\n  annotationTool.loadAllFrames(result.frames);\n});\n```\n\n**Supported shapes for OpenRV export:**\n- Curves (freehand drawings) → pen strokes\n- Lines → pen strokes (2-point)\n- Arrows → pen strokes (3 components: line + arrowhead)\n- Rectangles → pen strokes (closed 5-point path)\n- Circles → pen strokes (approximated as 33-point polygon)\n- Text → text annotations\n\n**Rotation support:** Shapes with rotation are fully supported. The rotation is \"baked in\" to the exported coordinates, including support for custom rotation centers. Text rotation only affects the anchor position since OpenRV text doesn't natively support rotation.\n\n**Coordinate system:** OpenRV uses Normalized Device Coordinates (NDC) centered at the image center (-1 to +1 range, Y+ is up), while sm-annotate uses 0-1 normalized coordinates with origin at top-left (Y+ is down). The converter handles this transformation automatically.\n\n**Ghost mode settings:** Ghost mode configuration (enabled state, frames before/after, opacity, tint colors) is preserved in OpenRV exports and restored on import. Settings are stored in the RVPaint `paint` component.\n\n**Note:** When importing from OpenRV, all pen strokes are converted to curves since OpenRV doesn't distinguish between shape types. Non-visual shapes (eraser, selection, compare, audio-peaks, image) are not exported. Files with multiple RVPaint blocks (common in real OpenRV sessions) are fully supported.\n\n#### Python CLI Tools\n\nStandalone Python scripts are available in the `openrv/` folder for command-line conversion:\n\n```bash\n# Convert sm-annotate JSON to OpenRV .rv format\npython3 openrv/convert_to_rv.py annotations.json output.rv \\\n  --media /path/to/video.mp4 --width 1920 --height 1080\n\n# Parse OpenRV .rv file to sm-annotate JSON\npython3 openrv/parse_rv.py input.rv output.json [--fps 25]\n\n# Use --frames-only to output just the frames array (for direct use with loadAllFrames)\npython3 openrv/parse_rv.py input.rv output.json --frames-only\n\n# Run round-trip tests\npython3 openrv/test_roundtrip.py\n```\n\nThe Python scripts mirror the TypeScript implementation and are useful for:\n- Batch conversion of annotation files\n- Integration with Python-based pipelines\n- Command-line workflows without Node.js\n\n### FFmpeg Frame Extraction\n\nFor frame-accurate video playback, SmAnnotate supports FFmpeg WASM-based frame extraction. This eliminates the ±1 frame drift common with HTML5 video's `requestVideoFrameCallback`.\n\n```javascript\nimport { FFmpegFrameExtractor } from '@lifeart/sm-annotate';\n\n// Create extractor instance\nconst extractor = new FFmpegFrameExtractor();\n\n// Load FFmpeg WASM (downloads ~30MB)\nawait extractor.load((progress) =\u003e {\n  console.log(`Loading: ${Math.round(progress.loaded * 100)}%`);\n});\n\n// Probe video for metadata (FPS, duration, dimensions)\nconst info = await extractor.probe(videoBlob);\nconsole.log(`FPS: ${info.fps}, Duration: ${info.duration}s`);\nconsole.log(`Dimensions: ${info.width}x${info.height}`);\nconsole.log(`Total frames: ${info.totalFrames}`);\n\n// Extract all frames as ImageBitmap objects\nconst frames = await extractor.extractFrames(videoBlob, {\n  format: 'png', // or 'jpeg'\n  onProgress: (progress) =\u003e {\n    console.log(`Extracting frame ${progress.loaded}/${progress.total}`);\n  }\n});\n\n// Connect to annotation tool for frame-accurate rendering\nannotationTool.setFFmpegFrameExtractor(extractor);\n\n// Access individual frames\nconst frame1 = frames.get(1); // 1-indexed\n```\n\n**Demo page behavior:**\n- FFmpeg loads automatically on page load\n- When selecting a video, FPS is auto-detected (no manual prompt needed)\n- Frame extraction starts automatically after video selection\n- If FFmpeg is still loading when you select a video, it queues and processes after load completes\n- Network errors allow retry via the load button\n\n**Requirements:**\n- Server must set CORS headers for SharedArrayBuffer support:\n  ```\n  Cross-Origin-Embedder-Policy: require-corp\n  Cross-Origin-Opener-Policy: same-origin\n  ```\n- Video files should be under ~100MB (WASM memory limit)\n\n### Frame Navigation\n\n```javascript\n// Navigate frames\nannotationTool.nextFrame();\nannotationTool.prevFrame();\n\n// Jump to annotated frames\nannotationTool.nextAnnotatedFrame();\nannotationTool.prevAnnotatedFrame();\n\n// Get list of frames with annotations\nconst annotatedFrames = annotationTool.getAnnotatedFrames();\n```\n\n### Ghost Mode (Onion Skinning)\n\nGhost mode displays annotations from adjacent frames as semi-transparent overlays, helping animators see the context of surrounding frames. This technique is also known as \"onion skinning\" in animation software.\n\n```javascript\n// Enable/disable ghost mode\nannotationTool.setGhostEnabled(true);\nannotationTool.setGhostEnabled(false);\n\n// Toggle ghost mode\nconst isEnabled = annotationTool.toggleGhost();\n\n// Check if ghost mode is enabled\nif (annotationTool.ghostEnabled) {\n  console.log('Ghost mode is on');\n}\n\n// Configure ghost mode settings\nannotationTool.setGhostConfig({\n  framesBefore: 2,    // Show 1-5 previous frames (default: 2)\n  framesAfter: 1,     // Show 1-5 next frames (default: 1)\n  opacity: 0.3,       // Base opacity 0.1-0.5 (default: 0.3)\n  tintBefore: 'rgba(255, 0, 0, 0.3)',  // Tint for previous frames (red)\n  tintAfter: 'rgba(0, 128, 0, 0.3)',   // Tint for next frames (green)\n});\n\n// Get current ghost configuration\nconst config = annotationTool.getGhostConfig();\n\n// Listen for ghost mode changes (returns unsubscribe function)\nconst unsubscribe = annotationTool.onGhostChange((enabled) =\u003e {\n  console.log('Ghost mode:', enabled ? 'on' : 'off');\n});\n// Later: unsubscribe();\n```\n\n**Ghost mode features:**\n- Previous frames shown with customizable red tint (default)\n- Next frames shown with customizable green tint (default)\n- Opacity decreases with distance from current frame\n- Settings are saved/restored with session data\n- Settings are preserved in OpenRV .rv file exports\n- Toolbar toggle button syncs with programmatic changes\n\n## Hotkeys\n\n### General\n\n| Key | Action |\n| --- | --- |\n| `Ctrl/Cmd + Z` | Undo last action |\n| `Backspace` / `Delete` | Delete selected shape (in move tool) |\n| `←` / `→` | Previous / Next frame |\n| `Space` | Play / Pause video |\n\n### Move Tool\n\n| Key | Action |\n| --- | --- |\n| `Ctrl/Cmd + D` | Duplicate selected shape |\n| `Ctrl/Cmd + Shift + →` | Copy all annotations to next frame |\n| `Ctrl/Cmd + Shift + ←` | Copy all annotations to previous frame |\n| `Backspace` / `Delete` | Delete selected shape |\n| `Shift` + drag handle | Resize while keeping aspect ratio |\n| `Shift` + drag rotation handle | Snap rotation to 15° increments |\n\n### Curve Tool\n\n| Key | Action |\n| --- | --- |\n| `Shift` | Magnifier x2 |\n| `r` | Red color |\n| `g` | Green color |\n| `b` | Blue color |\n| `y` | Yellow color |\n| `1` - `9` | Tool size |\n\n## Mouse/Touch Actions\n\n| Action | Result |\n| --- | --- |\n| Click on progress bar | Jump to frame |\n| Drag on progress bar | Scrub through frames |\n| Click on annotation marker | Jump to annotated frame |\n| Long press frame buttons | Jump to next/previous annotation |\n| Pinch (two fingers) | Zoom in/out (0.5x to 3x) |\n| Two-finger drag | Pan the canvas |\n| Tap collapse button | Toggle toolbar visibility |\n\n## Tools\n\n| Tool | Description |\n| --- | --- |\n| Rectangle | Draw rectangular shapes |\n| Circle | Draw circular/elliptical shapes |\n| Line | Draw straight lines |\n| Arrow | Draw arrows |\n| Curve | Freehand drawing |\n| Text | Add text annotations |\n| Eraser | Remove annotations |\n| Move | Reposition, resize, and rotate shapes; drag rotation handle to rotate, drag center point to change rotation pivot |\n| Selection | Crop and capture video frame area |\n| Compare | Split-view video comparison |\n| Opacity | Adjust overlay or selected shape opacity (off/25%/50%/70%/100%) |\n| Ghost | Toggle ghost mode (onion skinning) to see adjacent frame annotations |\n| Theme | Toggle between dark and light mode |\n\n## Development\n\n```bash\n# Install dependencies\nyarn install\n\n# Run development server\nyarn dev\n\n# Run tests\nyarn test\n\n# Run tests with coverage\nyarn test:coverage\n\n# Type check\nyarn typecheck\n\n# Build\nyarn build\n```\n\n## Contributing\n\nWe welcome contributions to improve the project. Please feel free to submit issues or pull requests for consideration.\n\n## License\n\nThis code is allowed for non-commercial use. For commercial use, users must contact the author.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flifeart%2Fsm-annotate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flifeart%2Fsm-annotate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flifeart%2Fsm-annotate/lists"}