https://github.com/phrogz/vunsq

Functional Video Sequencer
https://github.com/phrogz/vunsq

Last synced: 7 days ago
JSON representation

Functional Video Sequencer

• Host: GitHub
• URL: https://github.com/phrogz/vunsq
• Owner: Phrogz
• Created: 2017-06-09T21:51:20.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2017-09-14T15:05:59.000Z (about 7 years ago)
• Last Synced: 2024-10-20T13:26:37.690Z (19 days ago)
• Language: QML
• Size: 135 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Vunsq

Functional Video Sequencer

Like a mod-tracker, but:

* For timing and blending 2D video instead of sound, and

* Using generative, pixel-shading functions for the 'samples' instead of pre-recorded videos

# Core Concepts and Terminology

An **Effect** is a function like a pixel shader: it gets fed an x coordinate, a time, and (optionally) custom arguments, and it produces RGBA values for an entire light strand.

You can have a maximum of 128 different Effects.

An **Event** is an instance of an Effect. It specifies a time to start, a the strands it affects, and optional custom arguments.

Events reference Effects by numeric identifier.

A **Presentation** is the entire ‘movie'. It specifies a BPM it is associated with, and an ideal reference song to play with it.

A Presentation has  a **Timeline** that groups together all Events  to display.

Presentations can be played by the runtime along with a different song at a different BPM, speeding them up or slowing them down.

_Note: Effects are code in the host runtime, and not stored within a Presentation._

# JSON Example

{

   "bpm"      : 100.32,

   "length"   : 237483,

   "media"    : "CHVRCHES-LeaveATrace.m4a",

   "timeline" : [

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }],

      [{ "effect":2, "speed":2.5 }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":2, "speed":2.5 }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }],

      [{ "effect":2, "speed":2.5 }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }],

      [{ "effect":2, "speed":2.5 }]

   ]

}

# Binary Format

All multi-byte numbers are stored in little-endian format.

## Presentation

 bytes | field

:-----:|-----------------------------------

   4   | Presentation BPM (float)

   4   | Presentation duration (uint32)

   1   | Characters in Media URI (uint8)

   ~   | Media URI (UTF-8)

   ~   | Timeline Index

   ~   | Timeline

## Timeline Index

 bytes | field

:-----:|-----------------------------------------

   1   | Strand Count                    (uint8)

   4   | Offset to Strand 1 Timeline     (uint32)

   2   | Byte count of Strand 1 Timeline (uint16)

   4   | Offset to Strand 2 Timeline     (uint32)

   2   | Byte count of Strand 2 Timeline (uint16)

   4   | Offset to Strand … Timeline     (uint32)

   2   | Byte count of Strand … Timeline (uint16)

   ~   | (repeat for all strands)

## Timeline

 bytes | field

:-----:|-----------------------------------

   4   | Strand 1 Event Count (uint32)

   ~   | Event 1

   ~   | Event 2

   ~   | Event …

   4   | Strand 2 Event Count (uint32)

   ~   | Event 1

   ~   | Event 2

   ~   | Event …

   4   | Strand … Event Count (uint32)

   ~   | Event 1

   ~   | Event 2

   ~   | Event …

## Event

 bytes | field

:-----:|-----------------------------------

   1   | Effect#    (uint8; 128-255 only)

   4   | Start      (uint32 ms)

   4   | Speed      (float)

   1   | Arg Count  (uint8)

   1   | Arg 1      (uint8)

   1   | Arg 2      (uint8)

   1   | Arg …      (uint8)