{"id":21095463,"url":"https://github.com/sensorsini/v2e","last_synced_at":"2025-04-05T20:05:20.666Z","repository":{"id":40305396,"uuid":"190332735","full_name":"SensorsINI/v2e","owner":"SensorsINI","description":"V2E: From video frames to DVS events","archived":false,"fork":false,"pushed_at":"2023-11-09T17:35:10.000Z","size":35325,"stargazers_count":335,"open_issues_count":21,"forks_count":58,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-03-29T19:02:05.881Z","etag":null,"topics":["event-camera","pytorch","simulator"],"latest_commit_sha":null,"homepage":"https://sites.google.com/view/video2events/home","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SensorsINI.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-05T05:42:04.000Z","updated_at":"2025-03-23T21:40:06.000Z","dependencies_parsed_at":"2024-11-19T22:41:25.254Z","dependency_job_id":null,"html_url":"https://github.com/SensorsINI/v2e","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SensorsINI%2Fv2e","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SensorsINI%2Fv2e/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SensorsINI%2Fv2e/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SensorsINI%2Fv2e/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SensorsINI","download_url":"https://codeload.github.com/SensorsINI/v2e/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393568,"owners_count":20931812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["event-camera","pytorch","simulator"],"created_at":"2024-11-19T22:26:23.507Z","updated_at":"2025-04-05T20:05:20.645Z","avatar_url":"https://github.com/SensorsINI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# v2e [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1czx-GJnx-UkhFVBbfoACLVZs8cYlcr_M?usp=sharing)\n\nPython torch + opencv code to go from conventional stroboscopic video frames with low frame rate into realistic synthetic DVS event streams with much higher effective timing precision. v2e includes finite intensity-depenedent photoreceptor bandwidth, Gaussian pixel to pixel event threshold variation, and noise 'leak' events.\n\nSee the [v2e home page](https://sites.google.com/view/video2events/home) for videos and further information. \n\nOur paper (below) about _v2e_ **debunks incorrect assertions about event cameras that pervade the current computer vision literature.**\n\nSee [v2e-sample-input-data](https://drive.google.com/drive/folders/1oWxTB9sMPp6UylAdxg5O1ko_GaIXz7wo?usp=sharing) for sample input files.\n\n![v2e-tennis-example](media/v2e-tennis-split-screen.gif)\n\nVote for [new v2e features](https://docs.google.com/forms/d/e/1FAIpQLSdJoIH3wBkPANWTng56VeXiItkh_fl5Lz3QwZIpQ6ut1AMFCw/viewform?usp=sf_link).\n\nv2e development was supported by the Swiss National Competence Center for Robotics [(NCCR Robotics)](https://nccr-robotics.ch/).\n\n## News\n\nSee [changelog](https://github.com/SensorsINI/v2e/blob/master/CHANGELOG.md) for latest news\n\n## Contact\nYuhuang Hu (yuhuang.hu@ini.uzh.ch)\nTobi Delbruck (tobi@ini.uzh.ch)\n\n### Citation\nIf you use v2e, we appreciate a citation to the paper below. See the [v2e home page](https://sites.google.com/view/video2events/home) for futher background papers.\n\n+ Y. Hu, S-C. Liu, and T. Delbruck. v2e: From Video Frames to Realistic DVS Events. In _2021 IEEE/CVF  Conference  on  Computer  Vision  and  Pattern Recognition Workshops (CVPRW)_, URL: https://arxiv.org/abs/2006.07722, 2021\n\nTo reproduce the experiments of the paper, please find [this repository](https://github.com/SensorsINI/v2e_exps_public).\n\n### Advice about conversion time\nWe recommend running _v2e_ on a CUDA GPU or it will be very slow, particularly when using SuperSloMo upsampling. \nEven with a low-end GTX-1050, _v2e_ runs about 50-200X slower than real time \nusing 10X slowdown factor and 346x260 video.\n\nConversion speed depends linearly on the reciprocal of the desired DVS timestamp resolution.\nIf you demand fine resolution of e.g. 100us, \nthen expect many minutes of computing per second of source video. Running on Google colab\nwith GPU, it took 500s per second of 12FPS source video, because of the very high upsampling ratio\nof over 800X and the 220k frames that needed to be produced for DVS modeling.\n\nWe advise using the _--stop_ option for trial run before starting a long conversion.\n## Using v2e in Google CoLab\nIf you don't want to install, try [opening v2e in google colab](https://colab.research.google.com/drive/1czx-GJnx-UkhFVBbfoACLVZs8cYlcr_M?usp=sharing).\n\n## Local Installation\nThere are 3 general steps\n1. Make a _conda_ environment\n2. Install _pytoch_ and other _conda_ distributed packages to this environment\n3. Install the rest of the packages and _v2e_ to the conda enviroment with _pip_.\n\n_pip_ is needed because some packages are not availble from the conda repositories. It is important to go in this order because _conda_ in general is not aware of _pip_ installs.\n\n### Make conda environment\nInstall _v2e_ to a separate Python environment\nsuch as `conda` environment:\n\n```bash\nconda create -n v2e python=3.10  # create a new environment\nconda activate v2e  # activate the environment\n```\n### Ensure you have CUDA enabled GPU-accelerated _pytorch_\nSee https://pytorch.org/get-started/locally/ to generate the correct\nconda install command to enable GPU-accelerated CUDA. On windows 10, for example, this tool generated the following command. _v2e_ does not use _torchaudio_ so you can leave it out.\n```\nconda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch\n```\n\nYou may want to check this stackoverflow question:\nhttps://stackoverflow.com/questions/57238344/i-have-a-gpu-and-cuda-installed-in-windows-10-but-pytorchs-torch-cuda-is-availa\n\n### Use pip to install rest of packages and v2e\nAfter installing pytorch to your CUDA environment, to install v2e in developer mode (so your edits to source take effect immediately), run the following command in your terminal inside the activated conda environment. The `python -m pip install -e .` command runs _setup.py_ which installs more packages and adds a script to run _v2e_ from the command line to the _conda_ enviroment python path; see https://stackoverflow.com/questions/39023758/what-does-pip-install-dot-mean:\n```bash\nconda activate v2e # activate your env with pytoch already installed by conda\ngit clone https://github.com/SensorsINI/v2e\ncd v2e\npython -m pip install -e . # use pip to install requirements from setup.py in user mode (so your edits to source files in v2e take effect immediately\n```\n\n+ If you want an additional Windows GUI interface, you will need to install [Gooey](https://github.com/chriskiehl/Gooey) package. This package works the best on Windows:\n    ```bash\n    pip install Gooey\n    ```\n    On Linux, `Gooey` can be hard to install.\n\n    For a sample of conversion using the gooey GUI, see https://youtu.be/THJqRC_q2kY\n\n### Download SuperSloMo model\n\nWe use the excellent [Super SloMo](https://people.cs.umass.edu/~hzjiang/projects/superslomo/) framework to interpolate the APS frames.\nHowever, since APS frames only record light intensity, we  retrained it on grayscale images.\n\nDownload our pre-trained model checkpoint from Google Drive\n[SuperSloMo39.ckpt](https://drive.google.com/file/d/1ETID_4xqLpRBrRo1aOT7Yphs3QqWR_fx/view?usp=sharing) (151 MB) and save it to the _input_ folder. \nThe default value of --slomo_model argument is set to this location.\n\nSpecial thanks to Zhe He for recovering the checkpoint file.\n\n### Download sample input data\nThe sample input videos to try _v2e_ with are in [v2e-sample-input-data](https://drive.google.com/drive/folders/1oWxTB9sMPp6UylAdxg5O1ko_GaIXz7wo?usp=sharing) on google drive.\n\nDownload the [tennis.mov](https://drive.google.com/file/d/1dNUXJGlpEM51UVYH4-ZInN9pf0bHGgT_/view?usp=sharing)\nvideo and put in the _input_ folder\nto run the example below.\n\n## Usage\n\n_v2e_ serves multiple purposes. Please read to code if you would like to adapt it for your own application. Here, we only introduce the usage for generating DVS events from conventional video and from specific datasets.\n\n## Render emulated DVS events from conventional video.\n\n_v2e.py_ reads a standard video (e.g. in .avi, .mp4, .mov, or .wmv), or a folder of images, and generates emulated DVS events at upsampled timestamp resolution.\n\nDon't be intimidated by the huge number of options. Running _v2e.py_ with no arguments sets reasonable values and opens a file browser to let you select an input video. Inspect the logging output for hints.\n\n**Hint:** Note the options _[--dvs128 | --dvs240 | --dvs346 | --dvs640 | --dvs1024]_; they set output size and width to popular DVS cameras.\n\n**On headless platforms**, with no graphics output, use --no_preview option to suppress the OpenCV windows.\n\n```\nusage: v2e.py [-h] [-o OUTPUT_FOLDER] [--avi_frame_rate AVI_FRAME_RATE]\n              [--output_in_place [OUTPUT_IN_PLACE]] [--overwrite]\n              [--unique_output_folder [UNIQUE_OUTPUT_FOLDER]]\n              [--skip_video_output]\n              [--auto_timestamp_resolution [AUTO_TIMESTAMP_RESOLUTION]]\n              [--timestamp_resolution TIMESTAMP_RESOLUTION]\n              [--dvs_params DVS_PARAMS] [--pos_thres POS_THRES]\n              [--neg_thres NEG_THRES] [--sigma_thres SIGMA_THRES]\n              [--cutoff_hz CUTOFF_HZ] [--leak_rate_hz LEAK_RATE_HZ]\n              [--shot_noise_rate_hz SHOT_NOISE_RATE_HZ]\n              [--photoreceptor_noise]\n              [--leak_jitter_fraction LEAK_JITTER_FRACTION]\n              [--noise_rate_cov_decades NOISE_RATE_COV_DECADES]\n              [--refractory_period REFRACTORY_PERIOD]\n              [--dvs_emulator_seed DVS_EMULATOR_SEED]\n              [--show_dvs_model_state SHOW_DVS_MODEL_STATE [SHOW_DVS_MODEL_STATE ...]]\n              [--save_dvs_model_state]\n              [--record_single_pixel_states RECORD_SINGLE_PIXEL_STATES]\n              [--output_height OUTPUT_HEIGHT] [--output_width OUTPUT_WIDTH]\n              [--dvs128 | --dvs240 | --dvs346 | --dvs640 | --dvs1024]\n              [--disable_slomo] [--slomo_model SLOMO_MODEL]\n              [--batch_size BATCH_SIZE] [--vid_orig VID_ORIG]\n              [--vid_slomo VID_SLOMO] [--slomo_stats_plot] [-i INPUT]\n              [--input_frame_rate INPUT_FRAME_RATE]\n              [--input_slowmotion_factor INPUT_SLOWMOTION_FACTOR]\n              [--start_time START_TIME] [--stop_time STOP_TIME] [--crop CROP]\n              [--hdr] [--synthetic_input SYNTHETIC_INPUT]\n              [--dvs_exposure DVS_EXPOSURE [DVS_EXPOSURE ...]]\n              [--dvs_vid DVS_VID] [--dvs_vid_full_scale DVS_VID_FULL_SCALE]\n              [--no_preview] [--ddd_output] [--dvs_h5 DVS_H5]\n              [--dvs_aedat2 DVS_AEDAT2] [--dvs_text DVS_TEXT]\n              [--label_signal_noise] [--cs_lambda_pixels CS_LAMBDA_PIXELS]\n              [--cs_tau_p_ms CS_TAU_P_MS] [--scidvs]\nv2e: generate simulated DVS events from video.\noptional arguments:\n  -h, --help            show this help message and exit\nOutput: General:\n  -o OUTPUT_FOLDER, --output_folder OUTPUT_FOLDER\n                        folder to store outputs.\n  --avi_frame_rate AVI_FRAME_RATE\n                        frame rate of output AVI video files; only affects\n                        playback rate.\n  --output_in_place [OUTPUT_IN_PLACE]\n                        store output files in same folder as source video (in\n                        same folder as frames if using folder of frames).\n  --overwrite           overwrites files in existing folder (checks existence\n                        of non-empty output_folder).\n  --unique_output_folder [UNIQUE_OUTPUT_FOLDER]\n                        If specifying --output_folder, makes unique output\n                        folder based on output_folder, e.g. output1 (if non-\n                        empty output_folder already exists)\n  --skip_video_output   Skip producing video outputs, including the original\n                        video, SloMo video, and DVS video. This mode also\n                        prevents showing preview of output (cf --no_preview).\nDVS timestamp resolution:\n  --auto_timestamp_resolution [AUTO_TIMESTAMP_RESOLUTION]\n                        (Ignored by --disable_slomo or --synthetic_input.) If\n                        True (default), upsampling_factor is automatically\n                        determined to limit maximum movement between frames to\n                        1 pixel. If False, --timestamp_resolution sets the\n                        upsampling factor for input video. Can be combined\n                        with --timestamp_resolution to ensure DVS events have\n                        at most some resolution.\n  --timestamp_resolution TIMESTAMP_RESOLUTION\n                        (Ignored by --disable_slomo or --synthetic_input.)\n                        Desired DVS timestamp resolution in seconds;\n                        determines slow motion upsampling factor; the video\n                        will be upsampled from source fps to achieve the at\n                        least this timestamp resolution.I.e. slowdown_factor =\n                        (1/fps)/timestamp_resolution; using a high resolution\n                        e.g. of 1ms will result in slow rendering since it\n                        will force high upsampling ratio. Can be combind with\n                        --auto_timestamp_resolution to limit upsampling to a\n                        maximum limit value.\nDVS model:\n  --dvs_params DVS_PARAMS\n                        Easy optional setting of parameters for DVS\n                        model:None, 'clean', 'noisy'; 'clean' turns off noise,\n                        sets unlimited bandwidth and makes threshold variation\n                        small. 'noisy' sets limited bandwidth and adds leak\n                        events and shot noise.This option by default will\n                        disable user set DVS parameters. To use custom DVS\n                        paramters, use None here.\n  --pos_thres POS_THRES\n                        threshold in log_e intensity change to trigger a\n                        positive event.\n  --neg_thres NEG_THRES\n                        threshold in log_e intensity change to trigger a\n                        negative event.\n  --sigma_thres SIGMA_THRES\n                        1-std deviation threshold variation in log_e intensity\n                        change.\n  --cutoff_hz CUTOFF_HZ\n                        photoreceptor IIR lowpass filter cutoff-off 3dB\n                        frequency in Hz - see\n                        https://ieeexplore.ieee.org/document/4444573.CAUTION:\n                        See interaction with timestamp_resolution and\n                        auto_timestamp_resolution; check output logger\n                        warnings. The input sample rate (frame rate) must be\n                        fast enough to for accurate IIR lowpass filtering.\n  --leak_rate_hz LEAK_RATE_HZ\n                        leak event rate per pixel in Hz - see\n                        https://ieeexplore.ieee.org/abstract/document/7962235\n  --shot_noise_rate_hz SHOT_NOISE_RATE_HZ\n                        Temporal noise rate of ON+OFF events in darkest parts\n                        of scene; reduced in brightest parts.\n  --photoreceptor_noise\n                        Create temporal noise by injecting Gaussian noise to\n                        the log photoreceptor before lowpass filtering.This\n                        way, more accurate statistics of temporal noise will\n                        tend to result (in particular, alternating ON and OFF\n                        noise events)but the noise rate will only approximate\n                        the desired noise rate; the photoreceptor noise will\n                        be computed to result in the --shot_noise_rate noise\n                        value. Overrides the default shot noise mechanism\n                        reported in 2020 v2e paper.\n  --leak_jitter_fraction LEAK_JITTER_FRACTION\n                        Jitter of leak noise events relative to the (FPN)\n                        interval, drawn from normal distribution\n  --noise_rate_cov_decades NOISE_RATE_COV_DECADES\n                        Coefficient of Variation of noise rates (shot and\n                        leak) in log normal distribution decades across pixel\n                        arrayWARNING: currently only in leak events\n  --refractory_period REFRACTORY_PERIOD\n                        Refractory period in seconds, default is 0.5ms.The new\n                        event will be ignore if the previous event is\n                        triggered less than refractory_period ago.Set to 0 to\n                        disable this feature.\n  --dvs_emulator_seed DVS_EMULATOR_SEED\n                        Set to a integer \u003e0 to use a fixed random seed.default\n                        is 0 which means the random seed is not fixed.\n  --show_dvs_model_state SHOW_DVS_MODEL_STATE [SHOW_DVS_MODEL_STATE ...]\n                        One or more space separated list model states. Do not\n                        use '='. E.g. '--show_dvs_model_state all'. Possible\n                        models states are (without quotes) either 'all' or\n                        chosen from dict_keys(['new_frame', 'log_new_frame',\n                        'lp_log_frame', 'scidvs_highpass',\n                        'photoreceptor_noise_arr', 'cs_surround_frame',\n                        'c_minus_s_frame', 'base_log_frame', 'diff_frame'])\n  --save_dvs_model_state\n                        save the model states that are shown (cf\n                        --show_dvs_model_state) to avi files\n  --record_single_pixel_states RECORD_SINGLE_PIXEL_STATES\n                        Record internal states of a single pixel specified by\n                        (x,y) tuple to 'pixel-states.dat'.The file is a\n                        pickled binary dict that has the state arrays over\n                        time imcluding a time array.Pixel location can also be\n                        specified as x,y without ()\nDVS camera sizes (selecting --dvs346, --dvs640, etc. overrides --output_width and --output_height:\n  --output_height OUTPUT_HEIGHT\n                        Height of output DVS data in pixels. If None, same as\n                        input video. Use --output_height=260 for Davis346.\n  --output_width OUTPUT_WIDTH\n                        Width of output DVS data in pixels. If None, same as\n                        input video. Use --output_width=346 for Davis346.\n  --dvs128              Set size for 128x128 DVS (DVS128)\n  --dvs240              Set size for 240x180 DVS (DAVIS240)\n  --dvs346              Set size for 346x260 DVS (DAVIS346)\n  --dvs640              Set size for 640x480 DVS (DAVIS640)\n  --dvs1024             Set size for 1024x768 DVS (not supported for AEDAT-2.0\n                        output since there is no jAER DVS1024 camera\nSloMo upsampling (see also \"DVS timestamp resolution\" group):\n  --disable_slomo       Disables slomo interpolation; the output DVS events\n                        will have exactly the timestamp resolution of the\n                        source video (which is perhaps modified by\n                        --input_slowmotion_factor).\n  --slomo_model SLOMO_MODEL\n                        path of slomo_model checkpoint.\n  --batch_size BATCH_SIZE\n                        Batch size in frames for SuperSloMo. Batch size 8-16\n                        is recommended if your GPU has sufficient memory.\n  --vid_orig VID_ORIG   Output src video at same rate as slomo video (with\n                        duplicated frames). Specify emtpy string or 'None' to\n                        skip output.\n  --vid_slomo VID_SLOMO\n                        Output slomo of src video slowed down by\n                        slowdown_factor.Specify emtpy string or 'None' to skip\n                        output.\n  --slomo_stats_plot    show a plot of slomo statistics\nInput file handling:\n  -i INPUT, --input INPUT\n                        Input video file or a image folder; leave empty for\n                        file chooser dialog.If the input is a folder, the\n                        folder should contain a ordered list of image files.In\n                        addition, the user has to set the frame rate manually.\n  --input_frame_rate INPUT_FRAME_RATE\n                        Either override the video file metadata frame rate or\n                        manually define the video frame rate when the video is\n                        presented as a list of image files. Overrides the\n                        stored (metadata) frame rate of input video. This\n                        option overrides the --input_slowmotion_factor\n                        argument in case the input is from a video file.\n  --input_slowmotion_factor INPUT_SLOWMOTION_FACTOR\n                        (See --input_frame_rate argument too.) Sets the known slow-motion factor of the input video, \n                        i.e. how much the video is slowed down, i.e., \n                        the ratio of shooting frame rate to playback frame rate. \n                        input_slowmotion_factor\u003c1 for sped-up video and \n                        input_slowmotion_factor\u003e1 for slowmotion video.\n                        If an input video is shot at 120fps yet is presented as a 30fps video \n                        (has specified playback frame rate of 30Hz, \n                        according to file's FPS setting), \n                        then set --input_slowdown_factor=4.\n                        It means that each input frame represents (1/30)/4 s=(1/120)s.\n                        If input is video with intended frame intervals of \n                        1ms that is in AVI file \n                        with default 30 FPS playback spec, \n                        then use ((1/30)s)*(1000Hz)=33.33333.\n  --start_time START_TIME\n                        Start at this time in seconds in video. Use None to\n                        start at beginning of source video.\n  --stop_time STOP_TIME\n                        Stop at this time in seconds in video. Use None to end\n                        at end of source video.\n  --crop CROP           Crop input video by (left, right, top, bottom) pixels.\n                        E.g. CROP=(100,100,0,0) crops 100 pixels from left and\n                        right of input frames. CROP can also be specified as\n                        L,R,T,B without ()\n  --hdr                 Treat input video as high dynamic range (HDR)\n                        logarithmic, i.e. skip the linlog conversion step. Use\n                        --hdr for HDR input with floating point gray scale\n                        input videos. Units of log input are based on white\n                        255 pixels have values ln(255)=5.5441\nSynthetic input:\n  --synthetic_input SYNTHETIC_INPUT\n                        Input from class SYNTHETIC_INPUT that has methods\n                        next_frame() and total_frames(). Disables file input\n                        and SuperSloMo frame interpolation and the DVS\n                        timestamp resolution is set by the times returned by\n                        next_frame() method. SYNTHETIC_INPUT.next_frame()\n                        should return a tuple (frame, time) with frame having\n                        the correct resolution (see DVS model arguments) which\n                        is array[y][x] with pixel [0][0] at upper left corner\n                        and pixel values 0-255. The time is a float in\n                        seconds. SYNTHETIC_INPUT must be resolvable from the\n                        classpath. SYNTHETIC_INPUT is the module name without\n                        .py suffix. See example moving_dot.py.\nOutput: DVS video:\n  --dvs_exposure DVS_EXPOSURE [DVS_EXPOSURE ...]\n                        Mode to finish DVS frame event integration:\n                        \tduration time: Use fixed accumulation time in seconds, e.g. \n                        \t\t--dvs_exposure duration .005; \n                        \tcount n: Count n events per frame,e.g.\n                        \t\t--dvs_exposure count 5000;\n                        \tarea_count M N: frame ends when any area of N x N pixels fills with M events, e.g.\n                        \t\t-dvs_exposure area_count 500 64\n                        \tsource: each DVS frame is from one source frame (slomo or original, depending on if slomo is used)\n  --dvs_vid DVS_VID     Output DVS events as AVI video at frame_rate. To\n                        suppress, supply empty argument or 'None'.\n  --dvs_vid_full_scale DVS_VID_FULL_SCALE\n                        Set full scale event count histogram count for DVS\n                        videos to be this many ON or OFF events for full white\n                        or black.\n  --no_preview          disable preview in cv2 windows for faster processing.\nOutput: DVS events:\n  --ddd_output          Save frames, frame timestamp and corresponding event\n                        index in HDF5 format used for DDD17 and DDD20\n                        datasets. Default is False.\n  --dvs_h5 DVS_H5       Output DVS events as hdf5 event database.\n  --dvs_aedat2 DVS_AEDAT2\n                        Output DVS events as DAVIS346 camera AEDAT-2.0 event\n                        file for jAER; one file for real and one file for v2e\n                        events. To suppress, supply argument None.\n  --dvs_text DVS_TEXT   Output DVS events as text file with one event per line\n                        [timestamp (float s), x, y, polarity (0,1)].\n  --label_signal_noise  append to the --dvs_text file a column,containing list\n                        of signal and shot noise events. Each row of the CSV\n                        appends a 1 for signal and 0 for noise.** Notes: 1:\n                        requires activating --dvs_text option; 2: requires\n                        disabling --photoreceptor_noise option (because when\n                        noise arises from photoreceptor it is impossible to\n                        tell if event was caused by signal or noise.3: Only\n                        labels shot noise events (because leak noise events\n                        arise from leak and cannot be distinguished from\n                        photoreceptor input).\nCenter-Surround DVS:\n  --cs_lambda_pixels CS_LAMBDA_PIXELS\n                        space constant of surround in pixels, None to disable.\n                        This space constant lambda is sqrt(1/gR) where g is\n                        the transverse conductance and R is the lateral\n                        resistance.\n  --cs_tau_p_ms CS_TAU_P_MS\n                        time constant of photoreceptor center of diffuser in\n                        ms, or 0 to disable for instantaneous surround.\n                        Defined as C/g where C is capacitance and g is the\n                        transverse conductance from photoreceptor to\n                        horizontal cell network. This time isthe time constant\n                        for h cell diffuser network response time to global\n                        input to photoreceptors. If set to zero, then the\n                        simulation of diffuser runs until it converges, i.e.\n                        until the maximum change between timesteps is smaller\n                        than a threshold value\nSCIDVS pixel:\n  --scidvs              Simulate proposed SCIDVS pixel with nonlinear\n                        adapatation and high gain\nRun with no --input to open file dialog\n```\nYou can put [tennis.mov](https://drive.google.com/file/d/1dNUXJGlpEM51UVYH4-ZInN9pf0bHGgT_/view?usp=sharing) in the _input_ folder to try it out with the command line below.  Or leave out all options and just use the file chooser to select the movie.\n\nFrom root of v2e, run the following\n```\npython v2e.py -i input/tennis.mov --overwrite --timestamp_resolution=.003 --auto_timestamp_resolution=False --dvs_exposure duration 0.005 --output_folder=output/tennis --overwrite --pos_thres=.15 --neg_thres=.15 --sigma_thres=0.03 --dvs_aedat2 tennis.aedat --output_width=346 --output_height=260 --stop_time=3 --cutoff_hz=15 \n```\nRun the command above, and the following files will be created in a folder called _output/tennis_.\n\n```\ndvs-video.avi\ndvs-video-frame_times.txt\ntennis.aedat\nv2e-args.txt\nvideo_orig.avi\nvideo_slomo.avi\n```\n\n* _dvs-video.avi_: DVS video (with playback rate 30Hz) but with frame rate (DVS timestamp resolution) set by source video frame rate times slowdown_factor.\n* _dvs-video-frame_times.txt_: The times of the DVS frames. Useful when _--dvs_exposure count_ or _--dvs_exposure area-count_ methods are used.\n* _tennis.aedat_: AEDAT-2.0 file for playback and algorithm experiments in [jAER](https://jaerproject.net) (use the AEChip _Davis346Blue_ to play this file.)\n* _v2e-args.txt_: All the parameters and logging output from the run.\n* _video_orig.avi_: input video, but converted to luma and resized to output (width,height) and with repeated frames to allow comparison to _slomo.avi_.\n* _video_slomo.avi_: slow motion video (with playback rate 30Hz) but slowed down by slowdown_factor.\n\nThe [v2e site](https://sites.google.com/view/video2events/home) shows these videos.\n## Scripting v2e\nSee the [scripts folder](https://github.com/SensorsINI/v2e/blob/master/scripts)  for various samples of shell and cmd scripts to run v2e from terminal.\n\n### Synthetic input\n\nThere are also samples in the [scripts folder](https://github.com/SensorsINI/v2e/blob/master/scripts) of \npython modules to generate synthetic input to v2e, e.g. [particles.py](https://github.com/SensorsINI/v2e/blob/master/scripts/particles.py)\n\nYou can specify particles as the class that generates input frames to generate DVS events from using the command line option\n````shell\nv2e --synthetic_input scripts.particles ...\n````\nYou synthetic input class should subclass _base_synthetic_class.py_. You should override the constructor and the _next_frame()_ method.\n\n  * See [base_synthetic_input.py](https://github.com/SensorsINI/v2e/blob/master/scripts/base_synthetic_input.py) for more information.\n  * You can pass command line arguments into your class; see [particles.py](https://github.com/SensorsINI/v2e/blob/master/scripts/particles.py) for example.\n\n## Model parameters\n\nThe DVS ON and OFF threshold nominal values are set by _pos_thres_ and _neg_thres_. The pixel to pixel variation is set by _sigma_thres_. The pixel cutoff frequency in Hz is set by _cutoff_hz_. The leak event rate is set by _leak_rate_hz_. \n\nThe _-dvs_params_ argument sets reasonable DVS model parameters for high and low light conditions.\n\nSee our technical paper for futher information about these parameters.\n \n ### Automatic vs manual DVS timestamp resolution\n The output DVS timestamps will be quantized to some value depending on options chosen.\n \n  *  _--disable_slomo_ will disable slomo interpolation and the DVS events will have exactly the times of the input video, perhaps modified by --input_slowmotion_factor\n  *  _--timestamp_resolution=X_ will upsample as needed to obtain this desired timestamp resolution _X_ in seconds. If auto_timestamp_resolution is set, then timestamp_resolution will still set the minimum timestamp resolution, i.e. if automatic timestamping would result in 5ms timestamps but timestamp_resolution is 1ms, then 1ms will still be the timestamp resolution.\n  *  _--auto_timestamp_resolution_ will upsample in each _--batch_size_ frames using the computed optical flow to limit motion per frame to at most 1 pixel. In this case, turning on _--slomo_stats_plot_ will generate a plot like the following, which came from a driving video where the car sped up during part of the video:\n \n ![auto_slomo_stats](media/slomo_stats.png)\n \n This plot shows the actual timestamps of the interpolated frames (in orange) and the frame intervals for each batch of frames (in blue).\n \n### Photoreceptor lowpass filtering\n_v2e_ includes an intensity-dependent 1st-order lowpass filtering of light intensity; see the paper for details. \nIf you set a nonzero --cutofffreq_hz, then it is important that the sample rate be high enough to allow the IIR lowpass filters to update properly, i.e.\nthe time constant tau of the lowpass filters must be at least 3 times larger than the frame interval.\nCheck the console output for warnings about undersampling for lowpass filtering.\n\n\n \n ### Frame rate and DVS timestamp resolution in v2e\nThere are several different 'frame rates' in v2e. On opening the input video, v2e reads the frame rate of the video and assumes the video is shot in real time, except that you can specify a _--input_slowmotion_factor_ slowdown_factor if the video is already a slow-motion video. The desired DVS timestamp resolution is combined with the source frame rate to compute the slow-motion upsampling factor. The output DVS AVI video is then generated using a _--dvs-exposure_ method.\n\n * _--avi_frame_rate_: Just sets the frame rate for playback of output AVI files\n * _--dvs-exposure_: See next section\n * _--input_slowmotion_factor_: Specifies by what factor the input video is slowed down.\n \n### Effect of multiple events per (sub) frame\nAnytime a source (or upsampled source) video generates more than 1 event per frame, these events need to be distributed over the time between frames. v2e arbitrarily stacks them as shown in the example below, resulting in pyramids of events and periodic overall bursts of events at each frame.  I.e. v2e first computes the maximum number of events by any pixel, then it subdivides the interframe interval by this number, then it puts all pixels with 1 event at the frame, then pixels with 2 events have thier events placed at the first sub-interval and so on. The reduce this effect, use a smaller timestamp resolution.\n\n ![v2e_pyramid](media/v2e-pyr.gif)\n\n## DVS frame exposure modes\n\nThe DVS allows arbritrary frame rates. _v2e_ provides 3 methods to 'expose' DVS video frames, which are selected by the\n--dvs_exposure argument:\n 1. **Constant-Duration**: _--dvs_exposure _duration_ _T__:  - Each frame has constant duration _T_.\n 2. **Constant-Count**: _--dvs_exposure_count_ _N_:  - each frame has the same number _N_ of DVS events, as first described in Delbruck, Tobi. 2008. “Frame-Free Dynamic Digital Vision.” In Proceedings of Intl. Symp. on Secure-Life Electronics, Advanced Electronics for Quality Life and Society, 1:21–26. Tokyo, Japan: Tokyo. https://drive.google.com/open?id=0BzvXOhBHjRheTS1rSVlZN0l2MDg..\n 3. **Area-Event**: _--dvs_exposure_ _area_event_ _N_ _M_:  - frames are accumulated until any block of *M*x*M* pixels fills up with _N_ events, as first described in Liu, Min, and T. Delbruck. 2018. “Adaptive Time-Slice Block-Matching Optical Flow Algorithm for Dynamic Vision Sensors.” In Proceedings of British Machine Vision Conference (BMVC 2018). Newcastle upon Tyne, UK: Proceedings of BMVC 2018. https://doi.org/10.5167/uzh-168589.\n 4. **Source**: _--dvs_exposure source_: DVS timestamps are based on source video timestamps. Extra events per frame are spaced between frames accoruding to the maximum number of events for that frame.\n\n - _Constant-Duration_ is like normal video, i.e. sampled at regular, ideally Nyquist rate. \n - _Constant-Count_ frames have the same number of pixel brightness change events per frame. But if the scene is very textured (i.e. busy) then frame can get very brief, while parts of the input with only a small object moving can have very long frames.\n - _Area-Event_ compensates for this effect to some extent by concluding exposure when any block of pixels fills with a constant count.\n\n## DAVIS camera conversion Dataset\n\nv2e can convert recordings from\n [DDD20](https://sites.google.com/view/davis-driving-dataset-2020/home) and the original [DDD17](https://docs.google.com/document/d/1HM0CSmjO8nOpUeTvmPjopcBcVCk7KXvLUuiZFS6TWSg/pub) \nwhich are the first public end-to-end training datasets \nof automotive driving using a DAVIS event + frame camera. It lets you compare the real DVS data with the conversion. \nThis dataset is maintained by the Sensors Research Group of Institute of Neuroinformatics. \n\nFor your convenience, we offer via google drive one recording from _DDD20_ (our newer DDD dataset) of 800s of\nLos Angeles street driving. \nThe file is _aug04/rec1501902136.hdf5_ [[link]](https://drive.google.com/open?id=1KIaHsn72ZpVBZR6SGeFcd2lILyZoD2-5)\n  in Google Drive for you to try it with v2e (***Warning:*** 2GB 7z compressed, 5.4 GB uncompressed).\n\n```bash\nmkdir -p input\nmv rec1501902136.hdf5 ./input\n```\n\n**NOTE** you must run these scripts with the _-m package.script.py_ notation, not by directly pointing to the .py file.\n\n### Extract data from DDD recording\n\n_ddd_h5_extract_data.py_ extracts the DDD recording DVS events to jAER _.aedat_ and video _.avi_ files.\n\n```\nddd_extract_data.py -h\nusage: ddd_extract_data.py [-h] [-i INPUT] -o OUTPUT_FOLDER [--start_time START_TIME] [--stop_time STOP_TIME] [--rotate180] [--overwrite] [--dvs240] [--dvs346]\n\nExtract data from DDD recording\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -i INPUT, --input INPUT\n                        input video file; leave empty for file chooser dialog (default: None)\n  -o OUTPUT_FOLDER, --output_folder OUTPUT_FOLDER\n                        folder to store outputs (default: None)\n  --start_time START_TIME\n                        start at this time in seconds in video (default: None)\n  --stop_time STOP_TIME\n                        stop point of video stream (default: None)\n  --rotate180           rotate output 180 deg (default: False)\n  --overwrite           overwrites files in existing folder (checks existance of non-empty output_folder) (default: False)\n  --dvs240              Set size for 240x180 DVS (Davis240). Use for DDD17 recording. (default: False)\n  --dvs346              Set size for 346x260 DVS (Davis346). Use for DDD20 recording. (default: False)\n\nRun with no --input to open file dialog\n\n```\n\nYou can then synthesize events from the DDD APS video .avi file that is produced and compare with the real events extracted to the AEDAT-2.0 file that is also extracted.\n\n\n## Working with jAER DAVIS recordings\n\nDAVIS cameras like the one that recorded DDD17 and DDD20 are often used with [jAER](https://jaerproject.net) (although DDD recordings were made with custom python wrapper around caer). _v2e_ will output a jAER-compatible .aedat file in [AEDAT-2.0 format](https://inivation.com/support/software/fileformat/#aedat-20), which jAER uses.\n\nTo work with existing jAER DAVIS .aedat, you can export the DAVIS APS frames using the jAER EventFilter [DavisFrameAVIWriter](https://github.com/SensorsINI/jaer/blob/master/src/ch/unizh/ini/jaer/projects/davis/frames/DavisFrameAviWriter.java); see the [jAER user guide](https://docs.google.com/document/d/1fb7VA8tdoxuYqZfrPfT46_wiT1isQZwTHgX8O22dJ0Q/edit?usp=sharing), in particular, the [section about using DavisFrameAVIWriter](https://docs.google.com/document/d/1fb7VA8tdoxuYqZfrPfT46_wiT1isQZwTHgX8O22dJ0Q/edit#heading=h.g4cschniofmo). In DavisFrameAVIWriter, **don't forget to set the frameRate to the actual frame rate of the DAVIS frames** (which you can see at the top of the jAER display). This will make the conversion have approximately the correct DVS event timing. (jAER can drop APS frames if there are too many DVS events, so don't count on this.) Once you have the AVI from jAER, you can generate v2e events from it with _v2e.py_ and see how they compare with the original DVS events in jAER, by playing the exported v2e .aedat file in jAER.\n\nAn example of this conversion and comparison is on the [v2e home page](https://sites.google.com/view/video2events/home).\n\n## Technical Details ##\n\nSee the [v2e home page](https://sites.google.com/view/video2events/home).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsensorsini%2Fv2e","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsensorsini%2Fv2e","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsensorsini%2Fv2e/lists"}