https://github.com/eto-ai/spark-video

Processing videos on Apache Spark
https://github.com/eto-ai/spark-video

ffmpeg opencv spark

Last synced: about 2 months ago
JSON representation

Processing videos on Apache Spark

• Host: GitHub
• URL: https://github.com/eto-ai/spark-video
• Owner: eto-ai
• Created: 2021-11-12T22:26:38.000Z (about 3 years ago)
• Default Branch: main
• Last Pushed: 2022-02-14T15:34:29.000Z (almost 3 years ago)
• Last Synced: 2024-04-14T15:24:18.893Z (9 months ago)
• Topics: ffmpeg, opencv, spark
• Language: Scala
• Homepage:
• Size: 748 KB
• Stars: 10
• Watchers: 4
• Forks: 6
• Open Issues: 12
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# spark-video

## For User

### Local Development

Just copy and paste the python code from [bin/local_install.py](bin/local_install.py) to the pyspark REPL.

### Google Colab

See [notebooks/GoogleColabDemo.ipynb](notebooks/GoogleColabDemo.ipynb)

### Production: Databricks Runtime

Put these three lines in the init script:

```

sudo wget -O /databricks/jars/ffmpeg-4.4-1.5.6-linux-x86_64.jar https://repo1.maven.org/maven2/org/bytedeco/ffmpeg/4.4-1.5.6/ffmpeg-4.4-1.5.6-linux-x86_64.jar

sudo wget -O /databricks/jars/javacpp-1.5.6-linux-x86_64.jar https://repo1.maven.org/maven2/org/bytedeco/javacpp/1.5.6/javacpp-1.5.6-linux-x86_64.jar

sudo wget -O /databricks/jars/spark-video-assembly-0.0.4.jar https://github.com/eto-ai/spark-video/releases/download/v0.0.4/spark-video-assembly_2.12-0.0.4.jar

```

## For Developer

### Cheatsheet

```

bin/mill 'video[2.12].test'

bin/mill 'video[2.12].test.testOnly' '**.MLImageTest.scala'

bin/mill 'video[2.12].publishLocal'

bin/mill 'video[2.12].assembly'

bin/mill mill.scalalib.scalafmt.ScalafmtModule/checkFormatAll __.sources

bin/mill mill.scalalib.scalafmt.ScalafmtModule/reformatAll __.sources

```