{"id":19208278,"url":"https://github.com/elixir-webrtc/xav","last_synced_at":"2025-04-05T10:08:12.779Z","repository":{"id":147934901,"uuid":"612937179","full_name":"elixir-webrtc/xav","owner":"elixir-webrtc","description":"Elixir wrapper over FFmpeg for reading and decoding audio and video data","archived":false,"fork":false,"pushed_at":"2025-03-08T09:49:37.000Z","size":162887,"stargazers_count":62,"open_issues_count":2,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-29T09:09:25.699Z","etag":null,"topics":["audio","decoding","elixir","ffmpeg","reading","video"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elixir-webrtc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-12T12:40:02.000Z","updated_at":"2025-02-26T16:33:25.000Z","dependencies_parsed_at":"2024-02-13T14:43:03.670Z","dependency_job_id":"cec5c39d-e0c7-47ae-a4f6-1e21b5942715","html_url":"https://github.com/elixir-webrtc/xav","commit_stats":{"total_commits":94,"total_committers":6,"mean_commits":"15.666666666666666","dds":"0.11702127659574468","last_synced_commit":"a26f39a1d52cadcb87980a2615a848bda03da294"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-webrtc%2Fxav","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-webrtc%2Fxav/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-webrtc%2Fxav/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-webrtc%2Fxav/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elixir-webrtc","download_url":"https://codeload.github.com/elixir-webrtc/xav/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247318744,"owners_count":20919484,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio","decoding","elixir","ffmpeg","reading","video"],"created_at":"2024-11-09T13:25:37.392Z","updated_at":"2025-04-05T10:08:12.737Z","avatar_url":"https://github.com/elixir-webrtc.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Xav\n\n[![Hex.pm](https://img.shields.io/hexpm/v/xav.svg)](https://hex.pm/packages/xav)\n[![API Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/xav)\n[![CI](https://img.shields.io/github/actions/workflow/status/elixir-webrtc/xav/ci.yml?logo=github\u0026label=CI)](https://github.com/elixir-webrtc/xav/actions/workflows/ci.yml)\n[![codecov](https://codecov.io/gh/elixir-webrtc/xav/graph/badge.svg?token=2AG2acRhOf)](https://codecov.io/gh/elixir-webrtc/xav)\n\nElixir wrapper over FFmpeg for reading audio and video files.\n\nSee an interview with FFmpeg enthusiast:  https://youtu.be/9kaIXkImCAM\n\n## Installation\n\nMake sure you have installed FFMpeg (ver. 4.x - 7.x) development packages on your system\n(see [here](INSTALL.md) for installation one-liners) and add Xav to the list of your dependencies:\n\n```elixir\ndef deps do\n  [\n    {:xav, \"~\u003e 0.10.0\"},\n    # Add Nx if you want to have Xav.Frame.to_nx/1\n    {:nx, \"\u003e= 0.0.0\"}\n  ]\nend\n```\n\n## Usage\n\nDecode\n\n```elixir\ndecoder = Xav.Decoder.new(:vp8, out_format: :rgb24)\n{:ok, %Xav.Frame{} = frame} = Xav.Decoder.decode(decoder, \u003c\u003c\"somebinary\"\u003e\u003e)\n```\n\nDecode with audio resampling\n\n```elixir\ndecoder = Xav.Decoder.new(:opus, out_format: :flt, out_sample_rate: 16_000)\n{:ok, %Xav.Frame{} = frame} = Xav.Decoder.decode(decoder, \u003c\u003c\"somebinary\"\u003e\u003e)\n```\n\nRead from a file:\n\n```elixir\nr = Xav.Reader.new!(\"./some_mp4_file.mp4\")\n{:ok, %Xav.Frame{} = frame} = Xav.Reader.next_frame(r)\ntensor = Xav.Frame.to_nx(frame)\nKino.Image.new(tensor)\n```\n\nRead from a camera:\n\n```elixir\nr = Xav.Reader.new!(\"/dev/video0\", device?: true, out_format: :rgb24)\n{:ok, %Xav.Frame{} = frame} = Xav.Reader.next_frame(r)\ntensor = Xav.Frame.to_nx(frame)\nKino.Image.new(tensor)\n```\n\nSpeech to text:\n\n```elixir\n{:ok, whisper} = Bumblebee.load_model({:hf, \"openai/whisper-tiny\"})\n{:ok, featurizer} = Bumblebee.load_featurizer({:hf, \"openai/whisper-tiny\"})\n{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, \"openai/whisper-tiny\"})\n{:ok, generation_config} = Bumblebee.load_generation_config({:hf, \"openai/whisper-tiny\"})\n\nserving =\n  Bumblebee.Audio.speech_to_text_whisper(whisper, featurizer, tokenizer, generation_config,\n    defn_options: [compiler: EXLA]\n  )\n\n# Read a couple of frames.\n# See https://hexdocs.pm/bumblebee/Bumblebee.Audio.WhisperFeaturizer.html for default sampling rate.\nframes =\n    Xav.Reader.stream!(\"sample.mp3\", read: :audio, out_format: :flt, out_channels: 1, out_sample_rate: 16_000)\n    |\u003e Stream.take(200)\n    |\u003e Enum.map(fn frame -\u003e Xav.Frame.to_nx(frame) end)\n\nbatch = Nx.Batch.concatenate(frames)\nbatch = Nx.Defn.jit_apply(\u0026Function.identity/1, [batch])\nNx.Serving.run(serving, batch) \n```\n\n## Development\n\nTo make `clangd` aware of the header files used in your project, you can create a `compile_commands.json` file. \n`clangd` uses this file to know the compiler flags, include paths, and other compilation options for each source file. \n\n### Install bear\n\nThe easiest way to generate `compile_commands.json` from a Makefile is to use the `bear` tool. `bear` is a tool that records the compiler calls during a build and creates the `compile_commands.json` file.\n\nYou can install `bear` with your package manager:\n\n- __macOS__: brew install bear\n- __Ubuntu/Debian__: sudo apt install bear\n- __Fedora__: sudo dnf install bear\n\n### Generate compile_commands.json\n\nAfter installing bear, you can run it alongside your make command to capture the necessary information.\n\n```bash\nbear -- mix compile\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felixir-webrtc%2Fxav","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felixir-webrtc%2Fxav","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felixir-webrtc%2Fxav/lists"}