Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/algoflows/onvu-stream

ONVU Stream cloud native serverless upload and streaming playback service on AWS. It's a front-to-back full stack portfolio project.
https://github.com/algoflows/onvu-stream

aws cdk lambda-functions nx-workspace react sns vercel vite vitest

Last synced: about 14 hours ago
JSON representation

ONVU Stream cloud native serverless upload and streaming playback service on AWS. It's a front-to-back full stack portfolio project.

Awesome Lists containing this project

README 

        


  

  







![GitHub deployments](https://img.shields.io/github/deployments/algoflows/onvu-stream/production?label=vercel&logo=vercel&logoColor=white) 



[Project Planning](https://github.com/users/algoflows/projects/2/views/2) | [Milestones](https://github.com/algoflows/onvu-stream/milestones) | [Issues](https://github.com/algoflows/onvu-stream/issues)






# Stack



- [Amazon Web Services CDK](https://aws.amazon.com/cdk/)

  - Lambda 

  - S3, S3 Event Triggers

  - Media Converter Elements

  - Kenises Streaming

  - DynamoDB

- [Reactjs](https://vitejs.dev/) - Web client 

- [Tanstack Router](https://tanstack.com/) Advanced react router (built for dashboards and complex SPAs)

- [React Query](https://react-query-v3.tanstack.com/) (with jotai integration)

- [Jotai](https://jotai.org/) Atomic state management

- [Tailwind](https://tailwindcss.com/) shared UI library

- [NX.dev](https://nx.dev/) monorepo workspace

- [Cypress](https://www.cypress.io/) Visual testing framework

- [Framer Motion](https://www.framer.com/motion/) Animation library for react

- [CloudCraft](https://www.cloudcraft.co/) Visualise your AWS cloud environment

- [Dashjs](http://reference.dashif.org/dash.js/v4.5.1/samples/dash-if-reference-player/index.html) Media playing client

- [Google Shaka Player](https://github.com/shaka-project/shaka-player) MPEG-DASH player by Google

- [Shaka Player React](https://github.com/matvp91/shaka-player-react)



# AWS Architecture

  

  




# Breif



A requirement has been issued from the product team to improve the quality of the on demand video playback service from our cloud platform on low bandwidth links. It has been proposed that we can use MPEG Dash.



# Reqs



Design a simple web application that plays a video delivered by MPEG-Dash. It should store metadata about the video in the backend which is exposed by an API. Support your design with sample code and documentation.



# Tasks



### Platform

- [ ] Provision dynamodb table to store video metadata

- [ ] Provision POST Lambda to store video metadata and mpd reference url in dynamodb table

- [ ] Use mediainfo.js to extract mp4 metadata

- [ ] Provision GET lambda to get all videos

- [ ] Provision GET lambda to get single video ref

- Provision S3 storage bucket for video and asset storage (AWS VideoTranscoder solution)

  - [x] S3 input bucket (new uploads, mp4)

  - [x] S3 ouput bucket (after transcoding into mp4 dash)

  - [x] S3 thumnail bucket

- [x] Setup AWS env accounts (prod account, dev account, temp test account)



### Frontend

- [x] Scaffold Nextjs app (ATT changed for Vite powered React App)

- [x] Setup tailwind

- [x] Setup routing and navigation pages

- Screens/Pages

  - [x] Home

  - [x] Player

  - [ ] Upload

  - [ ] About (3d render of archicture and flow diagrams, iframe)

- Components

  - [x] Dash video player client

  - [x] RootLayout component

  - [ ] limit file upload size (300mb)



### Libs

- shared

  - [ ] config (constants etc..)

  - [ ] mocks

  - [ ] types

  - [ ] hooks/dal

  - [ ] loading spinner component

  - [x] card component

  - [ ] upload box component



### QA Testing



- [ ] SonarCloud code quality

- [ ] Test API Gatway and lambda

- [ ] Cypress

- [ ] Jest



### CICD



- Github Actions deployemnt

  - [ ] Deploy CDK

  - [x] Deploy React App (Vercel Git integration)

    - [x] dev  domain: https://onvu-stream-dev.vercel.app/

    - [x] test domain: https://onvu-stream-test.vercel.app/

    - [x] prod domain: https://onvu-stream.vercel.app/

- [x] NX Cloud build caching



### Documentation

- [x] Readme

- [x] Upload flow diagram

- [ ] Full Architecture diagram

  



### Stretch Goals

- Platform

  - [ ] Provision S3 storage bucket for thumbnails

  - [ ] AWS Media Converter Elements job creation DASH-ISO

  - [ ] Step functions sequential flow/job

  - [ ] Provision cognito userpool using CDK

  - [ ] AWS transcription service (extract keywords automatically and ability to see video transcription next to streaming client)

  - [ ] Create lambda to returned signed url for for video upload

  - [ ] Video upload using aws presigned url

- Frontend

  - [ ] Search box for video listings filtering

  - [ ] Authentication using Cognito NextAuth



### Data (Dynamodb)



| PK (videoId)|  SK              | length | type    | inputBucket| outputBucketMpd | coverImg | transcription |

| :-----:     | :--------------: | :-----:| :-----: | :-------:  | :------------:  | :-------:| :---------:   |

| 208439      | userId#createdAt | 4:20   | mpeg    | http://... | http://...      | http//...| fjlkajdjljfl  |



### Research, docs and resources

- [AWS Kinesis streaming mpeg-dash](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/dash-playback.html)

- [Streaming video from S3, CloudFront and React](https://www.youtube.com/watch?v=WP7Dpvrl8Ic)

- [KVS ingestion from RTSP cameras a Kinesis Video Streams tutorial](https://www.youtube.com/watch?v=nVxwX7Q9nPU)

- [Build a Real Time Data Streaming System with AWS Kinesis, Lambda Functions and a S3 Bucket](https://www.youtube.com/watch?v=We5Jr4GGLL0)

- [How to Build a File Upload System on AWS with React and a Serverless API | Lambda, S3, API Gateway](https://www.youtube.com/watch?v=IgAE-ycnb94)



## Development server



Run `nx serve my-app` for a dev server. Navigate to http://localhost:4200/. The app will automatically reload if you change any of the source files.



## Code scaffolding



Run `nx g @nrwl/react:component my-component --project=my-app` to generate a new component.



## Build



Run `nx build my-app` to build the project. The build artifacts will be stored in the `dist/` directory. Use the `--prod` flag for a production build.



## Running unit tests



Run `nx test my-app` to execute the unit tests via [Jest](https://jestjs.io).



Run `nx affected:test` to execute the unit tests affected by a change.



## Running end-to-end tests



Run `nx e2e my-app` to execute the end-to-end tests via [Cypress](https://www.cypress.io).



Run `nx affected:e2e` to execute the end-to-end tests affected by a change.



## Understand your workspace



Run `nx graph` to see a diagram of the dependencies of your projects.



## Further help



Visit the [Nx Documentation](https://nx.dev) to learn more.



## ☁ Nx Cloud



### Distributed Computation Caching & Distributed Task Execution






Nx Cloud pairs with Nx in order to enable you to build and test code more rapidly, by up to 10 times. Even teams that are new to Nx can connect to Nx Cloud and start saving time instantly.



Teams using Nx gain the advantage of building full-stack applications with their preferred framework alongside Nx’s advanced code generation and project dependency graph, plus a unified experience for both frontend and backend developers.



Visit [Nx Cloud](https://nx.app/) to learn more.