https://github.com/frankier/skelshop

📺 📰 🧑‍💼 Toolkit for skeleton & face analysis of talking heads (e.g. news) videos 🧑‍💼 📰 📺
https://github.com/frankier/skelshop

computer-vision facial-recognition openpose pose-estimation

Last synced: 8 days ago
JSON representation

📺 📰 🧑‍💼 Toolkit for skeleton & face analysis of talking heads (e.g. news) videos 🧑‍💼 📰 📺

• Host: GitHub
• URL: https://github.com/frankier/skelshop
• Owner: frankier
• License: mit
• Created: 2020-06-23T09:27:01.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2021-11-23T11:29:57.000Z (almost 3 years ago)
• Last Synced: 2024-04-17T12:20:34.086Z (7 months ago)
• Topics: computer-vision, facial-recognition, openpose, pose-estimation
• Language: Python
• Homepage: https://frankier.github.io/skelshop
• Size: 2.36 MB
• Stars: 5
• Watchers: 3
• Forks: 1
• Open Issues: 15
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


 SkelShop






  Docker/Singularity containers



|



  Documentation





SkelShop is a toolkit usable either as a suite of command line tools or a

Python library aimed at *offline analysis* of the ``talking heads'' genre of

videos, typically created in a television studio. This genre includes news

and current affairs programmes, variety shows and some documentaries. The

main attributes of this type of material is:

 * Multiple fixed place cameras in a studio with shot changes to different

   cameras and non-studio material.

 * Mainly upper bodies are visible.

 * Frontal shots are the most frequent type.

 * Faces are usually visible and most often frontal.

 * Occlusion is as often by an OSD (On Screen Display) than by some other

   object in the scene.

## Getting started

[See the documentation.](https://frankier.github.io/skelshop/)

## Features

 * Dump OpenPose skeletons to a fast-to-read HDF5 file format

 * A processing pipeline starting with shot segmentation

 * Apply black box person tracking on OpenPose skeletons

 * Draw skeletons over video and...

   * View the result in real time

   * Output the result to another video

 * Convert from some existing JSON based dump formats

 * Embed faces using dlib

   * Using OpenPose's face detection and keypoint estimation

   * Or using dlib's own face detection/keypoint estimation

   * Select best frames for best quality embeddings

 * Dump across heterogeneous HPC environments including GPU and CPU nodes.

 * Identify faces based on embeddings

   * Identify by direct comparison with references

   * Or cluster faces with no reference (also useful re-identitication)

   * Identify faces based on clusters, either with a reference or interactively

   * Quickly build up libraries of references using data from WikiData/WikiMedia commons

Here's a screenshot of the playsticks command:

![Screenshot of the playsticks

command](https://user-images.githubusercontent.com/299380/87277551-2d9f6180-c4eb-11ea-917c-4336ad36a97f.png)

[Find out more about what SkelShop can do in the documentation.](https://frankier.github.io/skelshop/)

## Contributions & Questions

Contributions are welcome! Feel free to use GitHub discussions to ask any

questions. [See also the contributing section of the

documentation.](https://frankier.github.io/skelshop/development/)

## Acknowledgments

Thanks to the authors of all the useful libraries I have used.

Some of the black box tracking code is based

on [this repository](https://github.com/lxy5513/cvToolkit).

Some code to do with clustering and usage of FAISS is based on [this repository](https://github.com/yl-1993/learn-to-cluster/).

[Icon by Adrien Coquet, FR from the Noun Project used under

CC-BY.](https://thenounproject.com/term/news/2673777)