{"id":28762230,"url":"https://github.com/hanksoong/charisma-predictor","last_synced_at":"2025-06-30T21:08:38.007Z","repository":{"id":295616821,"uuid":"990604429","full_name":"HANKSOONG/Charisma-Predictor","owner":"HANKSOONG","description":"Multimodal AI pipeline to predict Big Five personality traits and assess charismatic leadership using audio, text, and video inputs.","archived":false,"fork":false,"pushed_at":"2025-06-10T11:33:22.000Z","size":2008,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-17T08:07:31.178Z","etag":null,"topics":["audio-processing","big-five","charismatic-leadership","computer-vision","deep-learning","facial-landmarks","fusion-models","mediapipe","multimodal-learning","nlp","personality-prediction","pytorch","transformer"],"latest_commit_sha":null,"homepage":"https://github.com/HANKSOONG/Charisma-Predictor","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HANKSOONG.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-26T11:11:40.000Z","updated_at":"2025-06-11T00:08:18.000Z","dependencies_parsed_at":"2025-05-26T16:45:14.794Z","dependency_job_id":null,"html_url":"https://github.com/HANKSOONG/Charisma-Predictor","commit_stats":null,"previous_names":["hanksoong/charisma-predictor-multi-modal-ai-for-personality-leadership-assessment"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HANKSOONG/Charisma-Predictor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HANKSOONG%2FCharisma-Predictor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HANKSOONG%2FCharisma-Predictor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HANKSOONG%2FCharisma-Predictor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HANKSOONG%2FCharisma-Predictor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HANKSOONG","download_url":"https://codeload.github.com/HANKSOONG/Charisma-Predictor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HANKSOONG%2FCharisma-Predictor/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260318706,"owners_count":22991120,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio-processing","big-five","charismatic-leadership","computer-vision","deep-learning","facial-landmarks","fusion-models","mediapipe","multimodal-learning","nlp","personality-prediction","pytorch","transformer"],"created_at":"2025-06-17T08:07:31.068Z","updated_at":"2025-06-17T08:07:31.810Z","avatar_url":"https://github.com/HANKSOONG.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Charisma-Predictor: Multi-Modal AI for Personality \u0026 Leadership Assessment\n\n**Charisma-Predictor** is a multimodal AI pipeline developed as part of a Master's research project in Artificial Intelligence at Maastricht University. It received a final grade of **8.5 / 10**, based on methodology, implementation, and evaluation.\n\nThis repository contains the parts I personally developed: the **video model**, the **fusion logic**, and the **charisma scoring and visualization**.\n\n---\n\n## 🚀 Highlights\n\n*   **Video**: facial landmark tracking via MediaPipe + five sequence models (CNN, LSTM, GRU, Transformer, TCN)\n*   **Ensemble**: video outputs fused via weighted averaging (best MAE: 0.1189)\n*   **Fusion**: weighted averaging and Multi-Channel Weighted Fusion (MCWF)\n*   **Output**: Big Five personality prediction + charisma score ∈ \\[0, 1], classified into five levels\n*   **Optimization**: early stopping, learning rate scheduling (ReduceLROnPlateau)\n*   **Visuals**: leadership score distribution, confusion matrices\n*   **Achieved** up to **92.45% accuracy** on Big Five prediction via fusion (MCWF)\n\n---\n\n## 📊 Sample Output\n\n**Leadership Suitability Distribution:**\n\n![Leadership Histogram](figures/Leadership_Suitability_Distribution_true_label_vs_prediction.png)\n\n\u003e The fusion output shows high alignment between predicted and true leadership scores, confirming the effectiveness of cross-modal aggregation.\n\n**Fusion Model Personality Accuracy (MCWF):**\n\n![Fusion Confusion Matrix](figures/confusion_matrics_fusion.jpg)\n\n---\n\n## 👤 My Contribution\n\nThis repository reflects my direct contributions to the project:\n\n* Developed the full **video model pipeline**: feature extraction, five-model ensemble, evaluation\n* Designed and implemented **fusion logic**, including MCWF and weighted fusion strategies\n* Created the **charisma scoring system** using normalized Big Five trait correlations\n* Generated final evaluation metrics, plots, and analysis outputs\n\n---\n\n## 📂 Project Structure\n\n```\ncharisma-predictor/\n├── video_model/            # Facial landmark + sequence model ensemble\n├── fusion/                 # Fusion logic (weighted avg, MCWF)\n├── figures/                # Output plots (confusion matrices, histograms)\n├── results/                # Personality predictions + charisma scores\n├── text_and_audio/         # External references to group members' models\n│   └── README.md\n├── report_links/           # Final report (PDF)\n│   └── README.md\n├── LICENSE \n├── README.md               # You're reading it\n└── requirements.txt\n```\n\n---\n\n## 📅 Related Work by Team Members\n\nWhile this repo focuses on my implementation, the final fusion model also incorporated audio and text inputs from teammates:\n\n* 🔊 [Audio model (AST)](https://drive.google.com/drive/folders/1SoNqgf6J3f-QCa_LvFf0fSnW1xZLOgCV?usp=drive_link)\n* 📄 [Text model (BERT)](https://drive.google.com/drive/folders/1npBfmOsTbw5ziEsa_PnD_drb8xST2BSP?usp=drive_link)\n\n---\n\n## 🛠️ Run the Fusion Module\n\nThis repository includes the training code for the fusion model. To train the Multi-Channel Weighted Fusion (MCWF) model:\n\n```bash\npip install -r requirements.txt\ncd fusion\npython train_fusion.py\n```\n\n---\n\n## 🗋 Dataset\n\n* [First Impressions Dataset](https://chalearnlap.cvc.uab.cat/dataset/20/description/) – 10,000 annotated video clips\n\n---\n\n## 🤓 Methodology Summary\n\n**Model Workflow Overview:**\n\n*  **Video:** MediaPipe landmark sequences → five-model ensemble (CNN, LSTM, GRU, Transformer, TCN)\n*  **Audio:** AST + Random Forest (team contribution)\n*  **Text:** BERT-based personality estimation (team contribution)\n*  **Fusion:** Average, weighted, MCWF\n*  **Output:** Big Five scores → 0–1 charisma score → five-class suitability label\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhanksoong%2Fcharisma-predictor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhanksoong%2Fcharisma-predictor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhanksoong%2Fcharisma-predictor/lists"}