https://github.com/astrazeneca/multimodal-python-course

The purpose of the code is to facilitate a comprehensive understanding of multimodal data science applications within medical domain. The code serves to support the delivery of a cutting-edge workshop designed to introduce researchers to the rapidly evolving field of multimodal data science
https://github.com/astrazeneca/multimodal-python-course

Last synced: 3 months ago
JSON representation

The purpose of the code is to facilitate a comprehensive understanding of multimodal data science applications within medical domain. The code serves to support the delivery of a cutting-edge workshop designed to introduce researchers to the rapidly evolving field of multimodal data science

• Host: GitHub
• URL: https://github.com/astrazeneca/multimodal-python-course
• Owner: AstraZeneca
• License: apache-2.0
• Created: 2024-04-17T19:08:31.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2025-07-28T08:18:57.000Z (11 months ago)
• Last Synced: 2025-09-09T11:50:32.658Z (9 months ago)
• Language: Jupyter Notebook
• Size: 11.2 MB
• Stars: 10
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          


## Navigating the Multimodal Map: Insights into Foundation Models

### Venue

![course-image](img/logo.jpg)

Online training course run by the NextGen Data Scientists, AstraZeneca

### Trainers

Sylwia Majchrowska, Ricardo Mokhtari

### Course structure and links

Day | Title | Activity | Materials |

:---:|:-----:|:--------:|:---------:|

0 | Troubleshooting software installations | preparation | [Introduction and installations](notebooks/Day_0_Instalations.ipynb) |

1 | SAM Concept Cove | Session | [Materials](Day_1_SAM_Concept_Cove/Day_1_SAM_Concept_Cove.ipynb) |

2 | Multimodal data handling | Session | [Materials](Day_2_Data_Integration/Day_2_Data_Integration.ipynb) |

### References

1. LangSAM [Code](https://github.com/luca-medeiros/lang-segment-anything)

2. Grounding DiNO [Code](https://github.com/IDEA-Research/GroundingDINO) [Paper](https://arxiv.org/abs/2303.05499)

3. SAM [Code](https://github.com/facebookresearch/segment-anything) [Paper](https://arxiv.org/abs/2304.02643)

4. [Attention illustrated blog](https://towardsdatascience.com/illustrated-self-attention-2d627e33b20a)

5. [Attention video](https://www.youtube.com/watch?v=KmAISyVvE1Y)

6. [Another attention video](https://www.youtube.com/watch?v=eMlx5fFNoYc)

7. [Cross attention](https://www.youtube.com/watch?v=aw3H-wPuRcw&list=WL&index=37)

8. [Visualise a transformer](https://bbycroft.net/llm)

9. [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://arxiv.org/abs/2107.02314)

10. [MMML Tutorial - ICML 2023](https://cmu-multicomp-lab.github.io/mmml-tutorial/icml2023/)

11. [Multimodal data fusion – analysis](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10007548/)

12. [Fusion of Multi-Modal Data Stream for Clinical Event Prediction - Imon Banerjee, PhD](https://www.youtube.com/watch?v=3DroMVNb2vg)

13. [Data-Efficient Multimodal Fusion on a Single GPU](https://arxiv.org/pdf/2312.10144.pdf)

14. [Integrated multimodal artificial intelligence framework for healthcare applications](https://www.nature.com/articles/s41746-022-00689-4)

15. [Inferring multimodal latent topics from electronic health records](https://www.nature.com/articles/s41467-020-16378-3)

16. [Multimodal Risk Prediction with Physiological Signals, Medical Images and Clinical Notes](https://www.medrxiv.org/content/10.1101/2023.05.18.23290207v1)