{"id":19209182,"url":"https://github.com/hrolive/advanced-deep-learning-with-transformers","last_synced_at":"2025-04-10T16:05:21.416Z","repository":{"id":156425011,"uuid":"629982940","full_name":"HROlive/Advanced-Deep-Learning-with-Transformers","owner":"HROlive","description":"Workshop that will take you from Graph Neural Networks (GNNs) to Transformers, architectures which have led to numerous breakthrough achievements in a variety of fields such as Natural Language Processing (NLP), chemistry, and physics.","archived":false,"fork":false,"pushed_at":"2023-09-11T14:28:40.000Z","size":5106,"stargazers_count":11,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-24T13:51:16.279Z","etag":null,"topics":["anaconda","deep-learning","gnn","graphs","hpc","jupyter-notebook","machine-learning","neural-network","python","transformers"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HROlive.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-19T12:28:28.000Z","updated_at":"2024-12-31T07:31:51.000Z","dependencies_parsed_at":"2024-11-09T13:42:11.681Z","dependency_job_id":null,"html_url":"https://github.com/HROlive/Advanced-Deep-Learning-with-Transformers","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HROlive%2FAdvanced-Deep-Learning-with-Transformers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HROlive%2FAdvanced-Deep-Learning-with-Transformers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HROlive%2FAdvanced-Deep-Learning-with-Transformers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HROlive%2FAdvanced-Deep-Learning-with-Transformers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HROlive","download_url":"https://codeload.github.com/HROlive/Advanced-Deep-Learning-with-Transformers/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248250511,"owners_count":21072623,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anaconda","deep-learning","gnn","graphs","hpc","jupyter-notebook","machine-learning","neural-network","python","transformers"],"created_at":"2024-11-09T13:29:24.704Z","updated_at":"2025-04-10T16:05:21.398Z","avatar_url":"https://github.com/HROlive.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/banner.jpg\"\u003e\n\u003c/p\u003e\n\n## Table of Contents\n1. [Description](#description)\n2. [Information](#information)\n3. [File descriptions](#files)\n4. [Installing the dependencies](#dependencies)\n5. [Certificate](#certificate)\n\n\u003ca name=\"descripton\"\u003e\u003c/a\u003e\n## Description\n\nIn recent years, Graph Neural Networks (GNNs) and Transformers have led to numerous breakthrough achievements in a variety of fields such as Natural Language Processing (NLP), chemistry, and physics. By doing away with the need for fixed-size inputs, these architectures significantly extend the scope of problems to which deep learning can be applied.\n\nThis workshop will take you from the representation of graphs and finite sets as inputs for neural networks to the implementation of full GNNs for a variety of tasks. You will learn about the central concepts used in GNNs in a hands-on setting using Jupyter Notebooks and a series of coding exercises. While the workshop will use problems from the field of chemistry as an example for applications, the skills you learn can be transferred to any domain where finite sets or graph-based representations of data are appropriate. From GNNs, we will make the leap to Transformer architectures, and explain the conceptual ties between the two.\n\n\u003ca name=\"information\"\u003e\u003c/a\u003e\n## Information\n\nThe overall goals of this course were the following:\n\u003e - Workshop on graphs and how to represent them for neural networks;\n\u003e - Short recap of the morning concepts and an introduction to application in chemistry;\n\u003e - Working with molecules as graphs and deep dive into using Graph Neural Networks;\n\u003e - Implementing a GNN;\n\u003e - Overview of Transformers;\n\u003e - Implementing a Transformer from a GNN.\n\nMore detailed information and links for the course can be found on the [course website](https://hackmd.io/@enccs/transformers-april2023).\n\n\u003ca name=\"files\"\u003e\u003c/a\u003e\n## File descriptions\n\nThe training material(slides, recordings, notebooks, etc) can be found on [this website](https://enccs.github.io/gnn_transformers/).\n\nThe notebooks and exercises can be found in this repository and are organized in their respective folders, one for each session of the workshop.\nThis repository contains the notebooks for the ENCCS course on Graph Neural Networks and Transformers. There are two versions of the notebooks, one intended to be run on Google Colab (under the colab directory) and one intended to be run locally (under the session_1 to session_4 directories).\n\n\u003ca name=\"dependencies\"\u003e\u003c/a\u003e\n## Installing the dependencies\n\nYou need an Anaconda (or conda forge) installation before proceeding.\n\nIf you run the local files, you need to install the dependencies. These are in conda environment files. Depending on what GPU you have, run the below commands in a shell:\n\n #### NVIDIA GPU:\n```shell\nconda env create - f cuda_environment.yml\n```\n #### AMD GPU:\n```shell\nconda env create -f rocm_environment.yml\n```\n\nOnce the environment has been created, activate it by running\n\n```shell\nconda activate gnnt\n```\n    \nAnd start the jupyter notebook server:\n\n```shell\njupyter notebook\n```\n\n\u003ca name=\"certificate\"\u003e\u003c/a\u003e\n## Certificate\n\nThe certificate for the course can be found below:\n\n[\"Advanced Deep Learning with Transformers\" - EuroCC National Competence Centre Sweden](https://github.com/HROlive/Advanced-Deep-Learning-with-Transformers/blob/main/images/certificate.pdf) (Issued On: April 2023)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhrolive%2Fadvanced-deep-learning-with-transformers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhrolive%2Fadvanced-deep-learning-with-transformers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhrolive%2Fadvanced-deep-learning-with-transformers/lists"}