{"id":21578954,"url":"https://github.com/itrummer/lm4db","last_synced_at":"2025-03-18T07:40:46.001Z","repository":{"id":110457824,"uuid":"514375854","full_name":"itrummer/lm4db","owner":"itrummer","description":"Material for my VLDB'22 and BTW'23 tutorials on application for language models in data management","archived":false,"fork":false,"pushed_at":"2024-12-18T01:13:59.000Z","size":19725,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-24T14:18:41.966Z","etag":null,"topics":["bert-model","databases","gpt-3","language-model","ml","nlp"],"latest_commit_sha":null,"homepage":"https://itrummer.github.io/lm4db","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/itrummer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-15T18:58:48.000Z","updated_at":"2025-01-17T15:51:08.000Z","dependencies_parsed_at":"2024-08-01T00:41:09.109Z","dependency_job_id":null,"html_url":"https://github.com/itrummer/lm4db","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itrummer%2Flm4db","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itrummer%2Flm4db/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itrummer%2Flm4db/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itrummer%2Flm4db/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/itrummer","download_url":"https://codeload.github.com/itrummer/lm4db/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244180627,"owners_count":20411600,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert-model","databases","gpt-3","language-model","ml","nlp"],"created_at":"2024-11-24T13:12:03.155Z","updated_at":"2025-03-18T07:40:45.978Z","avatar_url":"https://github.com/itrummer.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n\u003ciframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/tYKH7Q5MDcg\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen\u003e\u003c/iframe\u003e\n\u003c/p\u003e\n\nThe introduction of Transformer-based language models has led to astonishing advances in the domain of natural language processing over the past years. Not only do such models dominate in a variety of standard benchmarks. The latest generation of language models can be specialized to novel, formerly unseen tasks with little to virtually no training data. \n\nIn this tutorial, I discuss the two key ideas enabling ultra-large language models: a new neural network architecture, the Transformer, and an unsupervised training process, based on the idea of transfer learning. After discussing the theoretical concepts behind language models, I demonstrate GPT-3 and other models and provide pointers on how to get access to this technology. Finally, I discuss novel use cases in data management that are enabled by language models, covering recent research and open problems.\n\n**Slides of the VLDB'22 tutorial (90 minutes) are [here](lm4dbtrummer.pdf).**\n\n**Slides of the BTW'23 tutorial (180 minutes) are [here](https://drive.google.com/file/d/1U-2j8oi5au3nuYwPIlhnno7c6UNDfifl/view?usp=sharing).**\n\n**Slides of the ICDE'24 tutorial (90 minutes) are [here](ICDEtutorial1.pdf).**\n\nPlease use the following citation to refer to this tutorial:\n```\n@article{Trummer2022e,\nauthor = {Trummer, Immanuel},\ndoi = {10.14778/3554821.3554896},\njournal = {PVLDB},\nnumber = {12},\npages = {3770 -- 3773},\ntitle = {From BERT to GPT-3 Codex: Harnessing the Potential of Very Large Language Models for Data Management},\nvolume = {15},\nyear = {2022}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitrummer%2Flm4db","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fitrummer%2Flm4db","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitrummer%2Flm4db/lists"}