{"id":19125311,"url":"https://github.com/brightmart/slot_filling_intent_joint_model","last_synced_at":"2025-07-15T20:34:43.325Z","repository":{"id":108167895,"uuid":"112477561","full_name":"brightmart/slot_filling_intent_joint_model","owner":"brightmart","description":"attention based joint model for intent detection and slot filling ","archived":false,"fork":false,"pushed_at":"2017-12-28T10:19:30.000Z","size":2544,"stargazers_count":178,"open_issues_count":4,"forks_count":67,"subscribers_count":20,"default_branch":"master","last_synced_at":"2024-12-11T12:24:59.013Z","etag":null,"topics":["attention-mechanism","bi-directional","cnn","intent-detection","joint-models","knowledge-representation","slot-filling"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brightmart.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-29T13:16:31.000Z","updated_at":"2024-11-21T07:49:46.000Z","dependencies_parsed_at":null,"dependency_job_id":"48973f36-89bc-4ae7-a8b9-76b5b2b1eb08","html_url":"https://github.com/brightmart/slot_filling_intent_joint_model","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brightmart%2Fslot_filling_intent_joint_model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brightmart%2Fslot_filling_intent_joint_model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brightmart%2Fslot_filling_intent_joint_model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brightmart%2Fslot_filling_intent_joint_model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brightmart","download_url":"https://codeload.github.com/brightmart/slot_filling_intent_joint_model/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230520393,"owners_count":18238948,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention-mechanism","bi-directional","cnn","intent-detection","joint-models","knowledge-representation","slot-filling"],"created_at":"2024-11-09T05:35:24.024Z","updated_at":"2024-12-20T01:15:54.341Z","avatar_url":"https://github.com/brightmart.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Joint model for intent detection and slot filling based on attention, input alignment and knowledge.\n\nwith ability to detect whether a input sentence is a noise input or meanfuling input by combine feature from domain detection, intent detection and slot filling.\n\nwith ability to assign possibility to a input sentence by using language model.\n\nIntroduction:\n-------------------------------------------------------------------------------------\n1.intent detection and slot filling joint model which share encoding information\n\n2.incorporate knowledge information with embedding for both intent detection and slot filling. this embedding share the same embedding space with slots output.\n\n3.use bi-direction RNN and CNN to do intent detection\n\n4.use slots middle output as a feature for intent detection to boost performance\n\n5.domain detection is availabile by using CNN, same structure as intent detection. domain is a high level concept which indicates \narea that intent(s) belongs to.\n\n6.similiarity module is used to detect most similiar training data for any user input\n\n7.toy task: input a sequence of natural number, such as [5,7,2,6,8,3].\nfor slot filling: count label each number to 0 or 1. if sum of a number together with its previous and next number is great than a threshold(such as 14), we mark it as 1. otherwise 0.\nin this case, output of slot filling will be:[0,0,1,1,1,0]\nfor intent detection, count how many number totally is marked as 1. in this case, output of intent will be:3.\n\nPerformance:\n-------------------------------------------------------------------------------------\n\ndataset1:\n|---slot_naive(V6)|---slot_alime(V7)----------|\n\n|------97.9%------|----99.8%%-----------------|\n\ndataset2:\n\n|---intent_tmall|---intent_tmall(similiarity)|---intent_alime|---intent_alime(similiarity)|---TextCNN---|---TextCNN(similiarity)|\n\n|------95.37%|------72.0%-----------------|----93.0%------|----62.9%-----------------|----95.70%-------|----73.5%-------------|\n\n\n\nUsage:\n-------------------------------------------------------------------------------------\n1.train the model: train() of xxx_train.py\n\n2.test the model: predict() of xxx_predict.py\n\n3. for model structure, you can check xxx_model.py\n\nDescription for different versions:\n-------------------------------------------------------------------------------------\nV0 (seq2seq version): use TextCNN for intent, use encoder-decoder(seq2seq) model for slots. train() and predict() for toy task is available under a1_joint_intent_slots_model.py\n\n-----------------------------\nV1 (naive version): \n\nuse bi-directional GRU to encode input. this is share between intent detection and slots filling.\n\nintent was predicted directically after fully connected layer based on sum up for different time step. \n\nslots were predicted directically after fully connected layer for each time step.\n\n-----------------------------\nV2 (simple version): \nadd knowledge to naive version. knowledge is embedding, and used as additional feature to make prediction both for intent and slots.\n\n-----------------------------\n\nV3 (p-BOW,TextCNN,similiarity module): \n\nuse positional bag of words to encoder input sentence. this is share between intent detection and slots filling.\n\nTextCNN is used for intent detection. knowlege is embedded, transformed and used as feature together with output of TextCNN to make \n\na prediction.\n\nsimiliarity module is used to detect the most similiar question for input sentence. it used the representation learned by positional \n\nbag of words. this module is useful when you want to check similiar question or when you want to know the coverage of your dataset;\n\nyou can get a prediction by simply use the intent(or called answer) for the most similiar question of the input sentence.\n\n-----------------------------\n\nV4(Ali me style TextCNN):\nword embedding is concated with knowledge embedding to get better representation for each word. 'Hopefully' to capture additional \n\ninfomration that is relevant to make prediction. \n\nother part is same as V3\n\n-----------------------------\n\nV5(TextCNN):\n\njust to make a comparision with V4 by not using any knowledge.\n\n-----------------------------\n\nV6(+domain version)\n\ndomain detection is predicted besides intent detection and slot filling.\n\n-----------------------------\n\nV7(+context window for slot filling)\n\nmainly change slot filling part: 1.word vector+symbol vector 2.context window 3.nolinear projection 4.bi-directional lstm\n\nfor intent and domain detection, use representation from concat of word vector and symbol vector.\n\n-----------------------------\nV8(intent condition on domain; slot filling condition on intent)\n\ngiven a domain, intent is limit to a subset of total intents; given a intent, slot name is limited to a subset of total slot names. this model doing this\n\nby providing hidden states of domain together with other features before doing intent detection; it works similiar for slot filling.\n\n-----------------------------\n\n![alt text](https://github.com/brightmart/slot_filling_intent_joint_model/blob/master/resources/JOINT_MODEL.JPG)\n\n![alt text](https://github.com/brightmart/slot_filling_intent_joint_model/blob/master/resources/TextCNN.JPG)\n\n![alt text](https://github.com/brightmart/slot_filling_intent_joint_model/blob/master/resources/slot_filling_alime.JPG)\n\nConclude:\n-------------------------------------------------------------------------------------\nDifferent models can be used for intent detection and slots filling. Some model's performance is strong than others in some dataset, while other model's peformance is better in other dataset. So we need to do experiment using different model to get a better performance.\n\nReference:\n-------------------------------------------------------------------------------------\n1.Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling,\n\nhttps://arxiv.org/pdf/1609.01454.pdf\n\n2.阿里AI Labs王刚解读9小时卖出百万台的“天猫精灵” | 高山大学（GASA）,\n\nhttp://www.sohu.com/a/206109679_473283\n\n3.史上最全！阿里智能人机交互的核心技术解析 \nhttps://yq.aliyun.com/articles/277907?spm=5176.100244.teamhomeleft.54.SKEyCU\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrightmart%2Fslot_filling_intent_joint_model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrightmart%2Fslot_filling_intent_joint_model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrightmart%2Fslot_filling_intent_joint_model/lists"}