{"id":24956747,"url":"https://github.com/changwookjun/transformer","last_synced_at":"2025-09-13T16:21:09.949Z","repository":{"id":273963988,"uuid":"158667289","full_name":"changwookjun/Transformer","owner":"changwookjun","description":"Chatbot using Tensorflow (Model is transformer) ko","archived":false,"fork":false,"pushed_at":"2018-12-10T02:29:04.000Z","size":539,"stargazers_count":29,"open_issues_count":1,"forks_count":13,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-24T16:45:57.103Z","etag":null,"topics":["bert","chatbot","korean-nlp","multihead-attention","self-attention","tensorflow","transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/changwookjun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-22T08:35:44.000Z","updated_at":"2024-10-20T16:55:15.000Z","dependencies_parsed_at":"2025-01-24T03:28:35.682Z","dependency_job_id":null,"html_url":"https://github.com/changwookjun/Transformer","commit_stats":null,"previous_names":["changwookjun/transformer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/changwookjun%2FTransformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/changwookjun%2FTransformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/changwookjun%2FTransformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/changwookjun%2FTransformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/changwookjun","download_url":"https://codeload.github.com/changwookjun/Transformer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248279196,"owners_count":21077406,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","chatbot","korean-nlp","multihead-attention","self-attention","tensorflow","transformer"],"created_at":"2025-02-03T06:41:04.913Z","updated_at":"2025-04-10T19:04:30.788Z","avatar_url":"https://github.com/changwookjun.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ChatBot(transformer) \nTransformer를 활용한 ChatBot을 개발함\n\n# Architecture \n![images](images/1.png)  \n\n# Attention \n![images](images/2.png)  \n\n# Scaled Dot-Product Attention \n![images](images/3.png)  \n\n#  Multi-Head Attention \n![images](images/4.png) \n\n#  Position-wise Feed-Forward Networks \n![images](images/5.png) \n\n#  Positional Encoding  \n![images](images/6.png) \n![images](images/7.png) \n\n# Regularization \n![images](images/8.png) \n\n\n\n# Learning Data\nTitle|Contents|Other\n--|--|--\n데이터 이름|Chatbot data\n데이터 용도|한국어 챗봇  학습을 목적으로 사용한다.\n데이터 권한|MIT 라이센스\n데이터 출처|https://github.com/songys/Chatbot_data (송영숙님)\n\n# Requirement\nPython 3.6.6   \ntensorflow 1.11   \nkonlpy   \npandas   \nsklearn   \n\n# Project Structure\n    .\n    ├── data_in                     # 데이터가 존재하는 영역\n        ├── ChatBotData.csv         # 전체 데이터\n        ├── ChatBotData.csv_short   # 축소된 데이터 (테스트 용도)\n        ├── README.md               # 데이터 저자 READMD 파일\n    ├── data_out                    # 출력 되는 모든 데이터가 모이는 영역\n        ├── vocabularyData.voc      # 사전 파일\n        ├── check_point             # check_point 저장 공간\n        ├── model                   # model 저장 공간\n    ├── configs.py                  # 모델 설정에 관한 소스\n    ├── data.py                     # data 전처리 및 모델에 주입되는 data set 만드는 소스\n    ├── main.py                     # 전체적인 프로그램이 시작되는 소스\n    ├── model.py                    # 모델이 들어 있는 소스\n    └── predict.py                  # 학습된 모델로 실행 해보는 소스      \n   \n\n# Config\ntf.app.flags.DEFINE_integer('batch_size', 64, 'batch size') # 배치 크기  \ntf.app.flags.DEFINE_integer('train_steps', 20000, 'train steps') # 학습 에포크  \ntf.app.flags.DEFINE_float('dropout_width', 0.5, 'dropout width') # 드롭아웃 크기 \ntf.app.flags.DEFINE_integer('hidden_size', 128, 'weights size') # 가중치 크기 # 논문 512 사용 \ntf.app.flags.DEFINE_float('learning_rate', 1e-3, 'learning rate') # 학습률  \ntf.app.flags.DEFINE_integer('shuffle_seek', 1000, 'shuffle random seek') # 셔플 시드값  \ntf.app.flags.DEFINE_integer('max_sequence_length', 25, 'max sequence length') # 시퀀스 길이  \ntf.app.flags.DEFINE_integer('embedding_size', 128, 'embedding size') # 임베딩 크기 # 논문 512 사용 학습 속도 및 성능 튜닝  \ntf.app.flags.DEFINE_integer('query_dimention', 128, 'q#uery dimention') # 논문 512 사용 학습 속도 및 성능 튜닝  \ntf.app.flags.DEFINE_integer('key_dimention', 128, 'key dimention') # 논문 512 사용 학습 속도 및 성능 튜닝  \ntf.app.flags.DEFINE_integer('value_dimention', 128, 'value dimention') # 논문 512 사용 학습 속도 및 성능 튜닝  \ntf.app.flags.DEFINE_integer('layers_size', 2, 'layers size') # 논문은 6개 레이어이나 2개 사용 학습 속도 및 성능 튜닝  \ntf.app.flags.DEFINE_integer('heads_size', 4, 'heads size')# 논문은 8개 헤더이나 4개 사용 학습 속도 및 성능 튜닝  \ntf.app.flags.DEFINE_string('data_path', './data_in/ChatBotData.csv', 'data path') #  데이터 위치  \ntf.app.flags.DEFINE_string('vocabulary_path', './data_out/vocabularyData.voc', 'vocabulary path') # 사전 위치  \ntf.app.flags.DEFINE_string('check_point_path', './data_out/check_point', 'check point path') # 체크 포인트 위치  \ntf.app.flags.DEFINE_boolean('tokenize_as_morph', False, 'set morph tokenize') # 형태소에 따른 토크나이징 사용 유무  \ntf.app.flags.DEFINE_boolean('conv_1d_layer', True, 'set conv 1d layer') # 논문의 두가지 방법중 두번째 conv1d 사용 유무  \ntf.app.flags.DEFINE_boolean('xavier_embedding', True, 'set init xavier embedding') # Xavier 초기화를 이용해서 임베딩 사용 유무  \n#tf.app.flags.DEFINE_boolean('mask_loss', True, 'set masking loss') # 로스에 마스킹 사용하여 (PAD, END) 로스 배제 사용 유무  \n\n# Usage\npython main.py\n\n# Predict\npython predict.py 남자친구가 너무 잘 생겼어\n\n# Reference\nTitle|Contents\n--|--\nData|[Chatbot data](https://github.com/songys/Chatbot_data)  \nPaper|[Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf)  \nPaper|[BERT: Pre-training of Deep Bidirectional Transformers forLanguage Understanding](https://arxiv.org/pdf/1810.04805.pdf)\n\n# Author\nChangWookJun / @changwookjun (changwookjun@gmail.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchangwookjun%2Ftransformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchangwookjun%2Ftransformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchangwookjun%2Ftransformer/lists"}