{"id":21665134,"url":"https://github.com/radi-cho/gatedtabtransformer","last_synced_at":"2025-07-06T01:05:57.871Z","repository":{"id":45378435,"uuid":"438372251","full_name":"radi-cho/GatedTabTransformer","owner":"radi-cho","description":"A deep learning tabular classification architecture inspired by TabTransformer with integrated gated multilayer perceptron.","archived":false,"fork":false,"pushed_at":"2023-02-04T09:59:50.000Z","size":63367,"stargazers_count":98,"open_issues_count":1,"forks_count":7,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-19T06:30:42.621Z","etag":null,"topics":["classification","machine-learning","tabular-data"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/radi-cho.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-12-14T19:16:28.000Z","updated_at":"2025-04-11T12:53:42.000Z","dependencies_parsed_at":"2025-04-12T00:41:37.295Z","dependency_job_id":null,"html_url":"https://github.com/radi-cho/GatedTabTransformer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/radi-cho/GatedTabTransformer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radi-cho%2FGatedTabTransformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radi-cho%2FGatedTabTransformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radi-cho%2FGatedTabTransformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radi-cho%2FGatedTabTransformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/radi-cho","download_url":"https://codeload.github.com/radi-cho/GatedTabTransformer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radi-cho%2FGatedTabTransformer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263833417,"owners_count":23517374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","machine-learning","tabular-data"],"created_at":"2024-11-25T10:49:08.431Z","updated_at":"2025-07-06T01:05:57.842Z","avatar_url":"https://github.com/radi-cho.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The GatedTabTransformer.\n\nA deep learning tabular classification architecture inspired by [TabTransformer](https://arxiv.org/abs/2012.06678) with integrated [gated](https://arxiv.org/abs/2105.08050) multilayer perceptron. Check out our paper on [arXiv](https://arxiv.org/abs/2201.00199). Applications and usage demonstrations are available [here](https://github.com/radi-cho/GatedTabTransformer-Applications).\n\n\u003cimg alt=\"Architecture\" src=\"./paper/media/GatedTabTransformer-architecture.png\" width=\"350px\"\u003e\u003c/img\u003e\n\n## Usage\n\n```python\nimport torch\nimport torch.nn as nn\nfrom gated_tab_transformer import GatedTabTransformer\n\nmodel = GatedTabTransformer(\n    categories = (10, 5, 6, 5, 8),      # tuple containing the number of unique values within each category\n    num_continuous = 10,                # number of continuous values\n    transformer_dim = 32,               # dimension, paper set at 32\n    dim_out = 1,                        # binary prediction, but could be anything\n    transformer_depth = 6,              # depth, paper recommended 6\n    transformer_heads = 8,              # heads, paper recommends 8\n    attn_dropout = 0.1,                 # post-attention dropout\n    ff_dropout = 0.1,                   # feed forward dropout\n    mlp_act = nn.LeakyReLU(0),          # activation for final mlp, defaults to relu, but could be anything else (selu, etc.)\n    mlp_depth=4,                        # mlp hidden layers depth\n    mlp_dimension=32,                   # dimension of mlp layers\n    gmlp_enabled=True                   # gmlp or standard mlp\n)\n\nx_categ = torch.randint(0, 5, (1, 5))   # category values, from 0 - max number of categories, in the order as passed into the constructor above\nx_cont = torch.randn(1, 10)             # assume continuous values are already normalized individually\n\npred = model(x_categ, x_cont)\nprint(pred)\n```\n\n## Citation\n\n```bibtex\n@misc{cholakov2022gatedtabtransformer,\n      title={The GatedTabTransformer. An enhanced deep learning architecture for tabular modeling}, \n      author={Radostin Cholakov and Todor Kolev},\n      year={2022},\n      eprint={2201.00199},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradi-cho%2Fgatedtabtransformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fradi-cho%2Fgatedtabtransformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradi-cho%2Fgatedtabtransformer/lists"}