{"id":31108736,"url":"https://github.com/lingbai-kong/CausalFormer","last_synced_at":"2025-09-17T06:45:46.631Z","repository":{"id":208374061,"uuid":"679695479","full_name":"lingbai-kong/CausalFormer","owner":"lingbai-kong","description":"PyTorch Implementation of CausalFormer: An Interpretable Transformer for Temporal Causal Discovery","archived":false,"fork":false,"pushed_at":"2025-04-13T09:09:53.000Z","size":4220,"stargazers_count":34,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-13T10:22:56.338Z","etag":null,"topics":["causal-discovery","datamining","interpretability","pytorch-implementation","time-series"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lingbai-kong.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-08-17T12:13:59.000Z","updated_at":"2025-04-13T09:09:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"216302b6-c95e-42c0-b3ff-6a89cdff542e","html_url":"https://github.com/lingbai-kong/CausalFormer","commit_stats":null,"previous_names":["lingbai-kong/causalformer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lingbai-kong/CausalFormer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingbai-kong%2FCausalFormer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingbai-kong%2FCausalFormer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingbai-kong%2FCausalFormer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingbai-kong%2FCausalFormer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lingbai-kong","download_url":"https://codeload.github.com/lingbai-kong/CausalFormer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingbai-kong%2FCausalFormer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275549019,"owners_count":25484678,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-17T02:00:09.119Z","response_time":84,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["causal-discovery","datamining","interpretability","pytorch-implementation","time-series"],"created_at":"2025-09-17T06:45:23.877Z","updated_at":"2025-09-17T06:45:46.624Z","avatar_url":"https://github.com/lingbai-kong.png","language":"Jupyter Notebook","funding_links":[],"categories":["🚀 GitHub Repositories"],"sub_categories":["🌟 **Real-World Magic**"],"readme":"# PyTorch Implementation of CausalFormer: An Interpretable Transformer for Temporal Causal Discovery\n\nOfficial PyTorch implementation for [CausalFormer: An Interpretable Transformer for Temporal Causal Discovery](https://www.computer.org/csdl/journal/tk/5555/01/10726725/21dMtID3eUw) ([arXiv](https://arxiv.org/abs/2406.16708)).\n\n## Requirements\n\n* Python \u003e= 3.5 (3.6 recommended)\n* PyTorch (tested with PyTorch 1.11.0)\n* Optional: CUDA (tested with CUDA 11.3)\n* networkx\n* numpy\n* pandas\n* scikit_learn\n\n## Folder Structure\n  ```\n  CausalFormer/\n  ├── base/ - abstract base classes\n  │   ├── base_data_loader.py\n  │   ├── base_model.py\n  │   └── base_trainer.py\n  ├── config/ - holds configuration for training\n  │   ├── config_basic_diamond_mediator.json\n  │   ├── config_basic_v_fork.json\n  │   ├── config_fMRI.json\n  │   └── config_lorenz.json\n  ├── data/ - default directory for storing input data\n  │   ├── basic\n  │   ├── fMRI\n  │   └── lorenz96\n  ├── data_loader/\n  ├── evaluator/\n  ├── experiments.ipynb\n  ├── explainer\n  │   └── explainer.py\n  ├── interpret.py - main script to start interpreting\n  ├── LICENSE\n  ├── logger/\n  ├── model/ - models, relevance propogation, losses, and metrics\n  │   ├── loss.py\n  │   ├── metric.py\n  │   ├── model.py\n  │   ├── NonParamRP.py\n  │   └── RRP.py\n  ├── parse_config.py\n  ├── README.md\n  ├── requirements.txt\n  ├── runner.py - integrated script to start running CausalFormer\n  ├── saved/\n  │   ├── models/ - trained models are saved here\n  │   └── log/ - default logdir for tensorboard\n  ├── trainer/\n  ├── train.py - main script to start training\n  └── utils\n  ```\n\n## Dataset\n\n- Synthetic datasets: [Basic causal structures with additive noise](https://dataverse.harvard.edu/dataverse/basic_causal_structures_additive_noise)\n\n- Lorenz96: \n\n  \u003e Lorenz, Edward N. \"Predictability: A problem partly solved.\" *Proc. Seminar on predictability*. Vol. 1. No. 1. 1996.\n\n  The Lorenz 96 model is a nonlinear model of climate dynamics as defined below. \n  $$\\frac{dx_{t,i}}{dt}=(x_{t,i+1}-x_{t,i-2})x_{t,i-1}-x_{t,i}+F$$\n  where $x_{t,i}$ is the data of time series $i$ at time slot $t$, and $F$ is a forcing constant that determines the level of non-linearity and chaos in the series. We simulate a Lorenz-96 model with 10 variables and $F\\in [ 30,40 ]$ over a time span of 1,000 units.\n\n- fMRI: [NetSim](https://www.fmrib.ox.ac.uk/datasets/netsim/index.html)\n\n### Dataset file format\n\n- Time Series: The time series file is a CSV containing multiple time series. The first row is the header with the names of the time series. Each column represents a time series.\n- Groundtruth Causal Graph: The groundtruth causal graph file contains tuples in the form of (i, j, t), where i is the cause, j is the effect, and t is the time lag.\n\n## Usage\n\nTry `python runner.py -c config/config_fMRI.json -t demo` to run code.\n\nChecking experiments.ipynb for more experiments running.\n\n### Config file format\nConfig files are in `.json` format:\n```javascript\n{\n  \"name\": \"Causality Learning\", // training session name\n  \"n_gpu\": 1,                   // number of GPUs to use for training.\n  \n  \"arch\": {\n    \"type\": \"PredictModel\",     // name of model architecture to train\n    \"args\": {\n      \"d_model\": 512,           // Dimension of the embedding vector. D_QK in paper\n      \"n_head\": 8,              // Number of attention heads. h in paper\n      \"n_layers\": 1,            // single transformer encoder layer\n      \"ffn_hidden\": 512,        // Hidden dimension in the feed forward layer. d_FFN in paper\n      \"drop_prob\": 0,           // Dropout probability (Not used in practice)\n      \"tau\": 10                 // Temperature hyperparameter for attention softmax\n    }                \n  },\n  \"data_loader\": {\n    \"type\": \"TimeseriesDataLoader\",    // selecting data loader\n    \"args\":{\n      \"data_dir\": \"data/\",             // dataset path\n      \"batch_size\": 64,                // batch size\n      \"time_step\": 32,                 // input window size. T in paper\n      \"output_window\": 31,             // output window size\n      \"feature_dim\": 1,                // input feature dim\n      \"output_dim\": 1,                 // output window size\n      \"shuffle\": true,                 // shuffle training data before splitting\n      \"validation_split\": 0.1          // size of validation dataset. float(portion) or int(number of samples)\n      \"num_workers\": 2,                // number of cpu processes to be used for data loading\n    }\n  },\n  \"optimizer\": {\n    \"type\": \"Adam\",\n    \"args\":{\n      \"lr\": 0.001,                     // learning rate\n      \"weight_decay\": 0,               // (optional) weight decay\n      \"amsgrad\": true\n    }\n  },\n  \"loss\": \"masked_mse_torch\",          // loss\n  \"metrics\": [\n    \"accuracy\", \"masked_mse_torch\"     // list of metrics to evaluate\n  ],                         \n  \"lr_scheduler\": {\n    \"type\": \"StepLR\",                  // learning rate scheduler\n    \"args\":{\n      \"step_size\": 50,          \n      \"gamma\": 0.1\n    }\n  },\n  \"trainer\": {\n    \"epochs\": 100,                     // number of training epochs\n    \"save_dir\": \"saved/\",              // checkpoints are saved in save_dir/models/name\n    \"save_freq\": 1,                    // save checkpoints every save_freq epochs\n    \"verbosity\": 2,                    // 0: quiet, 1: per epoch, 2: full\n    \"monitor\": \"min val_loss\"          // mode and metric for model performance monitoring. set 'off' to disable.\n    \"early_stop\": 10\t                 // number of epochs to wait before early stop. set 0 to disable.\n    \"lam\": 5e-4,                       // the coefficient for normalization\n    \"tensorboard\": true,               // enable tensorboard visualization\n  },\n  \"explainer\": {\n      \"m\":2,                           // number of top clusters of causal scores to consider.\n      \"n\":3                            // number of total clusters for k-means clustering.\n  }\n}\n```\n\n## License\nThis project is licensed under the  GPL-3.0 License. See LICENSE for more details\n\nThis project is based on the [pytorch-template](https://github.com/victoresque/pytorch-template) GitHub template.\n\n## Cite\n```\n@article{kong2024causalformer,\n  title={CausalFormer: An Interpretable Transformer for Temporal Causal Discovery},\n  author={Kong, Lingbai and Li, Wengen and Yang, Hanchen and Zhang, Yichao and Guan, Jihong and Zhou, Shuigeng},\n  journal={IEEE Transactions on Knowledge \\\u0026 Data Engineering},\n  number={01},\n  pages={1--14},\n  year={2024},\n  publisher={IEEE Computer Society}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flingbai-kong%2FCausalFormer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flingbai-kong%2FCausalFormer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flingbai-kong%2FCausalFormer/lists"}