{"id":15967416,"url":"https://github.com/bryanlimy/calciumgan","last_synced_at":"2025-03-26T15:32:32.669Z","repository":{"id":92146608,"uuid":"212600364","full_name":"bryanlimy/CalciumGAN","owner":"bryanlimy","description":"Synthesising Realistic Calcium Imaging Data of Neuronal Populations Using GAN.","archived":false,"fork":false,"pushed_at":"2023-07-06T22:00:09.000Z","size":6390,"stargazers_count":2,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-22T02:02:40.726Z","etag":null,"topics":["calcium-imaging","calcium-signals","gan","spike-analysis","spike-trains"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bryanlimy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-03T14:26:40.000Z","updated_at":"2021-12-04T03:47:33.000Z","dependencies_parsed_at":"2024-10-30T03:27:44.044Z","dependency_job_id":null,"html_url":"https://github.com/bryanlimy/CalciumGAN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bryanlimy%2FCalciumGAN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bryanlimy%2FCalciumGAN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bryanlimy%2FCalciumGAN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bryanlimy%2FCalciumGAN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bryanlimy","download_url":"https://codeload.github.com/bryanlimy/CalciumGAN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245681508,"owners_count":20655212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["calcium-imaging","calcium-signals","gan","spike-analysis","spike-trains"],"created_at":"2024-10-07T18:23:20.247Z","updated_at":"2025-03-26T15:32:32.644Z","avatar_url":"https://github.com/bryanlimy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Synthesising Realistic Calcium Imaging Data of Neuronal Populations Using GAN\n\nImplementation of CalciumGAN, used to obtain the results detailed in [CalciumGAN: A Generative Adversarial Network Model for Synthesising Realistic Calcium Imaging Data of Neuronal Populations](https://arxiv.org/abs/2009.02707).\n\nCiting this work\n```\n@article{li2020calciumgan,\n  title={CalciumGAN: A Generative Adversarial Network Model for Synthesising Realistic Calcium Imaging Data of Neuronal Populations},\n  author={Li, Bryan M and Amvrosiadis, Theoklitos and Rochefort, Nathalie and Onken, Arno},\n  journal={arXiv preprint arXiv:2009.02707},\n  year={2020}\n}\n```\n\n### Table of content\n- [1. Installation](#1-installation)\n- [2. Dataset](#2-dataset)\n- [3. Training model](#3-train-model)\n- [4. Spike analysis](#4-spike-analysis)\n- [5. Visualization and Profiling](#5-visualization-and-profiling)\n- [6. Hyper-parameter Search](#6-hyper-parameters-search)\n\n## 1. Installation\n\n### 1.1 Requirements\n- It is recommended to install the codebase in a virtual environment, \nsuch as [conda](https://conda.io/en/latest/).\n\n### 1.2 Quick install\n- Create a new `conda` environment in Python 3.6\n    ```bash\n    conda create -n calciumgan python=3.6\n    ```\n- Activate `calciumgan` virtual environment\n    ```bash\n    conda activate calciumgan\n    ```\n- Install all dependencies and packages with `setup.sh` script, works on both Linus and macOS.\n    ```bash\n    sh setup.sh\n    ```\n\n### 1.3 Manual setup\nInstall the following packages:\n- [TensorFlow](https://tensorflow.org)\n- [j-friedrich/OASIS](https://github.com/j-friedrich/OASIS)\n- [Neo](https://github.com/NeuralEnsemble/python-neo)\n- [Elephant](https://github.com/NeuralEnsemble/elephant)\n- packages in `requirements.txt`\n- code from [dg_python](https://github.com/mackelab/dg_python) are also being \nused for the dichotomized Gaussian model\n\n## 2. Dataset\n- Navigate to `dataset`\n    ```bash\n    cd dataset\n    ```\n\n### 2.1 Recorded Calcium Imaging Data\n- Place all raw calcium imaging data under `dataset/raw_data`\n- Apply OASIS to infer spike train\n    ```bash\n    python spike_train_inference.py --input_dir raw_data\n    ```\n- Generate `TFRecords` from a specific pickle file `--input`, normalize the \ndata, preform segmentation and store the `TFrecords` in `output_dir`. \nUse `--help` to see all available arguments.\n    ```bash\n    python generate_tfrecords.py --input raw_data/signals.pkl --output_dir tfrecords/sl2048 --sequence_length 2048 --normalize\n    ```\n\n#### 2.2 Dichotomized Gaussian Artificial Data\n- Generate artificial spike trains and calcium-like signals from the \nDichomotized Gaussian distribution with the mean and covariance of data in \n`--input`, save the the output pickle file to `--output`. `TFRecords` in `--output_dir`. Use `--help` to \nsee all available arguments.\n    ```bash\n    python generate_dg_data.py --input raw_data/signals.pkl --output dg.pkl\n    ```\n- Generate `TFRecords` from a specific pickle file `--input`, normalize the \ndata, preform segmentation and store the `TFrecords` in `output_dir`. \nUse `--help` to see all available arguments.\n    ```bash\n    python generate_tfrecords.py --input dg.pkl --output_dir tfrecords/sl2048_dg --sequence_length 2048 --normalize\n    ```\n\n## 3. Train model\n- To train CalciumGAN on the recorded calcium imaging data with the default \nhyper-parameters for 400 epochs. Checkpoints, generated data, model training\ninformation are stored in `--output_dir`.\n    ```bash\n    python main.py --input_dir dataset/tfrecords/sl2048 --output_dir runs/001 --epochs 400 --batch_size 128 --model calciumgan --algoirthm wgan-gp --noise_dim 32 --num_units 64 --kernel_size 24 --strides 2 --m 10 --layer_norm --mixed_precision --save_generated last \n    ```\n- Use `--help` to check all available arguments. Mixed precision compute, \nTensorBoard profiling, hyper-parameter search are some of the features built \ninto this codebase.\n- The training command applies to both recorded data and dischotomized \nGaussian artificial data.\n\n## 4. Spike analysis\n\n#### 4.1 Recorded Calcium Imaging Data\n- Deconvolve the calcium signals to spike trains from generated data in \n`--output_dir`, then compute various spikes statistics. \nUse `--help` to check all available arguments.\n    ```bash\n    python compute_metrics.py --output_dir runs/001\n    ```\n- All the plots can be found in `runs/001/metrics/plots`\n\n#### 4.2 Dichotomized Gaussian Artificial Data\n- Deconvolve the calcium signals to spike trains from generated data in \n`--output_dir`, then compute various spikes statistics. \nUse `--help` to check all available arguments.\n    ```bash\n    python compute_dg_metrics.py --output_dir runs/002\n    ```\n- PLots of mean and covariance can be found in `diagrams/`\n\n### 5. Visualization and Profiling\n- Run `tensorboard`\n    ```bash\n    tensorboard --logdir runs/001\n    ```\n- We have implemented profiling with \n[TensorFlow Profiler](https://www.tensorflow.org/tensorboard/tensorboard_profiling_keras) support.\nYou can enable profiling with `--profile` flag when training the model with `main.py`.\n\n### 6. Hyper-parameters Search\n- We have incorporated the [Hyperparameter Turning with Keras](https://www.tensorflow.org/tensorboard/hyperparameter_tuning_with_hparams) feature. \nModify the hyper-parameters you would like to test in `seasrch.py` and run\n    ```bash\n    python search.py --input_dir dataset/tfrecords/sl2048 --output_dir runs/hparams_search --epochs 400 --mixed_precision\n    ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbryanlimy%2Fcalciumgan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbryanlimy%2Fcalciumgan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbryanlimy%2Fcalciumgan/lists"}