https://github.com/csinva/mouse-brain-decoding

Decoding images from calcium recordings using data from stringer et al. 2018.
https://github.com/csinva/mouse-brain-decoding

ai artificial-intelligence calcium-imaging computer-vision decoding gan machine-learning ml neural-network neuroscience python pytorch vision

Last synced: 4 days ago
JSON representation

Decoding images from calcium recordings using data from stringer et al. 2018.

• Host: GitHub
• URL: https://github.com/csinva/mouse-brain-decoding
• Owner: csinva
• Created: 2019-02-23T18:20:36.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2019-03-28T05:13:16.000Z (almost 6 years ago)
• Last Synced: 2024-12-15T18:13:59.085Z (about 2 months ago)
• Topics: ai, artificial-intelligence, calcium-imaging, computer-vision, decoding, gan, machine-learning, ml, neural-network, neuroscience, python, pytorch, vision
• Language: Jupyter Notebook
• Homepage:
• Size: 955 KB
• Stars: 2
• Watchers: 5
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

        
# decoding natural images

This project is trying to decode grayscale naturalistic images from calcium imaging recordings in mouse cortex. The data is from [stringer et al 2018](https://github.com/MouseLand/stringer-pachitariu-et-al-2018b) and includes simultaneous recordings from 10,000 neurons.

The idea is to use a pre-trained gan from [this repo](https://github.com/csinva/pytorch_gan_pretrained) based on the [dcgan](https://github.com/pytorch/examples/tree/master/dcgan) and learn a linear mapping from the neurons to the latent space of the GAN generator.

So far the results are not very good. Some samples are shown below. Starting from the left, every two columns show the real data samples and the generated data samples:

| training                                            | validation                                           |

| ------------------------------------------------------------ | ------------------------------------------------------ |

| ![train](results/9900_samples.png) | ![val](results/9900_samples_val.png) |

# code

- the `train.py` file contains the bulk of the code for training

- the `scripts` folder contains files for sweeping over parameters

- many things left to do

  - try different loss functions for generating images that are more perceptially similar (e.g. vgg perceptual loss)

  - different types of normalization

  - augment data (add rotates, etc.)

  - try learning w/ poisson distr.