{"id":16525636,"url":"https://github.com/camdavidsonpilon/eem_analysis","last_synced_at":"2026-04-13T02:51:06.854Z","repository":{"id":66073653,"uuid":"254258364","full_name":"CamDavidsonPilon/eem_analysis","owner":"CamDavidsonPilon","description":null,"archived":false,"fork":false,"pushed_at":"2020-05-05T13:00:52.000Z","size":5332,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-13T13:31:14.222Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CamDavidsonPilon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-09T03:06:55.000Z","updated_at":"2023-09-08T18:05:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"4acfb7c3-4533-4850-9a96-f7ad56a59aa1","html_url":"https://github.com/CamDavidsonPilon/eem_analysis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CamDavidsonPilon%2Feem_analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CamDavidsonPilon%2Feem_analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CamDavidsonPilon%2Feem_analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CamDavidsonPilon%2Feem_analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CamDavidsonPilon","download_url":"https://codeload.github.com/CamDavidsonPilon/eem_analysis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241597080,"owners_count":19988187,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T17:04:25.043Z","updated_at":"2026-04-13T02:51:01.823Z","avatar_url":"https://github.com/CamDavidsonPilon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Autoencoding EEMs\n\n\n### Analysis of EEMs\n\nEEMs (excitation emission matrices) are measurements of a sample's fluorescence intensity at varying excitation and emission wavelengths.\n\nTraditionally, EEMs have been analyzed using linear matrix decomposition methods like PARAFAC. To interpret the decomposition, PARAFAC relies on some strong _chemical_ assumptions (not just statistical), namely:\n\n1. There are no inner filter effects occurring\n2. No quenching is present\n3. Beer-Lambert law is satisfied\n4. No additional scattering is present\n\n\nIf we generalize to non-linear decomposition, and ignore any attempt at interpretation, we can expand the models used. Namely, we can try a convolutional autoencoder to project the 2D EEMs to a lower space, and perform analysis there. The convolutional autoencoder has a much more accurate compression than alternative methods like PARAFAC. (This also means that the decompression is more accurate, as seen in the image below.)\n\n![comparison](https://i.imgur.com/2t2CdT4l.png)\n\nPARAFAC does do a better job when scattering is reduced. If we apply a naive Rayleigh scattering filter to our EEMS:\n\n![comparision2](https://i.imgur.com/XGMIzNnl.png)\n\n\nIn the comparison above, the convolutional autoencoder, henceforth CNN-AE, squeezes the 28x28 data into 12 dimensions. From these 12 dimensions, further dimensionality reduction can be applied, like PCA. The following figure is a PCA-reduced dataset of four vegetables' EEMS:\n\n![pca](https://i.imgur.com/AwDAdrVl.png)\n\nWe can clearly see the clusters of vegetables are almost perfectly separated, hence their original EEMs have enough information to distinguish vegetables.\n\n\n### Existing CNN-AE network\n\nEncoder -\u003e Decoder.\n\n\n![network](https://i.imgur.com/FRYHunI.png)\n\n\n### Installation\n\n1. Clone/download the repo to a local directory.\n2. Optional: create a virtualenv for this.\n3. From the command line:\n```\npython setup.py install\n```\n\n### Configuration\n\n1. Currently the supported EEMs must be NxN (a square). One can use image / scientific software to resize EEMs to be square. Change the `INPUTS` variable in `src/utils.py`.\n2. Data, in the form of csv (with `.csv` extension), should be put into the folder `data/flat_files`.\n3. To added labeling information, you can user `-` delimiters in the filename and edit the `Labels` in `src/utils.py`.\n\n### Running on an example dataset\n\n```\npython src/keras_conv_ae_training.py \u0026\u0026 python src/keras_encoder_reconstruction.py\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcamdavidsonpilon%2Feem_analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcamdavidsonpilon%2Feem_analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcamdavidsonpilon%2Feem_analysis/lists"}