https://github.com/marcinz20/anomaly-detection-in-credo-dataset
University project, which goal is to build a system, that detects anomalies in CREDO dataset
https://github.com/marcinz20/anomaly-detection-in-credo-dataset
credo data-analysis data-science encoder-decoder-model jupiter-notebook pca-analysis python3
Last synced: 3 months ago
JSON representation
University project, which goal is to build a system, that detects anomalies in CREDO dataset
- Host: GitHub
- URL: https://github.com/marcinz20/anomaly-detection-in-credo-dataset
- Owner: MarcinZ20
- License: mit
- Created: 2023-10-20T12:33:54.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-05-07T09:19:13.000Z (about 2 years ago)
- Last Synced: 2025-05-24T06:36:45.522Z (about 1 year ago)
- Topics: credo, data-analysis, data-science, encoder-decoder-model, jupiter-notebook, pca-analysis, python3
- Language: Jupyter Notebook
- Homepage:
- Size: 985 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Anomaly Detection in CREDO Dataset
The goal of the project is to perform anomaly detection on images using different Machine Learning techniques.
## Table of contents
- [Anomaly Detection in CREDO Dataset](#anomaly-detection-in-credo-dataset)
* [Description](#description)
* [Tech-stack](#tech-stack)
* [Project structure](#project-structure)
* [Run Locally](#run-locally)
* [Authors](#authors)
* [References](#references)
* [License](#license)
* [Acknowledgements](#acknowledgements)
## Description
This project focuses on analyzing anomalies in the CREADO dataset. It consists of images registered by CMOS sensors scattered around the world, capturing cosmic radiation particles. The primary objective is to identify and understand unusual patterns, detect outliers within the dataset and test different approaches.
To achieve this, we employed Python and implemented various anomaly detection techniques, including:
- Principal Component Analysis (PCA)
- Autoencoders
- 2D PCA
- Morphological Methods
The CREADO Anomaly Analysis Project provides a comprehensive exploration of anomaly detection in the context of cosmic radiation imagery. By leveraging PCA, Autoencoders, and 2D PCA, we aim to contribute valuable insights into identifying and understanding anomalies within the CREADO dataset.
## Tech-stack
## Project structure
├── LICENSE
├── Makefile <- Makefile with commands like `make data`
├── README.md <- The top-level README for developers using this project.
├── data
│ ├── processed <- The final, canonical data sets for modeling.
│ └── raw <- The original, immutable data dump.
│
├── docs <- A default Sphinx project; see sphinx-doc.org for details
│
├── models <- Trained and serialized models, model predictions, or model summaries
│
├── notebooks <- Jupyter notebooks. Naming convention is a number (for ordering),
│ the creator's initials, and a short `-` delimited description, e.g.
│ `1.0-jqp-initial-data-exploration`.
│
├── plots <- Plots extracted from notebooks
│
├── references <- Data dictionaries, manuals, and all other explanatory materials.
│
├── reports <- Generated analysis as HTML, PDF, LaTeX, etc.
│ └── figures <- Generated graphics and figures to be used in reporting
│
├── requirements.txt <- The requirements file for reproducing the analysis environment, e.g.
│ generated with `pip freeze > requirements.txt`
│
├── setup.py <- makes project pip installable (pip install -e .) so src can be imported
├── src <- Source code for use in this project.
│ ├── __init__.py <- Makes src a Python module
│ │
│ ├── data <- Scripts to download or generate data
│ │ └── make_dataset.py
│ │
│ ├── features <- Scripts to turn raw data into features for modeling
│ │ └── build_features.py
│ │
│ ├── models <- Scripts to train models and then use trained models to make
│ │ │ predictions
│ │
│ └── visualization <- Scripts to create exploratory and results oriented visualizations
│ └── visualize.py
│
└── tox.ini <- tox file with settings for running tox; see tox.readthedocs.io
## Run Locally
Clone the project
```bash
git clone https://github.com/MarcinZ20/Anomaly-Detection-in-CREDO-Dataset.git
```
Go to the project directory
```sh
cd Anomaly-Detection-in-CREDO-Dataset
```
Create environment
```sh
make create_environment
```
Install dependencies
```sh
make requirements
```
Verify installed environment
```sh
make test-environment
```
Create Dataset
```sh
make data
```
Images from `data/raw` should now be processed and loaded into `data/processed` directory
## Authors
- [@Marcin](https://www.github.com/MarcinZ20)
- [@Jan](https://www.github.com/tycjantyc)
## References
- PCA implementation algorithm used in the study: [PCA Implementation](https://github.com/Parveshdhull/FaceRecognitionUsing-PCA-2D-PCA-And-2D-Square-PCA)
- Cookiecutter Data Science project template: [Cookiecutter](https://www.cookiecutter.io)
## License
[](https://choosealicense.com/licenses/mit/)
## Acknowledgements
- [Awesome Readme Templates](https://awesomeopensource.com/project/elangosundar/awesome-README-templates)