{"id":20019302,"url":"https://github.com/jbris/data_mining_group_assignment","last_synced_at":"2025-03-02T03:26:48.460Z","repository":{"id":39728727,"uuid":"256433204","full_name":"JBris/data_mining_group_assignment","owner":"JBris","description":"Repo for data mining group assignment.","archived":false,"fork":false,"pushed_at":"2022-12-08T09:35:31.000Z","size":69,"stargazers_count":1,"open_issues_count":9,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-12T16:08:14.656Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JBris.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-17T07:29:12.000Z","updated_at":"2023-03-22T04:53:49.000Z","dependencies_parsed_at":"2023-01-25T06:00:37.340Z","dependency_job_id":null,"html_url":"https://github.com/JBris/data_mining_group_assignment","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JBris%2Fdata_mining_group_assignment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JBris%2Fdata_mining_group_assignment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JBris%2Fdata_mining_group_assignment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JBris%2Fdata_mining_group_assignment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JBris","download_url":"https://codeload.github.com/JBris/data_mining_group_assignment/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241453364,"owners_count":19965187,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T08:26:52.797Z","updated_at":"2025-03-02T03:26:48.437Z","avatar_url":"https://github.com/JBris.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# data_mining_group_assignment\n\n\n## Table of Contents  \n\n* [Introduction](#introduction) \n* [Data](#data)\n* [DataScience](#datascience)\n* [InfluxDB](#influxdb) \n* [PostgreSQL](#postgresql)  \n\n## Introduction\n\nThis repo contains public resources for the Data Mining group assignment.\n\n**As the provided data for this assignment is confidential, all data sets, images, and models will be inaccessible from this repo.** \n\nInfluxDB and Grafana are included in the Docker stack for database storage and visualization purposes. \n\nPostgres and Adminer are also included for those who are unfamiliar with Influx.\n\nIf you're using Docker, execute [build.sh](build.sh) to get started.\n\n## Data \n\nThe provided data set should be placed in the [data directory.](scripts/user/data), and renamed to *initial_dataset.csv*. \n\nThe [initial_processing script](scripts/user/data_processing/initial_processing.py) performs some initial processing on the data. It renames the columns, drops unneeded columns, converts channels and sites to factors, and factorizes the channel ID-key pairs.\n\n## DataScience \n\nThe datascience container offers both R and Python packages. A list of R and Python dependencies can be found in the [Dockerfile](Dockerfile). See [requirements.txt](scripts/user/requirements.txt) for the list of Python packages and their respective versions. See [packages.txt](scripts/user/packages.txt) for the list of R packages and their respective versions.\n\nAs the [docker-compose.yml](docker-compose.yml) file shows, this repo extends from the [rocker/tidyverse image](https://hub.docker.com/r/rocker/tidyverse) which already includes the tidyverse collection and RStudio server.\n\nIf you opt to use Docker, you can view the [Makefile](Makefile) for relevant Docker commands. The `make run` command will allow users to execute shell commands within the datascience container. The `make enter` command will allow users to directly enter the container by starting a new shell.\n\n## InfluxDB\n\nInfluxDB is a time series database. For those who are unfamiliar, more information can be found at [influxdata.com](https://www.influxdata.com/). InfluxDB can be combined with [Grafana](https://grafana.com/) to analyze and visualize the data. View the [.env.example file](.env.example) to configure your InfluxDB \u0026 Grafana versions and ports.\n\nCSV files can be easily imported to your InfluxDB instance using the [csv-to-influxdb](https://github.com/fabio-miranda/csv-to-influxdb) package.\n\n## PostgreSQL\n\nPostgres can be used instead of Influx if required. \n\nDatabase dumps can be imported and exported using `make dbimp` and `make dbexp` respectively. Dumps can be found in the `data` directory.\n\n`make csvimp` will import CSV files into the database. Ensure that the CSV data is placed in the [data directory](data). Supply the *f* argument to specify the file to use. E.g. `make csvimp f=processed_dataset`.\n \nThe following settings can be configured in your .env file:\n\n| Name          | Default Value |  \n| ------------- |:-------------:|  \n| DB_NAME       | project       | \n| DB_USER       | user          |    \n| DB_PASSWORD   | pass          |    \n| DB_ROOT_PASSWORD | password   |    \n| DB_HOST       | postgres      |    \n| DB_PORT       | 5432          |    \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjbris%2Fdata_mining_group_assignment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjbris%2Fdata_mining_group_assignment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjbris%2Fdata_mining_group_assignment/lists"}