{"id":24633321,"url":"https://github.com/pnb/dlwed17","last_synced_at":"2025-10-15T04:39:37.018Z","repository":{"id":136045361,"uuid":"95573601","full_name":"pnb/dlwed17","owner":"pnb","description":"Unsupervised Deep Autoencoders for Feature Extraction with Educational Data","archived":false,"fork":false,"pushed_at":"2017-07-05T15:51:40.000Z","size":4571,"stargazers_count":14,"open_issues_count":1,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-04T10:07:05.497Z","etag":null,"topics":["edm","educational-data-mining"],"latest_commit_sha":null,"homepage":"http://pnigel.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pnb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-06-27T15:32:08.000Z","updated_at":"2024-08-31T13:47:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"872d05c5-4497-4c79-9395-dc4ee53036e2","html_url":"https://github.com/pnb/dlwed17","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pnb/dlwed17","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pnb%2Fdlwed17","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pnb%2Fdlwed17/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pnb%2Fdlwed17/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pnb%2Fdlwed17/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pnb","download_url":"https://codeload.github.com/pnb/dlwed17/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pnb%2Fdlwed17/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279049858,"owners_count":26093368,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-15T02:00:07.814Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["edm","educational-data-mining"],"created_at":"2025-01-25T08:26:32.945Z","updated_at":"2025-10-15T04:39:36.993Z","avatar_url":"https://github.com/pnb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Unsupervised Deep Autoencoders for Feature Extraction with Educational Data\nThis repository contains the code for the paper (see `bosch-dlwed17-camera.pdf`) presented at the\n_Deep Learning with Educational Data_ workshop at the\n[2017 Educational Data Mining](http://educationaldatamining.org/EDM2017/) conference.\n\n#### Citation\nBosch, N., \u0026 Paquette, L. (2017). Unsupervised deep autoencoders for feature extraction with educational data. In Deep Learning with Educational Data Workshop at the 10th International Conference on Educational Data Mining.\n\n## Requirements\nThe code was tested with Keras 2.0.3 and Tensorflow 1.1.0 neural network libraries.\n\nData were from [Betty's Brain](http://www.teachableagents.org/research/bettysbrain.php). These data\nare required for the code to run, and are not publicly available. However, the code could be\n(relatively) easily adapted to another dataset.\n\n## Model-building steps\nModel building generally consists of data preprocessing, autoencoder feature extraction, and\nsupervised learning phases.\n\n### Data preprocessing\n1. `preprocess_bromp.py` - takes raw BROMP files created by the HART application and combines them\ninto an easily-used format\n2. `preprocess_timeseries.py` - creates timeseries (evenly spaced in time) data from Betty's Brain\ninteraction logs\n3. `preprocess_seq.py` - creates sequences suitable for training RNN models from the timeseries\ndata; sequences are saved to numpy binary files for faster loading later\n\n### Autoencoder feature extraction\n1. `ae_lstm.py` - this and similar files (e.g., `vae_lstm.py`) trains the autoencoders\n2. `extracy_embeddings.py` - takes a trained model, feeds in data sequences, and saves the\nembeddings generated by the model to be used as features for supervised models\n3. `align_embeddings+labels.py` - matches up BROMP affect/behavior labels to the embeddings\nextracted from a model, saving only the rows with labels to create a file with features and labels\nwhich can be used for supervised learning\n\n### Supervised learning\n1. `supervised/ae_feats_test.py` - trains a decision tree (CART) model with the autoencoder features\n2. `supervised/expert_feats_extract.py` - extracts some simple features with the traditional method\n(manual design by experts) of feature extraction for model building\n3. `supervised/expert_feats_test.py` - builds a model using the expert features to serve as a\nbaseline\n\n## Visualization\n`visualize_activations.py` generates images of model activations by feeding in a random subset of\nsamples to a trained autoencoder and creating histograms of the activations of every layer in the\nnetwork. For layers with several neurons (\u003e 15), a subset of neurons is sampled to create a more\ntractable image.\n\nThe model structure is also visualized (requires the `pydot` package).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpnb%2Fdlwed17","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpnb%2Fdlwed17","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpnb%2Fdlwed17/lists"}