{"id":19643853,"url":"https://github.com/logicalclocks/aml_end_to_end","last_synced_at":"2025-04-28T13:31:18.512Z","repository":{"id":40557397,"uuid":"317789755","full_name":"logicalclocks/aml_end_to_end","owner":"logicalclocks","description":"AML End to End Example","archived":true,"fork":false,"pushed_at":"2022-06-21T16:02:39.000Z","size":149848,"stargazers_count":53,"open_issues_count":0,"forks_count":18,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-14T19:52:24.073Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/logicalclocks.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-02T07:57:49.000Z","updated_at":"2025-03-03T12:07:59.000Z","dependencies_parsed_at":"2022-07-27T09:22:23.730Z","dependency_job_id":null,"html_url":"https://github.com/logicalclocks/aml_end_to_end","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logicalclocks%2Faml_end_to_end","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logicalclocks%2Faml_end_to_end/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logicalclocks%2Faml_end_to_end/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logicalclocks%2Faml_end_to_end/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/logicalclocks","download_url":"https://codeload.github.com/logicalclocks/aml_end_to_end/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251319757,"owners_count":21570451,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T14:24:18.731Z","updated_at":"2025-04-28T13:31:13.471Z","avatar_url":"https://github.com/logicalclocks.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AML End to End Example\n\n## Project description\nThis project demonstrates end to end pipeline how to train binary anti money laundering (AML) classifier based on \nGenerative Adversarial Networks (GANs) and Graph embeddings. Proposed solution includes following sub sections:  \n\n* Data ingestion - We will use sample of transactions data generated by [AMLSim](https://github.com/IBM/AMLSim) \n* Feature store – We use Hopsworks Feature Store to compute features, organize them as feature groups and store for \ndownstream analysis, such as creating training datasets for model training, as well as retrieving them      \n* Graph Embeddings - We will use [StellarGraph](https://github.com/stellargraph/stellargraph) library to compute graph \nembeddings.\n* Anomaly detection model - We will use keras implementation of [adversarial anomaly detection](https://arxiv.org/pdf/1905.11034.pdf) that was adapted to tabular data.\n* Hyper parameter tuning - We will use [Maggy](https://github.com/logicalclocks/maggy) to conduct experiments for \nhyperparameter tuning.  \n* Model serving - We will use Hopsworks model server to predict anomalous transactions. \n\n## Demo dataset\nA sample of transaction data is provided in the folder ./demodata, including upload alert_transactions.csv, party.csv and transactions.csv. \n\n## Anomaly detection model\nKeras implementation of [adversarial anomaly detection](https://arxiv.org/pdf/1905.11034.pdf) is provided in the folder\n./adversarialaml. To use this library install as python library from https://github.com/logicalclocks/AMLend2end.git.\n![intall_python_lib_from_git.gif](./images/intall_python_lib_from_git.gif)\n\n## End to End pipeline\nTo successfully complete this tutorial clone this repository to your Hopsworks project ![git_clone.gif](./images/git_clone.gif)\n \n### Jupyter notebooks step by step   \nRun jupyter notebooks in the following order:\n1) 1_transaction_feature_engineering_ingestion.ipynb \n2) 2_prep_training_dataset_for_embeddings.ipynb\n3) 3_maggy_node_embeddings.ipynb\n4) 4_compute_node_embeddings.ipynb \n5) 5_predict_and_create_node_embeddings_fg.ipynb\n6) 6_create_anomaly_detection_td.ipynb\n7) 7_maggy_adversarial_aml.ipynb\n8) 8_train_adversarial_aml.ipynb\n9) 9_aml_model_server.ipynb\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flogicalclocks%2Faml_end_to_end","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flogicalclocks%2Faml_end_to_end","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flogicalclocks%2Faml_end_to_end/lists"}