{"id":18508856,"url":"https://github.com/hfawaz/ijcnn19attacks","last_synced_at":"2025-04-09T03:32:14.899Z","repository":{"id":41570907,"uuid":"174534892","full_name":"hfawaz/ijcnn19attacks","owner":"hfawaz","description":"Adversarial Attacks on Deep Neural Networks for Time Series Classification","archived":false,"fork":false,"pushed_at":"2020-07-02T16:26:43.000Z","size":5003,"stargazers_count":76,"open_issues_count":4,"forks_count":28,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-03-23T22:42:25.046Z","etag":null,"topics":["adversarial-attacks","adversarial-examples","deep-learning","deep-neural-networks","time-series-classification"],"latest_commit_sha":null,"homepage":"https://germain-forestier.info/src/ijcnn2019/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hfawaz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-08T12:34:32.000Z","updated_at":"2025-02-18T15:07:32.000Z","dependencies_parsed_at":"2022-09-04T18:22:31.417Z","dependency_job_id":null,"html_url":"https://github.com/hfawaz/ijcnn19attacks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hfawaz%2Fijcnn19attacks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hfawaz%2Fijcnn19attacks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hfawaz%2Fijcnn19attacks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hfawaz%2Fijcnn19attacks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hfawaz","download_url":"https://codeload.github.com/hfawaz/ijcnn19attacks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247973918,"owners_count":21026736,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-attacks","adversarial-examples","deep-learning","deep-neural-networks","time-series-classification"],"created_at":"2024-11-06T15:15:38.526Z","updated_at":"2025-04-09T03:32:13.958Z","avatar_url":"https://github.com/hfawaz.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Adversarial Attacks on Deep Neural Networks for Time Series Classification\nThis is the companion repository for our paper also available on [ArXiv](https://arxiv.org/abs/1903.07054) titled \"Adversarial Attacks on Deep Neural Networks for Time Series Classification\". \nThis paper has been accepted at the [IEEE International Joint Conference on Neural Networks (IJCNN) 2019](https://www.ijcnn.org/). \n\n## Approach \n![fgsm](https://github.com/hfawaz/ijcnn19attacks/blob/master/img/pert-example.png)\n\n## Data \nThe data used in this project comes from the [UCR archive](https://www.cs.ucr.edu/~eamonn/time_series_data/UCR_TS_Archive_2015.zip), which contains the 85 univariate time series datasets. \n\n## Pre-trained models\nYou can download the pre-trained ResNet models for each dataset in the archive [here](https://germain-forestier.info/src/ijcnn2019/pre-trained-resnet.zip). \nThese are the models used to generate the adversarial time series examples. \nThey are published for reproducibility, nevertheless the code can be applied to any model in the [h5py](http://docs.h5py.org/en/latest/build.html) format. \n\n## Prerequisites\nAll python packages needed are listed in [pip-requirements.txt](https://github.com/hfawaz/ijcnn19ensemble/blob/master/src/utils/pip-requirements.txt) file and can be installed simply using the pip command.\n\n* [numpy](http://www.numpy.org/)  \n* [pandas](https://pandas.pydata.org/)  \n* [sklearn](http://scikit-learn.org/stable/)  \n* [scipy](https://www.scipy.org/)  \n* [matplotlib](https://matplotlib.org/)  \n* [tensorflow-gpu](https://www.tensorflow.org/)  \n* [keras](https://keras.io/)  \n* [h5py](http://docs.h5py.org/en/latest/build.html)\n\nNote that for convience we copied some of the code from the [Cleverhans API](https://github.com/tensorflow/cleverhans) and modified it to adapt it for time series data instead of images. \n\n## Code\nTo perform the ```fgsm``` (or ```bim```) attack on the datasets in the UCR archive you can run the following command: \n```\npython3 main.py attack fgsm \n```\n\nOnce the perturbed time series are generated, you can launch this command to plot and visualize the difference: \n```\npython3 main.py draw fgsm \n```\n\nIf you want to visualize the noise as well you can run this command: \n```\npython3 main.py noise fgsm \n```\n\nFinally to plot the Multi-Dimensional Scaling (MDS) and visualize the difference between an original and perturbed dataset, you can issue this command: \n```\npython3 main.py mds fgsm \n```\n\n## Results \n\nThe following animation shows how the accuracy and the time series shape variation with respect to the amount of perturbation allowed. \n![fgsm](https://github.com/hfawaz/ijcnn19attacks/blob/master/img/acc-ts.gif)\n\nThe folllowing table shows the accuracy over the 85 datasets with and without adversarial perturbation, using both attacks FGSM and BIM for two models ResNet (white-box mode) and FCN (black-box mode).\nThe raw csv results can be found [here](https://github.com/hfawaz/ijcnn19attacks/blob/master/results/all-results.csv).\nFor example column 'resnet_ori' shows the original accuracy of ResNet over the 85 datasets, while column 'resnet_fgsm_adv' shows the accuracy after performing the FGSM attack.\n\n|                                | resnet_ori | resnet_fgsm_adv | resnet_bim_adv | fcn_ori | fcn_fgsm_adv | fcn_bim_adv | \n|--------------------------------|------------|-----------------|----------------|---------|--------------|-------------| \n| 50words                        | 73.2       | 17.1            | **8.8**            | 45.5    | **7.7**          | 8.8         | \n| Adiac                          | 83.1       | 3.1             | **1.5**            | 84.7    | 2.8          | **2.0**         | \n| ArrowHead                      | 85.1       | 33.1            | **14.3**           | 82.3    | 41.7         | **29.1**        | \n| Beef                           | 76.7       | 20.0            | **10.0**           | 70.0    | **26.7**         | 36.7        | \n| BeetleFly                      | 85.0       | **15.0**            | **15.0**           | 90.0    | **25.0**         | **25.0**        | \n| BirdChicken                    | 95.0       | 55.0            | **15.0**           | 100.0   | 60.0         | **45.0**        | \n| Car                            | 93.3       | 21.7            | **6.7**            | 90.0    | 21.7         | **11.7**        | \n| CBF                            | 98.9       | 86.1            | **84.8**           | 99.4    | 95.3         | **94.7**        | \n| ChlorineConcentration          | 83.5       | 12.3            | **11.8**           | 82.4    | **12.3**         | 12.5        | \n| CinC_ECG_torso                 | 83.8       | 25.4            | **23.3**           | 83.8    | 25.7         | **23.6**        | \n| Coffee                         | 100.0      | 50.0            | **35.7**           | 100.0   | 75.0         | **64.3**        | \n| Computers                      | 81.2       | 40.8            | **24.0**           | 81.6    | 58.4         | **30.8**        | \n| Cricket_X                      | 79.0       | 35.4            | **20.8**           | 79.5    | 43.8         | **34.1**        | \n| Cricket_Y                      | 80.5       | 24.9            | **13.8**           | 76.7    | 28.5         | **20.8**        | \n| Cricket_Z                      | 81.5       | 27.7            | **16.2**           | 80.3    | 35.4         | **26.2**        | \n| DiatomSizeReduction            | 30.1       | 46.7            | **34.6**           | 30.4    | **43.1**         | 57.8        | \n| DistalPhalanxOutlineAgeGroup   | 79.8       | **16.0**            | 17.0           | 82.8    | **16.8**         | 17.5        | \n| DistalPhalanxOutlineCorrect    | 82.0       | 35.2            | **20.7**           | 79.8    | 35.8         | **25.3**        | \n| DistalPhalanxTW                | 74.8       | **9.8**             | 12.5           | 75.8    | **11.2**         | 12.2        | \n| Earthquakes                    | 78.6       | 51.2            | **48.8**           | 78.3    | 68.9         | **69.6**        | \n| ECG200                         | 89.0       | 61.0            | **46.0**           | 89.0    | 74.0         | **66.0**        | \n| ECG5000                        | 93.5       | 76.1            | **36.4**           | 93.9    | 90.0         | **88.0**        | \n| ECGFiveDays                    | 96.2       | 30.2            | **3.9**            | 99.0    | 51.2         | **31.4**        | \n| ElectricDevices                | 73.5       | 48.6            | **31.2**           | 70.9    | 50.3         | **48.9**        | \n| FaceAll                        | 85.5       | 76.7            | **72.5**           | 95.7    | 90.2         | **89.6**        | \n| FaceFour                       | 95.5       | 71.6            | **43.2**           | 92.0    | 71.6         | **70.5**        | \n| FacesUCR                       | 95.3       | 79.4            | **76.1**           | 94.7    | **86.4**         | 85.9        | \n| FISH                           | 97.7       | 12.6            | **4.0**            | 96.0    | 12.6         | **9.7**         | \n| FordA                          | 91.8       | 33.9            | **21.6**           | 90.1    | 59.6         | **57.3**        | \n| FordB                          | 91.1       | 27.8            | **14.3**           | 88.2    | 70.0         | **67.7**        | \n| Gun_Point                      | 99.3       | 31.3            | **6.7**            | 100.0   | 62.0         | **16.0**        | \n| Ham                            | 80.0       | 21.0            | **20.0**           | 71.4    | **27.6**         | **27.6**        | \n| HandOutlines                   | 86.0       | **36.2**            | **36.2**           | 74.6    | **36.2**         | **36.2**        | \n| Haptics                        | 51.6       | 19.2            | **14.6**           | 48.7    | 18.8         | **17.9**        | \n| Herring                        | 64.1       | 43.8            | **35.9**           | 65.6    | 59.4         | **57.8**        | \n| InlineSkate                    | 37.8       | 14.9            | **12.5**           | 32.4    | **9.6**          | 11.1        | \n| InsectWingbeatSound            | 50.6       | 17.7            | **15.7**           | 39.3    | **11.5**         | 12.1        | \n| ItalyPowerDemand               | 95.9       | 92.5            | **91.6**           | 96.1    | 89.8         | **89.6**        | \n| LargeKitchenAppliances         | 90.4       | 74.7            | **65.3**           | 89.6    | 66.4         | **63.5**        | \n| Lighting2                      | 77.0       | **42.6**            | **42.6**           | 73.8    | 41.0         | **39.3**        | \n| Lighting7                      | 78.1       | 50.7            | **35.6**           | 80.8    | 57.5         | **54.8**        | \n| MALLAT                         | 96.6       | 33.0            | **4.6**            | 97.0    | 32.6         | **24.2**        | \n| Meat                           | 98.3       | 35.0            | **1.7**            | 81.7    | **1.7**          | 31.7        | \n| MedicalImages                  | 76.2       | 52.1            | **28.7**           | 77.9    | 60.9         | **57.6**        | \n| MiddlePhalanxOutlineAgeGroup   | 74.2       | 58.0            | **12.8**           | 72.8    | 62.0         | **54.0**        | \n| MiddlePhalanxOutlineCorrect    | 80.5       | 29.8            | **19.5**           | 80.7    | 25.8         | **20.2**        | \n| MiddlePhalanxTW                | 60.9       | **13.3**            | 14.5           | 58.4    | **21.1**         | 24.3        | \n| MoteStrain                     | 92.4       | 74.3            | **68.8**           | 93.4    | 80.5         | **77.4**        | \n| NonInvasiveFatalECG_Thorax1    | 94.6       | 5.5             | **2.4**            | 95.6    | 7.4          | **5.1**         | \n| NonInvasiveFatalECG_Thorax2    | 94.4       | 5.2             | **1.2**            | 95.6    | 4.4          | **1.6**         | \n| OliveOil                       | 86.7       | 20.0            | **3.3**            | 86.7    | **13.3**         | **13.3**        | \n| OSULeaf                        | 97.9       | 15.7            | **0.0**            | 98.3    | 17.4         | **6.6**         | \n| PhalangesOutlinesCorrect       | 85.7       | 36.8            | **16.2**           | 81.5    | 35.9         | **24.9**        | \n| Phoneme                        | 33.3       | 15.0            | **10.3**           | 32.1    | 21.0         | **15.5**        | \n| Plane                          | 100.0      | 81.0            | **56.2**           | 100.0   | 58.1         | **56.2**        | \n| ProximalPhalanxOutlineAgeGroup | 83.9       | 46.3            | **8.3**            | 81.5    | 46.8         | **9.8**         | \n| ProximalPhalanxOutlineCorrect  | 91.4       | 32.0            | **10.7**           | 91.1    | 35.7         | **20.6**        | \n| ProximalPhalanxTW              | 77.8       | **10.2**            | 11.8           | 81.0    | 15.0         | **13.0**        | \n| RefrigerationDevices           | 51.7       | 32.0            | **30.1**           | 50.7    | **38.4**         | 40.0        | \n| ScreenType                     | 60.8       | 31.7            | **25.9**           | 60.8    | 36.5         | **28.0**        | \n| ShapeletSim                    | 100.0      | 53.9            | **36.1**           | 75.6    | 60.0         | **58.3**        | \n| ShapesAll                      | 91.7       | 5.2             | **1.0**            | 89.5    | 6.7          | **6.3**         | \n| SmallKitchenAppliances         | 78.9       | 40.5            | **21.9**           | 78.7    | 47.5         | **28.8**        | \n| SonyAIBORobotSurface           | 96.8       | 83.9            | **82.2**           | 96.0    | 85.0         | **84.2**        | \n| SonyAIBORobotSurfaceII         | 98.6       | 89.2            | **88.7**           | 98.1    | **91.5**         | 91.6        | \n| StarLightCurves                | 97.2       | 58.8            | **57.7**           | 96.6    | 73.0         | **60.1**        | \n| Strawberry                     | 96.2       | 21.9            | **3.8**            | 95.8    | 14.4         | **13.7**        | \n| SwedishLeaf                    | 95.4       | 31.2            | **16.0**           | 97.3    | 34.6         | **30.4**        | \n| Symbols                        | 92.7       | 36.6            | **12.9**           | 94.3    | 58.4         | **28.3**        | \n| synthetic_control              | 100.0      | 94.3            | **94.0**           | 98.3    | **94.7**         | 95.3        | \n| ToeSegmentation1               | 96.9       | 62.3            | **46.9**           | 96.1    | 63.2         | **57.5**        | \n| ToeSegmentation2               | 91.5       | 63.8            | **53.8**           | 90.8    | 54.6         | **52.3**        | \n| Trace                          | 100.0      | 58.0            | **52.0**           | 100.0   | **47.0**         | 52.0        | \n| TwoLeadECG                     | 100.0      | 5.3             | **0.4**            | 100.0   | 13.0         | **5.2**         | \n| Two_Patterns                   | 100.0      | 98.2            | **96.7**           | 86.8    | 82.9         | **82.6**        | \n| UWaveGestureLibraryAll         | 86.2       | 21.8            | **7.1**            | 81.7    | 25.3         | **22.3**        | \n| uWaveGestureLibrary_X          | 78.0       | 32.1            | **11.1**           | 75.7    | 32.7         | **27.2**        | \n| uWaveGestureLibrary_Y          | 66.7       | 27.7            | **14.9**           | 63.9    | 29.6         | **22.4**        | \n| uWaveGestureLibrary_Z          | 75.0       | 37.0            | **14.0**           | 72.0    | 27.1         | **21.0**        | \n| wafer                          | 99.8       | 86.6            | **7.3**            | 99.7    | **64.3**         | 81.2        | \n| Wine                           | 61.1       | **38.9**            | **38.9**           | 55.6    | **38.9**         | **38.9**        | \n| WordsSynonyms                  | 62.5       | 15.7            | **13.5**           | 55.0    | **9.7**          | 12.7        | \n| Worms                          | 64.6       | 27.6            | **19.9**           | 66.9    | 27.1         | **23.8**        | \n| WormsTwoClass                  | 74.6       | 45.3            | **31.5**           | 74.6    | 55.8         | **44.8**        | \n| yoga                           | 87.2       | 45.4            | **12.8**           | 84.1    | 44.9         | **19.2**        | \n\n\n## Reference\n\nIf you re-use this work, please cite:\n\n```\n@InProceedings{IsmailFawaz2019adversarial,\n  Title                    = {Adversarial Attacks on Deep Neural Networks for Time Series Classification},\n  Author                   = {Ismail Fawaz, Hassan and Forestier, Germain and Weber, Jonathan and Idoumghar, Lhassane and Muller, Pierre-Alain},\n  booktitle                = {IEEE International Joint Conference on Neural Networks},\n  Year                     = {2019}\n}\n```\n\n## Acknowledgement\n\nWe would like to thank NVIDIA Corporation for the Quadro P6000 grant and the Mésocentre of Strasbourg for providing access to the cluster.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhfawaz%2Fijcnn19attacks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhfawaz%2Fijcnn19attacks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhfawaz%2Fijcnn19attacks/lists"}