{"id":19072257,"url":"https://github.com/giordano-lucas/dex-cyclic-arbitrage","last_synced_at":"2026-02-04T02:40:51.188Z","repository":{"id":171696198,"uuid":"438590939","full_name":"giordano-lucas/dex-cyclic-arbitrage","owner":"giordano-lucas","description":"DEX Cyclic Arbitrage Analysis","archived":false,"fork":false,"pushed_at":"2022-01-14T15:12:53.000Z","size":109901,"stargazers_count":14,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-22T04:17:32.444Z","etag":null,"topics":["arbitrage","cycles","dex","uniswap"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/giordano-lucas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-15T10:40:08.000Z","updated_at":"2025-01-28T20:28:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"69445b65-8a46-4fed-90e1-93bf4634c746","html_url":"https://github.com/giordano-lucas/dex-cyclic-arbitrage","commit_stats":null,"previous_names":["giordano-lucas/dex-cyclic-arbitrage"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/giordano-lucas/dex-cyclic-arbitrage","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giordano-lucas%2Fdex-cyclic-arbitrage","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giordano-lucas%2Fdex-cyclic-arbitrage/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giordano-lucas%2Fdex-cyclic-arbitrage/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giordano-lucas%2Fdex-cyclic-arbitrage/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/giordano-lucas","download_url":"https://codeload.github.com/giordano-lucas/dex-cyclic-arbitrage/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giordano-lucas%2Fdex-cyclic-arbitrage/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265224177,"owners_count":23730348,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arbitrage","cycles","dex","uniswap"],"created_at":"2024-11-09T01:36:16.383Z","updated_at":"2026-02-04T02:40:51.147Z","avatar_url":"https://github.com/giordano-lucas.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg width=\"1665\" alt=\"Screenshot 2021-12-29 at 09 52 48\" src=\"https://user-images.githubusercontent.com/43466781/147819628-ed23274e-8d05-487d-b499-2b7a51e36eee.png\"\u003e\n\n:point_right: Read our **data story** online [using the following link](https://giordano-lucas.github.io/dex-cyclic-arbitrage/) :rocket: \n\n# Abstract\n\nThe paper Cyclic Arbitrage in Decentralized Exchange Markets[[1]](https://arxiv.org/abs/2105.02784) showed a large number of occurrences of cyclic arbitrages in decentralised exchanges compared to centralised ones. In their work, they mainly focus on analysing these cycles in terms of length, token distribution, daily patens and profitability.\n\nHowever, the factors driving their appearance have not been studied yet.  To this end, we propose to extend the work of [[1]](https://arxiv.org/abs/2105.02784) on Uniswap data. Moreover, we also plan to study the predictive power of these factors in a binary classification setting. It will allow determining whether or not a cycle can actually be implemented and generate a positive return, which has an inherent market value.\n\n# Goal\n\nThe goal of this project is to study exploited cyclic arbitrage in decentralised exchanges. We already have access to the [Cyclic transaction dataset](https://disco.ethz.ch/misc/uniswap/cycles_in_Uniswap.json) which contains cyclic arbitrages that were exploited. We intend to extract features out of events (trade rates, trade volumes, liquidity) preceding the arbitrages. \n\nThese features could potentially be high dimensional (depending on the length of the time series) and we will need to use dimensionality reduction techinques to create an embedding to build a relevant set of features of our future machine learning models.\n\nThen, we will cluster the arbitrages based on the computed features. Ideally, we would like to observe meaningful clusterings: profitable cycles get clustered together, cycles having similar duration (how long it is profitable) also end up in the same cluster, etc. Once meaningful clusters are obtained, it gets interesting to use the same features in a prediction model having profitability of the arbitrage as a target.\n\n# Methods\n\n1. **Data preprocessing**: \n\n    1. Keep only cycles of length 3.\n    2. Filter out illiquid tokens.\n    3. Log-transformation for heavy-tailed features\n    4. Token-based standard scaling\n    5. Zero padding for length standardisation.\n\n2. **Cycles embedding**:\n\n    1. After preprocessing an autencoder is built.\n    2. Multiple architectures are tested (linear, multilayer densly connected, convolutional layer). \n    3. Their performance is compared to a classical PCA approach. \n    4. In part 4. Profitablity prediction, the performance of the different embeddings techniques is evaluated on the accuracy of the task.\n\n3. **Cycles clustering**:\n\n    1. Use the embedding, a KMeans clustering is constructed. \n    2. Clusters in the training set are analysed \n    3. Based on the test set results, we can understand whether or not there is predictibility in the results obtained in point 2.\n\n4. **Cycle profitablity prediction**:\n\n    1. Study profitability prediction for arbitrage cycles.\n    2. Multiples models are tested (logistic regression, SVM).\n    3. The impact of adding token encoding to the models is tested.\n    4. The performance of the different embeddings is evaluated.\n\n# Notes to the reader\n\nEach folder contains a decidaced `README` where extra instruction and details are given.\n\n## Organisation of the repository\n\n    .\n    ├── data                                      # Data folder\n    │ ├── uniswap_raw_data                        # data fetched from bitquery\n    │ │  ├── uniswap_raw_data_0_1000.json.gz      # example of file\n    │ │  ├── ...\n    │ ├── liquid                                  # directory containing datasets corresponding to liquid cycles\n    │ │ ├── uniswap_data_liquid.csv               # csv version of the dataset fetched from biquery (filter out illiquid cycles)\n    │ │ ├── additional_features_train_liquid.csv  # file used by the clustering and prediction task with extra features (train)\n    │ │ ├── additional_features_test_liquid.csv   # file used by the clustering and prediction task with extra features (test)\n    │ │ ├── ML_features                           # directory for ML features\n    │ │ │ ├── ...\n    │ │ ├── pca                                   # directory containing the encoded features from the PCA model\n    │ │ │ ├── ...\n    │ │ ├── rule_based                            # directory containing the encoded features from the Rule-Based model\n    │ │ │ ├── ...\n    │ ├── full                                    # similar architecture as for the liquid folder but for the full dataset\n    │ │ ├── ...\n    │ ├── cycles_in_Uniswap.json                  # dataset from the paper\n    │ ├── filtered_cycles.json                    # only cycles of length 3 \n    ├── data_acquisition                    # Scripts to fetch the datasets (from bitquery and from the paper)\n    ├── data_exploration                    # Contains visualisations of the datasets\n    ├── data_processing                     # All scripts to process the raw data into usables features for ML\n    ├── models                              # all ML related tasks\n    │ ├── clustering                        # files related to the clustering task\n    │ ├── embedding                         # files related to the embedding task\n    │ ├── prediction                        # files related to the profitablity prediction task\n    ├── figures                             # Contains the ouput images and html used for the data story\n    ├── requirements.txt                    # Dependencies file\n    └── README.md               \n    \n## How to run the code \n\n1. Follow the steps in [Data Acquisition](data_acquisition/README.md) to download the raw datasets \n2. Follow the steps in [Data Processing](data_processing/README.md) to generate the preprocessed data\n3. **Data exploration**: run the `data_exploration/data_exploration.ipynb` notebook to see the data exploration steps taken.\n4. **Embeddings**: open the `models/embedding` folder:\n    1. Autoencoder: follow the steps in [Train Autoencoders](models/embedding/autoencoder/README.md) to understand how to train and use the available autoencoders\n    2. PCA: run the `pca_embedding.ipynb` notebook to create the `PCA` embedding.\n    3. Rule-based: follow the steps in [Build Rule-based features](models/embedding/rule_based/README.md) to generate preprocessed data usefull for performance comparision.\n5. **Clustering**: run the `models/clustering/Kmeans.ipynb` notebook to see the code related to the clustering.\n6. **Profitablity prediction**: run the `models/prediction/prediction.ipynb` notebook for the profitablity prediction task.\n\n## Dependencies requirement\n\nIn the repository, we provide a `requirement.txt` file from which you can create a virtual python environment.\n\n## Side note on the Scitas Cluster\n\nIf you want to run our code in the scitas cluster, you will need several additional steps for the  set-up:\n\n1. Create a compatible Jupyter/Tensorflow environment using the [following official tutotrial](https://scitas-data.epfl.ch/confluence/display/DOC/How+to+use+Jupyter+and+Tensorflow+on+Izar)\n2. To be able to import `talos` on the Scitas cluster, we need to update line 8 of `opt/venv-gcc/lib/python3.7/site-packages/kerasplotlib/traininglog.py` from `from keras.callbacks import Callback` to `from tensorflow.keras.callbacks import Callback`\n\n# Timeline and contributions :\n\n## Week 1 : Data Acquisition \u0026 Setup (1)\n\n| Task                                | Team member(s)                  | work hours  |\n| :-----------------------------------|:--------------------------------| -----------:|\n| Literature research                 | Lucas \u0026 Augustin                | 3h          |\n| API choice and query design         | Lucas \u0026 Augustin                | 4h          |\n| EPFL Cluster \u0026 environment setup    | Lucas                           | 2h          |\n| Data fetching script test           | Augustin                        | 3h          |\n| Data fetching validation            | Augustin                        | 2h          |\n| Data fetching improvements          | Augustin                        | 2h          |\n\n\n## Week 2 : Data preprocessing\n\n| Task                                     | Team member(s)                  | work hours  |\n| :----------------------------------------|:--------------------------------| -----------:|\n| Data cleaning                            | Augustin                        | 5h          |\n| Data exploration paper dataset           | Augustin                        | 2h          |\n| Data exploration                         | Lucas                           | 3h          |\n| Raw data =\u003e embedding format             | Lucas                           | 3h          |\n\n\n## Week 3 : Embedding \u0026 Clustering \n\n| Task                                     | Team member(s)                  | work hours  |\n| :----------------------------------------|:--------------------------------| -----------:|\n| Autencoder keras basic code              | Lucas                           | 3h          |\n| Comparision with PCA and debugging       | Lucas                           | 1h          |\n| K-means                                  | Augustin                        | 2h          |\n\n## Week 4 : Clustering analysis, Profitablity prediction \u0026 report writing\n\n| Task                                    | Team member(s)                  | work hours  |\n| :---------------------------------------|:--------------------------------| -----------:|\n| Clustering analysis                     | Lucas                           | 4h          |\n| Profitablity prediction setup           | Augustin                        | 2h          |\n| Github pages setup                      | Lucas                           | 2h          |\n| Data story (1)                          | Lucas                           | 5h          |\n| Data story (2)                          | Augustin                        | 2h          |\n\n### Week 5 : Improvements in data processing \u0026 report writing\n\n| Task                                    | Team member(s)                  | work hours  |\n| :---------------------------------------|:--------------------------------| -----------:|\n| Token based scaling                     | Lucas \u0026 Augustin                |    5h        |\n| Token one hot encoding                  | Lucas                           |    1h        |\n| Token encoding in profitablity prediction   | Augustin                    |    1h        |\n| Deep NN for   profitablity prediction   | Augustin                        |    1h        |\n| Better data processing                  | Augustin                        |    2h        |\n| Improved data exploration               | Lucas                           |    3h        |\n| Better understanding of PCA output      | Augustin                        |    1h        |\n| Autencoder testing                      | Augustin                        |    2h        |\n| Data story (3)                          | Lucas                           |    1h        |\n| Add ruled based indicators for autoencoder performance comparision | Lucas |    2h       |        \n\n### Week 6 : Hyperparameter opmisation, improvements \u0026 report writing\n\n| Task                                    | Team member(s)                  | work hours   |\n| :---------------------------------------|:--------------------------------| ------------:|\n| Filter illiquid data \u0026 debug            | Lucas                           |    3h        |\n| Update architecture for liquid data     | Augustin                        |    3h        |\n| Research on attention learning          | Lucas                           |    2h        |\n| Data processing simpler pipeline        | Augustin                        |    2h        |\n| Autencoder improvement and debug        | Augustin                        |    3h        |\n| Autencoder manual tests for several architectures        | Augustin       |    8h        |\n| Testing optimizers                      | Augustin                        |    2h        |\n| Talos setup                             | Lucas                           |    2h        |\n| Running Talos                           | Augustin                        |    1h        |\n| Hyperparameter opmisation               | Lucas \u0026 Augustin                |    4h        |\n| Reporting losses                        | Augustin                        |    1h        |\n| Kmeans : better silouhette analysis     | Lucas                           |    3h        |\n| Kmeans : update results for liquid data | Lucas                           |    4h        |\n| PCA embedding                           | Lucas                           |    1h        |\n| Ruled based data : pandas-ta implementation    | Lucas                    |    1h        |\n| Ruled based data : pandas implementation       | Lucas                    |    3h        |\n| Ruled based data : code optimisation           | Lucas                    |    3h        |\n| Ruled based data : performance comparision with AE  | Lucas               |    1h        |\n| Repository cleaning                     | Lucas \u0026 Augustin                |    1h        |\n| Notebook comments and markdown          | Lucas \u0026 Augustin                |    4h        |\n| Data story (4)                          | Lucas \u0026 Augustin                |    6h        |\n\n## Total contribution:\n\n| Team member                     | work hours   |\n|:--------------------------------| ------------:|\n| Lucas Giordano                  |    81h       |\n| Augustin Kapps                  |    60h       |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgiordano-lucas%2Fdex-cyclic-arbitrage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgiordano-lucas%2Fdex-cyclic-arbitrage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgiordano-lucas%2Fdex-cyclic-arbitrage/lists"}