{"id":15637906,"url":"https://github.com/philipperemy/lead-lag","last_synced_at":"2025-05-07T16:07:57.629Z","repository":{"id":43807124,"uuid":"143992785","full_name":"philipperemy/lead-lag","owner":"philipperemy","description":"Estimation of the lead-lag parameter from non-synchronous data.","archived":false,"fork":false,"pushed_at":"2024-05-07T23:00:50.000Z","size":13733,"stargazers_count":116,"open_issues_count":4,"forks_count":53,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-01-03T08:11:50.515Z","etag":null,"topics":["lead-lag","mathematics","statistics"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/philipperemy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["philipperemy"]}},"created_at":"2018-08-08T09:39:48.000Z","updated_at":"2024-12-26T19:12:01.000Z","dependencies_parsed_at":"2024-10-23T04:25:17.517Z","dependency_job_id":null,"html_url":"https://github.com/philipperemy/lead-lag","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philipperemy%2Flead-lag","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philipperemy%2Flead-lag/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philipperemy%2Flead-lag/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philipperemy%2Flead-lag/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/philipperemy","download_url":"https://codeload.github.com/philipperemy/lead-lag/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233352253,"owners_count":18663264,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lead-lag","mathematics","statistics"],"created_at":"2024-10-03T11:14:56.087Z","updated_at":"2025-01-10T13:34:30.991Z","avatar_url":"https://github.com/philipperemy.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/philipperemy"],"categories":[],"sub_categories":[],"readme":"## Estimation of the lead-lag parameter from non-synchronous data [[paper](https://arxiv.org/abs/1303.4871)]\n\n*Works on Linux, MacOS and Windows (Microsoft Visual C++ 14.0 or greater is required for Windows).*\n\n\u003e Abstract: We propose a simple continuous time model for modeling the lead-lag effect between two financial\nassets. A two-dimensional process (Xt, Yt) reproduces a lead-lag effect if, for some time shift\nϑ ∈ R, the process (Xt, Yt+ϑ) is a semi-martingale with respect to a certain filtration. The\nvalue of the time shift ϑ is the lead-lag parameter. Depending on the underlying filtration,\nthe standard no-arbitrage case is obtained for ϑ = 0. We study the problem of estimating the\nunknown parameter ϑ ∈ R, given randomly sampled non-synchronous data from (Xt) and (Yt).\nBy applying a certain contrast optimization based on a modified version of the Hayashi–Yoshida\ncovariation estimator, we obtain a consistent estimator of the lead-lag parameter, together with\nan explicit rate of convergence governed by the sparsity of the sampling design. The complexity is\n**O(n.LOG(n))**.\n\n### API\n\nCalculate the time lag (delay) in seconds between two time series in Python using the `lead_lag` module.\n\n```python\nlead_lag.lag(ts1: pd.Series, ts2: pd.Series, max_lag: Union[float, int]) -\u003e Optional[float]\n```\n\n#### Arguments\n- `ts1`: This is a Pandas Series containing the first time series data.\n- `ts2`: This is another Pandas Series containing the second time series data.\n- `max_lag`: defines a time interval within which the optimal lag is sought: `[-max_lag, max_lag]`.\n\nIt's important to note that the timestamps in the two input series, need not be synchronized. This means that the data points in these series don't have to occur at the exact same times. The module can handle non-synchronous data. However, for efficiency reasons, the smallest achievable lag is set at 100 microseconds.\n\n#### Returns\nThe signed estimated `lag`, expressed in seconds. \n- If the calculated lag is positive, it implies that `ts1` leads `ts2`.\n- If the calculated lag is negative, it implies that `ts2` leads `ts1`.\n-  If any issues arise during the calculation, the function returns `None`.\n\nThere is also a `lead_lag.LeadLag` object, which offers more features. Refer to the examples to learn how to use it.\n\n### Example\n\nHere's how the library operates using the simplest example:\n\n```python\nfrom datetime import datetime, timedelta\n\nimport numpy as np\nimport pandas as pd\n\nfrom lead_lag import lag\n\nts = pd.Series(\n    data=np.cumsum(np.random.uniform(low=-1, high=1, size=1000)),\n    index=[datetime(2022, 1, 1, 12, 0, 0) - timedelta(seconds=i) for i in range(1000)]\n)\nprint('lag=', lag(ts, ts.shift(-9), max_lag=10))\n# lag= 9.0\n```\n\n### Installation\n\nFollow those steps to install it:\n\n```bash\n# 1. PyPI. Recommended for all platforms (Windows, Linux, MacOS)\npip install lead_lag\n\n# 2. From the repository link.\n# Via SSH.\npip install git+ssh://git@github.com/philipperemy/lead-lag \n# Or via HTTPS.\npip install git+https://github.com/philipperemy/lead-lag.git\n\n# 3. From the sources.\ngit clone https://github.com/philipperemy/lead-lag.git \u0026\u0026 cd lead-lag\nvirtualenv -p python3 venv \u0026\u0026 source venv/bin/activate\nmake\n```\n\n### More Examples\n\nYou can gain further insight into the library's functionality by exploring the contents of the [examples](examples) directory. In particular:\n\n- There's an example titled \"FTX vs Bitmex lead lag,\" which examines the relationship between FTX and Bitmex exchanges. In this case, FTX leads Bitmex, and the calculated lag amounts to 90 milliseconds. This observation is based on data from September 27th, 2022.\n- Another example named \"Bitflyer vs Btcbox lead lag\" investigates the lead-lag dynamics between Bitflyer and Btcbox exchanges. The findings suggest that Bitflyer leads Btcbox, and the computed lag stands at 15 seconds. The data for this comparison pertains to the year 2018.\n\nYou can also run the Jupyter Notebook [lead_lag_example_2.ipynb](notebooks/lead_lag_example_2.ipynb):\n\n```bash\nmake jupyter\n```\n\n### Numerical Illustrations\n\n#### Non-synchronous data (generated from the Brownian Bachelier model)\n\nWe simulate a lead-lag Bachelier model without drift with: \n\nN = 10,000 (grid on which we sample random arriving times for both X and Y), #I = 500, #J = 3,000, ρ = 0.80, x0 = 1.0, y0 = 2.1, s1 = 1.0, s2 = 1.5, lead_lag = 200 (X is the leader, Y the lagger), finite grid Gn = [0, 400]. \n\nWe show a realization of the process (Xt, Yt) and its corresponding Constrast vs Lag plot:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figures/Figure_1.png\" width=\"400\"\u003e\n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figures/Figure_2.png\" width=\"400\"\u003e\n\u003c/p\u003e\n\n*The contrast is just a positive definitive cross correlation quantity.*\n\nClearly, the argmax of the contrast is located around the correct value (lead_lag = 200). We also observe some\npersistence in the contrast (I may have forgotten an extra term in the modified HY estimator). Even though X has a\nsampling rate 7x lower than Y, the estimator can still pick up the correct value. We can also normalize the contrast to\nhave an unbiased estimation of the cross correlation function rho for different lags. In theory this function should be\na Dirac centered around the lead_lag parameter with ρ(lead_lag) = 0.8 and 0 elsewhere.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figures/Figure_3.png\" width=\"370\"\u003e\n\u003c/p\u003e\n\nWe can also look at negative lags and define the LLR (standing for Lead/Lag Ratio) to measure the lead/lag\nrelationships. If LLR \u003e 1, then X is the leader and Y the lagger and vice versa for LLR \u003c= 1. In our case, for the\nrealization of our process (X,Y), we find LLR ~ 8.03.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figures/Figure_4.png\" width=\"350\"\u003e\n\u003c/p\u003e\n\n#### Non-synchronous data (Bitcoin markets)\n\nWe now consider a real world use case where we have two Japanese bitcoin exchanges: bitflyer and btcbox. The former has\nhigher liquidity hence we expect it to lead the latter. If we plot the prices of BTC/JPY for both exchanges for a\nspecific day, we get:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figures/Figure_5.png\" width=\"400\"\u003e\n\u003c/p\u003e\n\nSo which one leads? We apply the same lead lag procedure using the constrast quantity computed on a\ngrid `Gn = ]-40,40[` (unit is second here).\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figures/Figure_6.png\" width=\"400\"\u003e\n\u003c/p\u003e\n\nThe contrast is maximized for ϑ = 15 seconds. This promptly means that bitflyer is the leader as expected and that\nbtcbox takes on average 15 seconds to reflect any changes on its price.\n\n### References\n\n- [High-Frequency Covariance Estimates With Noisy and Asynchronous Financial Data](https://www.princeton.edu/~yacine/QMLE2D.pdf)\n- [On covariance estimation of non-synchronously observed diffusion](http://www.ms.u-tokyo.ac.jp/~nakahiro/mypapers_for_personal_use/hayyos03.pdf)\n- [Estimation of the lead-lag parameter from non-synchronous data](https://arxiv.org/pdf/1303.4871.pdf)\n- https://stats.stackexchange.com/questions/235697/semi-martingale-vs-martingale-what-is-the-difference?rq=1\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphilipperemy%2Flead-lag","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fphilipperemy%2Flead-lag","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphilipperemy%2Flead-lag/lists"}