{"id":13449701,"url":"https://github.com/Heerozh/spectre","last_synced_at":"2025-03-22T22:33:43.358Z","repository":{"id":37677860,"uuid":"213599662","full_name":"Heerozh/spectre","owner":"Heerozh","description":"GPU-accelerated Factors analysis library and Backtester","archived":false,"fork":false,"pushed_at":"2023-11-28T12:14:48.000Z","size":3517,"stargazers_count":638,"open_issues_count":8,"forks_count":108,"subscribers_count":20,"default_branch":"master","last_synced_at":"2024-10-28T16:45:21.095Z","etag":null,"topics":["algorithmic-trading","backtester","backtesting","factor-analysis","quantitative-analysis","spectre"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Heerozh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-10-08T09:20:02.000Z","updated_at":"2024-10-24T09:00:41.000Z","dependencies_parsed_at":"2024-01-18T17:54:10.888Z","dependency_job_id":null,"html_url":"https://github.com/Heerozh/spectre","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Heerozh%2Fspectre","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Heerozh%2Fspectre/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Heerozh%2Fspectre/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Heerozh%2Fspectre/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Heerozh","download_url":"https://codeload.github.com/Heerozh/spectre/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245029284,"owners_count":20549681,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithmic-trading","backtester","backtesting","factor-analysis","quantitative-analysis","spectre"],"created_at":"2024-07-31T06:00:51.771Z","updated_at":"2025-03-22T22:33:42.487Z","avatar_url":"https://github.com/Heerozh.png","language":"Python","funding_links":[],"categories":["Analytics","Python","金融股票","Analytic tools","Python：量化金融第一生态"],"sub_categories":["Optimization","Factor Analysis","网络服务_其他","三、技术指标与因子分析"],"readme":"[![Coverage Status](https://coveralls.io/repos/github/Heerozh/spectre/badge.svg?branch=master)](https://coveralls.io/github/Heerozh/spectre?branch=master)\n\n# ||spectre\n\nspectre is a **GPU-accelerated Parallel** quantitative trading library, focused on **performance**.\n\n  * Fast GPU Factor Engine, see below [Benchmarks](#benchmarks)\n  * Pure python code, based on PyTorch, so it can integrate DL model very smoothly.\n  * Compatible with `alphalens` and `pyfolio`\n\nPython 3.7+, PyTorch 1.3+, Pandas 1.0+ recommended\n\n\n# Installation\n\n```bash\npip install --no-deps git+git://github.com/Heerozh/spectre.git\n```\n\nDependencies:\n\n```bash\nconda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch\nconda install pyarrow pandas tqdm plotly requests\n```\n\n# Benchmarks\n\nMy Machine：\n- i9-7900X @ 3.30GHz, 20 Cores\n- DDR4 3800MHz\n- 3090: GIGABYTE GeForce RTX 3090 GAMING OC 24G\n- 2080Ti: RTX 2080Ti Founders\n\nRunning on Quandl 5 years, 3196 Assets, total 3,637,344 bars.\n\n|                |     spectre (CUDA/3090)       |     spectre (CUDA/2080Ti)     |       spectre (CPU)        |   zipline.pipeline    |\n|----------------|-------------------------------|-------------------------------|----------------------------|-----------------------|\n|SMA(100)        | 87.9 ms ± 3.35 ms (**33.9x**) | 144 ms ± 974 µs (**20.7x**)   | 2.68 s ± 36.1 ms (1.11x)   | 2.98 s ± 14.4 ms (1x) |\n|EMA(50) win=229 | 166 ms ± 3.25 ms (**50.5x**)  | 270 ms ± 1.89 ms (**31.0x**)  | 4.37 s ± 46.4 ms (1.74x)   | 8.38 s ± 56.8 ms (1x) |\n|(MACD+RSI+STOCHF).rank.zscore | 184 ms ± 7.83 ms (**77.7x**) | 282 ms ± 1.33 ms (**50.7x**) | 6.01 s ± 28.1 (2.38x)   | 14.3 s ± 277 ms (1x) |\n\n* The CUDA memory used in the spectre benchmark is 1.8G, returned by cuda.max_memory_allocated().\n* Benchmarks excluded the initial run (no copy data to VRAM, about saving 300ms).\n\n# Quick Start\n\n## DataLoader\n\nFirst of all is data, you can use [CsvDirLoader](#csvdirloader) read your csv files.\n\nspectre also has built-in Yahoo downloader, `symbols=None` will download all SP500 components.\n\n```python\nfrom spectre.data import YahooDownloader\nYahooDownloader.ingest(start_date=\"2001\", save_to=\"./prices/yahoo\", symbols=None, skip_exists=True)\n```\n\nYou can use `spectre.data.ArrowLoader('./prices/yahoo/yahoo.feather')` load those data now.\n\n## Factor and FactorEngine\n\n```python\nfrom spectre import factors\nfrom spectre.data import ArrowLoader\nloader = ArrowLoader('./prices/yahoo/yahoo.feather')\nengine = factors.FactorEngine(loader)\nengine.to_cuda()\nengine.add(factors.SMA(5), 'ma5')\nengine.add(factors.OHLCV.close, 'close')\ndf = engine.run('2019-01-11', '2019-01-15')\ndf\n```\n\n\n|                         |         |        ma5|\t   close|\n|-------------------------|---------|-----------|-----------|\n|**date**                 |**asset**|           |\t        |\n|2019-01-14 00:00:00+00:00|        A|  68.842003|  70.379997|\n|                         |     AAPL| 151.615997| 152.289993|\n|                         |      ABC|  75.835999|  76.559998|\n|                         |      ABT|  69.056000|  69.330002|\n|                         |     ADBE| 234.537994| 237.550003|\n|                      ...|      ...|        ...|\t     ...|\n|2019-01-15 00:00:00+00:00|      XYL|  68.322006|  69.160004|\n|                         |      YUM|  91.010002|  90.000000|\n|                         |      ZBH| 102.932007| 102.690002|\n|                         |     ZION|  43.760002|  44.320000|\n|                         |      ZTS|  85.846001|  84.500000|\n\n\n## Factor Analysis\n\n\n```python\nfrom spectre import factors\nimport math\n\nrisk_free_rate = 0.04 / 252\nexcess_logret = factors.LogReturns() - math.log(1 + risk_free_rate)\nuniverse = factors.AverageDollarVolume(win=120).top(100)\n\n# Barra MOMENTUM\nema126 = factors.EMA(half_life=126, inputs=[excess_logret])\nrstr = ema126.shift(11).sum(252)\nMOMENTUM = rstr\n\n# Barra Volatility\nema42 = factors.EMA(half_life=42, inputs=[excess_logret])\ndastd = factors.STDDEV(252, inputs=[ema42])\nVOLATILITY = dastd\n\n# run engine\nfrom spectre.data import ArrowLoader\nloader = ArrowLoader('./prices/yahoo/yahoo.feather')\nengine = factors.FactorEngine(loader)\n\nengine.set_filter( universe )\nengine.add( MOMENTUM, 'MOMENTUM' )\nengine.add( VOLATILITY, 'VOLATILITY' )\n\nengine.to_cuda()\n%time factor_data, mean_return = engine.full_run(\"2013-01-02\", \"2018-01-19\", periods=(1,5,10,))\n```\n\n\u003cimg src=\"https://github.com/Heerozh/spectre/raw/media/full_run.png\" width=\"800\" height=\"600\"\u003e\n\n### Diagram\n\nYou can also view your factor structure graphically:\n\n```python\nfactors.BBANDS(win=5).normalized().rank().zscore().show_graph()\n```\n\n\u003cimg src=\"https://github.com/Heerozh/spectre/raw/media/factor_diagram.png\" width=\"800\" height=\"360\"\u003e\n\nThe thickness of the line represents the length of the Rolling Window, kind of like \"bandwidth\".\n\nIf `engine.to_cuda(enable_stream=True)`, the calculation of the branches will be performed\nsimultaneously, but the VRAM usage will increase proportionally. \n\n### Compatible with alphalens\n\nThe return value of `full_run` is compatible with `alphalens`:\n```python\nimport alphalens as al\n...\nfactor_data, _ = engine.full_run(\"2013-01-02\", \"2018-01-19\")\nclean_data = factor_data[['{factor_name}', 'Returns']].droplevel(0, axis=1)\nal.tears.create_full_tear_sheet(clean_data)\n```\n\n\n## Back-testing\n\nBack-testing uses FactorEngine's results as data, market event as triggers.\n\nYou can find other examples in the `./examples` directory.\n\n```python\nfrom spectre import factors, trading\nfrom spectre.data import ArrowLoader\nimport pandas as pd, math\n\n\nclass MyAlg(trading.CustomAlgorithm):\n    def initialize(self):\n        # your factors\n        risk_free_rate = 0.04 / 252\n        excess_logret = factors.LogReturns() - math.log(1 + risk_free_rate)\n        universe = factors.AverageDollarVolume(win=120).top(100)\n\n        # Barra MOMENTUM Risk Factor\n        ema126 = factors.EMA(half_life=126, inputs=[excess_logret])\n        rstr = ema126.shift(11).sum(252)\n        MOMENTUM = rstr.zscore(mask=universe)\n\n        # Barra Volatility Risk Factor\n        ema42 = factors.EMA(half_life=42, inputs=[excess_logret])\n        dastd = factors.STDDEV(252, inputs=[ema42])\n        VOLATILITY = dastd.zscore(mask=universe)\n\n        # setup engine\n        engine = self.get_factor_engine()\n        engine.to_cuda()\n        engine.set_filter( universe )\n        engine.add( (MOMENTUM + VOLATILITY).to_weight(), 'alpha_weight' )\n\n        # schedule rebalance before market close\n        self.schedule_rebalance(trading.event.MarketClose(self.rebalance, offset_ns=-10000))\n\n        # simulation parameters\n        self.blotter.capital_base = 1000000\n        self.blotter.set_commission(percentage=0, per_share=0.005, minimum=1)\n        # self.blotter.set_slippage(percentage=0, per_share=0.4)\n\n    def rebalance(self, data: 'pd.DataFrame', history: 'pd.DataFrame'):\n        data = data.fillna(0)\n        self.blotter.batch_order_target_percent(data.index, data.alpha_weight)\n\n        # closing asset position that are no longer in our universe.\n        removes = self.blotter.portfolio.positions.keys() - set(data.index)\n        self.blotter.batch_order_target_percent(removes, [0] * len(removes))\n\n        # record data for debugging / plotting\n        self.record(aapl_weight=data.loc['AAPL', 'alpha_weight'],\n                    aapl_price=self.blotter.get_price('AAPL'))\n\n    def terminate(self, records: 'pd.DataFrame'):\n        # plotting results\n        self.plot(benchmark='SPY')\n\n        # plotting the relationship between AAPL price and weight\n        ax1 = records.aapl_price.plot()\n        ax2 = ax1.twinx()\n        records.aapl_weight.plot(ax=ax2, style='g-')\n\nloader = ArrowLoader('./prices/yahoo/yahoo.feather')\n%time results = trading.run_backtest(loader, MyAlg, '2014-01-01', '2019-01-01')\n```\n\n\u003cimg src=\"https://github.com/Heerozh/spectre/raw/media/backtest.png\" width=\"800\" height=\"630\"\u003e\n\nIt awful but you get the idea.\n\nThe return value of `run_backtest` is compatible with `pyfolio`:\n```python\nimport pyfolio as pf\npf.create_full_tear_sheet(results.returns, positions=results.positions.value, transactions=results.transactions,\n                          live_start_date='2017-01-03')\n```\n\n# API\n\n## Note\n\n### Differences to zipline:\n* In order to GPU optimize, the `CustomFactor.compute` function calculates the results of all bars \n  at once, so you need to be careful to prevent Look-Ahead Bias, because the inputs are not just \n  historical data. Also using `engine.test_lookahead_bias` do some tests.\n* spectre's normally using float32 data type for GPU performance.\n* spectre FactorEngine arranges data by bars, so `Return(win=10)` means 10 bars return, may \n  actually be more than 10 days if some assets not open trading in period. You can change this \n  behavior by aligning data: filling missing bars with NaNs in your DataLoader, please refer to the \n  `align_by_time` parameter of `CsvDirLoader`.\n\n\n### Differences to common chart:\n* If there is adjustments data, the prices is re-adjusted every day, so the factor you got, like MA, \n  will be different from the stock chart software which only adjusted according to last day.\n  If you want adjusted by last day, use like 'AdjustedColumnDataFactor(OHLCV.close)' as input data.\n  This will speeds up a lot because it only needs to be adjusted once, but brings Look-Ahead Bias.\n* Factors that uses the close data will be delayed by 1 bar.\n* spectre's `EMA` uses the algorithm same as `zipline` and `Dataframe.ewm(span=...)`, when `span` is \n  greater than 100, it will be slightly different from common EMA.\n* spectre's `RSI` uses the algorithm same as `zipline`, for consistency in benchmarks. \n\n\n## Factors\n\n## Built-in Technical Indicator Factors list\n\n```python\nReturns(inputs=[OHLCV.close])\nLogReturns(inputs=[OHLCV.close])\nSimpleMovingAverage = MA = SMA(win=5, inputs=[OHLCV.close])\nVWAP(inputs=[OHLCV.close, OHLCV.volume])\nExponentialWeightedMovingAverage = EMA(span=5, inputs=[OHLCV.close])\nAverageDollarVolume(win=5, inputs=[OHLCV.close, OHLCV.volume])\nAnnualizedVolatility(win=20, inputs=[Returns(win=2), 252])\nBollingerBands = BBANDS(win=20, inputs=[OHLCV.close, 2])\nMovingAverageConvergenceDivergenceSignal = MACD(12, 26, 9, inputs=[OHLCV.close])\nTrueRange = TRANGE(inputs=[OHLCV.high, OHLCV.low, OHLCV.close])\nRSI(win=14, inputs=[OHLCV.close])\nFastStochasticOscillator = STOCHF(win=14, inputs=[OHLCV.high, OHLCV.low, OHLCV.close])\n\nStandardDeviation = STDDEV(win=5, inputs=[OHLCV.close])\nRollingHigh = MAX(win=5, inputs=[OHLCV.close])\nRollingLow = MIN(win=5, inputs=[OHLCV.close])\n```\n\n## Factors Common Methods\n\n```python\n# Standardization\nnew_factor = factor.rank(mask=filter)   \nnew_factor = factor.demean(mask=filter, groupby: 'dict or column_name'=None)\nnew_factor = factor.zscore(mask=filter)\nnew_factor = factor.to_weight(mask=filter, demean=True)  # return a weight that sum(abs(weight)) = 1\n\n# Quick computation\nnew_factor = factor1 + factor1\nnew_factor = factor.abs()\nnew_factor = factor.sum()\n\n# To filter (Comparison operator):\nnew_filter = (factor1 \u003c factor2) | (factor1 \u003e 0)\nnew_filter[n_features] = factor.one_hot()  # one-hot encoding\nnew_filter = factor.any(win=5)\nnew_filter = factor.all(win=5)\n# Rank filter\nnew_filter = factor.top(n)\nnew_filter = factor.bottom(n)\n# Specific assets\nnew_filter = StaticAssets({'AAPL', 'MSFT'})\n\n# Local filter\nnew_factor = factor.filter(some_filter)   # fills elements of self with NaN where mask is False\n\n# Multiple returns selecting\nnew_factor = factor[0]\n\n# Others\nnew_factor = factor.shift(1)\nnew_factor = factor.quantile(bins=5)  # factor value quantile groupby datetime\nnew_factor = factor.fill_na(0)\nnew_factor = factor.fill_na(ffill=True)  # propagate last valid observation forward to next valid\n\n```\n\n\n## Dataloader\n\n### CsvDirLoader\n`loader = spectre.data.CsvDirLoader(prices_path: str, prices_by_year=False, earliest_date: pd.Timestamp = None,\n              dividends_path=None, splits_path=None, file_pattern='*.csv', calender_asset: str = None, align_by_time=False,\n              ohlcv=('open', 'high', 'low', 'close', 'volume'), adjustments=None,\n              split_ratio_is_inverse=False, split_ratio_is_fraction=False,\n              prices_index='date', dividends_index='exDate', splits_index='exDate', **read_csv)`\n\nRead CSV files in the directory, each file represents an asset.\n\nReading csv is very slow, so you also need to use [ArrowLoader](#arrowloader).\n\n**prices_path:** Prices csv folder. When encountering duplicate datetime in `prices_index`, \n    Loader will keep the last, drop others.\\\n**prices_index:** `index_col`for csv in `prices_path`\\\n**prices_by_year:** If prices file name like 'spy_2017.csv', set this to True\\\n**ohlcv:** Required, OHLCV column names. When you don't need to use `adjustments` and\n    `factors.OHLCV`, you can set this to None.\\\n**adjustments:** Optional, list, `dividend amount` and `splits ratio` column names.\\\n**dividends_path:** Dividends csv folder, structured as one csv per asset.\n    For duplicate data, loader will first drop the exact same rows, and then for the same\n    `dividends_index` but different 'dividend amount(`adjustments[0]`)' rows, loader will sum them up.\n    If `dividends_path` not set, the `adjustments[0]` column is considered to be included\n    in the prices csv.\\\n**dividends_index:** `index_col`for csv in `dividends_path`.\\\n**splits_path:** Splits csv folder, structured as one csv per asset.\n    When encountering duplicate datetime in `splits_index`, Loader will use the last\n    non-NaN 'split ratio', drop others.\n    If `splits_path` not set, the `adjustments[1]` column is considered to be included\n    in the prices csv.\\\n**splits_index:** `index_col`for csv in `splits_path`.\\\n**split_ratio_is_inverse:** If split ratio calculated by to/from, set to True.\n    For example, 2-for-1 split, to/form = 2, 1-for-15 Reverse Split, to/form = 0.6666...\\\n**split_ratio_is_fraction:** If split ratio in csv is fraction string, like `1/3`, set to True.\\\n**file_pattern:** csv file name pattern, default is '*.csv'.\\\n**earliest_date:** Data before this date will not be read, save memory.\\\n**calender_asset:** Asset name as trading calendar, like 'SPY', for clean up non-trading\n    time data.\\\n**align_by_time:** If True and `calender_asset` is not None, the index of datetime will be the same \n   for all assets, if some assets have no data at that time, NaNs will be filled. The benefit is \n   that the columns of data matrix in `CustomFactor.compute` will also be aligned.\\\n**\\*\\*read_csv:** Parameters for all csv when calling `pd.read_csv`.\n `parse_dates` or `date_parser` is required.\n\nExample for load [IEX](https://github.com/Heerozh/iex_fetcher) CSV files:\n\n```python\nusecols = {'date', 'uOpen', 'uHigh', 'uLow', 'uClose', 'uVolume', 'exDate', 'amount', 'ratio'}\ncsv_loader = spectre.data.CsvDirLoader(\n    './iex/daily/', calender_asset='SPY', \n    dividends_path='./iex/dividends/', \n    splits_path='./iex/splits/',\n    ohlcv=('uOpen', 'uHigh', 'uLow', 'uClose', 'uVolume'), adjustments=('amount', 'ratio'),\n    prices_index='date', dividends_index='exDate', splits_index='exDate', \n    parse_dates=True, usecols=lambda x: x in usecols,\n    dtype={'uOpen': np.float32, 'uHigh': np.float32, 'uLow': np.float32, 'uClose': np.float32, \n           'uVolume': np.float64, 'amount': np.float64, 'ratio': np.float64})\n```\n\n### ArrowLoader\n\nIngest data from other DataLoader into a feather file, speed up reading speed a lot.\n\n3GB data takes about 7 seconds on initial load.\n\n**Ingest**\n`spectre.data.ArrowLoader.ingest(source=CsvDirLoader(...), save_to='./filename.feather')`\n\n**Read**\n`loader = spectre.data.ArrowLoader('./filename.feather')`\n\n### QuandlLoader\n\n**no longer updated, only contain prices before 2018**\n\nDownload 'WIKI_PRICES.zip' (You need an account):\n`https://www.quandl.com/api/v3/datatables/WIKI/PRICES.csv?qopts.export=true\u0026api_key=[yourapi_key]`\n\n```python\nfrom spectre.data import ArrowLoader, QuandlLoader\nArrowLoader.ingest(source=QuandlLoader('WIKI_PRICES.zip'),\n                   save_to='wiki_prices.feather')\n```\n\n### How to write your own DataLoader\n\nInherit from `DataLoader`, overriding the `_load` method, read data into a large `DataFrame`, \nindex is `MultiIndex ['date', 'asset']`, where date is `Datetime` type, `asset` is `string` type, \nand then call `self._format(df, split_ratio_is_inverse)` to format the data. \nAlso call `test_load` in your test case to do basic format testing.\n\nFor example, suppose you have a csv file that contains data for all assets:\n```python\nclass YourLoader(spectre.data.DataLoader):\n    @property\n    def last_modified(self) -\u003e float:\n        return os.path.getmtime(self._path)\n\n    def __init__(self, file: str, calender_asset='SPY') -\u003e None:\n        super().__init__(file,\n                         ohlcv=('open', 'high', 'low', 'close', 'volume'),\n                         adjustments=('ex-dividend', 'split_ratio'))\n        self._calender = calender_asset\n\n    def _load(self) -\u003e pd.DataFrame:\n        df = pd.read_csv(self._path, parse_dates=['date'],\n                         usecols=['asset', 'date', 'open', 'high', 'low', 'close',\n                                  'volume', 'ex-dividend', 'split_ratio', ],\n                         dtype={\n                             'open': np.float32, 'high': np.float32, 'low': np.float32,\n                             'close': np.float32, 'volume': np.float64,\n                             'ex-dividend': np.float64, 'split_ratio': np.float64\n                         })\n\n        df.set_index(['date', 'asset'], inplace=True)\n        df = self._format(df, split_ratio_is_inverse=True)\n        if self._calender:\n            df = self._align_to(df, self._calender)\n\n        return df\n```\n\n## FactorEngine\n\nA fast factor calculation pipeline.\n\n### FactorEngine.__init__\n\n`engine = FactorEngine(loader: DataLoader)`\n\n### FactorEngine.add\n\n`engine.add(factor, column_name)`\n\nAdd a factor to engine.\n\n\n### FactorEngine.set_filter\n\n`engine.set_filter(factor: FilterFactor or None)`\n\nSet the Global Filter, engine deletes rows which Global Filter returns as False at the last step, \naffect all factors.\n\n\n### FactorEngine.align_by_time\n\n`engine.align_by_time = bool`\n\nSame as `CsvDirLoader(align_by_time=True)`, but it's dynamic. Notes: Very slow on large amounts of \ndata, and if the data source is already aligned, this method cannot make it return to unaligned. \n\n\n### FactorEngine.clear\n\n`engine.clear()`\n\nRemove global filter, and all factors.\n\n\n### FactorEngine.to_cuda\n\n`engine.to_cuda(enable_stream=False)`\n\nSwitch to GPU mode.\n\nSet enable_stream to True allows pipeline branches to calculation simultaneously.\nHowever, this will lead to more VRAM usage and may also affect performance.\n\n### FactorEngine.to_cpu\n\n`engine.to_cpu()`\n\nSwitch to CPU mode.\n\n\n### FactorEngine.run\n\n`df = engine.run(start_time, end_time, delay_factor=True)`\n\nRun the engine to calculate the factor data, return a DataFrame. The column is each added factor.\n\n#### *Auto Delay\nBy default, `delay_factor` is True, it means enable auto-delay. If 'high, low, close, volume' data \nis used by a terminal factor (including its upstream), that factor will be delayed by `shift(1)` \nin the last step, because in theory you can't trade on this factor before it generated. Others\nwill not be delayed, in order to provide the latest data as much as possible.\n\nSet to `False` to force engine not delay any factors.\n\n\n### FactorEngine.plot_chart\n\n`engine.plot_chart(start_time, end_time, trace_types=None, styles=None, delay_factor=True)`\n    \nPlotting common stock price chart for researching.\n\n`trace_types`: `dict(factor_name=plotly_trace_type)`, trace type can be 'Bar', or 'Scatter', \ndefault is 'Scatter'.\n\n`styles`: `dict(factor_name=plotly_trace_styles)`, add the trace styles, please refer \nto plotly documentation: [Scatter traces](https://plot.ly/python/reference/#scatter)\n\n\n```python\nrsi = factors.RSI()\nbuy_signal = (rsi.shift(1) \u003c 30) \u0026 (rsi \u003e 30)\n\nengine = factors.FactorEngine(loader)\nengine.timezone = 'America/New_York'\nengine.set_filter(factors.StaticAssets({'NVDA', 'MSFT'}))\nengine.add(factors.MA(20), 'MA20')\nengine.add(rsi, 'RSI')\nengine.add(factors.OHLCV.close.filter(buy_signal), 'Buy')\nengine.to_cuda()\n_ = engine.plot_chart('2017', '2018', styles={\n    'MA20': {\n              'line': {'dash': 'dash'}\n            },\n    'RSI': {\n              'yaxis': 'y3',  # y1: price axis, y2: volume axis, yN: add new y-axis\n              'line': {'width': 1}\n           },\n    'Buy': { \n              'mode': 'markers', \n              'marker': { 'symbol': 'triangle-up', 'size': 10, 'color': 'rgba(0, 0, 255, 0.5)' }\n           }\n})\n```\n\n\u003cimg src=\"https://github.com/Heerozh/spectre/raw/media/chart.png\" width=\"730\" height=\"1000\"\u003e\n\n\n### FactorEngine.full_run\n\n`factor_data, mean_returns = engine.full_run(\n    start_time, end_time, trade_at='close', periods=(1, 4, 9),\n    quantiles=5, filter_zscore=20, demean=True, preview=True)`\n    \nNot only run the engine, but also run factor analysis.\n\n\n### FactorEngine.get_price_matrix\n\n`df_prices = engine.get_price_matrix(start_time, end_time, prices: ColumnDataFactor = OHLCV.close)`\n\nGet the adjusted historical prices matrix which columns is all assets.\n\nIf global filter is setted, all unfiltered assets from `start_time` to `end_time` will be included.\n\n\n### FactorEngine.test_lookahead_bias\n\n`engine.test_lookahead_bias(start_time, end_time)`\n\nRun the engine to test if there is a lookahead bias.\n\nFill random values to second half of the ohlcv data, and then check if there are differences between\nthe two runs in the first half.\n\n## ColumnDataFactor\n\nYou can use `ColumnDataFactor` to represents data from any column in the `DataLoader`, for example:\n\n`spectre.factors.ColumnDataFactor(inputs=['col_name'])`\n\n`factors.OHLCV.close` is just a sugar way to write \n`spectre.factors.ColumnDataFactor (inputs = [data_loader.ohlcv[3]])`.\n\n\n## How to write your own factor\n\nInherit from `factors.CustomFactor`, write `compute` function.\n\nAll `inputs` will pass to compute function.\n\n### win = 1\nWhen `win = 1`, the `inputs` data is tensor type, the first dimension of data is the asset, the \nsecond dimension is each bar price data. Note that if the data is `align_by_time=False`, the number \nof bars for each asset is different and not aligned (for example, the time for each price in bar_t3 \ncolumn may be inconsistent).\n\n        +-----------------------------------+\n        |            bar_t1    bar_t3       |\n        |               |         |         |\n        |               v         v         |\n        | asset 1--\u003e [[1.1, 1.2, 1.3, ...], |\n        | asset 2--\u003e  [  5,   6,   7, ...]] |\n        +-----------------------------------+\nExample of LogReturns:\n```python\nfrom spectre import factors \nimport torch\nclass LogReturns(factors.CustomFactor):\n    inputs = [factors.Returns(2, inputs=[factors.OHLCV.close])]\n    win = 1\n\n    def compute(self, change: torch.Tensor) -\u003e torch.Tensor:\n        return (change + 1).log()\n```\n\n### win \u003e 1\nIf rolling window is required(`win \u003e 1`), all `inputs` data will be wrapped into\n`spectre.parallel.Rolling`.\n\nThis is just an unfolded `tensor` data, but because the data is very large after unfolded, for\nbetter performance and saving VRAM, the rolling class automatically splits the data into multiple\nsmall chunks. You need to use the `agg` method to operating `tensor`.\n```python\nfrom spectre import factors, parallel\nclass OvernightReturn(factors.CustomFactor):\n    inputs = [factors.OHLCV.open, factors.OHLCV.close]\n    win = 2\n\n    def compute(self, opens: parallel.Rolling, closes: parallel.Rolling) -\u003e torch.Tensor:\n        ret = opens.last() / closes.first() - 1\n        return ret\n```\nThe `closes.first()` above is just a helper method for `closes.agg(lambda x: x[:, :, 0])`,\nwhere `x[:, :, 0]` return the first element of rolling window. The first dimension of `x` is the\nasset, the second dimension is each bar, and the third dimension is the bar price and historical \nprice with `win` length, and `Rolling.agg` runs on all the chunks and combines them.\n\n        +------------------win=3-------------------+\n        |          history_t-2 curr_bar_value      |\n        |              |          |                |\n        |              v          v                |\n        | asset 1--\u003e[[[nan, nan, 1.1],  \u003c--bar_t1  |\n        |             [nan, 1.1, 1.2],  \u003c--bar_t2  |\n        |             [1.1, 1.2, 1.3]], \u003c--bar_t3  |\n        |                                          |\n        | asset 2--\u003e [[nan, nan,   5],  \u003c--bar_t1  |\n        |             [nan,   5,   6],  \u003c--bar_t2  |\n        |             [  5,   6,   7]]] \u003c--bar_t3  |\n        +------------------------------------------+\n\n`Rolling.agg` can carry multiple `Rolling` objects, such as\n```python\nweighted_mean = lambda _close, _volume: (_close * _volume).sum(dim=2) / _volume.sum(dim=2)\nclose.agg(weighted_mean, volume)\n```\n\n### Using Pandas Series\n\nCustomFactor's inputs data is a matrix without DataFrame's Index information.\nIf you need index, or not familiar with PyTorch, here is a another way:\n\n```python\nfrom spectre import factors\nclass YourFactor(factors.CustomFactor):\n\n    def compute(self, data: torch.Tensor) -\u003e torch.Tensor:\n        # convert to pd.Series data\n        pd_series = self._revert_to_series(data)\n        # ...\n        # convert back to grouped tensor\n        return self._regroup(pd_series)\n```\nThis method is completely non-parallel and inefficient, but easy to write.\n\n## Back-testing\n\n[Quick Start](#back-testing) contains easy-to-understand examples, please read first.\n\nThe `spectre.trading.CustomAlgorithm` currently does not supports live trading,\nwill implement it in the future.\n\n### CustomAlgorithm.initialize\n\n`alg.initialize(self)` **Callback**\n\nCalled when back-testing starts, at least you need use `get_factor_engine` to add factors\nand call `schedule_rebalance` here.\n\n\n### CustomAlgorithm.terminate\n\n`alg.terminate(self, records: pd.DataFrame)` **Callback**\n\nCalled when back-testing ends.\n\n\n### rebalance callback\n\n`rebalance(self, data: pd.DataFrame, history: pd.DataFrame)` **Callback**\n\nThe function name does not have to be 'rebalance', it can be specified in `schedule_rebalance`.\n`data` is the factors data of last bar returned by `FactorEngine`; \n`history` same as `data`, but contains previous data, please refer to `set_history_window`.\n\nPut calculations into the `FactorEngine` as much as possible can improve backtest performance.\n\n\n### CustomAlgorithm.get_factor_engine\n\n`self.get_factor_engine(name: str = None)`\n**context:** *initialize, rebalance, terminate*\n\nGet the factor engine of this trading algorithm. But note that you can add factors or filter only \nduring `initialize`, otherwise it will cause unexpected effects.\n\nThe algorithm has a default engine, `name` can be None.\nBut if you created multiple engines using `create_factor_engine`, you need to specify which one.\n\n\n### CustomAlgorithm.create_factor_engine\n\n`self.create_factor_engine(name: str, loader: DataLoader = None)`\n**context:** *initialize*\n\nCreate another engine, generally used when you need multiple data sources.\n\n\n### CustomAlgorithm.set_history_window\n\n`self.set_history_window(offset: pd.DateOffset=None)`\n**context:** *initialize*\n\nSet the length of historical data passed to each `rebalance` call. **SLOW**\n\nDefault: If None, pass all available historical data, so there will be no historical data on the\nfirst day, one historical row on the next day, and so on.\n\n\n### CustomAlgorithm.schedule_rebalance\n\n`self.schedule_rebalance(event: Event)`\n**context:** *initialize*\n\nSchedule `rebalance` to be called when an event occurs.\nEvents are: `MarketOpen`, `MarketClose`, `EveryBarData`,\nFor example:\n```python\nalg.schedule_rebalance(trading.event.MarketClose(self.any_function))\n```\n\nThe `Market*` events has `offset_ns` parameter `MarketClose(self.any_function, offset_ns=-1000)`, \na negative value of `offset_ns` means 'before', in backtest mode, the magnitude of the value has no\neffect.\n\n\n### CustomAlgorithm.schedule\n\n`self.schedule(event: Event)`\n**context:** *initialize*\n   \nSchedule an event, callback is `callback(source: \"Any class who fired this event\")`\n\n\n### CustomAlgorithm.empty_cache_after_run\n\n`self.empty_cache_after_run = True`\n**context:** *initialize*\n\nEmpty engine's cache after factor calculation. \nIf you need more VRMA in rebalance context, or wanna play 3D game when backtesting, set it to \nTrue will help.\n\n\n### CustomAlgorithm.stop_event_manager\n\n`alg.stop_event_manager()`\n**context:** *all*\n\nStop backtesting or live trading.\n\n\n### CustomAlgorithm.fire_event\n\n`alg.fire_event(event_type: Type[Event])`\n**context:** *all*\n\nTrigger a type of event (any subclasses that inherit from `Event`）, \nfor example: `alg.fire_event(MarketClose)`, (do not do this, do not fire built-in events)\n\n\n### CustomAlgorithm.results\n\n`self.results`\n**context:** *terminate*\n\nGet back-test results, same as the return value of [trading.run_backtest](#spectretradingrun_backtest)\n\n\n### CustomAlgorithm.plot\n\n`self.plot(annual_risk_free=0.04, benchmark: Union[pd.Series, str] = None)`\n**context:** *terminate*\n\nPlot a simple portfolio cumulative return chart.\\\n`benchmark`: `pd.Series` of benchmark daily return, or an asset name.\n\n\n\n### CustomAlgorithm.current\n\n`self.current`\n**context:** *rebalance*\n\nCurrent datetime, Read-Only.\n\n\n### CustomAlgorithm.get_price_matrix\n\n`self.get_price_matrix(length: pd.DateOffset, name: str = None, prices=OHLCV.close)`\n**context:** *rebalance*\n\nHelp method for calling `engine.get_price_matrix`, `name` specifies which engine.\n\nReturns the historical asset prices, adjusted and filtered by the current time.\n**Slow**\n\n\n### CustomAlgorithm.record\n\n`self.record(**kwargs)`\n**context:** *rebalance*\n\nRecord the data and pass all when calling `terminate`, use `column = value` format.\n\n\n### SimulationBlotter.set_commission\n\n`self.blotter.set_commission(percentage=0, per_share=0.005, minimum=1)`\n**context:** *initialize*\n\npercentage: percentage part, calculated by `percentage * price * shares`\\\nper_share: calculated by `per_share * shares`\\\nminimum: minimum commission if above sum does not exceed\n\ncommission = max(percentage_part + per_share_part, minimum)\n\n\n### SimulationBlotter.set_slippage\n\n`self.blotter.set_slippage(percentage=0, per_share=0.01)`\n**context:** *initialize, rebalance*\n\nMarket impact add to the price.\n\n\n### SimulationBlotter.set_short_fee\n\n`self.blotter.set_short_fee(percentage=0)`\n**context:** *initialize*\n\nSet the transaction fees which only charged for sell orders.\n\n\n### SimulationBlotter.daily_curb\n\n`self.blotter.daily_curb = float` \n**context:** *initialize, rebalance*\n\nLimit on trading a specific asset if today to previous day return \u003e= ±value. **SLOW**\n\n\n### SimulationBlotter.order_target\n\n`self.blotter.order_target(asset: str, target: number)` \n**context:** *rebalance*\n\nPlace an order on an asset to target number of shares in position, negative number means short.\n\nIf asset cannot be traded or limited by `daily_curb`, it will return False.\n\n\n### SimulationBlotter.batch_order_target\n\n`self.blotter.batch_order_target(asset: Iterable[str], target: Iterable[float])` \n**context:** *rebalance*\n\nSame as `SimulationBlotter.order_target`, but for multiple assets.\n\nReturn value is a list of skipped assets, which indicate that they cannot be traded or limited by \n`daily_curb`.\n\n\n### SimulationBlotter.order_target_percent\n\n`self.blotter.order_target_percent(asset: str, pct: float)` \n**context:** *rebalance*\n\nPlace an order on an asset to target percentage of portfolio net value, negative number means short.\n\nIf asset cannot be traded or limited by `daily_curb`, it will return False.\n\n\n### SimulationBlotter.batch_order_target_percent\n\n`self.blotter.batch_order_target_percent(asset: Iterable[str], pct: Iterable[float])` \n**context:** *rebalance*\n\nSame as `SimulationBlotter.order_target_percent`, but for multiple assets and better performance.\n\nReturn value is a list of skipped assets, which indicate that they cannot be traded or limited by \n`daily_curb`.\n\n\n### SimulationBlotter.order\n\n`self.blotter.order(asset: str, amount: int)` \n**context:** *rebalance*\n\nOrder a certain amount of an asset, negative number means short.\n\nIf asset cannot be traded or limited by `daily_curb`, it will return False.\n\n\n### SimulationBlotter.get_price\n\n`float = self.blotter.get_price(asset: Union[str, Iterable])` \n**context:** *rebalance*\n\nGet current price of assert. \n*Notice: Batch calls are slow, You can add prices as factor to get the price,\nlike: `engine.add(OHLCV.close, 'prices')`*\n\n\n### SimulationBlotter.portfolio.set_stop_model\n\n`self.blotter.portfolio.set_stop_model(model: StopModel)` \n**context:** *initialize*\n\nSet stop tracking model for positions, models are:\n\n`trading.StopModel(ratio, callback)`\\\n`trading.TrailingStopModel(ratio, callback)`\\\n`trading.PnLDecayTrailingStopModel` and `trading.TimeDecayTrailingStopModel`\n\nStop loss example:\n```python\nclass Backtester(trading.CustomAlgorithm):\n    def initialize(self):\n        ...\n        self.blotter.portfolio.set_stop_model(trading.TrailingStopModel(-0.1, self.stop))\n\n    def stop(self, asset, amount):\n        self.blotter.order(asset, amount)\n        self.record(...)\n\n    def rebalance(self, data, history):\n        self.blotter.portfolio.check_stop_trigger()\n        ...\n```\n\n#### PnLDecayTrailingStopModel\n`trading.PnLDecayTrailingStopModel(ratio, pnl_target, callback, decay_rate=0.05, max_decay=0)`\n\nThis is a model that can stop gain and stop loss at the same time.\n\nExponential decay to the stop ratio: `ratio * decay_rate ^ (PnL% / PnL_target%)`, \nSo `PnLDecayTrailingStopModel(-0.1, 0.1, callback)` means initial stop loss is -10%, and the \n`ratio` will decrease when profit% approaches the target +10%. If recorded high profit% exceeds 10%, \nany drawdown will trigger a stop loss.\n\n#### TimeDecayTrailingStopModel\n`trading.TimeDecayTrailingStopModel(ratio, period_target: pd.Timedelta, callback, decay_rate=0.05, \nmax_decay=0)`\n\nSame as `PnLDecayTrailingStopModel`, but target is time period.\n\n\n### SimulationBlotter.get_returns\n\n`self.blotter.get_returns()`\n**context:** *rebalance, terminate*\n\nGet the portfolio returns, use `(self.blotter.get_returns() + 1).prod()` to get current cumulative\nreturn.\n\n\n### SimulationBlotter.portfolio Read Only Properties\n\n**context:** *rebalance, terminate*\n\n`self.blotter.portfolio.positions` Current positions, `Dict[asset, Position]` type.\n```python\nclass Position:\n    shares = None\n    average_price = None\n    last_price = None\n    unrealized = None\n    realized = None\n```\n\n`self.blotter.portfolio.value` Current portfolio value\n\n`self.blotter.portfolio.cash` Current portfolio cash\n\n`self.blotter.portfolio.leverage` Current portfolio leverage\n\n\n### spectre.trading.run_backtest\n\n`results = trading.run_backtest(loader: DataLoader, alg_type: Type[CustomAlgorithm], start, end)`\n\nRun backtest, return value is namedtuple:\n\n**results.returns:** daily return\n\n**results.positions:** daily positions\n\n**results.transactions:** full transactions with all orders\n\n\n\n# Copyright \u0026 Thanks\nCopyright (C) 2019-2020, by Zhang Jianhao (heeroz@gmail.com), All rights reserved.\n\nThanks to [JetBrains](https://www.jetbrains.com/?from=spectre)'s support.\n\n------------\n\u003e *A spectre is haunting Market — the spectre of capitalism.*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHeerozh%2Fspectre","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHeerozh%2Fspectre","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHeerozh%2Fspectre/lists"}