{"id":13802354,"url":"https://github.com/peterhinch/micropython-filters","last_synced_at":"2025-04-30T15:31:24.247Z","repository":{"id":27222608,"uuid":"30693766","full_name":"peterhinch/micropython-filters","owner":"peterhinch","description":"Digital filters impemented in MicroPython's inline ARM Thumb assembler (e.g. Pyboard, RP2).","archived":false,"fork":false,"pushed_at":"2022-01-26T18:29:50.000Z","size":556,"stargazers_count":64,"open_issues_count":3,"forks_count":13,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-08-05T00:07:10.822Z","etag":null,"topics":["assembler","fir-filter","micropython"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/peterhinch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-02-12T08:50:19.000Z","updated_at":"2024-05-26T17:36:05.000Z","dependencies_parsed_at":"2022-09-01T02:41:48.916Z","dependency_job_id":null,"html_url":"https://github.com/peterhinch/micropython-filters","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterhinch%2Fmicropython-filters","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterhinch%2Fmicropython-filters/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterhinch%2Fmicropython-filters/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterhinch%2Fmicropython-filters/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/peterhinch","download_url":"https://codeload.github.com/peterhinch/micropython-filters/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224215504,"owners_count":17274798,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["assembler","fir-filter","micropython"],"created_at":"2024-08-04T00:01:42.614Z","updated_at":"2024-11-12T04:21:07.704Z","avatar_url":"https://github.com/peterhinch.png","language":"Python","funding_links":[],"categories":["Libraries"],"sub_categories":["Mathematics"],"readme":"# Fast Filters for the Pyboard\n\nV0.92 20th December 2021 Updated to improve portability.  \nAuthor: Peter Hinch\n\n# Introduction\n\nThis repository is intended for high speed filtering of integer data acquired\nfrom transducers. For those unfamiliar with digital filtering, please see the\nlast section of this doc.\n\nThe repository comprises two sections.\n\n## Realtime filtering\n\nThis handles a continuous stream of samples and conceptually works as follows\n(pseudocode):\n```python\nfilt = MyFilter(args)\nwhile True:\n    s = MyTransducer.get()  # Acquire an integer sample\n    t = filt(s)  # Filter it\n    # Use the filtered result\n```\nSupport is provided for moving average and FIR (finite impulse response)\nfiltering. FIR filters may be defined using a web application to provide low\npass, high pass or bandpass characteristics.\n\nThe `fir.py` and `avg.py` modules use ARM Thumb V6 inline assembly language for\nperformance and can run on Pyboards (Thumb V7) and also the Raspberry Pico\n(V6). The `fir_py.py` module is written in pure Python using the Viper emitter\nfor performance. It is therefore portable, but runs at about 33% of the speed\nof the assembler version.\n\nFilter functions may be called from hard ISR's.\n\nThe following images show Bode and Nyquist plots of measured results from a\nPyboard 1.1. The test signal was fed into a Pyboard ADC, with the resultant\nsignal on the DAC being plotted. -60dB is the noise floor of my home-brew\nnetwork analyser.\n\n![Image](./images/lpf_bode.jpg)  \n\n![Image](./images/lpf_nyquist.jpg)  \n\n## Non-realtime filtering\n\nThis processes a set of samples in a buffer, for example processing sample sets\nacquired by `ADC.read_timed()`. It requires the ARMV7 assembler and is\ntherefore restricted to Pyboard and similar targets. The files and docs are in\nthe `non_realtime` directory. See [the docs](./non_realtime/FILT.md).\n\nThe algorithm can be configured for continuous (circular buffer) or\ndiscontinuous sample sets. It can optionally perform decimation. In addition to\nFIR filtering it can be employed for related functions such as convolution,\ncross- and auto-correlation.\n\nThe remainder of this README describes the realtime options.\n\n# Designing the filter\n\nThe first stage in designing a filter is to determine the coefficients. FIR\nfilters can be designed for low pass, high pass, bandpass or band stop\napplications and the site [TFilter](http://t-filter.engineerjs.com/) enables\nthese to be computed. I have provided a utility `coeff_format.py` to simplify\nthe conversion of coefficients into Python code. Use the website cited above\nand set it to provide integer coefficients. Cut and paste the list of\ncoefficients into a file: this will have one integer per line. Then run:\n```bash\npython3 coeff_format.py inputfilename outputfilename.py\n```\nThe result will be Python code defining the array.\n\n## Coefficient order\n\nThe website cited above generates symmetrical (linear phase) sets of\ncoefficients. In other words, for a set of n coefficients,\n`coeff[x] == coeff[n-x]`. For coefficient arrays lacking this symmetry note\nthat the code applies coefficients to samples such that the oldest sample is\nmultiplied by `coeff[0]` and so on with the newest getting `coeff[n]`.\n\n## Scaling\n\nCalculations are based on 32 bit signed arithmetic. Given that few transducers\noffer more than 16 bit precision there is a lot of headroom. Nevertheless\noverflow can occur depending on the coefficient size. The maximum output from\nthe multiplication is `max(data)*max(coeffs)` but the subsequent addition offers\nfurther scope for overflow. In applications with analog output there is little\npoint in creating results with more precision than the output DAC. The `fir()`\nfunction includes scaling by performing an arithmetic right shift on the result\nof each multiplication. This can be in the range of 0-31 bits, although 20 bits\nis a typical maximum.  \nFor an analytical way to determine the minimum scaling required to prevent\noverflow see Appendix 1. The `lpf.py` example applies a scaling of 16 bits to\npreserve the 12 bit resolution of the ADC which is then scaled in Python to\nmatch the DAC.\n\n## Solutions\n\nTwo MicroPython solutions are offered. The choice depends on the platform in\nuse. `fir_py.py` uses the Viper code emitter and should run on any platform.\n`fir.py` uses inline Arm Thumb assembler and will run on hosts using ARM V6 or\nlater. This includes all Pyboards and boards using the Raspberry RP2 chip (e.g.\nthe Raspberry Pico). Using the Assembler version is slightly more inolved but\nit runs about three times faster (on the order of 15μs).\n\n## Portable FIR using Viper\n\nThe `fir_py` module uses a closure to enable the function to retain state\nbetween calls. Usage is as follows:\n```python\nfrom fir_py import create_fir\nfrom array import array\n# 21 tap LPF. Figures from TFilter.\ncoeffs = array('i', (-1318, -3829, -4009, -717, 3359, 2177, -3706, -5613,\n                    4154, 20372, 28471, 20372, 4154, -5613, -3706, 2177,\n                    3359, -717, -4009, -3829, -1318))\nfir = create_fir(coeffs, 0)  # Instantiate fir function. No scaling.\n\nprint(fir(1))\nfor n in range(len(coeffs)+3):\n    print(fir(0))\n```\nThis example simulates an impulse function passing through the filter. The\noutcome simply replays the coefficients followed by zeros once the impulse has\ncleared the filter.\n\nThe `create_fir` function takes the following mandatory positional args:\n 1. `coeffs` A 32 bit integer array of coefficients.\n 2. `shift` The result of each multiplication is shifted right by `shift` bits\n before adding to the result. See Scaling above.\n\nNote that Viper can issue very confusing error messages. If these occur, check\nthe data types passed to `create_fir` and `fir`.\n\n## FIR using ARM Thumb Assembler\n\nIn addition to the coefficient array the Assembler version requires the user to\npass an array to hold the set of samples\n\nThe `fir.fir()` function takes three arguments:\n 1. An integer array of length equal to the number of coeffcients + 3.\n 2. An integer array of coefficients.\n 3. The new data value.\n \nThe function returns an integer which is the current filtered value.  \nThe array must be initialised as follows:\n 1. `data[0]` should be set to len(data).\n 2. `data[1]` is a scaling value in range 0..31: see scaling above.\n 3. Other elements of the data array must be zero.\n\nUsage is along these lines:\n```python\nfrom fir import fir\nfrom array import array\n# 21 tap LPF. Figures from TFilter.\ncoeffs = array('i', (-1318, -3829, -4009, -717, 3359, 2177, -3706, -5613,\n                    4154, 20372, 28471, 20372, 4154, -5613, -3706, 2177,\n                    3359, -717, -4009, -3829, -1318))\nncoeffs = len(coeffs)\ndata = array('i', (0 for _ in range(ncoeffs + 3)))\ndata[0] = ncoeffs\ndata[1] = 0  # No scaling\n\nprint(fir(data, coeffs, 1))\nfor n in range(ncoeffs + 3):\n    print(fir(data, coeffs, 0))\n```\nThis example simulates an impulse function passing through the filter. The\noutcome simply replays the coefficients followed by zeros once the impulse has\ncleared the filter.\n\n## Demo program\n\nThe file `lpf.py` uses a Pyboard as a low pass filter with a cutoff of 40Hz. It\nprocesses an analog input presented on pin X7, filters it, and outputs the\nresult on DAC2 (X6). For convenience the code includes a swept frequency\noscillator with output on DAC1 (X5). By linking X5 and X7 the filtered result\ncan be viewed on X6.\n\nThe filter uses Timer 4 to sample the incoming data at 2KHz.\nThe program generates a swept frequency sine wave on DAC1 and reads it using\nthe ADC on pin X7. The filtered signal is output on DAC2. The incoming signal\nis sampled at 2KHz by means of Timer 4, with the FIR filter operating in the\ntimer's callback handler.\n\nWhen using the oscillator to test filters you may see occasional transients\noccurring in the stopband. These are a consequence of transient frequency\ncomponents caused by the step changes in the oscillator frequency: this can be\ndemonstrated by increasing the delay between frequency changes. Ideally the\noscillator would issue a slow, continuous sweep.\n\nfirtest.py illustrates the FIR operation and computes execution times with\ndifferent sets of coefficients.\n\n## Performance\n\nThese results were measured on a Pyboard 1.1 running firmware V1.17. Times are\nin μs and were measured using `firtest.py` (adapted to run the Viper version).\nThe accuracy of these timings is suspect as they varied between runs - and it\nmakes no sense for the 41 tap filter to run faster than the 21 tap. However\nthey give an indication of performance.\n\n| Taps | Asm | Viper |\n|:----:|:---:|:-----:|\n| 21   | 18  |  33   |\n| 41   |  9  |  32   |\n| 109  | 30  |  93   |\n\n# Moving average\n\nA moving average is a degenerate case of an FIR filter with unity coefficients.\nAs such it can run faster. On the Pyboard 1.1 the moving average takes about\n8μs for a typical set of coefficients.\n\nThe Raspberry Pico ARM V6 assembler doesn't support integer division. A special\nversion `avg_pico.py` runs on the Pico. This offers scaling using a right shift\noperation which produces correct resullts if the number of entries is a power\nof 2. Alternatively with a scaling factor of 0 the result is `N*average` where\n`N` is the number of entries.\n\nOn Pyboards and other ARMV7 targets, the file `avg.py` produces expected values\nfor all `N`. Both versions provide a function `avg`.\n\n## Moving Average Usage\n\nThe `avg` function takes two arguments, or three in the Pico case:\n 1. An integer array of length equal to the no. of entries to average +3.\n 2. The new data value.\n 3. The scaling value (number of bits to shift right) - Pico only.\n\nThe function returns an integer which is the current filtered value.  \nInitially all elements of the data array must be zero, except `data[0]` which\nshould be set to `len(data)`  \nThe test scripts `avgtest.py` and `avgtest_pico.py` illustrate its operation.\n```python\nfrom avg_pico import avg\n```\n\n# Absolute Beginners\n\nData arriving from transducers often needs to be filtered to render it useful.\nReasons include reducing noise (random perturbations) in the data, isolating a\nparticular signal or shaping the response to sudden changes in the data value.\nA common approach to reducing noise is to take a moving average of the last N\nsamples. While this is computationally simple and hence fast, it is a\nrelatively crude form of filtering because the oldest sample in the set has the\nsame weight as the most recent. This is often non-optimal.\n\nFIR (finite impulse response) filters can be viewed as an extension of the\nmoving average concept where each sample is allocated a different weight\ndepending on its age. These weights are defined by a set of coefficients. The\nresult is calculated by multiplying each sample by its coefficient before\nadding them; a moving average corresponds to the situation where all\ncoefficients are set to 1. By adjusting the coefficients you can alter the\nrelative weights of the data values, with the most recent having a different\nweight to the next most recent, and so on.\n\nIn practice FIR filters can be designed to produce a range of filter types:\nlow pass, high pass, bandpass, band stop and so on. They can also be tailored\nto produce a specific response to sudden changes (impulse response). The\nprocess of computing the coefficients is complex, but the link above provides a\nsimple GUI approach. Set the application to produce 16 or 32 bit integer\nvalues, set your desired characteristics and press \"Design Filter\". Then\nproceed as suggested above to convert the results to Python code.\n\nThe term \"finite impulse response\" describes the response of a filter to a\nbrief (one sample) pulse. In an FIR filter with N coefficients the response\ndrops to zero after N samples because the impulse has passed through the\nfilter. This contrasts with an IIR (infinite impulse response) filter where the\nresponse theoretically continues forever. Analog circuits such as a CR network\ncan have an IIR response, as do some digital filters.\n\n# Appendix 1 Determining scaling\n\nThe following calculation determines the number of bits required to represent\nthe outcome when a worst-case signal passes through an FIR filter. The ADC is\nassumed to be biassed for symmetrical output. The worst-case signal, at some\namount of shift through the filter, has maximum positive excursion matching\npositive coefficients and maximum negative excursion matching negative\ncoefficients. This will produce the largest possible positive sum. By symmetry\na negative result of equal magnitude could result, where a negative signal\nmatches a positive coefficient and vice-versa.\n\nThere are two places where overflow can occur: in the multiplication and in the\nsubsequent addition. The former cannot be compensated: the coefficients need to\nbe reduced in size. The latter can be compensated by performing a right shift\nafter the multiplication, and the Assembler routine provides for this.\n\nThe following code calculates the number of bits required to accommodate this\nresult. On 32-bit platforms, small integers occupy 31 bits holding values up to\n+-2^30. Consequently if this script indicates that 33 bits are required,\nscaling of at least 2 bits must be applied to guarantee no overflow.\n\n```python\nfrom math import log\n\n# Return no. of bits to contain a positive integer\ndef nbits(n : int) -\u003e int:\n    return int(log(n) // log(2)) + 1\n\ndef get_shift(coeffs : list, adcbits : int =12):\n    # Assume ADC is biassed for equal + and - excursions\n    maxadc = (2 ** (adcbits - 1) - 1)  # 2047 for 12 bit ADC\n    lv = sorted([abs(x) * maxadc for x in coeffs], reverse=True)\n    # Add 1 to allow for equal negative swing\n    print(\"Max no. of bits for multiply\", nbits(lv[0]) + 1)\n    print(\"Max no. of bits for sum of products\", nbits(sum(lv) + 1))\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterhinch%2Fmicropython-filters","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeterhinch%2Fmicropython-filters","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterhinch%2Fmicropython-filters/lists"}