{"id":20346658,"url":"https://github.com/arthurfdlr/shorttime_fouriertransform","last_synced_at":"2025-09-22T08:23:23.138Z","repository":{"id":112737081,"uuid":"346865696","full_name":"ArthurFDLR/ShortTime_FourierTransform","owner":"ArthurFDLR","description":null,"archived":false,"fork":false,"pushed_at":"2021-05-29T16:49:47.000Z","size":72266,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-04T15:48:30.042Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ArthurFDLR.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-11T23:10:55.000Z","updated_at":"2024-07-15T21:08:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"ef8a0937-a4c3-4976-a141-581ec291f9a0","html_url":"https://github.com/ArthurFDLR/ShortTime_FourierTransform","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ArthurFDLR/ShortTime_FourierTransform","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArthurFDLR%2FShortTime_FourierTransform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArthurFDLR%2FShortTime_FourierTransform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArthurFDLR%2FShortTime_FourierTransform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArthurFDLR%2FShortTime_FourierTransform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ArthurFDLR","download_url":"https://codeload.github.com/ArthurFDLR/ShortTime_FourierTransform/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArthurFDLR%2FShortTime_FourierTransform/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276369381,"owners_count":25630201,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-22T02:00:08.972Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T22:13:45.955Z","updated_at":"2025-09-22T08:23:23.118Z","avatar_url":"https://github.com/ArthurFDLR.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align = \"center\"\u003e The Short-Time Fourier Transform \u003c/h1\u003e\n\n[![MIT License][license-shield]][license-url]\n[![LinkedIn][linkedin-shield]][linkedin-url]\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\".\\spectrograms\\fftInPlace\\SAINtJHN_AllAroundTheRoses_Intro_spectrogram_fftin.png\" alt=\"SAINtJHN-AllAroundTheRoses introductionm spectrogram\" width=\"80%\" style=\"border-radius: 5px;\"\u003e\n\u003c/p\u003e\n\nThis work focuses on implementing one of the most commonly used time-frequency decomposition tools: the Short-Time Fourier Transform (STFT). We have already extensively covered the STFT in our course summary reports, so we will not dive into the mathematical details. However, we will compare the performances of a naive implementation and one of many optimized algorithms. The naive transform is based on the direct implementation of the mathematical formula of the discrete Fourier transform. The most commonly used optimization technique is the Cooley–Tukey algorithm. We will use this fast Fourier transform to implement an accelerated version of the STFT.\n\n- [Algorithms description and implementation](#algorithms-description-and-implementation)\n  - [The DFT-based STFT](#the-dft-based-stft)\n  - [The Cooley–Tukey algorithm](#the-cooleytukey-algorithm)\n  - [Algorithms integration](#algorithms-integration)\n- [Results](#results)\n  - [Spectrograms](#spectrograms)\n  - [Performance](#performance)\n\n## Algorithms description and implementation\n\n### The DFT-based STFT\n\nThe STFT algorithm lies on the Fourier transform computation over short segments of a signal to analyze. This produces the frequency spectrum on each segment which can be placed one after the other to generate a time-frequency representation. While any windowing function can be used, we will consider a rectangular window to keep our program simple and efficient. Thus, there are two main parameters to our algorithm: the window width and the number of steps between two segments. Naturally, the Fourier transform generates complex values.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\".\\.github\\iscola_stft.png\" alt=\"Short-time Fourier Transform of a linear chirp\" width=\"70%\"\u003e\n\u003c/p\u003e\n\nOur program is implemented in C++. All signals are stored as vectors from the standard library. Besides, the complex numbers C++ library is also leverage to simplify the implementation. Finally, the STFT results are stored as a vector of complex value vectors. The behavior of the STFT is thus straightforward to implement considering a Fourier transform function _Fft::dft_.\n\n\u003cdetails\u003e\n    \u003csummary\u003eSee implementation\u003c/summary\u003e\n\n```c++\ntypedef std::vector\u003cstd::complex\u003cfloat\u003e\u003e vec_complex;\ntypedef std::vector\u003cfloat\u003e vec_real;\ntypedef std::vector\u003cvec_complex\u003e mat_complex;\n\t\nFft::mat_complex Fft::stft_dft(Fft::vec_real \u0026vec, size_t window_size, size_t window_step){\n\tFft::mat_complex spectrogram = Fft::mat_complex();\n\tfor (size_t begin = 0; begin \u003c vec.size() - window_size; begin+=window_step)\n\t{\n\t\tspectrogram.push_back(Fft::dft(vec, begin, window_size));\n\t}\n\treturn spectrogram;\n}\n```\n\u003c/details\u003e\n\nWe now need a DFT function to complete our naive STFT implementation. The kᵗʰ element of the DFT of a discrete signal xₜ of size _N_ is defined as\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\".\\.github\\eq_1.png\" alt=\"Short-time Fourier Transform of a linear chirp\" width=\"170px\"\u003e\n\u003c/p\u003e\n\u003c!--- $$ X_{k} = \\sum_{t=0}^{N-1}x_{t}e^{-j2\\pi k t/N} $$ --\u003e\n\nOur function must iterate over each frequency step _k_ and, in an inner loop, iterate over a time step _t_ to perform the summation.\n\n\u003cdetails\u003e\n    \u003csummary\u003eSee implementation\u003c/summary\u003e\n\n```c++\nFft::vec_complex Fft::dft(const Fft::vec_complex \u0026signal, size_t vec_begin, size_t vec_size) {\n\tFft::vec_complex output;\n\tfor (size_t k = 0; k \u003c vec_size/2.; k++) {\n\t\tstd::complex\u003cfloat\u003e sum = 0.;\n\t\tfor (size_t t = 0; t \u003c vec_size; t++) {\n\t\t\tfloat angle = 2 * PI * t * k / vec_size;\n\t\t\tsum += signal[t+vec_begin] * std::exp(std::complex\u003cfloat\u003e(0, -angle));\n\t\t}\n\t\toutput.push_back(sum);\n\t}\n\treturn output;\n}\n```\n\u003c/details\u003e\n\nWe now have a functional STFT transform. However, it is awfully slow. The time complexity of the DFT operation is O(N²). The decomposition of a ∼9s requires a computation time of 4 minutes! We will dive deeper in the results and performance later on. Although, first, let us introduce our optimization algorithm.\n\n### The Cooley–Tukey algorithm\n\nThe computation of each term of the DFT is very close. They only differ by a complex angle in their summation. The Cooley–Tukey algorithm play with this characteristic to reduce the number of operation through the principle of dividing to conquer. The simplest form of this method is based on a binary separation, also called radix-2 decimation-in-time. Indeed, we can split our summation in an odd-indexed part and an even-indexed part of the signal, ∀ k ∈ [0, N−1],\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\".\\.github\\eq_2.png\" alt=\"Short-time Fourier Transform of a linear chirp\" width=\"470px\"\u003e\n\u003c/p\u003e\n\u003c!--- \n\\begin{equation*}\n\\begin{split}\nX_k\n\u0026= \\sum_{t=0}^{\\frac{N}{2}-1} x_{2t} \\cdot e^{-j 2\\pi k (2t) / N}  +   \\sum_{t=0}^{\\frac{N}{2}-1} x_{2t+1} \\cdot e^{-j 2\\pi k (2t+1) / N} \\\\[2mm]\n\u0026= \\underbrace{\\sum_{t=0}^{\\frac{N}{2}-1} x_{2t} \\cdot e^{-j 2\\pi k (2t) / N}}_{\\mathrm{DFT\\;of\\;even-indexed\\;part\\;of\\;} x_t}\n+ e^{-\\frac{j2\\pi}{N}k}\n\\underbrace{\\sum_{t=0}^{\\frac{N}{2}-1} x_{2t+1} \\cdot e^{-j 2\\pi k (2t+1) / N}}_{\\mathrm{DFT\\;of\\;odd-indexed\\;part\\;of\\;} x_t} \\\\[2mm]\n\u0026=  E_k + e^{-\\frac{j2\\pi}{N}k} O_k\n\\end{split}\n\\end{equation*}\n --\u003e\n\nSimilarly, we can calculate, ∀ k ∈ [0, N−1],\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\".\\.github\\eq_3.png\" alt=\"Short-time Fourier Transform of a linear chirp\" width=\"180px\"\u003e\n\u003c/p\u003e\n\u003c!-- $$ X_{k+ \\frac{N}{2}} =  E_k - e^{-\\frac{j2\\pi}{N}k} O_k $$ --\u003e\n\nThis set of formulas is perfectly adapted to a simple recursive implementation.\n\n\u003cdetails\u003e\n    \u003csummary\u003eSee implementation\u003c/summary\u003e\n\n```c++\nvoid Fft::transform_radix2(Fft::vec_complex \u0026 x) {\n    int n = x.size();\n    if (n == 1)\n        return;\n\n    Fft::vec_complex x_even(n / 2), x_odd(n / 2);\n    for (int i = 0; 2 * i \u003c n; i++) {\n        x_even[i] = x[2*i];\n        x_odd[i] = x[2*i+1];\n    }\n    Fft::transform_radix2(x_even);\n    Fft::transform_radix2(x_odd);\n\n    std::complex\u003cfloat\u003e w = 1.;\n    std::complex\u003cfloat\u003e w_n = std::exp(std::complex\u003cfloat\u003e(0, - 2. * PI / n));\n    for (int k = 0; 2 * k \u003c n; k++) {\n        x[k] = x_even[k] + w * x_odd[k];\n        x[k + n/2] = x_even[k] - w * x_odd[k];\n        w *= w_n;\n    }\n}\n```\n\u003c/details\u003e\n\nNote that given the algorithm's structure, the size of the input signal necessarily is a power of 2. This is not an issue given the STFT use-case. Indeed, we can easily choose an adequate window size in power of two.\n\nThis algorithm can also be implemented using in-place computation. The creation of the temporary vectors xₑ and xₒ can thus be avoided. This can heavily improve the space complexity of the program. In addition, while the time complexity O(N⋅ log(N)) theoretically remains unchanged, we would certainly observe a performance improvement as the vector creation process has linear time complexity. Finally, an in-place computation allows to calculate and store one for all the nᵗʰ roots of unity wₙ. However, the in-place implementation implies complicated operation, which we will not discuss here. For the sake of efficiency comparison, we will use [this open-source](https://www.nayuki.io/page/free-small-fft-in-multiple-languages) in-place implementation of the radix-2 transformation.\n\n### Algorithms integration\n\nThe algorithms and partial source code presented above are all implemented as part of a simple C++ library called \\textbf{Fft}. The functions defined in the associated scope are included into our main C++ file \\textit{compute\\_spectrogram.cpp}. Besides, the open-source library AudioFile from Adam Stark is used to digest WAV files as the subject of our time-frequency analysis. The imported file sampling rate is crucial to define the temporal step and the size of the frequency bins of our time-frequency representation. However, we also have to define the window size _M_, and the window steps _R_ of the representation.\n\nWe already discussed in length the uncertainty principles of the short-time Fourier transform in our course summaries. Proper window size is critical for a good representation. Wave files are usually sampled at a rate of fₛ = 44.1 kHz. After some experimentation, we found a window size of M=2¹⁰ = 1024 yields good results. The window step is arbitrarily set to half the window size R=M/2.\n\nAfter computation, the absolute value of the STFT result is stored in a CSV file. The first line contains the temporal step and the frequency step of the discrete representation. A Python script then generates a proper spectrogram visualization from the data provided through the CSV file. The whole process is streamlined using a _makefile_ which allows using a single command to generate a spectrogram. From the main directory,\n\n``` make file=./audio\\_files/my\\_audio.wav ```\n\n## Results\n\nWe can now test and compare our three algorithms on several audio files available in [./audio_files](./audio_files/). All audio files are encoded on 32-bit float at a sampling rate of 41.1 kHz. We have selected a human natural speech sample, a couple of electronic pieces of music with specific characteristics, and a nature audio recording of birds. \n\n### Spectrograms\n\nThe spectrograms shown below are produced using our FFT algorithm (out-of-place computation). However, the spectrogram representation is (fortunately) strictly independent to the Fourier transform algorithm used. Still, all spectrogram produced during this study are available in [./spectrograms](./spectrograms/)\n\n\u003cul\u003e\n\u003cli\u003e\n\"Are we headed towards a future where an AI will be able to out-think us in every way? Then the answer is unequivocally yes.\" - Elon Musk\n    \nThe time-frequency analysis of natural language speech is one of the most powerful techniques for transcription. Heavy work is currently put into the application of CNN on such representation. We will dive deeper into this technique in our research project. \n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\".\\spectrograms\\fftInPlace\\ConsciousnessAI_Musk_spectrogram_fftin.png\" alt=\"SAINtJHN-AllAroundTheRoses introductionm spectrogram\" width=\"80%\" style=\"border-radius: 5px;\"\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\nIntroduction of the song All Around The Roses from SAINtJHN:\n    \nThis xylophone sample perfectly illustrates the use of the time-frequency representation to decompose music instruments notes in a fundamental frequency and all the harmonics that compose its timbre.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\".\\spectrograms\\fftInPlace\\SAINtJHN_AllAroundTheRoses_Intro_spectrogram_fftin.png\" alt=\"SAINtJHN-AllAroundTheRoses introductionm spectrogram\" width=\"80%\" style=\"border-radius: 5px;\"\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\nIntroduction of the song Sacrificial from Rezz:\n    \nWe can observe the frequency crescendo of this song until the introductory drop around 5s.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\".\\spectrograms\\fftInPlace\\Sacrificial_Rezz_Intro_spectrogram_fftin.png\" alt=\"SAINtJHN-AllAroundTheRoses introductionm spectrogram\" width=\"80%\" style=\"border-radius: 5px;\"\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\nBird Sounds Spectacular : Morning Bird Song from Paul Dinning - Wildlife in Cornwall:\n    \nMultiple birds and background noise of nearby rad traffic can be heard simultaneously in this sample. Nevertheless, we can identify patterns associated with specific bird songs. Most notably, one is repeated at 13s and 18s. The road noise is contained in the lower frequency range.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\".\\spectrograms\\fftInPlace\\birds_spectrogram_fftin.png\" alt=\"SAINtJHN-AllAroundTheRoses introductionm spectrogram\" width=\"80%\" style=\"border-radius: 5px;\"\u003e\n\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\n### Performance\n\n| | FFT in-place | FFT out-of-place | DFT |\n| -: | :-: | :-: | :-: |\nSAINtJHN - All Around The Roses (11s) | 313  | 853  | 60913  |\nRezz - Sacrificial (8s) | 241  | 648  | 46159  |\nMorning Bird Song (39s) | 1152  | 3069  | 2178394  |\nElon Musk about Consciousness \\| AI (9s) | 268  | 739  | 52204  |\n\nAs expected, the radix-2 algorithm allows a way faster computation. Given the better time complexity, the execution time reduction factor is proportional to the logarithm of the analyzed signal's length. Finally, we can note that the gain in performance due to the in-place computation is significant. It reduces the execution time of a factor of 3 in comparison to the out-of-place algorithm.\n\n\u003c!-- MARKDOWN LINKS \u0026 IMAGES --\u003e\n[license-shield]: https://img.shields.io/github/license/ArthurFDLR/ShortTime_FourierTransform?style=for-the-badge\n[license-url]: https://github.com/ArthurFDLR/ShortTime_FourierTransform/blob/master/LICENSE\n[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge\u0026logo=linkedin\u0026colorB=555\n[linkedin-url]: https://linkedin.com/in/arthurfdlr/","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farthurfdlr%2Fshorttime_fouriertransform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farthurfdlr%2Fshorttime_fouriertransform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farthurfdlr%2Fshorttime_fouriertransform/lists"}