{"id":24098484,"url":"https://github.com/matlab-deep-learning/fourier-neural-operator","last_synced_at":"2025-06-26T16:33:57.756Z","repository":{"id":118344217,"uuid":"588071252","full_name":"matlab-deep-learning/fourier-neural-operator","owner":"matlab-deep-learning","description":null,"archived":false,"fork":false,"pushed_at":"2023-01-12T09:25:50.000Z","size":45,"stargazers_count":19,"open_issues_count":1,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-02T05:02:25.988Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"MATLAB","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matlab-deep-learning.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"License.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-12T08:59:42.000Z","updated_at":"2025-03-26T16:47:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"bf674753-90bb-414d-bc3a-ee77194b658c","html_url":"https://github.com/matlab-deep-learning/fourier-neural-operator","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/matlab-deep-learning/fourier-neural-operator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Ffourier-neural-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Ffourier-neural-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Ffourier-neural-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Ffourier-neural-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matlab-deep-learning","download_url":"https://codeload.github.com/matlab-deep-learning/fourier-neural-operator/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Ffourier-neural-operator/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262102650,"owners_count":23259303,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-10T14:46:13.942Z","updated_at":"2025-06-26T16:33:57.747Z","avatar_url":"https://github.com/matlab-deep-learning.png","language":"MATLAB","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fourier Neural Operator\n\n[![View \u003cFile Exchange Title\u003e on File Exchange](https://www.mathworks.com/matlabcentral/images/matlab-file-exchange.svg)](https://www.mathworks.com/matlabcentral/fileexchange/####-file-exchange-title)  \n\u003c!-- Add this icon to the README if this repo also appears on File Exchange via the \"Connect to GitHub\" feature --\u003e \n\nThe Fourier Neural Operator (FNO) [1] is a neural operator with an integral kernel parameterized in Fourier space. This allows for an expressive and efficient architecture. Applications of the FNO include weather forecasting and, more generically, finding efficient solutions to the Navier-Stokes equations which govern fluid flow.\n\n## Setup \nAdd `fno` directory to the path.\n\n```matlab\naddpath(genpath('fno'));\n```\n\n## Requirements\n\nRequires:\n- [MATLAB](https://www.mathworks.com/products/matlab.html) (R2021b or newer)\n- [Deep Learning Toolbox\u0026trade;](https://www.mathworks.com/products/deep-learning.html)\n\n## References\n[1] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima\nAnandkumar. Fourier Neural Operator for Parametric Partial Differential Equations. In International Conference on\nLearning Representations (ICLR), 2021a. (https://openreview.net/forum?id=c8P9NQVtmnO)\n\n# Example: Fourier Neural Operator for 1d Burgers' Equation\n\nIn this example we apply the Fourier Neural Operator to learn the one-dimensional Burgers' equation with the following definition:\n\n\u003e \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;\\frac{\\partial\u0026space;u}{\\partial\u0026space;t}+u\\frac{\\partial\u0026space;u}{\\partial\u0026space;x}=\\frac{1}{Re}\\frac{\\partial^2\u0026space;u}{\\partial\u0026space;x^2\u0026space;}\"/\u003e,    \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;x\\in\u0026space;(0,1),~t\\in\u0026space;(0,1]\"/\u003e\n\n\u003e \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;u(x,0)=u_0\u0026space;(x)\"/\u003e,    \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;x\\in\u0026space;(0,1)\"/\u003e\n\nwhere \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;u=u\\left(x,t\\right)\"/\u003e and \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;\\textrm{Re}\"/\u003e is the Reynolds number. Periodic boundary conditions are imposed across the spatial domain. We learn the operator mapping the initial condition \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;u_0\"/\u003e to the solution at time \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;t=1\"/\u003e: \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;u_0\u0026space;\\longmapsto\u0026space;u\\left(x,1\\right)\"/\u003e.\n\n## Data preparation\n\nWe use the `burgers_data_R10.mat`, which contains initial velocities \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;u_0\"/\u003e and solutions \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;u\\left(x,1\\right)\"/\u003e of the Burgers' equation. We then use these as training inputs and targets respectively. The network inputs include the spatial domain \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;x=\\left(0,1\\right)\"/\u003e at the desired discretization. In this example we choose a grid size of \u003cimg src=\"https://latex.codecogs.com/gif.latex?\\inline\u0026space;h=2^{10}\"/\u003e.\n\n```matlab:Code\n% Setup.\naddpath(genpath('fno'));\n\n% Download training data.\ndataDir = fullfile('data');\nif ~isfolder(dataDir)\n    mkdir(dataDir);\nend\ndataFile = fullfile(dataDir,'burgers_data_R10.mat');\nif ~exist(dataFile, 'file')\n    location = 'https://ssd.mathworks.com/supportfiles/nnet/data/burgers1d/burgers_data_R10.mat';\n    websave(dataFile, location); \nend\ndata = load(dataFile, 'a', 'u');\nx = data.a;\nt = data.u;\n\n% Specify the number of observations in training and test data, respectively. \nnumTrain = 1e3;\nnumTest = 1e2;\n\n% Specify grid size and downsampling factor.\nh = 2^10;\nn = size(x,2);\nns = floor(n./h);\n\n% Downsample the data for training.\nxTrain = x(1:numTrain, 1:ns:n);\ntTrain = t(1:numTrain, 1:ns:n);\nxTest = x(end-numTest+1:end, 1:ns:n);\ntTest = t(end-numTest+1:end, 1:ns:n);\n\n% Define the grid over the spatial domain x.\nxmax = 1;\nxgrid = linspace(0, xmax, h);\n\n% Combine initial velocities and spatial grid to create network\n% predictors.\nxTrain = cat(3, xTrain, repmat(xgrid, [numTrain 1]));\nxTest = cat(3, xTest, repmat(xgrid, [numTest 1]));\n```\n\n## Define network architecture\n\nHere we create a `dlnetwork` for the Burgers' equation problem. The network accepts inputs of dimension `[h 2 miniBatchSize]`, and returns outputs of dimension `[h 1 miniBatchSize]`. The network consists os multiple blocks which combine spectral convolution with regular, linear convolution. The convolution in Fourier space filters out higher order oscillations in the solution, while the linear convolution learns local correlations.\n\n```matlab:Code\nnumModes = 16;\nwidth = 64;\n\nlg = layerGraph([ ...\n    convolution1dLayer(1, width, Name='fc0')\n    \n    spectralConvolution1dLayer(width, numModes, Name='specConv1')\n    additionLayer(2, Name='add1')\n    reluLayer(Name='relu1')\n    \n    spectralConvolution1dLayer(width, numModes, Name='specConv2')\n    additionLayer(2, Name='add2')\n    reluLayer(Name='relu2')\n    \n    spectralConvolution1dLayer(width, numModes, Name='specConv3')\n    additionLayer(2, Name='add3')\n    reluLayer(Name='relu3')\n    \n    spectralConvolution1dLayer(width, numModes, Name='specConv4')\n    additionLayer(2, Name='add4')\n    \n    convolution1dLayer(1, 128, Name='fc5')\n    reluLayer(Name='relu5')\n    convolution1dLayer(1, 1, Name='fc6')\n    ]);\n\nlg = addLayers(lg, convolution1dLayer(1, width, Name='fc1'));\nlg = connectLayers(lg, 'fc0', 'fc1');\nlg = connectLayers(lg, 'fc1', 'add1/in2');\n\nlg = addLayers(lg, convolution1dLayer(1, width, Name='fc2'));\nlg = connectLayers(lg, 'relu1', 'fc2');\nlg = connectLayers(lg, 'fc2', 'add2/in2');\n\nlg = addLayers(lg, convolution1dLayer(1, width, Name='fc3'));\nlg = connectLayers(lg, 'relu2', 'fc3');\nlg = connectLayers(lg, 'fc3', 'add3/in2');\n\nlg = addLayers(lg, convolution1dLayer(1, width, Name='fc4'));\nlg = connectLayers(lg, 'relu3', 'fc4');\nlg = connectLayers(lg, 'fc4', 'add4/in2');\n\nnumInputChannels = 2;\nXInit = dlarray(ones([h numInputChannels 1]), 'SCB');\nnet = dlnetwork(lg, XInit);\n\nanalyzeNetwork(net)\n```\n\n## Training options\n\nThe network is trained using the standard SGDM algorithm, where the learn rate is decreased every `stepSize` iterations.\n\n```matlab:Code\nexecutionEnvironment = \"gpu\";\n\nbatchSize = 20;\nlearnRate = 1e-3;\nmomentum = 0.9;\n\nnumEpochs = 20;\nstepSize = 100;\ngamma = 0.5;\nexpNum = 1;\ncheckpoint = false;\nexpDir = sprintf( 'checkpoints/run%g', expNum );\nif ~isfolder( expDir ) \u0026\u0026 checkpoint\n    mkdir(expDir)\nend\n\nvel = [];\ntotalIter = 0;\n\nnumTrain = size(xTrain,1);\nnumIterPerEpoch = floor(numTrain./batchSize);\n```\n\n## Training loop\n\nTrain the network.\n\n```matlab:Code\nif executionEnvironment == \"gpu\" \u0026\u0026 canUseGPU\n    xTrain = gpuArray(xTrain);\n    xTest = gpuArray(xTest);\nend\n\nstart = tic;\nfigure;\nclf\nlineLossTrain = animatedline('Color', [0 0.4470 0.7410]);\nlineLossTest = animatedline('Color', 'k', 'LineStyle', '--');\nylim([0 inf])\nxlabel(\"Iteration\")\nylabel(\"Loss\")\ngrid on\n\n% Compute initial validation loss.\ny = net.predict( dlarray(xTest, 'BSC') );\nyTest = extractdata(permute(stripdims(y), [3 1 2]));\nrelLossTest = relativeL2Loss(yTest , tTest);\naddpoints(lineLossTest, 0, double(relLossTest/size(xTest,1)))\n\n% Main loop.\nlossfun = dlaccelerate(@modelLoss);\nfor epoch = 1:numEpochs\n    % Shuffle the data.\n    dataIdx = randperm(numTrain);\n    \n    for iter = 1:numIterPerEpoch\n        % Get mini-batch data.\n        batchIdx = (1:batchSize) + (iter-1)*batchSize;\n        idx = dataIdx(batchIdx);\n        X = dlarray( xTrain(batchIdx, :, :), 'BSC' );\n        T = tTrain(batchIdx, :);\n        \n        % Compute loss and gradients.\n        [loss, dnet] = dlfeval(lossfun, X, T, net);\n\n        % Update model parameters using SGDM update rule.\n        [net, vel] = sgdmupdate(net, dnet, vel, learnRate, momentum);\n\n        % Plot training progress.\n        totalIter = totalIter + 1;\n        D = duration(0,0,toc(start),'Format','hh:mm:ss');\n        addpoints(lineLossTrain,totalIter,double(extractdata(loss/batchSize)))\n        title(\"Epoch: \" + epoch + \", Elapsed: \" + string(D))\n        drawnow\n        \n        % Learn rate scheduling.\n        if mod(totalIter, stepSize) == 0\n            learnRate = gamma.*learnRate;\n        end\n    end\n    % Compute validation loss and MSE.\n    y = net.predict( dlarray(xTest, 'BSC') );\n    yTest = extractdata(permute(stripdims(y), [3 1 2]));\n    relLossTest = relativeL2Loss( yTest , tTest );\n    mseTest = mean( (yTest(:) - tTest(:)).^2 );\n    \n    % Display progress.\n    D = duration(0,0,toc(start),'Format','hh:mm:ss');\n    numTest = size(xTest, 1);\n    fprintf('Epoch = %g, train loss = %g, val loss = %g, val mse = %g, total time = %s. \\n', ...\n        epoch, extractdata(loss)/batchSize, relLossTest/numTest, mseTest/numTest, string(D));\n    addpoints(lineLossTest, totalIter, double(relLossTest/numTest))\n    \n    % Checkpoints.\n    if checkpoint\n        filename = sprintf('checkpoints/run%g/epoch%g.mat', expNum, epoch);\n        save(filename, 'net', 'epoch', 'vel', 'totalIter', 'relLossTest', 'mseTest', 'learnRate');\n    end\nend\n```\n\n```text:Output\nEpoch = 1, train loss = 0.226405, val loss = 0.175389, val mse = 7.73286e-05, total time = 00:00:13. \nEpoch = 2, train loss = 0.153691, val loss = 0.145805, val mse = 5.99213e-05, total time = 00:00:22. \nEpoch = 3, train loss = 0.0923258, val loss = 0.0904608, val mse = 2.49174e-05, total time = 00:00:27. \nEpoch = 4, train loss = 0.102122, val loss = 0.0639219, val mse = 1.43723e-05, total time = 00:00:32. \nEpoch = 5, train loss = 0.0346076, val loss = 0.0393621, val mse = 9.33419e-06, total time = 00:00:36. \nEpoch = 6, train loss = 0.0361029, val loss = 0.032303, val mse = 7.10724e-06, total time = 00:00:45. \nEpoch = 7, train loss = 0.0270364, val loss = 0.0296161, val mse = 6.43696e-06, total time = 00:00:50. \nEpoch = 8, train loss = 0.0263171, val loss = 0.0283881, val mse = 5.92292e-06, total time = 00:00:54. \nEpoch = 9, train loss = 0.0248211, val loss = 0.0261364, val mse = 5.54218e-06, total time = 00:00:58. \nEpoch = 10, train loss = 0.0243392, val loss = 0.0253596, val mse = 5.32946e-06, total time = 00:01:03. \nEpoch = 11, train loss = 0.0236119, val loss = 0.0250861, val mse = 5.22886e-06, total time = 00:01:07. \nEpoch = 12, train loss = 0.023318, val loss = 0.024752, val mse = 5.12552e-06, total time = 00:01:11. \nEpoch = 13, train loss = 0.0230901, val loss = 0.0243369, val mse = 5.04185e-06, total time = 00:01:16. \nEpoch = 14, train loss = 0.0229644, val loss = 0.0241713, val mse = 4.99882e-06, total time = 00:01:20. \nEpoch = 15, train loss = 0.0228391, val loss = 0.0240904, val mse = 4.99516e-06, total time = 00:01:25. \nEpoch = 16, train loss = 0.022768, val loss = 0.0240143, val mse = 4.97173e-06, total time = 00:01:29. \nEpoch = 17, train loss = 0.0228152, val loss = 0.023916, val mse = 4.95474e-06, total time = 00:01:33. \nEpoch = 18, train loss = 0.022787, val loss = 0.0238792, val mse = 4.94643e-06, total time = 00:01:38. \nEpoch = 19, train loss = 0.0227602, val loss = 0.023865, val mse = 4.93665e-06, total time = 00:01:42. \nEpoch = 20, train loss = 0.0227464, val loss = 0.0238451, val mse = 4.93358e-06, total time = 00:01:46. \n```\n\n![figure_0.png](images/figure_0.png)\n\n## Test on unseen, higher resolution data\n\nHere we take the trained network and test on unseen data with a higher spatial resolution than the training data. This is an example of zero-shot super-resolution.\n\n```matlab:Code\ngridHighRes = linspace(0, xmax, n);\n\nidxToPlot = numTrain+(1:4);\nfigure;\nfor p = 1:4\n    xn = dlarray(cat(1, x(idxToPlot(p),:), gridHighRes),'CSB');\n    yn = predict(net, xn);\n\n    subplot(2, 2, p)\n    plot(gridHighRes, t(idxToPlot(p),:)), hold on, plot(gridHighRes, extractdata(yn))\n    axis tight\n    xlabel('x')\n    ylabel('U')\nend\n```\n\n![figure_1.png](images/figure_1.png)\n\n### Helper functions\n\n```matlab:Code\nfunction [loss, grad] = modelLoss(x, t, net)\ny = net.forward(x);\ny = permute(stripdims(y), [3 1 2]);\ny = stripdims(y);\n\nloss = relativeL2Loss(y, t);\n\ngrad = dlgradient(loss, net.Learnables);\nend\n\nfunction loss = relativeL2Loss(y, t)\ndiffNorms = normFcn( (y - t) );\ntNorms = normFcn( t );\n\nloss = sum(diffNorms./tNorms, 1);\nend\n\nfunction n = normFcn(x)\nn = sqrt( sum(x.^2, 2) );\nend\n```\n\nCopyright 2022-2023 The MathWorks, Inc.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatlab-deep-learning%2Ffourier-neural-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatlab-deep-learning%2Ffourier-neural-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatlab-deep-learning%2Ffourier-neural-operator/lists"}