{"id":19761919,"url":"https://github.com/pswpswpsw/nif","last_synced_at":"2025-04-30T14:30:42.651Z","repository":{"id":47647068,"uuid":"460537754","full_name":"pswpswpsw/nif","owner":"pswpswpsw","description":"A library for dimensionality reduction on spatial-temporal PDE","archived":false,"fork":false,"pushed_at":"2024-04-08T22:24:38.000Z","size":86530,"stargazers_count":64,"open_issues_count":1,"forks_count":13,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-11T11:10:12.644Z","etag":null,"topics":["deep-learning","dimensionality-reduction","pde","reduced-order-modeling","tensorflow"],"latest_commit_sha":null,"homepage":"https://pswpswpsw.github.io/nif","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-2.1","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pswpswpsw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-02-17T17:27:11.000Z","updated_at":"2025-03-13T04:37:20.000Z","dependencies_parsed_at":"2024-04-08T01:37:24.002Z","dependency_job_id":"65366181-14ff-4051-bb8a-395522bae0b4","html_url":"https://github.com/pswpswpsw/nif","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pswpswpsw%2Fnif","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pswpswpsw%2Fnif/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pswpswpsw%2Fnif/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pswpswpsw%2Fnif/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pswpswpsw","download_url":"https://codeload.github.com/pswpswpsw/nif/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251721381,"owners_count":21632827,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","dimensionality-reduction","pde","reduced-order-modeling","tensorflow"],"created_at":"2024-11-12T03:42:47.197Z","updated_at":"2025-04-30T14:30:37.620Z","avatar_url":"https://github.com/pswpswpsw.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Neural Implicit Flow (NIF): mesh-agnostic dimensionality reduction\n![example workflow](https://github.com/pswpswpsw/nif/workflows/Tests/badge.svg)\n[![License](https://img.shields.io/github/license/pswpswpsw/nif)](https://github.com/pswpswpsw/nif/blob/main/LICENSE)[![DOI](https://zenodo.org/badge/460537754.svg)](https://zenodo.org/badge/latestdoi/460537754)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./misc/myimage.gif\" alt=\"animated\" /\u003e\n\u003c/p\u003e\n\n- NIF is a mesh-agnostic dimensionality reduction paradigm for parametric spatial temporal fields. For decades, dimensionality reduction (e.g., proper orthogonal decomposition, convolutional autoencoders) has been the very first step in reduced-order modeling of any large-scale spatial-temporal dynamics. \n\n- Unfortunately, these frameworks are either not extendable to realistic industry scenario, e.g., adaptive mesh refinement, or cannot preceed nonlinear operations without resorting to lossy interpolation on a uniform grid. Details can be found in our [paper](https://arxiv.org/pdf/2204.03216.pdf).\n\n- NIF is built on top of Keras, in order to minimize user's efforts in using the code and maximize the existing functionality in Keras. \n\n\n## Highlights\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./misc/compare.gif\" alt=\"animated\" /\u003e\n\u003c/p\u003e\n\n\n- Built on top of **Tensorflow 2.x** with **Keras model subclassing**, hassle-free for many up-to-date advanced concepts and features\n\n    ```python\n    from nif import NIF\n    \n    # set up the configurations, loading dataset, etc...\n    \n    model_ori = NIF(...)\n    model_opt = model_ori.build()\n    \n    model_opt.compile(optimizer, loss='mse')\n    model_opt.fit(...)\n    \n    model_opt.predict(...)\n    ```\n\n- **Distributed learning**: data parallelism across multiple GPUs on a single node\n\n    ```python\n    enable_multi_gpu = True\n    cm = tf.distribute.MirroredStrategy().scope() if enable_multi_gpu else contextlib.nullcontext()\n    with cm:\n        \n        # ...\n        model.fit(...)\n        # ...\n    ```\n\n- Flexible training schedule: e.g., first some standard optimizer (e.g., Adam) then **fine-tunning with L-BFGS**\n  \n    ```python\n    from nif.optimizers import TFPLBFGS\n    \n    # load previous model\n    new_model_ori = NIF(cfg_shape_net, cfg_parameter_net, mixed_policy)\n    new_model.load_weights(...)\n    \n    # prepare the dataset\n    data_feature = ... #\n    data_label = ... # \n    \n    # fine tune with L-BFGS\n    loss_fun = tf.keras.losses.MeanSquaredError()\n    fine_tuner = TFPLBFGS(new_model, loss_fun, data_feature, data_label, display_epoch=10)\n    fine_tuner.minimize(rounds=200, max_iter=1000)\n    new_model.save_weights(\"./fine-tuned/ckpt\")\n    ```\n  \n- Templates for many useful customized callbacks\n\n   ```python\n   # setting up the model\n   # ...\n\n   # - tensorboard\n   tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=\"./tb-logs\", update_freq='epoch')\n\n   # - printing, model save checkpoints etc.\n   class LossAndErrorPrintingCallback(tf.keras.callbacks.Callback):\n   # ....\n\n   # - learning rate schedule\n   def scheduler(epoch, lr):\n       if epoch \u003c 1000:\n           return lr\n       else:\n           return 1e-4\n   scheduler_callback = tf.keras.callbacks.LearningRateScheduler(scheduler)\n\n   # - collecting callbacks into model.fit(...)\n\n   callbacks = [tensorboard_callback, LossAndErrorPrintingCallback(), scheduler_callback]\n   model_opt.fit(train_dataset, epochs=nepoch, batch_size=batch_size,\n             shuffle=False, verbose=0, callbacks=callbacks)\n   ```\n   \n- Simple extraction of subnetworks\n\n    ```python\n    model_ori = NIF(...)\n    \n    # ....\n    \n    # extract latent space encoder network\n    model_p_to_lr = model_ori.model_p_to_lr()\n    lr_pred = model_p_to_lr.predict(...)\n    \n    # extract latent-to-weight network: from latent representation to weights and biase of shapenet\n    model_lr_to_w = model_ori.model_lr_to_w()\n    w_pred = model_lr_to_w.predict(...)\n    \n    # extract shapenet: inputs are weights and spatial coordinates, output is the field of interests\n    model_x_to_u_given_w = model_ori.model_x_to_u_given_w()\n    u_pred = model_x_to_u_given_w.predict(...)\n    ```\n\n- Get input-output Jacobian or Hessian.\n    ```python\n    model = ... # your keras.Model\n    x = ... # your dataset\n    # define both the indices of target and source \n    \n    x_index = [0,1,2,3]\n    y_index = [0,1,2,3,4]\n    \n    # wrap up keras.Model using JacobianLayer \n    from nif.layers import JacobianLayer\n    y_and_dydx_layer = JacobianLayer(model, y_index, x_index)\n    \n    y, dydx = y_and_dydx_layer(x)\n    \n    model_with_jacobian = Model([x], [y, dydx])\n    \n    # wrap up keras.Model using HessianLayer\n    from nif.layers import HessianLayer\n    y_and_dydx_and_dy2dx2_layer = HessianLayer(model, y_index, x_index)\n    \n    y, dydx, dy2dx2 = y_and_dydx_and_dy2dx2_layer(x)\n    \n    model_with_jacobian_and_hessian = Model([x], [y, dydx, dy2dx2])\n    \n    ```\n\n- Data normalization for multi-scale problem\n\n    - just simply feed `n_para`: number of parameters, `n_x`: input dimension of shapenet, `n_target`: output dimension of shapenet, and `raw_data`: numpy array with shape = `(number of pointwise data points, number of features, target, coordinates, etc.)`\n\n    ```python\n    from nif.data import PointWiseData\n    data_n, mean, std = PointWiseData.minmax_normalize(raw_data=data, n_para=1, n_x=3, n_target=1) \n    ```\n\n- Large-scale training with tfrecord converter\n\n    - all you need is to prepare a BIG npz file that contains all the point-wise data\n    - `.get_tfr_meta_dataset` will read all files under the searched directory that ends with `.tfrecord`\n\n    ```python\n    from nif.data.tfr_dataset import TFRDataset\n    fh = TFRDataset(n_feature=4, n_target=3)\n    \n    # generating tfrecord files from a single big npz file (say gigabytes)\n    fh.create_from_npz(...)\n\n    # prepare some model\n    model = ...\n    model.compile(...)\n\n    # obtaining a meta dataset\n    meta_dataset = fh.get_tfr_meta_dataset(...)\n\n    # start sub-dataset-batching\n    for batch_file in meta_dataset:\n        batch_dataset = fh.gen_dataset_from_batch_file(batch_file, batch_size)\n        model.fit(...)\n    ```\n- Save and load models (via Checkpoints only)\n    ```python\n    # save the config  \n    model.save_config(\"config.json\")\n  \n    # save the weights\n    model.save_weights(\"./saved_weights/ckpt-{}/ckpt\".format(epoch)\")\n    \n    # load the config\n    with open(\"config.json\", \"r\") as f:\n        config = json.load(f) \t\n    model_ori = nif.NIF(**config)\n    model = model_ori.model()\n  \n    # load the weights\n    model.load_weights(\"./saved_weights/ckpt-999/ckpt\")\n    ```\n- Network pruning and quantization\n\n\n## Google Colab Tutorial\n\n1. **Hello world! A simple fitting on 1D travelling wave** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pswpswpsw/nif/blob/main/tutorial/1_simple_1d_wave.ipynb)\n    - learn how to use class `nif.NIF`\n    - model checkpoints/restoration\n    - mixed precision training\n    - L-BFGS fine tuning\n\n2. **Tackling multi-scale data** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pswpswpsw/nif/blob/main/tutorial/2_multi_scale_NIF.ipynb)\n\n    - learn how to use class `nif.NIFMultiScale`\n    - demonstrate the effectiveness of learning high frequency data\n\n3. **Learning linear representation** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pswpswpsw/nif/blob/main/tutorial/3_multi_scale_linear_NIF.ipynb)\n    - learn how to use class `nif.NIFMultiScaleLastLayerParameterized`\n    - demonstrate on a (shortened) flow over a cylinder case from an AMR solver\n\n4. **Getting input-output derivatives is super easy** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pswpswpsw/nif/blob/main/tutorial/4_get_gradients_by_wrapping_model_with_layer.ipynb)\n    - learn how to use `nif.layers.JacobianLayer`, `nif.layers.HessianLayer`\n\n5. **Scaling to hundreds of GB data** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pswpswpsw/nif/blob/main/tutorial/5_large_scale_training_on_tensorflow_record_data.ipynb)\n    - learn how to use `nif.data.tfr_dataset.TFRDataset` to create `tfrecord` from `npz`\n    - learn how to perform sub-dataset-batch training with `model.fit`\n\n6. **Revisit NIF on multi-scale data with regularization** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pswpswpsw/nif/blob/main/tutorial/6_revisit_multi_scale_NIF_with_L1_L2_regularization.ipynb)\n    - learn how to use L1 or L2 regularization for weights and bias in ParameterNet.\n    - a demonstration for the failure of NIF-Multiscale in terms of increasing spatial interpolation when dealing with\n      high-frequency signal.\n        - this means you need to be cautious about increasing spatial sampling resolution when dealing with\n          high-frequency signal.\n    - learn that L2 or L1 regularization does not seem to help resolving the above issue.\n\n7. **NIF Compression** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pswpswpsw/nif/blob/main/tutorial/7_model_pruning_and_quantization.ipynb)\n    - learn how to use low magnititute pruning and quantization to compress ParameterNet\n\n8. **Revisit NIF on multi-scale data: Sobolov training helps removing spurious signals** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pswpswpsw/nif/blob/main/tutorial/8_revisit_multi_scale_NIF_with_sobolov_training.ipynb)\n    - learn how to use `nif.layers.JacobianLayer` to perform Sobolov training\n    - learn how to monitor different loss terms using customized Keras metrics\n    - learn that feeding derivative information to the system help resolve the super-resolution issue\n\n## Requirements\n\n```python\nmatplotlib\nnumpy\ntensorflow-gpu\ntensorflow_probability==0.18.0\ntensorflow_model_optimization==0.7.3\n```\n\n## Issues, bugs, requests, ideas\n\nUse the [issues](https://github.com/pswpswpsw/nif/issues) tracker to report bugs.\n\n## How to cite\n\nIf you find NIF is helpful to you, you can cite our JMLR [paper](https://www.jmlr.org/papers/volume24/22-0365/22-0365.pdf) in the following bibtex format\n\n   ```\n@article{JMLR:v24:22-0365,\n  author  = {Shaowu Pan and Steven L. Brunton and J. Nathan Kutz},\n  title   = {Neural Implicit Flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data},\n  journal = {Journal of Machine Learning Research},\n  year    = {2023},\n  volume  = {24},\n  number  = {41},\n  pages   = {1--60},\n  url     = {http://jmlr.org/papers/v24/22-0365.html}\n}\n   ```\n\n## Contributors\n\n- [Shaowu Pan](http://www.shaowupan.com)\n- [Yaser Afshar](https://github.com/yafshar)\n\n\n## License\n\n[LGPL-2.1 License](https://github.com/pswpswpsw/nif/blob/main/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpswpswpsw%2Fnif","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpswpswpsw%2Fnif","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpswpswpsw%2Fnif/lists"}