{"id":13689450,"url":"https://github.com/taki0112/Tensorflow-Cookbook","last_synced_at":"2025-05-01T23:34:32.580Z","repository":{"id":98366712,"uuid":"171020125","full_name":"taki0112/Tensorflow-Cookbook","owner":"taki0112","description":"Simple Tensorflow Cookbook for easy-to-use","archived":false,"fork":false,"pushed_at":"2020-02-09T06:12:49.000Z","size":3165,"stargazers_count":2777,"open_issues_count":1,"forks_count":472,"subscribers_count":118,"default_branch":"master","last_synced_at":"2024-11-11T21:10:21.251Z","etag":null,"topics":["tensorflow","tensorflow-cookbook","tensorflow-examples"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/taki0112.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-16T15:43:31.000Z","updated_at":"2024-11-10T12:59:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"5feb5e49-e6e3-4ea1-92d9-0e99e8549dbe","html_url":"https://github.com/taki0112/Tensorflow-Cookbook","commit_stats":{"total_commits":109,"total_committers":3,"mean_commits":"36.333333333333336","dds":0.08256880733944949,"last_synced_commit":"580e5fc26e8f24023d1da6d095452fb6d5a121c7"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taki0112%2FTensorflow-Cookbook","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taki0112%2FTensorflow-Cookbook/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taki0112%2FTensorflow-Cookbook/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taki0112%2FTensorflow-Cookbook/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/taki0112","download_url":"https://codeload.github.com/taki0112/Tensorflow-Cookbook/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224282276,"owners_count":17285798,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["tensorflow","tensorflow-cookbook","tensorflow-examples"],"created_at":"2024-08-02T15:01:48.396Z","updated_at":"2024-11-12T13:31:46.148Z","avatar_url":"https://github.com/taki0112.png","language":"Python","funding_links":[],"categories":["Machine Learning Tutorials","Python"],"sub_categories":["Data Management"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"./assets/tf-cook.png\" height = '300px'\u003e\n\u003c/div\u003e\n\n\n# [Web page](http://bit.ly/jhkim_tf_cookbook)\n# [Tensorflow 2 Cookbook](https://github.com/taki0112/Tensorflow2-Cookbook)\n\n## Contributions\nIn now, this repo contains general architectures and functions that are useful for the GAN and classificstion.\n\nI will continue to add useful things to other areas.\n\nAlso, your pull requests and issues are always welcome.\n\nAnd write what you want to implement on the issue. I'll implement it.\n\n# How to use\n## Import\n* `ops.py`\n  * **operations**\n  * from ops import *\n* `utils.py`\n  * **image processing**\n  * from utils import *\n  \n## Network template\n```python\ndef network(x, is_training=True, reuse=False, scope=\"network\"):\n    with tf.variable_scope(scope, reuse=reuse):\n        x = conv(...)\n        \n        ...\n        \n        return logit\n```\n\n## Insert data to network using DatasetAPI\n```python\nImage_Data_Class = ImageData(img_size, img_ch, augment_flag)\n\ntrainA_dataset = ['./dataset/cat/trainA/a.jpg', \n                  './dataset/cat/trainA/b.png', \n                  './dataset/cat/trainA/c.jpeg', \n                  ...]\ntrainA = tf.data.Dataset.from_tensor_slices(trainA_dataset)\ntrainA = trainA.map(Image_Data_Class.image_processing, num_parallel_calls=16)\ntrainA = trainA.shuffle(buffer_size=10000).prefetch(buffer_size=batch_size).batch(batch_size).repeat()\n\ntrainA_iterator = trainA.make_one_shot_iterator()\ndata_A = trainA_iterator.get_next()\n\nlogit = network(data_A)\n```\n* See [this](https://github.com/taki0112/Tensorflow-DatasetAPI) for more information.\n\n## Option\n* `padding='SAME'`\n  * pad = ceil[ (kernel - stride) / 2 ]\n* `pad_type`\n  * 'zero' or 'reflect'\n* `sn`\n  * use [spectral_normalization](https://arxiv.org/pdf/1802.05957.pdf) or not\n\n## Caution\n* If you don't want to share variable, **set all scope names differently.**\n\n---\n## Weight\n```python\nweight_init = tf.truncated_normal_initializer(mean=0.0, stddev=0.02)\nweight_regularizer = tf.contrib.layers.l2_regularizer(0.0001)\nweight_regularizer_fully = tf.contrib.layers.l2_regularizer(0.0001)\n```\n### Initialization\n* `Xavier` : tf.contrib.layers.xavier_initializer()\n  ```python\n  \n    USE \"\"\"tf.contrib.layers.variance_scaling_initializer()\"\"\"\n    \n    if uniform :\n      factor = gain * gain\n      mode = 'FAN_AVG'\n    else :\n      factor = (gain * gain) / 1.3\n      mode = 'FAN_AVG'\n  ```\n* `He` : tf.contrib.layers.variance_scaling_initializer()\n  ```python\n    if uniform :\n      factor = gain * gain\n      mode = 'FAN_IN'\n    else :\n      factor = (gain * gain) / 1.3\n      mode = 'FAN_OUT'\n  ```\n* `Normal` : tf.random_normal_initializer(mean=0.0, stddev=0.02)\n* `Truncated_normal` : tf.truncated_normal_initializer(mean=0.0, stddev=0.02)\n* `Orthogonal` : tf.orthogonal_initializer(1.0) / # if relu = sqrt(2), the others = 1.0\n\n### Regularization\n* `l2_decay` : tf.contrib.layers.l2_regularizer(0.0001)\n* `orthogonal_regularizer` : orthogonal_regularizer(0.0001) \u0026 orthogonal_regularizer_fully(0.0001)\n\n## Convolution\n### basic conv\n```python\nx = conv(x, channels=64, kernel=3, stride=2, pad=1, pad_type='reflect', use_bias=True, sn=True, scope='conv')\n```\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/vdumoulin/conv_arithmetic/raw/master/gif/padding_strides.gif\" width = '300px'\u003e\n\u003c/div\u003e\n\n### partial conv (NVIDIA [Partial Convolution](https://github.com/NVIDIA/partialconv))\n```python\nx = partial_conv(x, channels=64, kernel=3, stride=2, use_bias=True, padding='SAME', sn=True, scope='partial_conv')\n```\n\n![p_conv](https://github.com/taki0112/partial_conv-Tensorflow/raw/master/assets/partial_conv.png)\n![p_result](https://github.com/taki0112/partial_conv-Tensorflow/raw/master/assets/classification.png)\n\n### dilated conv\n```python\nx = dilate_conv(x, channels=64, kernel=3, rate=2, use_bias=True, padding='VALID', sn=True, scope='dilate_conv')\n```\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/vdumoulin/conv_arithmetic/raw/master/gif/dilation.gif\" width = '300px'\u003e\n\u003c/div\u003e\n\n---\n\n## Deconvolution\n### basic deconv\n```python\nx = deconv(x, channels=64, kernel=3, stride=1, padding='SAME', use_bias=True, sn=True, scope='deconv')\n```\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/vdumoulin/conv_arithmetic/raw/master/gif/padding_strides_transposed.gif\" width = '300px'\u003e\n\u003c/div\u003e\n\n---\n\n## Fully-connected\n```python\nx = fully_connected(x, units=64, use_bias=True, sn=True, scope='fully_connected')\n```\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://stanford.edu/~shervine/teaching/cs-230/illustrations/fully-connected.png\"\u003e\n\u003c/div\u003e\n\n---\n\n## Pixel shuffle\n```python\nx = conv_pixel_shuffle_down(x, scale_factor=2, use_bias=True, sn=True, scope='pixel_shuffle_down')\nx = conv_pixel_shuffle_up(x, scale_factor=2, use_bias=True, sn=True, scope='pixel_shuffle_up')\n```\n* `down` ===\u003e [height, width] -\u003e [**height // scale_factor, width // scale_factor**]\n* `up` ===\u003e [height, width] -\u003e [**height \\* scale_factor, width \\* scale_factor**]\n\n![pixel_shuffle](./assets/pixel_shuffle.png)\n\n\n---\n\n## Block\n### residual block\n```python\nx = resblock(x, channels=64, is_training=is_training, use_bias=True, sn=True, scope='residual_block')\nx = resblock_down(x, channels=64, is_training=is_training, use_bias=True, sn=True, scope='residual_block_down')\nx = resblock_up(x, channels=64, is_training=is_training, use_bias=True, sn=True, scope='residual_block_up')\n```\n* `down` ===\u003e [height, width] -\u003e [**height // 2, width // 2**]\n* `up` ===\u003e [height, width] -\u003e [**height \\* 2, width \\* 2**]\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://cdn-images-1.medium.com/max/1600/1*FqmD91PvbH7NKCnQWFJxvg.png\"\u003e\n\u003c/div\u003e\n\n### dense block\n```python\nx = denseblock(x, channels=64, n_db=6, is_training=is_training, use_bias=True, sn=True, scope='denseblock')\n```\n* `n_db` ===\u003e The number of dense-block\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/taki0112/Densenet-Tensorflow/raw/master/assests/Denseblock.JPG\" height = '400px'\u003e\n\u003c/div\u003e\n\n### residual-dense block\n```python\nx = res_denseblock(x, channels=64, n_rdb=20, n_rdb_conv=6, is_training=is_training, use_bias=True, sn=True, scope='res_denseblock')\n```\n* `n_rdb` ===\u003e The number of RDB\n* `n_rdb_conv` ===\u003e per RDB conv layer\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=./assets/compare.png height = '400px'\u003e\n  \u003cimg src=./assets/rdn.png height = '350px' width='800px'\u003e\n  \u003cimg src=./assets/rdb.png height = '250px' width='650px'\u003e\n\u003c/div\u003e\n\n### attention block\n```python\nx = self_attention(x, use_bias=True, sn=True, scope='self_attention')\nx = self_attention_with_pooling(x, use_bias=True, sn=True, scope='self_attention_version_2')\n\nx = squeeze_excitation(x, ratio=16, use_bias=True, sn=True, scope='squeeze_excitation')\n\nx = convolution_block_attention(x, ratio=16, use_bias=True, sn=True, scope='convolution_block_attention')\n\nx = global_context_block(x, use_bias=True, sn=True, scope='gc_block')\n\nx = srm_block(x, use_bias=False, is_training=is_training, scope='srm_block')\n```\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/taki0112/Self-Attention-GAN-Tensorflow/raw/master/assests/framework.PNG\"\u003e\n\u003c/div\u003e\n\n---\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/hujie-frank/SENet/blob/master/figures/SE-Inception-module.jpg\" width=\"420\"\u003e\n  \u003cimg src=\"https://github.com/hujie-frank/SENet/blob/master/figures/SE-ResNet-module.jpg\"  width=\"420\"\u003e\n\u003c/div\u003e\n\n---\n\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://bloglunit.files.wordpress.com/2018/08/screen-shot-2018-08-22-at-8-42-27-pm.png?w=2800\"\u003e\n  \u003cimg src=\"https://bloglunit.files.wordpress.com/2018/08/screen-shot-2018-08-22-at-8-47-09-pm.png?w=2800\"\u003e\n\u003c/div\u003e\n\n\n---\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=./assets/gcb.png\u003e\n\u003c/div\u003e\n\n---\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=./assets/srm.png height='350' width='500'\u003e\n\u003c/div\u003e\n\n---\n\n## Normalization\n```python\nx = batch_norm(x, is_training=is_training, scope='batch_norm')\nx = layer_norm(x, scope='layer_norm')\nx = instance_norm(x, scope='instance_norm')\nx = group_norm(x, groups=32, scope='group_norm')\n\nx = pixel_norm(x)\n\nx = batch_instance_norm(x, scope='batch_instance_norm')\nx = layer_instance_norm(x, scope='layer_instance_norm')\nx = switch_norm(x, scope='switch_norm')\n\nx = condition_batch_norm(x, z, is_training=is_training, scope='condition_batch_norm'):\n\nx = adaptive_instance_norm(x, gamma, beta)\nx = adaptive_layer_instance_norm(x, gamma, beta, smoothing=True, scope='adaLIN')\n\n```\n* See [this](https://github.com/taki0112/BigGAN-Tensorflow) for how to use `condition_batch_norm`\n* See [this](https://github.com/taki0112/MUNIT-Tensorflow) for how to use `adaptive_instance_norm`\n* See [this](https://github.com/taki0112/UGATIT) for how to use `adaptive_layer_instance_norm` \u0026 `layer_instance_norm`\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/taki0112/Group_Normalization-Tensorflow/raw/master/assests/norm.png\"\u003e\n\u003c/div\u003e\n\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/taki0112/Switchable_Normalization-Tensorflow/raw/master/assests/teaser.png\"\u003e\n\u003c/div\u003e\n\n---\n\n## Activation\n```python\nx = relu(x)\nx = lrelu(x, alpha=0.2)\nx = tanh(x)\nx = sigmoid(x)\nx = swish(x)\nx = elu(x)\n```\n\n---\n\n## Pooling \u0026 Resize\n```python\nx = nearest_up_sample(x, scale_factor=2)\nx = bilinear_up_sample(x, scale_factor=2)\nx = nearest_down_sample(x, scale_factor=2)\nx = bilinear_down_sample(x, scale_factor=2)\n\nx = max_pooling(x, pool_size=2)\nx = avg_pooling(x, pool_size=2)\n\nx = global_max_pooling(x)\nx = global_avg_pooling(x)\n\nx = flatten(x)\nx = hw_flatten(x)\n```\n\n---\n\n## Loss\n### classification loss\n```python\nloss, accuracy = classification_loss(logit, label)\n\nloss = dice_loss(n_classes=10, logit, label)\n```\n\n### regularization loss\n```python\ng_reg_loss = regularization_loss('generator')\nd_reg_loss = regularization_loss('discriminator')\n```\n\n* If you want to use `regularizer`, then you should write it\n\n### pixel loss\n```python\nloss = L1_loss(x, y)\nloss = L2_loss(x, y)\nloss = huber_loss(x, y)\nloss = histogram_loss(x, y)\n\nloss = gram_style_loss(x, y)\n\nloss = color_consistency_loss(x, y)\n```\n* `histogram_loss` means the difference in the color distribution of the image pixel values.\n* `gram_style_loss` means the difference between the styles using gram matrix.\n* `color_consistency_loss` means the color difference between the generated image and the input image.\n\n### gan loss\n```python\nd_loss = discriminator_loss(Ra=True, loss_func='wgan-gp', real=real_logit, fake=fake_logit)\ng_loss = generator_loss(Ra=True, loss_func='wgan-gp', real=real_logit, fake=fake_logit)\n```\n* `Ra`\n  * use [relativistic gan](https://arxiv.org/pdf/1807.00734.pdf) or not\n* `loss_func`\n  * gan\n  * lsgan\n  * hinge\n  * wgan-gp\n  * dragan\n* See [this](https://github.com/taki0112/BigGAN-Tensorflow/blob/master/BigGAN_512.py#L180) for how to use `gradient_penalty`\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=./assets/relativistic.png\u003e\n\u003c/div\u003e\n\n### [vdb loss](https://arxiv.org/abs/1810.00821)\n```python\nd_bottleneck_loss = vdb_loss(real_mu, real_logvar, i_c) + vdb_loss(fake_mu, fake_logvar, i_c)\n```\n\n### kl-divergence (z ~ N(0, 1))\n```python\nloss = kl_loss(mean, logvar)\n```\n\n---\n\n## Author\n[Junho Kim](http://bit.ly/jhkim_ai)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaki0112%2FTensorflow-Cookbook","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftaki0112%2FTensorflow-Cookbook","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaki0112%2FTensorflow-Cookbook/lists"}