{"id":17912681,"url":"https://github.com/xiangronglin/grayscale-conversion","last_synced_at":"2025-03-23T22:34:56.625Z","repository":{"id":40409661,"uuid":"440781070","full_name":"XiangRongLin/grayscale-conversion","owner":"XiangRongLin","description":"grayscale conversion optimized with OpenMP, SIMD and CUDA","archived":false,"fork":false,"pushed_at":"2022-06-26T07:48:56.000Z","size":35922,"stargazers_count":6,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-19T00:06:55.841Z","etag":null,"topics":["cuda","grayscale","hpc","openmp","simd"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/XiangRongLin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-22T08:05:19.000Z","updated_at":"2024-07-28T14:12:52.000Z","dependencies_parsed_at":"2022-09-01T06:12:17.157Z","dependency_job_id":null,"html_url":"https://github.com/XiangRongLin/grayscale-conversion","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XiangRongLin%2Fgrayscale-conversion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XiangRongLin%2Fgrayscale-conversion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XiangRongLin%2Fgrayscale-conversion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XiangRongLin%2Fgrayscale-conversion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/XiangRongLin","download_url":"https://codeload.github.com/XiangRongLin/grayscale-conversion/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245180339,"owners_count":20573651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cuda","grayscale","hpc","openmp","simd"],"created_at":"2024-10-28T19:46:32.292Z","updated_at":"2025-03-23T22:34:51.613Z","avatar_url":"https://github.com/XiangRongLin.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"Report by [Dennis Weggenmann](https://github.com/DennisWeggenmann) and [Xiang Rong Lin](https://github.com/XiangRongLin) for the lecture \"High Performance Computing\" during the winter semester 21/22 at \"Hochschule für Technik Stuttgart\"\n\nhttps://github.com/sspeiser/hpc-uebungen\n\n# Motivation\nRGB to Grayscale conversation is an embarrassingly parallel Problem. So it's perfect for multithreaded programms. Each Pixel can be calculated/converted independently. \n# GPU\n\n## Problems\n- memory transfer between Host and Device\n- when to start measure time ? Before the kernel launch, before the memcopy or before the allocation \n- The first cuda function calls takes a lot of time. For benchmarking reasons i just call a cudaFree(0).\nNow the first cuda call (most of the time its cudaMalloc) takes for example 0,023 seconds instead of 0,153 seconds\n- cudaHostRegister was not able to allocate enough memory but after trying it 10 times its working\n\n## Implementation\n### GPU Workflow\nHost = CPU \n\nDevice = GPU\n\nThe GPU needs to acces the Data from the main memory. Then the CPU instructs the GPU. The calculation will be  parallel  executed in cores and the result will be copied back to the main memory.\n\n![cuda_profiling](images/workflow.png)\nhttps://www.academia.edu/20415057/Parallel_Implementation_of_Grayscale_Conversion_in_Graphics_Hardware\n\n\n\nThread\n\nEach Thread gets mapped to one cuda core\n\nBlocks:\n\nThreads are grouped into Blocks\n\nGrids:\n\nBlocks are grouped into a Grid\n\nEach Kernel launch creates one single Grid\n\n\n![cuda_profiling](images/Block-thread.svg)\n\nhttps://developer-blogs.nvidia.com/wp-content/uploads/2020/06/kernel-execution-on-gpu-1-625x438.png\n\nWe have an input image which is loaded by stbi_load and returns an unsigned char.\nThen we allocate memory for our grayscale image which the results will be copied in\n\nSo now we need to allocate memory for our RGB image on the Device. We do this with CudaMalloc\n```C\ncudaMalloc(\u0026device_rgb, sizeof(uchar3) * pixel_size*3 );\n```\nwe also allocate memory for our greyimage on the Device with\n```C\ncudaMalloc(\u0026device_grey, sizeof(unsigned char) * pixel_size);\n```\nwith cudaMemcpy the Imagedata will be copied to the memory we allocated for our RBG image. We also have to pass the size of the copied data and in which direction we are going to copy.\n```C\ncudaMemcpy(device_rgb, Image, sizeof(unsigned char) * pixel_size*3 , cudaMemcpyHostToDevice);\n```\nNow the data is in the GPU memory and we are able to launch our kernel.\n### We are on the Device now\n\nA Kernelfunction looks like a normal function but has a __global__\nkeyword befor it. With the global identifer we define a function that will run on the Device.\n```C\nConvertToGrey\u003c\u003c\u003cGrid, Block\u003e\u003e\u003e(device_rgb, device_grey, rows, columns);\n```\nAs parameters we pass our already allocated device_rgb and device_grey references and the rows and columns which are basically the width and height of our image.\nIf we take a look at the Kernel function the first thing we see is this\n```C\nint index_x = threadIdx.x + blockIdx.x * blockDim.x;\n```\nThreadIdx, blockIdx and blockDim are cuda variables. We can access them when we run on the Device.\nthreadIdx : thread index in the block\nblockIdx : block index in the grid\nblockDim : number of threads by blocks.\n\nWe want the unique Grid index of a thread because threadIdx is only unique in its own Thread Block. So we multiply the Blocks index with the block dimension and add the Threadindex.\nWe do the same for the y index. And now we have the current pixel location\n\n1d coordinate of the greyscale image\n```C\nint output_offset = index_y * columns + index_x;\n```\nnow we write the result into the outputimage\n```C\n output[output_offset] = rgb.x * 0.299f +rgb.y* 0.587f +rgb.z * 0.114f \n```\n### we are back on the Host now\nThe kernel call is asynchronous. But in our case this doesnt bother us because we only have one stream so cudaMemcpy waits until the GPU has finished.\nNow we copy the data back from the Device to the Host\n```C\ncudaMemcpy(host_grey, device_grey, sizeof(unsigned char) * pixel_size, cudaMemcpyDeviceToHost);\n```\nin the end we need to free the allocated memory on the Device\n```C\ncudaFree(device_rgb);\ncudaFree(device_grey);\n```\n\n## Review\nLooking at the Performance its interessting for a 27000x6000 pixel image the calculation takes 0,007 seconds but here is the catch. With nvprof or nvvp we can \nsee that the GPU is only 1,8% of the time busy with computing. The rest is allocation and memory transfer.\n\n|Allocation in seconds |memcopy HtoD in seconds |memcopy DtoH in seconds |Kernel in seconds\n|---|---|---|---|\n|0.018321 |0.054598|0.048913|0.00754453|\n\n![cuda_profiling](images/cuda_profiling.png)\n\nThe orange bars are all the called cuda functions like cudamalloc.\nThe blue bar is the kernel activity\n\nThere is also a huge overhead from the cudaMemcpy call and the actual Memcpy operation. \n\n\n## Memory on the GPU\nThe GPU has different kinds of memory. The biggest one is the global memory. Its located on the device Dram. There are other types of memory(like Local memory, constant memory) located on the Dram but we dont need them now. For optimisation is the On Chip memory more interessting especially the Shared Memory. Its very fast so why we didnt used it to make the Kernel function even faster.\nFor this specific task there is no performance gain from using shared memory instead of the global memory because the shared memory doesnt reuse any data so the number of global memory reads stays the same. Also the calculation time only takes arround 2% of the time so maybe its better to focus on the other 98%. \n\n## Memorytransfer between GPU and CPU\nThere is no free lunch\nIn general avoid memory transfer between device and\nhost. Its recommended to to copy the data to the device. \nThen calculate on the device and then copy the data back.\n\nlike in ([greyscale.cu](cuda/greyscale.cu))\n\n### pinned memory\nGeneraly pinned memory is recommended if we want to overlap copy and compute\n![pinnendMemory](images/pagableDataTransfer.png)\nhttps://developer-blogs.nvidia.com/wp-content/uploads/2012/12/pinned-1024x541.jpg\n\nInstead of malloc() we could use cudaMallocHost(). This will allocate the data in the Pinned Memory. I didnt find a way to direclty allocate the Image with the stb_image.h functionality. But luckily CUDA offers cudaHostRegister which will pin memory that is already allocated.\ninstead of allocating host_grey with malloc we use cudaMallocHost which will allocate in the Pinnend Memory.\n\nSo the Memory transfer between Host and Device should be faster. \n\n![cuda_profiling](images/cuda_profiling_pinned.png)\n\nwell yes and no. The Memcpy HtoD is faster compared to the nonpinnend version(see [greyscale.cu](cuda/greyscale.cu)). (37 ms vs 55 ms) the cudaMemcpy call still takes (100 ms instead of 120 ms in the nonpinnend version). The Memcpy DtoH is also faster (12 ms v 37 ms) and for the cudaMemcpy call (19,8 ms vs 58 ms)\n\nSo we get a minimal transfer speed-up but the Host allocation takes now longer (27 ms)\n\n|Allocation in seconds |memcopy HtoD in seconds |memcopy DtoH in seconds |Kernel in seconds |\n|---|---|---|---|\n|0.292566 |0.032401|0.0121894|0.0075523|0.086|\n\n\n### final runtime with CUDA allocation, memory transfer and kernel execution time in seconds done on a GTX 1060 6GB\n\n|greyscale|greyscalePinnedMemory| greyscaleV2(using pinned memory too)|\n|---|---|---|\n|0.16000 |0.16200| 0.1100|\n\n## Possible Solution and idea for future research\nIt would be interessting to see how a grayscale conversation performs on a M1 with a Unified Memory Architecture. It could be faster since the GPU and CPU using that common pool of memory. There should\nbe no need to transfer the data. The GPU will let the CPU know when its done\n\n\n# CPU\n## Problem\n- naive solution is single threaded\n- processors can calculate 128/256 bit at once, but only part of it is used in a single iteration\n- data is in rgbrgbrgbrgb format, but rrrrggggbbbb is needed\n- memory is not aligned, meaning that in order to read 8-bytes we may need to actually read 16-bytes\n\n## Solution attempt\n### Multithreaded\nThe easiest place to optimize it, is utilizing all cores of a CPU and thus convert it to a multithreaded application.\nThis is done with OpenMP by adding the pragma `#pragma omp parallel for collapse(2)` (see [openmp_baseline.c](cpu/algorithms/openmp_baseline.c))).\n`omp parallel for` parallelizes the `for` loop with `collapse(2)` collapsing both loops and thus parallelizing both.\nThis gives a more than 6 times performance boost.\n\nIn the next step the memory access can optimized.\nCurrently each thread calculates the grey value for a random pixel, depending on how it is scheduled by openMP.\nFor this it needs to load 3 unsigned char, so 24 bytes from memory.\nBut a CPU preloads more data into the cache anticipating that it will be needed.\nThis behavior can be used to by having each thread operating on a continuous section, thus using the data that is already in the CPU Cache (see [memory.c](cpu/algorithms/memory.c)).\n\n### SIMD FMA\nAll references to intrinsic functions can be looked up here: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html\n\nThe next place to optimize is utilizing the full register of the CPU by using Single Instruction Multiple Data (SIMD).\nFor example we could add 16 8-bit integers at once in a 128 bit register instead of only a single one, thus theoretically creating a 16 time speedup.\nAdditionally one can use a dedicated arithmetic logic unit that multiplies two numbers and add it to an accumulator, knows as MAC-unit (multiplier-accumulator).\nThis takes the form of \"fused multiply add\" (FMA), which additionally only rounds at the end, thus combining two operations into one.\n\nThe data being in the form of rgbrgbrgbrgb appears for the first time.\nFor FMA a whole register need to be filled with only red, green or blue values, meaning we want the data in rrrrggggbbbb format.\nThis problem is ignored for now, by just setting the register with the appropriate values, which comes with its own performance problems, because with the data being spread out like this, multiple reads may be necessary.\n```C\nr_vector = _mm_set_ps(img[(i * channels)], img[(i + 1) * channels], img[(i + 2) * channels], img[(i + 3) * channels]);\ng_vector = _mm_set_ps(img[(i * channels) + 1], img[(i + 1) * channels + 1], img[(i + 2) * channels + 1], img[(i + 3) * channels + 1]);\nb_vector = _mm_set_ps(img[(i * channels) + 2], img[(i + 1) * channels + 2], img[(i + 2) * channels + 2], img[(i + 3) * channels + 2]);\n```\n\nWith the data in the correct format the multiplication is very simple\n```C\ngray_vector = _mm_setzero_ps();\ngray_vector = _mm_fmadd_ps(r_vector, r_factor, gray_vector);\ngray_vector = _mm_fmadd_ps(g_vector, g_factor, gray_vector);\ngray_vector = _mm_fmadd_ps(b_vector, b_factor, gray_vector);\n```\n\nFull code see [memory_simd_fma.c](cpu/algorithms/memory_simd_fma.c)\n\nA problem with FMA is, that the basic FMA instruction set only supports working with 32-bit and 64-bit floating point numbers.\nThis means that with a 128-bit register a maximum of 4 pixel can be calculated at once.\n\n### SIMD SSE\nThis implementation if completly copied from a Stackoverflow post by [Rotem](https://stackoverflow.com/users/4926757/rotem): https://stackoverflow.com/a/57844027/13516981\nOnly modification made was making it compatible with pure C, since it was using C++ features (see [memory_simd_sse.c](cpu/algorithms/memory_simd_sse.c))\n\nIt has 2 major optimization areas.\n\nFirst it utilizes shuffle (`_mm_shuffle_epi8`), concat (`_mm_alignr_epi8`) and shift (`_mm_slli_si128`) functions to solve the problem of rearranging the bytes from rgbrgbrgbrgb to rrrrggggbbbb.\nFor example one can group the bytes according to their color like this.\n```C\nconst __m128i shuffle_mask = _mm_set_epi8(9, 6, 3, 0, 11, 8, 5, 2, 10, 7, 4, 1, 9, 6, 3, 0);\n\n__m128i r3_r2_r1_r0_b3_b2_b1_b0_g3_g2_g1_g0_r3_r2_r1_r0 = _mm_shuffle_epi8(r5_b4_g4_r4_b3_g3_r3_b2_g2_r2_b1_g1_r1_b0_g0_r0, shuffle_mask);\n```\nOr like this\n```C\n// The 12 is the amount of bytes to shift the result\n__m128i b7_g7_r7_b6_g6_r6_b5_g5_r5_b4_g4_r4 = _mm_alignr_epi8(b7_g7_r7_b6_g6_r6_b5_g5, r5_b4_g4_r4_b3_g3_r3_b2_g2_r2_b1_g1_r1_b0_g0_r0, 12);\n```\nIf the bytes are not at the start or end in order to concatenate them they are shifted like this\n```C\n// 8 bytes to the left\n__m128i g3_g2_g1_g0_r3_r2_r1_r0_zz_zz_zz_zz_zz_zz_zz_zz = _mm_slli_si128(r3_r2_r1_r0_b3_b2_b1_b0_g3_g2_g1_g0_r3_r2_r1_r0, 8);\n// 4 bytes to the right\n__m128i zz_zz_zz_zz_r7_r6_r5_r4_b7_b6_b5_b4_g7_g6_g5_g4 = _mm_srli_si128(r7_r6_r5_r4_b7_b6_b5_b4_g7_g6_g5_g4_r7_r6_r5_r4, 4);\n```\n\nAdditionally it sacrifices some accuracy by calculating the gray value with 16-bit integers instead of 32-bit floats.\nBut the most important for me is, that it showed me how to work with the bit modification functions in a structured manner by naming the variables according to the bytes it contains.\n\n### SIMD AVX\nWith this knowledge the next step is using the AVX instruction set, which operates on 256-bit registers unlike 128-bit in SSE.\nThis in theory allows one to process twice the amount of pixels at once.\nUnfortunately many AVX functions behave slightly different compared to their SSE counterpart in the form of only operating within 128-bit lanes instead of across the whole 256-bit register.\nThis means that it is not possible to shuffle a byte from the lower lane to the upper lane with `_mm256_shuffle_epi8`.\nSo in a register with `gA_rA_b9_g9_r9_b8_g8_r8_b7_g7_r7_b6_g6_r6_b5_g5_r5_b4_g4_r4_b3_g3_r3_b2_g2_r2_b1_g1_r1_b0_g0_r0`, where the lane split is between g5 and r5, it is not possible to group all red values because r6, r7, r8 and r9 are in the upper lane whereas the other values are in the lower lane.\n\nBecause of this for AVX a different set of functions is used in order to group the bytes of each color.\nThe central function is `_mm256_blendv_epi8(__m256i a, __m256i b, __m256i mask)` which allows to combine parts of the first register with the second one according to the mask.\n```C\n__m256i g4_g3_g2_g1_g0_b4_b3_b2_b1_b0_r5_r4_r3_r2_r1_r0_b9_b8_b7_b6_b5_rA_r9_r8_r7_r6_r5_r4_r3_r2_r1_r0 =\n    _mm256_blendv_epi8(\n        g4_g3_g2_g1_g0_b4_b3_b2_b1_b0_r5_r4_r3_r2_r1_r0_b9_b8_b7_b6_b5_rA_r9_r8_r7_r6_gA_g9_g8_g7_g6_g5,\n        b9_b8_b7_b6_b5_rA_r9_r8_r7_r6_gA_g9_g8_g7_g6_g5_g4_g3_g2_g1_g0_b4_b3_b2_b1_b0_r5_r4_r3_r2_r1_r0,\n        _mm256_set_epi8(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /**/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 128, 128, 128, 128, 128));\n```\nIn this example we want to combine the group of red bytes `r5_r4_r3_r2_r1_r0` of the second register with the group of `rA_r9_r8_r7_r6` in the first one.\nThey are deliberately lined up, so that in the first register before the group of `rA_r9_r8_r7_r6` is exactly enough space to fit in the first 6 bytes `r5_r4_r3_r2_r1_r0`.\nAccordingly the mask is set to use all values of the first register, except the first 6 ones.\n\nThis setup is done through `_mm256_shuffle_epi8` with which we can still shuffle inside the lanes in order to group the bytes.\nAfterwards we can use `_mm256_permute2x128_si256(__m256i a, __m256i b, 1)` in order to swap the lanes so that we can blend them.\nIn some places `_mm256_alignr_epi8` is also used.\nLike in SSE it concatenates registers, although it only works on 128-bit lanes.\n\nFor the exact steps see [memory_simd_avx.c](cpu/algorithms/memory_simd_avx.c) where the variable names reflect the outcome of a function.\n\nAnother change is aligning the memory of the input and output image.\nThis allows using the aligned load and store functions instead of the unaligned ones (`_mm256_load_si256` instead of `_mm256_loadu_si256`).\nWith this we may avoid loading data that is spread across boundaries of a address and thus reducing memory transfer.\nThe output is aligned with by allocating the memory with `aligned_alloc(32, size)` instead of `malloc(size)`.\nFor the input image the library used to load the image `stb_image.h`, allows to override the default `malloc` function used.\nWe do so be defining following values before the include of that library.\n```C\n#define STBI_MALLOC(sz)           aligned_alloc(32, size)\n```\n\n## Benchmarks\nWith \n- AMD Ryzen 5 3600 6-Core Processor \n- cygwin gcc 11\n- compiled `gcc -fopenmp grayscale.c -lm -march=native -O3`\n- 20 runs each\n- 27000x6000 pixel image https://photojournal.jpl.nasa.gov/catalog/?IDNumber=PIA03239\n\n### Baseline\n|time in s|megapixel per s|\n|---|---|\n|2.739997|56.3852|\n\n### openmp baseline\n\n|thread number|time in s|megapixel per s|\n|---|---|---|\n|12|0.411790|375.1795|\n|32|0.414154|373.0381|\n|64|0.414555|372.6776|\n|128|0.430195|359.1287|\n\n### memory\n\n|thread number|time in s|megapixel per s|\n|---|---|---|\n|32|0.032053|4820.0608|\n|64|0.031559|4895.3717|\n|128|0.030711|5030.6157|\n|256|0.032755|4716.6270|\n\n### FMA\n\n|thread number|time in s|megapixel per s|\n|---|---|---|\n|12|0.035387|4365.8509|\n|32|0.035855|4308.9137|\n|64|0.035242|4383.8510|\n|128|0.034176|4520.5836|\n|256|0.035138|4396.8387|\n\n### SSE\n\n|thread number|time in s|megapixel per s|\n|---|---|---|\n|12|0.030364|5088.0302|\n|32|0.030036|5143.7032|\n|64|0.030248|5107.6352|\n|128|0.030838|5009.9306|\n|256|0.032062|4818.7077|\n\n### AVX\n\n|thread number|time in s|megapixel per s|\n|---|---|---|\n|12|0.030029|5144.9279|\n|32|0.029775|5188.7483|\n|64|0.030188|5117.7360|\n|128|0.030685|5034.9111|\n|256|0.032302|4782.7716|\n\n### CPU comparison\n|CPU|algorithm|thread number|time in s|megapixel per s|\n|---|---|---|---|---|\n|AMD Ryzen 5 3600 (6 Core)|memory|128|0.030711|5030.6157|\n|AMD Ryzen 5 3600 (6 Core)|simd_sse|32|0.030036|5143.7032|\n|AMD Ryzen 5 3600 (6 Core)|simd_avx|32|0.029775|5188.7483|\n|Intel Core i7-4710HQ (4 Core)|memory|128|0.080105|1928.6639|\n|Intel Core i7-4710HQ (4 Core)|simd_sse|128|0.056655|2726.9456|\n|Intel Core i7-4710HQ (4 Core)|simd_avx|128|0.055232|2797.2128|\n|Intel Core i9-9880H (8 Core)|memory|128|0.038570|4005.5441|\n|Intel Core i9-9880H (8 Core)|simd_sse|64|0.041884|3688.6061|\n|Intel Core i9-9880H (8 Core)|simd_avx|128|0.027279|5663.5644|\n\n## Review\nA review of the AVX variant\n### Memory Bottleneck\nThe memory access should not be the bottleneck.\nThere are many indicator for this.\n\nFirst one being that the Intel Core i9-9880H performs better than the AMD Ryzen 5 3600 even though in has a memory bandwidth of only [39.74 GiB/s](https://en.wikichip.org/wiki/intel/core_i9/i9-9880h) compared to [47.68 GiB/s](https://en.wikichip.org/wiki/amd/ryzen_5/3600).\n\nSecond one being that the alignment of the memory in the AVX step, did not change the performance in any noticeable way.\nOne can test it out by replacing `_mm256_load_si256` with `_mm256_loadu_si256` and `_mm256_store_si256` with `_mm256_storeu_si256` and reverting the changes to align the memory.\n\nLastly doing a calculation of the theoretical amount of transferred data, we are below the available bandwidth.\n```\npixel = 27000*6000 = 162000000\nduration = 30ms = 0.03s\npixel_per_iteration = 32\nbytes_per_pixel = 3\niterations = pixel / pixel_per_iteration = 5062500\nbytes_read_per_iteration = 128 // it is not bytes_per_pixel*pixel_per_iteration=3*32=96, because we are effectively reading so many bytes in this sequence: 32 - 16 - 32 - 16. But because each read is 32-bytes, it results in 4*32=128 bytes read. Because the CPU address width is 8-byte there are no considerations here with 32-byte aligned memory.\nbytes_read = bytes_read_per_iteration * iterations = 128 * 5062500  = 648000000\nbytes_written = pixel * bytes_per_pixel = 162000000 * 3 = 486000000\nbytes_transferred = bytes_read + bytes_written = 648000000 + 486000000 = 1134000000\ntransfer_rate = bytes_transferred / duration = 1134000000 (byte) / 0.03s = 37800000000 b/s = 35,20 Gb/s \n```\n\n### Latency and Throughput\nEach SIMD functions has a different latency and throughput.\nLatency means how many clock cycles it takes for the calculation to be complete and throughput means how much of a clock cycle the operations takes.\nTaking [_mm256_load_si256](https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm256_load_si256\u0026ig_expand=1935,515,305,4291) as example on the Haswell architecture.\nIt has a latency of 1 and throughput of 0.25.\nThis mean we can load 4 independent datasets in a single clock cycle.\nThis is a source of optimization, that was not done here.\nIt could also be the source of the different benchmark results for the different algorithms between the CPUs.\nInstead functions were selected purely on being able to arrange the bytes in a way that was needed.\nThe linked intrinsic guide is for Intel, so one for AMD would need to be found first.\n\n### Number of Threads\nThis is something that can be further investigated.\nCurrently they were just discovered through trial and error without putting much thought behind them.\n\n# Conclusion\nGrayscale conversion is not suited to be computed with the GPU due to transfer 4 bytes of data for each pixel, which are 5 floating point operations in order to convert it to grayscale.\nThe CPU on the other hand is very suited with the easiest performance improvement being to parallelize it and grouping the memory access. This can be easily done because each calculation is independent from another one.\nSIMD can be used to further improve it, but only in the very smallest margins at the cost of huge developer overhead.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiangronglin%2Fgrayscale-conversion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxiangronglin%2Fgrayscale-conversion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiangronglin%2Fgrayscale-conversion/lists"}