{"id":20694734,"url":"https://github.com/cse-25/rt_linux_latency","last_synced_at":"2026-05-09T07:33:33.437Z","repository":{"id":261265062,"uuid":"883727714","full_name":"CSE-25/RT_Linux_Latency","owner":"CSE-25","description":"A Case study on latency difference between Desktop Linux Kernel and Linux RT Kernel patch","archived":false,"fork":false,"pushed_at":"2024-11-06T06:24:43.000Z","size":24,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-01-17T18:01:53.797Z","etag":null,"topics":["bash","kernel","linux","realtime"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CSE-25.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-05T13:23:10.000Z","updated_at":"2024-11-06T06:24:47.000Z","dependencies_parsed_at":"2024-11-05T16:41:47.683Z","dependency_job_id":"9840f0b8-e6e1-4ca6-a8ca-8da37c0e9719","html_url":"https://github.com/CSE-25/RT_Linux_Latency","commit_stats":null,"previous_names":["cse-25/rt_linux_latency"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSE-25%2FRT_Linux_Latency","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSE-25%2FRT_Linux_Latency/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSE-25%2FRT_Linux_Latency/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSE-25%2FRT_Linux_Latency/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CSE-25","download_url":"https://codeload.github.com/CSE-25/RT_Linux_Latency/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242958495,"owners_count":20212812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","kernel","linux","realtime"],"created_at":"2024-11-17T00:06:13.469Z","updated_at":"2025-12-16T06:12:18.440Z","avatar_url":"https://github.com/CSE-25.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e A Case Study On Latency in Real Time Linux \u003c/h1\u003e\nIn order to fulfill the requirements of a real-time system, a system must react to an external event like an interrupt within a defined time frame.\n\n\u003ch2\u003eSteps to give a Linux kernel, Real Time properties\u003c/h2\u003e\n\u003cul\u003e\n  \u003cli\u003edownload the kernel\u003c/li\u003e\n  \u003cli\u003edownload the PREEMPT_RT patch\u003c/li\u003e\n  \u003cli\u003epatch the kernel\u003c/li\u003e\n  \u003cli\u003ebuild the kernel\u003c/li\u003e\n  \u003cli\u003erestart your system\u003c/li\u003e\n  \u003cli\u003echoose the RT kernel\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch3\u003eDownload the kernel\u003c/h3\u003e\n\nFirst select the appropriate RT kernel, whose version is as close as possible to the kernel version of the respective distribution.\n\n\u003e [!TIP]\n\u003e ### How to check kernel version on Linux?\n\u003e ```bash\n\u003e uname -r\n\u003e ```\n\u003e or\n\u003e ```bash\n\u003e cat /proc/version\n\u003e ```\n\nSuppose we get \u003cbr\u003e\n`Linux version 5.15.0-122-generic` \u003cbr\u003e\nThis means you have kernel major version 5, minor version 15, patch 0 and sublevel 122\n\n\u003ch3\u003eDownload the PREEMPT_RT patch\u003c/h3\u003e\n\nGet the kernel that is closest to the current non-RT version from [Linux_RT](https://cdn.kernel.org/pub/linux/kernel/projects/rt/) page. \u003cbr\u003e\n`patch-5.15.167-rt79.patch.xz` is the closest from the page.\n\n\u003ch3\u003ePatch the kernel\u003c/h3\u003e\n\nWith the following commands the appropriate source code is downloaded, the patch is downloaded, the archive is unpacked and the patch applied:\n\n```bash\nmkdir -p /usr/src/kernels\ncd /usr/src/kernels\nwget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.122.tar.gz\nwget https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.15/patch-5.15.167-rt79.patch.xz\ntar xf linux-5.15.122.tar.gz\nmv linux-5.15.122 linux-5.15.122-rt79\ncd linux-5.15.122-rt79\nxz -d ../patch-5.15.167-rt79.patch.xz\npatch -p1 \u003c../patch-5.15.167-rt79.patch\n```\n\nTo ensure that the RT kernel supports the current distribution as well as possible, the first step is to take over its kernel configuration, which is located in the /bootdirectory of Debian, as with many other distributions, and which matches the kernel version number. \n\n\u003cbr\u003e\n\nIn this case, it is the file `/boot/config-5.15.0-122-generic`, which is now copied to the root directory of the kernel source code under the name `.config`:\n\n```bash\ncp /boot/config-5.15.0-122-generic .config\n```\n\n\u003ch3\u003eBuild the kernel\u003c/h3\u003e\n\nIn the last step, before the kernel can be compiled, the new kernel has to be configured so that the functionality imported with the RT patch is also used. The command make menuconfig is called and we select Processor type and features -\u003e Preemption Model -\u003e Fully Preemptible Kernel (RT).\n\nThe kernel is then compiled, installed, and the system must be rebooted.\n\n```bash\nmake -j4 \nmake modules_install install\n```\n\n\u003ch3\u003eRestart your system and choose the RT Kernel\u003c/h3\u003e\n\n```bash\nsudo reboot\n```\n\n\u003e [!IMPORTANT]  \n\u003e ### Verifying the system’s response behavior.\n\u003e After rebooting, you should make sure that the new kernel is properly configured. This is indicated by whether the flags PREEMPT and RT are included in the output of the program uname.\n\u003e ```bash\n\u003e  uname -v | cut -d\" \" -f1-4 \n\u003e ```\n\u003e If the Output is something like `#1 SMP PREEMPT RT`, kernel is properly configured. \n\n\u003ch2\u003eExpected Time Span\u003c/h2\u003e\n\nThe expected time span can be calculated with the rule of thumb\n\n```\nmaximum_latency = clock_interval * 10⁵\n```\n\nThis means that, for example, in a system with a clock frequency of 1 GHz and thus a clock interval of 1 ns, a maximum latency of less than 100 µs can be expected.\n\n\u003ch2\u003ePlotting latency\u003c/h2\u003e\n\n- For measuring latency we are using `Cyclictest` from `rt-tests` package.\n- It is one of the most frequently used tools for evaluating the relative performance of real-time systems.\n- Cyclictest accurately and repeatedly measures the difference between a thread's intended wake-up time and the time at which it actually wakes up from clock_nanosleep in order to provide statistics about the system's latencies.\n- It can measure latencies in real-time systems caused by the hardware, the firmware, and the operating system.\n\n\u003ch3\u003eInstalling rt-tests\u003c/h3\u003e\n\nEither clone the git repo and compile\n\n```git\ngit clone git://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git\n```\n```bash\nsudo apt-get install build-essential libnuma-dev\nmake\n```\n(or)\n\nInstall from apt in debian\n\n```shell\nsudo apt install rt-tests\n```\n\n\u003e [!NOTE]\n\u003e We can use `cyclictest` command directly if installed via apt. We have to use `sudo ./cyclictest` from the appropriate directory if we are working with compiled src code.\n\n\u003ch3\u003ePlotting Script\u003c/h3\u003e\n\nRun the [`plot.bash`](./plot.bash) Script file to produce `plot.png` and cpu core wise logs.\n\n**Code**\n```bash\n#!/bin/bash\n\n# 1. Run cyclictest\ncyclictest -l100000 -m -Sp90 -i200 -h400 -q \u003eoutput \n\n# 2. Get maximum latency\nmax=`grep \"Max Latencies\" output | tr \" \" \"\\n\" | sort -n | tail -1 | sed s/^0*//`\n\n# 3. Grep data lines, remove empty lines and create a common field separator\ngrep -v -e \"^#\" -e \"^$\" output | tr \" \" \"\\t\" \u003ehistogram \n\n# 4. Set the number of cores, for example\ncores=4\n\n# 5. Create two-column data sets with latency classes and frequency values for each core, for example\nfor i in `seq 1 $cores`\ndo\n  column=`expr $i + 1`\n  cut -f1,$column histogram \u003ehistogram$i\ndone\n\n# 6. Create plot command header\necho -n -e \"set title \\\"Latency plot\\\"\\n\\\nset terminal png\\n\\\nset xlabel \\\"Latency (us), max $max us\\\"\\n\\\nset logscale y\\n\\\nset xrange [0:400]\\n\\\nset yrange [0.8:*]\\n\\\nset ylabel \\\"Number of latency samples\\\"\\n\\\nset output \\\"plot.png\\\"\\n\\\nplot \" \u003eplotcmd\n\n# 7. Append plot command data references\nfor i in `seq 1 $cores`\ndo\n  if test $i != 1\n  then\n    echo -n \", \" \u003e\u003eplotcmd\n  fi\n  cpuno=`expr $i - 1`\n  if test $cpuno -lt 10\n  then\n    title=\" CPU$cpuno\"\n   else\n    title=\"CPU$cpuno\"\n  fi\n  echo -n \"\\\"histogram$i\\\" using 1:2 title \\\"$title\\\" with histeps\" \u003e\u003eplotcmd\ndone\n\n# 8. Execute plot command\ngnuplot -persist \u003cplotcmd\n\n```\n\u003e [!IMPORTANT]\n\u003e `gnuplot` package should be installed before running this script.\n\u003e ```bash\n\u003e sudo apt install gnuplot\n\u003e ```\n\n\n\u003ch2\u003eExperiment 1: RT Kernel Raspberry Pi 5 vs Non RT Kernel Ryzen 9 5900HX Laptop\u003c/h2\u003e\n\n\u003ch3\u003eDevice 1: Rapberrypi with RT Kernel\u003c/h3\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/user-attachments/assets/ac935f9c-bc67-482a-8e16-fe529318795c\" width=\"800px\"/\u003e\n\u003c/div\u003e\n\n\u003ch3\u003eDevice 2: Laptop with Non- RT Kernel\u003c/h3\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/user-attachments/assets/61c118e6-d89c-4d48-b491-3e90efd612a3\" width=\"800px\"/\u003e\n\u003c/div\u003e\n\n\u003ch3\u003e Experiment Results Comparision \u003c/h3\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n**10,000 samples**\n\n| Without RT Kernel | With RT Kernel |\n| --- | --- |\n| ![10k_wo_rt](https://github.com/user-attachments/assets/435b0b54-9222-462b-8b66-b551a3aa26eb) | ![10k_w_rt](https://github.com/user-attachments/assets/80ac620a-6d36-4fb7-ad00-91db7787c728) |\n\n\u003cbr\u003e\n\n**100,000 samples**\n\n| Without RT Kernel | With RT Kernel |\n| --- | --- |\n| ![100k_wo_rt](https://github.com/user-attachments/assets/6f8b2829-3345-4646-b770-98398700ccaa) | ![100k_w_rt](https://github.com/user-attachments/assets/51fc6bb8-aa0c-4d7d-aee7-b56b7c33b0da) |\n\n\u003cbr\u003e\n\n**1,000,000 samples**\n\n| Without RT Kernel | With RT Kernel |\n| --- | --- |\n| ![1m_wo_rt](https://github.com/user-attachments/assets/03af4d1d-1128-484a-a096-40d01e056591) | ![1m_w_rt](https://github.com/user-attachments/assets/852ef32b-d749-45d0-926b-ff8b1f49621f) |\n\n\u003c/div\u003e\n\n\u003cbr\u003e\n\n\u003ch2\u003eReferences\u003c/h2\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \n| Description | Link |\n| --- | --- |\n| `Real Time Applications` | \u003chttps://wiki.linuxfoundation.org/realtime/documentation/howto/applications/start\u003e |\n| `Raspberrypi applying kernel patches` | \u003chttps://www.raspberrypi.com/documentation/computers/linux_kernel.html#apply-patches\u003e |\n| `rt-tests src code` | \u003chttps://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git/tree/src\u003e |\n| `Cyclictest Documentation` | \u003chttps://wiki.linuxfoundation.org/realtime/documentation/howto/tools/cyclictest/test-design#option-selection\u003e |\n| `Deadline Scheduling in RT Linux` | \u003chttps://www.kernel.org/doc/Documentation/scheduler/sched-deadline.txt\u003e |\n| `RT Kernel installation in x86 systems and plotting latency` | \u003chttps://medium.com/@patdhlk/realtime-linux-e97628b51d5d\u003e |\n\n\u003c/div\u003e\n\n\u003cbr\u003e\n\n\u003ch2\u003e Contributors \u003c/h2\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \n| Name | Roll No. |\n| --- | --- |\n| Abhinav R | `CB.EN.U4CSE21001` |\n| Ashwin Narayanan S | `CB.EN.U4CSE21008` |\n| Hariharan A | `CB.EN.U4CSE21021` |\n| Balaji K | `CB.EN.U4CSE21110` |\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcse-25%2Frt_linux_latency","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcse-25%2Frt_linux_latency","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcse-25%2Frt_linux_latency/lists"}