{"id":13464223,"url":"https://github.com/tinyclub/linux-lab","last_synced_at":"2025-05-14T17:02:32.279Z","repository":{"id":37663511,"uuid":"63026379","full_name":"tinyclub/linux-lab","owner":"tinyclub","description":"Docker/Qemu Based Linux Kernel Learning, Development and Testing Environment; New Linux ELF Video Course from this project author: https://www.cctalk.com/m/group/88089283","archived":false,"fork":false,"pushed_at":"2024-10-25T15:31:34.000Z","size":153128,"stargazers_count":1177,"open_issues_count":1,"forks_count":175,"subscribers_count":61,"default_branch":"master","last_synced_at":"2025-04-13T01:58:06.815Z","etag":null,"topics":["buildroot","busybox","cross-compiler","docker","embedded-linux","lab","linux-kernel","qemu","uboot"],"latest_commit_sha":null,"homepage":"http://tinylab.org/linux-lab","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinyclub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-07-11T01:12:09.000Z","updated_at":"2025-04-05T12:21:55.000Z","dependencies_parsed_at":"2023-01-30T23:31:31.793Z","dependency_job_id":"2402b215-c827-447c-a182-c0163c16b1f5","html_url":"https://github.com/tinyclub/linux-lab","commit_stats":null,"previous_names":[],"tags_count":56,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyclub%2Flinux-lab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyclub%2Flinux-lab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyclub%2Flinux-lab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyclub%2Flinux-lab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinyclub","download_url":"https://codeload.github.com/tinyclub/linux-lab/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254190366,"owners_count":22029629,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["buildroot","busybox","cross-compiler","docker","embedded-linux","lab","linux-kernel","qemu","uboot"],"created_at":"2024-07-31T14:00:37.278Z","updated_at":"2025-05-14T17:02:32.211Z","avatar_url":"https://github.com/tinyclub.png","language":"Makefile","funding_links":[],"categories":["Makefile","docker","Projects"],"sub_categories":["Linux kernel"],"readme":"\u003c!-- metadata start --\u003e\u003c!--\n% Linux Lab v1.4 Manual\n% [TinyLab Community | Tinylab.org][044]\n% \\today\n--\u003e\u003c!-- metadata end --\u003e\n\n![Linux Lab Logo](doc/images/linux-lab-logo.jpg)\n\n**Subscribe Wechat**：\n\n![Wechat Public](doc/images/tinylab-wechat.jpg)\n\n\u003c!-- toc start --\u003e\n\n# Table of Content\n\n- [1. Linux Lab Overview](#1-linux-lab-overview)\n - [1.1 Project Introduction](#11-project-introduction)\n - [1.2 Project Homepage](#12-project-homepage)\n - [1.3 Demonstration](#13-demonstration)\n - [1.3.1 Free Video Courses In Chinese](#131-free-video-courses-in-chinese)\n - [1.3.2 Non-Free Video Courses In Chinese](#132-non-free-video-courses-in-chinese)\n - [1.4 Project Functions](#14-project-functions)\n - [1.5 Project History](#15-project-history)\n - [1.5.1 Project Origins](#151-project-origins)\n - [1.5.2 Problems Solved](#152-problems-solved)\n - [1.5.3 Project Born](#153-project-born)\n- [2. Linux Lab Installation](#2-linux-lab-installation)\n - [2.1 Hardware and Software Requirement](#21-hardware-and-software-requirement)\n - [2.2 Docker Installation](#22-docker-installation)\n - [2.3 Choose a working directory](#23-choose-a-working-directory)\n - [2.4 Switch to normal user](#24-switch-to-normal-user)\n - [2.5 Download the lab](#25-download-the-lab)\n - [2.6 Run and login the lab](#26-run-and-login-the-lab)\n - [2.7 Update and rerun the lab](#27-update-and-rerun-the-lab)\n - [2.8 Quickstart: Boot a board](#28-quickstart-boot-a-board)\n- [3. Linux Lab Kickstart](#3-linux-lab-kickstart)\n - [3.1 Using boards](#31-using-boards)\n - [3.1.1 List available boards](#311-list-available-boards)\n - [3.1.2 Choosing a board](#312-choosing-a-board)\n - [3.1.2.1 Real board](#3121-real-board)\n - [3.1.2.2 Virtual board](#3122-virtual-board)\n - [3.1.2.3 Buy one](#3123-buy-one)\n - [3.1.3 Using as plugins](#313-using-as-plugins)\n - [3.1.4 Configure boards](#314-configure-boards)\n - [3.2 Build in one command](#32-build-in-one-command)\n - [3.3 Detailed Operations](#33-detailed-operations)\n - [3.3.1 Downloading](#331-downloading)\n - [3.3.2 Checking out](#332-checking-out)\n - [3.3.3 Patching](#333-patching)\n - [3.3.4 Configuration](#334-configuration)\n - [3.3.4.1 Default Configuration](#3341-default-configuration)\n - [3.3.4.2 Manual Configuration](#3342-manual-configuration)\n - [3.3.4.3 Old default configuration](#3343-old-default-configuration)\n - [3.3.5 Building](#335-building)\n - [3.3.6 Saving](#336-saving)\n - [3.3.7 Booting](#337-booting)\n- [4. Linux Lab Advance](#4-linux-lab-advance)\n - [4.1 Using Linux Kernel](#41-using-linux-kernel)\n - [4.1.1 non-interactive configuration](#411-non-interactive-configuration)\n - [4.1.2 using kernel modules](#412-using-kernel-modules)\n - [4.1.3 using kernel features](#413-using-kernel-features)\n - [4.1.3.1 list supported kernel features](#4131-list-supported-kernel-features)\n - [4.1.3.2 using kernel modules](#4132-using-kernel-modules)\n - [4.1.3.3 using rust feature](#4133-using-rust-feature)\n - [4.1.3.4 using kft feature](#4134-using-kft-feature)\n - [4.1.3.5 using rt feature](#4135-using-rt-feature)\n - [4.1.3.6 persist or clear feature setting](#4136-persist-or-clear-feature-setting)\n - [4.1.4 Create new development branch](#414-create-new-development-branch)\n - [4.1.5 Use standalone git repository](#415-use-standalone-git-repository)\n - [4.2 Using U-Boot Bootloader](#42-using-u-boot-bootloader)\n - [4.3 Using QEMU Emulator](#43-using-qemu-emulator)\n - [4.4 Using Toolchains](#44-using-toolchains)\n - [4.5 Using Rootfs](#45-using-rootfs)\n - [4.6 Debugging Linux and U-Boot](#46-debugging-linux-and-u-boot)\n - [4.6.1 Debugging Linux](#461-debugging-linux)\n - [4.6.2 Debugging U-Boot](#462-debugging-u-boot)\n - [4.7 Test Automation](#47-test-automation)\n - [4.8 File Sharing](#48-file-sharing)\n - [4.8.1 Install files to rootfs](#481-install-files-to-rootfs)\n - [4.8.2 Share with NFS](#482-share-with-nfs)\n - [4.8.3 Transfer via tftp](#483-transfer-via-tftp)\n - [4.8.4 Share with 9p virtio](#484-share-with-9p-virtio)\n - [4.9 Learning Assembly](#49-learning-assembly)\n - [4.10 Learning C](#410-learning-c)\n - [4.10.1 Host build and Run](#4101-host-build-and-run)\n - [4.10.2 Cross build and Run](#4102-cross-build-and-run)\n - [4.11 Running any make goals](#411-running-any-make-goals)\n - [4.12 Speed up kernel development](#412-speed-up-kernel-development)\n - [4.12.1 Speed up compiling and save disk life](#4121-speed-up-compiling-and-save-disk-life)\n - [4.12.2 ONESHOT Mode](#4122-oneshot-mode)\n - [4.12.3 Nolibc Mode](#4123-nolibc-mode)\n - [4.12.4 Tiny Mode](#4124-tiny-mode)\n - [4.13 More Usage](#413-more-usage)\n- [5. Linux Lab Development](#5-linux-lab-development)\n - [5.1 Choose a board supported by QEMU](#51-choose-a-board-supported-by-qemu)\n - [5.2 Create the board directory](#52-create-the-board-directory)\n - [5.3 Clone a Makefile from an existing board](#53-clone-a-makefile-from-an-existing-board)\n - [5.4 Configure the variables from scratch](#54-configure-the-variables-from-scratch)\n - [5.5 At the same time, prepare the configs](#55-at-the-same-time-prepare-the-configs)\n - [5.6 Choose the versions of kernel, rootfs and U-Boot](#56-choose-the-versions-of-kernel-rootfs-and-u-boot)\n - [5.7 Configure, build and boot them](#57-configure-build-and-boot-them)\n - [5.8 Save the images and configs](#58-save-the-images-and-configs)\n - [5.9 Upload everything](#59-upload-everything)\n- [6. FAQs](#6-faqs)\n - [6.1 Docker Issues](#61-docker-issues)\n - [6.1.1 Speed up docker images downloading](#611-speed-up-docker-images-downloading)\n - [6.1.2 Docker network conflicts with LAN](#612-docker-network-conflicts-with-lan)\n - [6.1.3 Why not allow running Linux Lab in local host](#613-why-not-allow-running-linux-lab-in-local-host)\n - [6.1.4 Run tools without sudo](#614-run-tools-without-sudo)\n - [6.1.5 Network not work](#615-network-not-work)\n - [6.1.6 Client.Timeout exceeded while waiting headers](#616-clienttimeout-exceeded-while-waiting-headers)\n - [6.1.7 Restart Linux Lab after host system shutdown or reboot](#617-restart-linux-lab-after-host-system-shutdown-or-reboot)\n - [6.1.8 the following directives are specified both as a flag and in the configuration file](#618-the-following-directives-are-specified-both-as-a-flag-and-in-the-configuration-file)\n - [6.1.9 pathspec FETCH_HEAD did not match any file known to git](#619-pathspec-fetch_head-did-not-match-any-file-known-to-git)\n - [6.1.10 Docker not work in Ubuntu 20.04](#6110-docker-not-work-in-ubuntu-2004)\n - [6.1.11 Error creating aufs mount](#6111-error-creating-aufs-mount)\n - [6.2 QEMU Issues](#62-qemu-issues)\n - [6.2.1 Why kvm speedding up is disabled](#621-why-kvm-speedding-up-is-disabled)\n - [6.2.2 Poweroff hang](#622-poweroff-hang)\n - [6.2.3 How to exit QEMU](#623-how-to-exit-qemu)\n - [6.2.4 Boot with missing sdl2 libraries failure](#624-boot-with-missing-sdl2-libraries-failure)\n - [6.3 Environment Issues](#63-environment-issues)\n - [6.3.1 NFS/tftpboot not work](#631-nfstftpboot-not-work)\n - [6.3.2 How to switch Windows in VIM](#632-how-to-switch-windows-in-vim)\n - [6.3.3 How to delete typo in shell command line](#633-how-to-delete-typo-in-shell-command-line)\n - [6.3.4 Language input switch shortcuts](#634-language-input-switch-shortcuts)\n - [6.3.5 How to tune the screen size](#635-how-to-tune-the-screen-size)\n - [6.3.6 How to work in fullscreen mode](#636-how-to-work-in-fullscreen-mode)\n - [6.3.7 How to record video](#637-how-to-record-video)\n - [6.3.8 Linux Lab not response](#638-linux-lab-not-response)\n - [6.3.9 VNC login with failures](#639-vnc-login-with-failures)\n - [6.3.10 Ubuntu Snap Issues](#6310-ubuntu-snap-issues)\n - [6.3.11 How to exit fullscreen mode of vnc clients](#6311-how-to-exit-fullscreen-mode-of-vnc-clients)\n - [6.4 Lab Issues](#64-lab-issues)\n - [6.4.1 No working init found](#641-no-working-init-found)\n - [6.4.2 linux/compiler-gcc7.h: No such file or directory](#642-linuxcompiler-gcc7h-no-such-file-or-directory)\n - [6.4.3 linux-lab/configs: Permission denied](#643-linux-labconfigs-permission-denied)\n - [6.4.4 scripts/Makefile.headersinst: Missing UAPI file](#644-scriptsmakefileheadersinst-missing-uapi-file)\n - [6.4.5 unable to create file: net/netfilter/xt_dscp.c](#645-unable-to-create-file-netnetfilterxt_dscpc)\n - [6.4.6 how to run as root](#646-how-to-run-as-root)\n - [6.4.7 not in supported list](#647-not-in-supported-list)\n - [6.4.8 is not a valid rootfs directory](#648-is-not-a-valid-rootfs-directory)\n- [7. Contact and Sponsor](#7-contact-and-sponsor)\n - [7.1 Contact Us](#71-contact-us)\n - [7.2 Give me a star](#72-give-me-a-star)\n - [7.3 Buy our products](#73-buy-our-products)\n - [7.4 Sponsor](#74-sponsor)\n - [7.4.1 Sponsor via wechat](#741-sponsor-via-wechat)\n - [7.4.2 Sponsor list](#742-sponsor-list)\n\n\u003c!-- toc end --\u003e\n\n# 1. Linux Lab Overview\n\n## 1.1 Project Introduction\n\nThis project aims to create a Docker and QEMU based Linux development Lab to easier the learning, development and testing of [Linux Kernel][040].\n\nLinux Lab is open source with no warranty – use at your own risk.\n\n[![Docker QEMU Linux Lab](doc/images/linux-lab.png)][043]\n\n**Friendly Notice**: TinyLab Community have developed [Linux Lab Disk][023], you can buy from [TinyLab.org's Taobao Shop][024] or [Bilibili Shop][025].\n\n## 1.2 Project Homepage\n\n* Homepage\n * \u003chttps://tinylab.org/linux-lab/\u003e\n * \u003chttps://oschina.net/p/linux-lab\u003e\n\n* Repository\n * \u003chttps://gitee.com/tinylab/linux-lab\u003e\n * \u003chttps://github.com/tinyclub/linux-lab\u003e\n\nRelated Projects:\n\n* Cloud Lab\n * Linux Lab Running Environment Manager, provide GUI and CLI, support local and remote login\n * \u003chttps://tinylab.org/cloud-lab\u003e\n\n* Linux 0.11 Lab\n * Learning Linux 0.11, only available in Linux Lab Disk from now on\n * Download it to `labs/linux-0.11-lab` and use it in Linux Lab directly\n * \u003chttps://tinylab.org/linux-0.11-lab\u003e\n\n* CS630 QEMU Lab\n * Learning X86 Linux Assembly, only available in Linux Lab Disk from now on\n * Download it to `labs/cs630-qemu-lab` and use it in Linux Lab directly\n * \u003chttps://tinylab.org/cs630-qemu-lab\u003e\n\n* RVOS Lab\n * Learning RISC-V OS course, merged in [Linux Lab Disk][029]\n * Download it to `src/examples` and use it in Linux Lab directly\n * \u003chttps://gitee.com/tinylab/rvos-lab\u003e\n\n* GUI Lab\n * Learning embedded GUI (e.g. Guilite)，merged in [Linux Lab Disk][029]\n * Download it to `src/examples` and use it in Linux Lab directly\n * \u003chttps://gitee.com/tinylab/gui-lab\u003e\n\n* RISC-V Linux\n * Learning RISC-V Linux kernel, merged in [Linux Lab Disk][029]\n * Download it to `src/examples` and use it in Linux Lab directly\n * \u003chttps://gitee.com/tinylab/riscv-linux\u003e\n\n* RISC-V Lab\n * Learning embedded RISC-V software development，merged in [Linux Lab Disk][029] for RISC-V\n * \u003chttps://gitee.com/tinylab/riscv-lab\u003e\n\n* ARM Lab\n * Learning embedded ARM software development，merged in [Linux Lab Disk][029] for ARM\n * \u003chttps://gitee.com/tinylab/arm-lab\u003e\n\n## 1.3 Demonstration\n\n### 1.3.1 Free Video Courses In Chinese\n\n* [Linux Lab Open Videos][036]\n * Linux Lab Introduction\n * Loongson Linux Development\n * Linux Lab Disk Demonstration\n * Linux Lab Release Meeting Replay Videos\n * Rust For Linux Introduction\n\n* [Linux Kernel Observation][051]\n\n* [RISC-V Linux Kernel Investigation][052]\n\n* RISC-V Linux System Development Course\n * [Part I: Embedded Quickstart][https://space.bilibili.com/687228362/channel/collectiondetail?sid=1750690], Require [Linux RISC-V Disk][023]\n * [Part II: Embedded Practice][https://space.bilibili.com/687228362/channel/collectiondetail?sid=2021659], Require [Tiny RISC-V Box][055]\n * [Part III: Embedded Advance][https://space.bilibili.com/687228362/channel/collectiondetail?sid=3128538], Require [Tiny RISC-V Box][055]\n\n### 1.3.2 Non-Free Video Courses In Chinese\n\n* [《The Perspective of Linux ELF》][035]\n * Learn Linux ELF by practice, with hundreds of examples, all verified in Linux Lab\n\n* [《Rust Language Quickstart》][006]\n * Rust course for C programmer, with examples verified in Linux Lab\n\n* [《Software Reverse Engineering Quickstart》][037]\n * Learn reverse engineering by practice, with examples verified in Linux Lab\n\n* [《Linux Kernel Livepatch Introduction》][038]\n * Learn Linux live patching in AArch64 by practice, with examples verified in Linux Lab\n\n## 1.4 Project Functions\n\nNow, Linux Lab becomes an intergrated Linux learning, development and testing environment, it supports:\n\n| Items | Description |\n|------------|---------------------------------------------------------------------------------------------------|\n| Boards | QEMU based, 7+ main Architectures, 20+ popular boards; Several real boards supported too |\n| Components | Uboot, Linux / Modules, Buildroot, Qemu, Linux v0.11, v2.6.10 ~ 5.x supported |\n| Prebuilt | All of above components have been prebuilt |\n| Rootfs | Support include initrd, harddisk, mmc and nfs, Debian availab for ARM |\n| Docker | Cross toolchains from gcc-4.3 available in one command, external ones configurable |\n| Access | Accessible from local or remote, include CLI and GUI, support bash, ssh, vnc, web vnc and web ssh |\n| Network | Builtin bridge networking, every board has network (except Raspi3) |\n| Boot | Support serial port, curses (bash/ssh friendly) and graphic booting |\n| Testing | Support automatic testing via `make test` target |\n| Debugging | debuggable via `make debug` target |\n\nContinue reading for more features and usage.\n\n## 1.5 Project History\n\n### 1.5.1 Project Origins\n\nAbout 10 years ago (2010), a tinylinux proposal: [Work on Tiny Linux Kernel][010] accepted by Embedded\nLinux Foundation, therefore I have worked on this project for serveral months.\n\n### 1.5.2 Problems Solved\n\nDuring the project cycle, several scripts written to verify if the adding tiny features (e.g. [gc-sections][021])\nbreaks the other kernel features on the main CPU architectures.\n\nThese scripts uses qemu-system-ARCH as the cpu/board simulator, basic boot+function tests have been done for ftrace+perf, accordingly, defconfigs,\nrootfs, test scripts have been prepared, at that time, all of them were simply put in a directory, without a design or holistic consideration.\n\n### 1.5.3 Project Born\n\nThey have slept in my harddisk for several years without any attention, until one day, docker and novnc came to my world, at first, [Linux 0.11 Lab][004] was born, after that, Linux Lab was designed to unify all of the above scripts, defconfigs, rootfs and test scripts.\n\n# 2. Linux Lab Installation\n\nLinux Lab uses Docker, if have already installed Docker and configured the best mirror site of docker images, it is very easy to install Linux Lab.\n\nIf really a Linux newbie or simply don't want to spend time on boring installation, buy the instant Linux Lab Disk:\n\n[![Linux Lab Disk](doc/images/linux-lab-disk-demo.png)][023]\n\nIt supports:\n\n* Capacity\n * From 32G to 512G and even 1T, 2T, 4T\n* Products\n * High Speed U Disk, Solid U Disk, Portable disk, Solid disk (NVME / SATA)\n* Systems\n * Top6 Linux Distributions and even more based on your requirement\n * Include Ubuntu 18.04-22.04, Deepin 20.8+, Fedora 37+, Mint 21.1+, Kali, Manjaro\n* Features\n * Boot from any powered-off 64bit X86 Machine, include PC, Laptop and MacBook\n * Boot from any running Windows, Linux and run in parallel with them\n * Switch from or to any running Windows, Linux without poweroff\n * Multiple Linux Lab Disks can boot or switch from/to each other\n * Support timezone setting of different systems transparently, without manual setting\n * Share files and clipboards automatically between the main system and our disk system\n * Support transparent compress, use 128G as ~256G capacity\n * Support memory compiling, speedup compiling and save disk erase life\n * Support factory restore, allow restore factory system in some cases\n * Support volatile memory booting, allow read and write from memory, faster and longer\n * Merged in many labs, such as Linux Lab, Linux 0.11 Lab, be able to learn Linux kernel, embedded Linux, Uboot, Assembly, C, Python, Database, Network and so forth\n* Where to buy\n * [Taobao shop of TinyLab.org Community][023]\n * [Bilibili Shop][025]\n* Product details\n * \u003chttps://tinylab.org/linux-lab-disk\u003e\n * Introduce and demonstrate the features, functions and usage of Linux Lab Disk\n\n## 2.1 Hardware and Software Requirement\n\nLinux Lab is a full embedded Linux development system, it needs enough calculation capacity and disk \u0026 memory storage space, to avoid potential extension issues, here is the recommended configuration:\n\n| Hardware | Requirement | Description |\n|-----------|------------------|------------------------------------------------------|\n| Processor | X86_64, \u003e 1.5GHz | Must choose 64bit X86 while using virtual machine |\n| Disk | \u003e= 50G | System (25G), Docker Images(~5G), Linux Lab (20G) |\n| Memory | \u003e= 4G | Lower than 4G may have many unpredictable exceptions |\n\nIf often use, please increase disk storage to 100G~200G, memory storage to 8G, cpu cores to 4 and above.\n\nCurrently, all of the X86_64 systems support Docker should be able to run Linux Lab, include Windows, Linux and MacOS, all of the popular Linux distributions may have been tried by different users.\n\nWelcome to take a look at [the systems running Linux Lab][016] and share yours, for example:\n\n $ cd /path/to/cloud-lab\n $ tools/docker/env\n System: Ubuntu 16.04.6 LTS\n Linux: 4.4.0-176-generic\n Docker: Docker version 18.09.4, build d14af54\n\n## 2.2 Docker Installation\n\nDocker is required by Linux Lab, please install it at first:\n\n - Linux, Mac OSX, Windows 10\n\n [Docker CE][026]\n\n - older Windows (include some older Windows 10)\n\n [Docker Toolbox][011]; Install Ubuntu via Virtualbox or Vmware Virtual Machine\n\nBefore running Linux Lab, please refer to section 6.1.4 and make sure the following command works without sudo and without any issue:\n\n $ docker run hello-world\n\nIn China, to use docker service normally, please **must** configure one of chinese docker mirror sites, for example:\n\n* [Aliyun Docker Mirror Documentation][018]\n * For non Univerisity users, require login with freely registered account\n\n* [USTC Docker Mirror Documentation][020]\n * For Univerisity users\n\nMore docker related issues, such as download slowly, download timeout and download errors, are cleary documented in the 6.1 section of FAQs.\n\nThe other issues, please read the [official docker docs][007].\n\n**Notes for Ubuntu Users**\n - [doc/install/ubuntu-docker.md][003]\n\n**Notes for Arch Users**\n - [doc/install/arch-docker.md][001]\n\n**Notes for Manjaro Users**\n - [doc/install/manjaro-docker.md][002]\n\n**Notes for Windows Users**:\n\n - Please make sure your Windows version support docker: [Official Docker Documentation][007] and determine Docker Desktop or Docker Toolbox should be used\n\n - Linux Lab only tested with 'Git Bash' in Windows, please must use with it\n - After installing [Git For Windows][017], \"Git Bash Here\" will come out in right-button press menu\n\n## 2.3 Choose a working directory\n\nPlease simply choose one directory in `~/Downloads` or `~/Documents` or create a new `~/Develop` directory.\n\n $ mkdir ~/Develop\n $ cd ~/Develop\n\nFor Windows and Mac OSX, to compile Linux normally, please refer to section 5.7.1 and enable building cache:\n\n## 2.4 Switch to normal user\n\nBefore downloading Linux Lab, please **MUST** switch to normal user.\n\nCheck who am i, `0` means root, non-zero means normal user:\n\n $ id -u `whoami`\n 1000\n\nIf current user is `root`, switch to a normal one:\n\n # id -u `whoami`\n 0\n # sudo -su \u003cUSER\u003e\n\nIf no normal user exists, create new:\n\n $ sudo useradd --create-home --shell /bin/bash --user-group --groups adm,sudo laber\n $ sudo passwd laber\n $ sudo -su laber\n $ whoami\n laber\n\n## 2.5 Download the lab\n\nUse Ubuntu system as an example:\n\nDownload cloud lab framework with normal user, pull images and checkout linux-lab repository:\n\n $ git clone https://gitee.com/tinylab/cloud-lab.git\n $ cd cloud-lab/\n\nIf cloned source code with `root` account, please **MUST** switch to normal user and change their owner:\n\n $ sudo -su \u003cUSER\u003e\n $ sudo chown -R \u003cUSER\u003e:\u003cUSER\u003e -R cloud-lab/{*,.git}\n\n## 2.6 Run and login the lab\n\nLaunch the lab and login with the user and password printed in the console:\n\n $ tools/docker/run linux-lab\n\nLogin with Bash:\n\n $ tools/docker/bash\n\nRe-login the lab via web browser:\n\n $ tools/docker/webvnc\n\nThe other login methods:\n\n $ tools/docker/vnc\n $ tools/docker/ssh\n $ tools/docker/webssh\n\nChoose one of the methods:\n\n $ tools/docker/login list # List, choose and record\n $ tools/docker/login vnc # Choose one directly and record for late login\n\nSummary of login methods:\n\n| Login Method | Description | Default User | Where |\n|--------------|-------------|--------------|-----------------------|\n| bash | docker bash | Ubuntu | localhost |\n| ssh | normal ssh | Ubuntu | localhost |\n| vnc | normal vnc | Ubuntu | localhost+VNC client |\n| webvnc | web desktop | Ubuntu | anywhere via internet |\n| webssh | web ssh | Ubuntu | anywhere via internet |\n\nSince vnc clients differs from operating systems, we use webvnc by default to make sure auto login vnc for all systems.\n\nIf really want to use local vnc clients, please install a vnc client, for example: `vinagre`, then specify it like this:\n\n $ tools/docker/vnc vinagre\n\nIf the above command not work normally, based on the information printed above, please configure the vnc client yourself.\n\n**Notes**:\n\n* vinagre has fullscreen mode, but not enabled by default, which can be enabled through menu: `View -\u003e Fullscreen`, but must enable `Keyboard shortcuts` at first, otherwise, no way exit fullscreen except via `sudo pkill x11vnc`.\n* The directly connected ssh and vnc may not always work, please use one of the other three methods instead.\n\n## 2.7 Update and rerun the lab\n\nUsually, only need to update Linux Lab itself, to get the new boards support or related fixups:\n\n $ cd /path/to/cloud-lab/labs/linux-lab/\n $ git checkout master\n $ git pull\n\nSometimes, need to update Cloud Lab, to fix up potential running issues or getting newer docker image:\n\n $ cd /path/to/cloud-lab\n $ git checkout master\n $ git pull\n\nIf modified the running environment of Linux Lab locally and want to reuse it in the future, save the container (very slow, not recommend if not necessary):\n\n $ tools/docker/save linux-lab\n $ git checkout -- configs/linux-lab/docker/name\n\nThen rerurn Linux lab:\n\n $ tools/docker/rerun linux-lab\n\n## 2.8 Quickstart: Boot a board\n\nGet into the lab environment, switch directory:\n\n $ cd /labs/linux-lab\n\nIssue the following command to boot the prebuilt kernel and rootfs on the default `vexpress-a9` board:\n\n $ make boot\n\nLogin as `root` user without password(password is empty), just input `root` and press Enter:\n\n Welcome to Linux Lab\n\n linux-lab login: root\n\n # uname -a\n Linux linux-lab 5.1.0 #3 SMP Thu May 30 08:44:37 UTC 2019 armv7l GNU/Linux\n #\n # poweroff\n #\n\nShutdown the board with the `poweroff` command.\n\n**Notes*: If some boards not support `poweroff`, please press `CTRL+a x`. Of course, open another terminal and issue kill or pkill command also can quit qemu.\n\n# 3. Linux Lab Kickstart\n\n## 3.1 Using boards\n\n### 3.1.1 List available boards\n\nList builtin boards:\n\n $ make list\n [ aarch64/raspi3 ]:\n ARCH = arm64\n CPU ?= cortex-a53\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/mmcblk0 /dev/ram0\n ROOTDEV ?= /dev/mmcblk0\n [ aarch64/virt ]:\n ARCH = arm64\n CPU ?= cortex-a57\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/sda /dev/vda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/vda\n [ arm/mcimx6ul-evk ]:\n ARCH = arm\n CPU ?= cortex-a9\n LINUX ?= v5.4\n ROOTDEV_LIST := /dev/mmcblk0 /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/mmcblk0\n [ arm/versatilepb ]:\n ARCH = arm\n CPU ?= arm926t\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/sda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n [ arm/vexpress-a9 ]:\n ARCH = arm\n CPU ?= cortex-a9\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/mmcblk0 /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n [ i386/pc ]:\n ARCH = x86\n CPU ?= qemu32\n LINUX ?= v5.1\n ROOTDEV_LIST ?= /dev/hda /dev/ram0 /dev/nfs\n ROOTDEV_LIST[LINUX_v2.6.34.9] ?= /dev/sda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/hda\n [ mips64el/ls2k ]:\n ARCH = mips\n CPU ?= mips64r2\n LINUX ?= loongnix-release-1903\n LINUX[LINUX_loongnix-release-1903] := 04b98684\n ROOTDEV_LIST := /dev/sda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n [ mips64el/ls3a7a ]:\n ARCH = mips\n CPU ?= mips64r2\n LINUX ?= loongnix-release-1903\n LINUX[LINUX_loongnix-release-1903] := 04b98684\n ROOTDEV_LIST ?= /dev/sda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n [ mipsel/ls1b ]:\n ARCH = mips\n CPU ?= mips32r2\n LINUX ?= v5.2\n ROOTDEV_LIST ?= /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n [ mipsel/ls232 ]:\n ARCH = mips\n CPU ?= mips32r2\n LINUX ?= v2.6.32-r190726\n ROOTDEV_LIST := /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n [ mipsel/malta ]:\n ARCH = mips\n CPU ?= mips32r2\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/hda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n [ ppc/g3beige ]:\n ARCH = powerpc\n CPU ?= generic\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/hda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n [ riscv32/virt ]:\n ARCH = riscv\n CPU ?= any\n LINUX ?= v5.0.13\n ROOTDEV_LIST := /dev/vda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/vda\n [ riscv64/virt ]:\n ARCH = riscv\n CPU ?= any\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/vda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/vda\n [ x86_64/pc ]:\n ARCH = x86\n CPU ?= qemu64\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/hda /dev/ram0 /dev/nfs\n ROOTDEV_LIST[LINUX_v3.2] := /dev/sda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n [ csky/virt ]:\n ARCH = csky\n CPU ?= ck810\n LINUX ?= v4.9.56\n ROOTDEV ?= /dev/nfs\n\n`ARCH`, `FILTER` arguments are supported:\n\n $ make list ARCH=arm\n $ make list FILTER=virt\n\nand more:\n\n $ make list-board # only ARCH\n $ make list-short # ARCH and Linux\n $ make list-base # no plugin\n $ make list-plugin # only plugin\n $ make list-full # everything\n $ make list-real # real hardware boards\n $ make list-virt # only virtual boards\n\n### 3.1.2 Choosing a board\n\n#### 3.1.2.1 Real board\n\nFrom version v0.6, to support learn external devices, Linux Lab adds real hardware board support, to use such boards, please buy them and connect them to your develop host correctly.\n\nOnly list real boards:\n\n $ make list-real\n [ arm/ebf-imx6ull ]:\n ARCH = arm\n CPU ?= cortex-a9\n LINUX ?= v4.19.35\n ROOTDEV_LIST := /dev/mmcblk0 /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/mmcblk0\n\nBecause real hardware boards differs from each other, so, board specific document are recommended, for example: `boards/arm/ebf-imx6ull/README.md`.\n\n[![Linux Lab Board - IMX6ULL](doc/images/ebf-imx6ull.png)][023]\n\n#### 3.1.2.2 Virtual board\n\nBy default, the default virtual board: `vexpress-a9` is used, we can configure, build and boot for a specific board with `BOARD`, for example:\n\n $ make BOARD=malta\n $ make boot\n\nIf several boards have the same name, please specify the architecture to distinguish:\n\n $ make BOARD=mipsel/malta\n\nCurrently, such boards have the same name:\n\n $ make list FILTER=virt\n [ aarch64/virt ]:\n ARCH = arm64\n CPU ?= cortex-a57\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/sda /dev/vda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/vda\n [ riscv32/virt ]:\n ARCH = riscv\n CPU ?= any\n LINUX ?= v5.0.13\n ROOTDEV_LIST := /dev/vda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/vda\n [ riscv64/virt ]:\n ARCH = riscv\n CPU ?= any\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/vda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/vda\n\n $ make list FILTER=/pc\n [ i386/pc ]:\n ARCH = x86\n CPU ?= qemu32\n LINUX ?= v5.1\n ROOTDEV_LIST ?= /dev/hda /dev/ram0 /dev/nfs\n ROOTDEV_LIST[LINUX_v2.6.34.9] ?= /dev/sda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/hda\n [ x86_64/pc ]:\n ARCH = x86\n CPU ?= qemu64\n LINUX ?= v5.1\n ROOTDEV_LIST := /dev/hda /dev/ram0 /dev/nfs\n ROOTDEV_LIST[LINUX_v3.2] := /dev/sda /dev/ram0 /dev/nfs\n ROOTDEV ?= /dev/ram0\n\nUse them like this:\n\n $ make BOARD=x86_64/pc\n $ make BOARD=riscv64/virt\n\nIf using `board`, it only works on-the-fly, the setting will not be saved, this is helpful to run multiple boards at the same and not to disrupt each other:\n\n $ make board=malta boot\n\nThis allows to run multi boards in different terminals or background at the same time.\n\nCheck the board specific configuration:\n\n $ cat boards/arm/vexpress-a9/Makefile\n\n**Notes**: More money are required to maintain this project, only one virtual board is free now, the others are non-free, buy one as you want.\n\n### 3.1.2.3 Buy one\n\nAll supported real hardware boards, virtual hardware boards support and the related Linux Lab Disk will be put in [TinyLab.org's Taobao Shop][024] or [Bilibili Shop][025], after bought them, please contact with wechat: `tinylab` and join in the development group.\n\n### 3.1.3 Using as plugins\n\nThe 'Plugin' feature is supported by Linux Lab, to allow boards being added and maintained in standalone git repositories. Standalone repository is very important to ensure Linux Lab itself not grow up big and big while more and more boards being added in.\n\nBook examples or the boards with a whole new CPU architecture benefit from such feature a lot, for book examples may use many boards and a new CPU architecture may need require lots of new packages (such as cross toolchains and the architecture specific QEMU system tool).\n\nHere maintains the available plugins:\n\n- [C-Sky Linux][013]\n- [Loongson Linux][012]\n\nThe Loongson plugin has been merged into v5.0.\n\n### 3.1.4 Configure boards\n\nEvery board has its own configuration, some can be changed on demand, for example, memory size, linux version, buildroot version, qemu version and the other external devices, such as serial port, network devices and so on.\n\nThe configure method is very simple, just edit it by referring to current values (`boards/\u003cBOARD\u003e/Makefile`), this command open local configuration (`boards/\u003cBOARD\u003e/.labconfig`) via vim:\n\n $ make edit\n\nBut please don't make a big change once, we often only need to tune Linux version, this command is better for such case:\n\n $ make list-linux\n v4.12 v4.5.5 v5.0.10 [v5.1]\n $ make config LINUX=v5.0.10\n $ make list-linux\n v4.12 v4.5.5 [v5.0.10] v5.1\n\nIf want to upstream your local changes, please use `board-edit` and `board-config`, otherwise, `edit` and `config` are preferrable, for they will avoid conflicts while pulling remote updates.\n\n## 3.2 Build in one command\n\nv0.3+ version add target dependency by default, so, if want to compile a kernel, just run:\n\n $ make kernel-build\n\n Or\n\n $ make build kernel\n\nIt will do everything required, of course, we still be able to run the targets explicitly.\n\nAnd futher, with the timestamping support, finished targets will not be run again during the late operations, if still want, just clean the stamp and run it again:\n\n $ make cleanstamp kernel-build\n $ make kernel-build\n\n Or\n\n $ make force-kernel-build\n\nTo clean all of the stamp files:\n\n $ make cleanstamp kernel\n\nThis function also support uboot, root and qemu.\n\n## 3.3 Detailed Operations\n\n### 3.3.1 Downloading\n\nDownload board specific package and the kernel, buildroot source code:\n\n $ make source APP=bsp,kernel,root,uboot\n Or\n $ make source APP=all\n Or\n $ make source all\n\nDownload one by one:\n\n $ make bsp-source\n $ make kernel-source\n $ make root-source\n $ make uboot-source\n\n Or\n\n $ make source bsp\n $ make source kernel\n $ make source root\n $ make source uboot\n\nAfter v0.5, the source code are downloaded in `src/`, before, they are saved in the root directory of Linux Lab.\n\n**Notes**: the source code will be downloaded to `build/src` when `CACHE_SRC` or `ONESHOT` is configured to 1, please save or backup the data inside manually, otherwise, they will be lost after system poweroff.\n\n### 3.3.2 Checking out\n\nCheckout the target version of kernel and builroot:\n\n $ make checkout APP=kernel,root\n\nCheckout them one by one:\n\n $ make kernel-checkout\n $ make root-checkout\n\n Or\n\n $ make checkout kernel\n $ make checkout root\n\nIf checkout not work due to local changes, save changes and run to get a clean environment:\n\n $ make kernel-cleanup\n $ make root-cleanup\n\n Or\n\n $ make cleanup kernel\n $ make cleanup root\n\nThe same to QEMU and U-Boot.\n\n### 3.3.3 Patching\n\nApply available patches in `boards/\u003cBOARD\u003e/bsp/patch/linux` and `src/patch/linux/`:\n\n $ make kernel-patch\n\n Or\n\n $ make patch kernel\n\n### 3.3.4 Configuration\n\n#### 3.3.4.1 Default Configuration\n\nConfigure kernel and buildroot with defconfig:\n\n $ make defconfig APP=kernel,root\n\nConfigure one by one, by default, use the defconfig in `boards/\u003cBOARD\u003e/bsp/`:\n\n $ make kernel-defconfig\n $ make root-defconfig\n\n Or\n\n $ make defconfig kernel\n $ make defconfig root\n\nConfigure with specified defconfig:\n\n $ make B=raspi3\n $ make kernel-defconfig bcmrpi3_defconfig\n $ make root-defconfig raspberrypi3_64_defconfig\n\nIf only defconfig name specified, search boards/\u003cBOARD\u003e at first, and then the default configs path of buildroot, u-boot and linux-stable respectivly: src/buildroot/configs, src/u-boot/configs, src/linux-stable/arch/\u003cARCH\u003e/configs.\n\n#### 3.3.4.2 Manual Configuration\n\n $ make kernel-menuconfig\n $ make root-menuconfig\n\n Or\n\n $ make menuconfig kernel\n $ make menuconfig root\n\n#### 3.3.4.3 Old default configuration\n\n $ make kernel-olddefconfig\n $ make root-olddefconfig\n $ make uboot-olddefconfig\n\n Or\n\n $ make olddefconfig kernel\n $ make olddefconfig root\n $ make olddefconfig uboot\n\n### 3.3.5 Building\n\nBuild kernel and buildroot together:\n\n $ make build APP=kernel,root\n\nBuild them one by one:\n\n $ make kernel-build # make kernel\n $ make root-build # make root\n\n Or\n\n $ make build kernel\n $ make build root\n\nAfter v0.5, the building result are stored in `build/`, before they are put in `output/`.\n\n### 3.3.6 Saving\n\nSave all of the configs and rootfs/kernel/dtb images:\n\n $ make save APP=kernel,root\n $ make saveconfig APP=kernel,root\n\nSave configs and images to `boards/\u003cBOARD\u003e/bsp/`:\n\n $ make kernel-saveconfig\n $ make root-saveconfig\n $ make root-save\n $ make kernel-save\n\n Or\n\n $ make saveconfig kernel\n $ make saveconfig root\n $ make save kernel\n $ make save root\n\n### 3.3.7 Booting\n\nBoot with serial port (nographic) by default, exit with `CTRL+a x`, `poweroff`, `reboot` or `pkill qemu` (See [poweroff hang](#poweroff-hang)):\n\n $ make boot\n\nBoot with graphic (Exit with `CTRL+ALT+2 quit`):\n\n $ make b=pc boot G=1 LINUX=v5.1 BUILDROOT=2019.11\n $ make b=versatilepb boot G=1 LINUX=v5.1 BUILDROOT=2016.05\n $ make b=g3beige boot G=1 LINUX=v5.1 BUILDROOT=2016.05\n $ make b=malta boot G=1 LINUX=v2.6.36 BUILDROOT=2016.05\n $ make b=vexpress-a9 boot G=1 LINUX=v4.6.7 BUILDROOT=2016.05 // LINUX=v3.18.39 works too\n\n**Note**:\n\n* real graphic boot require LCD and keyboard drivers, the above boards work well, with Linux v5.1, `raspi3` and `malta` has tty0 console but without keyboard input.\n\n* new buildroot config files set tty console to serial with (`BR2_TARGET_GENERIC_GETTY_PORT=\"ttyAMA0\"`), to enable console with G=1, please change the `getty` line in `/etc/inittab`, for example, replace `ttyAMA0` with `console`, we can also simply switch to the serial console via the Qemu 'View' menu.\n\n`vexpress-a9` and `virt` has no LCD support by default, but for the latest qemu, it is able to boot\nwith G=1 and switch to serial console via the 'View' menu, this can not be used to test LCD and\nkeyboard drivers. `QOPTS` specify the additional QEMU options.\n\n $ make b=vexpress-a9 CONSOLE=ttyAMA0 boot G=1 LINUX=v5.1\n $ make b=raspi3 CONSOLE=ttyAMA0 QOPTS=\"-serial vc -serial vc\" boot G=1 LINUX=v5.1\n\nBoot with curses graphic (friendly to bash/ssh login, not work for all boards, exit with `ESC+2 quit` or `ALT+2 quit`):\n\n $ make b=pc boot G=2 LINUX=v4.6.7\n\nBoot with PreBuilt Kernel, Dtb and Rootfs:\n\n $ make boot kernel=old dtb=old root=old\n\nBoot with new kernel, dtb and rootfs if exists:\n\n $ make boot kernel=new dtb=new root=new\n\nBoot with new kernel and uboot, build them if not exists:\n\n $ make boot BUILD=kernel,uboot\n\nBoot without Uboot (only `versatilepb` and `vexpress-a9` boards tested):\n\n $ make boot U=0\n\nBoot with different rootfs (depends on board, check `/dev/` after boot):\n\n $ make boot ROOTDEV=ram0 // support by all boards, basic boot method\n $ make boot ROOTDEV=nfs // depends on network driver, only raspi3 not work\n $ make boot ROOTDEV=sda\n $ make boot ROOTDEV=mmcblk0\n $ make boot ROOTDEV=vda // virtio based block device\n\nBoot with extra kernel command line (KCLI = Additional Kernel Command LIne):\n\n $ make boot ROOTDEV=nfs KCLI=\"init=/bin/bash\"\n\nList supported options:\n\n $ make list ROOTDEV\n $ make list BOOTDEV\n $ make list CCORI\n $ make list NETDEV\n $ make list linux\n $ make list uboot\n $ make list qemu\n\nAnd more `\u003cxxx\u003e-list` are also supported with `list \u003cxxx\u003e`, for example:\n\n $ make list features\n $ make list modules\n $ make list gcc\n\n# 4. Linux Lab Advance\n\n## 4.1 Using Linux Kernel\n\n### 4.1.1 non-interactive configuration\n\nA tool named `scripts/config` in Linux kernel is helpful to get/set the kernel\nconfig options non-interactively, based on it, both of `kernel-getconfig`\nand `kernel-setconfig` are added to tune the kernel options, with them, we\ncan simply \"enable/disable/setstr/setval/getstate\" of a kernel option or many\nat the same time:\n\nGet state of a kernel module:\n\n $ make kernel-getconfig m=minix_fs\n Getting kernel config: MINIX_FS ...\n\n build/aarch64/linux-v5.1-virt/.config:CONFIG_MINIX_FS=m\n\nEnable a kernel module:\n\n $ make kernel-setconfig m=minix_fs\n Setting kernel config: m=minix_fs ...\n\n build/aarch64/linux-v5.1-virt/.config:CONFIG_MINIX_FS=m\n\n Enable new kernel config: minix_fs ...\n\nMore control commands of `kernel-setconfig` including `y, n, c, o, s, v`:\n\n| Option | Description |\n|--------|--------------------------------------------------------------|\n| `y` | build the modules in kernel or enable anther kernel options. |\n| `c` | build the modules as pluginable modules, just like `m`. |\n| `o` | build the modules as pluginable modules, just like `m`. |\n| `n` | disable a kernel option. |\n| `s` | `RTC_SYSTOHC_DEVICE=\"rtc0\"`, set the rtc device to rtc0 |\n| `v` | `PANIC_TIMEOUT=5`, set the kernel panic timeout to 5 secs. |\n\nOperates many options in one command line:\n\n $ make kernel-setconfig m=tun,minix_fs y=ikconfig v=panic_timeout=5 s=DEFAULT_HOSTNAME=linux-lab n=debug_info\n $ make kernel-getconfig o=tun,minix,ikconfig,panic_timeout,hostname\n\n### 4.1.2 using kernel modules\n\nBuild all internel kernel modules:\n\n $ make modules\n $ make modules-install\n $ make root-rebuild // not need for nfs boot\n $ make boot\n\nList available modules in `src/modules/`, `boards/\u003cBOARD\u003e/bsp/modules/`:\n\n $ make modules-list\n\nIf `m` argument specified, list available modules in `src/modules/`, `boards/\u003cBOARD\u003e/bsp/modules/` and `src/linux-stable/`:\n\n $ make modules-list m=hello\n 1 m=hello ; M=$PWD/src/modules/hello\n $ make modules-list m=tun,minix\n 1 c=TUN ; m=tun ; M=drivers/net\n 2 c=MINIX_FS ; m=minix ; M=fs/minix\n\nEnable one kernel module:\n\n $ make kernel-getconfig m=minix_fs\n Getting kernel config: MINIX_FS ...\n\n build/aarch64/linux-v5.1-virt/.config:CONFIG_MINIX_FS=m\n\n $ make kernel-setconfig m=minix_fs\n Setting kernel config: m=minix_fs ...\n\n build/aarch64/linux-v5.1-virt/.config:CONFIG_MINIX_FS=m\n\n Enable new kernel config: minix_fs ...\n\nBuild one kernel module (e.g. minix.ko):\n\n $ make modules M=fs/minix/\n Or\n $ make modules m=minix\n\nInstall and clean the module:\n\n $ make modules-install M=fs/minix/\n $ make modules-clean M=fs/minix/\n\nMore flexible usage:\n\n $ make kernel-setconfig m=tun\n $ make kernel tun.ko M=drivers/net\n $ make kernel drivers/net/tun.ko\n\nBuild external kernel modules (the same as internel modules):\n\n $ make modules m=hello\n Or\n $ make kernel x=$PWD/modules/hello/hello.ko\n\n### 4.1.3 using kernel features\n\n#### 4.1.3.1 list supported kernel features\n\nKernel features are abstracted in `src/feature/linux/`, including their\nconfigurations patchset, it can be used to manage both of the out-of-mainline\nand in-mainline features.\n\n $ make feature-list\n [ /labs/linux-lab/src/feature/linux ]:\n + 9pnet\n + core\n - debug\n - module\n + ftrace\n - v2.6.36\n * env.g3beige\n * env.malta\n * env.pc\n * env.versatilepb\n - v2.6.37\n * env.g3beige\n + gcs\n - v2.6.36\n * env.g3beige\n * env.malta\n * env.pc\n * env.versatilepb\n + kft\n - v2.6.36\n * env.malta\n * env.pc\n + uksm\n - v2.6.38\n\nVerified boards and Linux versions are recorded there, so, it should work\nwithout any issue if the environment not changed.\n\n#### 4.1.3.2 using kernel modules\n\nFor example, to enable kernel modules support, simply do:\n\n // only upper case 'FEATURE' will be saved\n $ make feature FEATURE=module\n $ make kernel-olddefconfig\n $ make kernel\n\n#### 4.1.3.3 using rust feature\n\nUse `x86_64/pc` as an example：\n\n $ make BOARD=x86_64/pc\n\nswitch to v6.1.1 Linux:\n\n $ make config LINUX=v6.1.1\n\nCompile the kernel, and test it with one of the simplest module - `rust_minimal`:\n\n // clean up everything for a whole new test\n $ make kernel-cleanall\n\n // this 'f' variable will not be saved for standalone make targets\n $ make test f=rust m=rust_minimal\n\n#### 4.1.3.4 using kft feature\n\nFor `kft` feature in v2.6.36 for malta board:\n\n $ make cleanall b=malta\n $ make test b=malta f=kft LINUX=v2.6.36\n\n#### 4.1.3.5 using rt feature\n\nLinux officially provide RT Preemption support, but many patches are outside of mainline kernel, to use it:\n\n $ make feature-list f=rt\n $ make test b=i386/pc f=rt LINUX=v5.2\n\n#### 4.1.3.6 persist or clear feature setting\n\nClear feature setting (reset feature saved in .labconfig):\n\n $ make feature FEATURE=rust\n $ make feature FEATURE=\n\nThe above function is the same as 'make config'.\n\n### 4.1.4 Create new development branch\n\nIf want to use a new development branch, please follow such steps:\n\nAt first, Get into `src/linux-stable` or another directory specified with `KERNEL_SRC`, checkout a development branch from a specific version:\n\n $ cd src/linux-stable\n $ git checkout -b linux-v5.1-dev v5.1\n\nAnd then, clone the necessary configurations and directories for our new branch.\n\n $ make kernel-clone LINUX=v5.1 LINUX_NEW=linux-v5.1-dev\n\nThe v5.1 must be the already supported version, if not, please use the near one in supported list, for example, `i386/pc` board support such versions:\n\n $ make b=i386/pc list linux\n v2.6.10 v2.6.11.12 v2.6.12.6 v2.6.21.5 v2.6.24.7 v2.6.34.9 v2.6.35.14 v2.6.36 v4.6.7 [v5.1] v5.2\n\nIf want to develop v2.6.38, please try to clone one from v2.6.36:\n\n $ cd src/linux-stable\n $ git checkout -b linux-v2.6.38-dev v2.6.38\n $ make kernel-clone LINUX=v2.6.36 LINUX_NEW=linux-v2.6.38-dev\n\nIn development, please commit asap, and also, please use such commands carefully to avoid destroy your important changes:\n\n* kernel-checkout, checkout a specified kernel version, may override your changes\n* kernel-cleanup, clean up git repository, may remove your changes\n* kernel-clean, clean building history\n* kernel-cleanall, clean both of the building history and the source code changes\n\n### 4.1.5 Use standalone git repository\n\nv0.8 starts to add `KERNEL_FORK`, allows to configure the third party Linux source code repository, has added openEuler and wsl2, both of them support `x86_64/pc` and the former support `aarch64/virt` too.\n\nFor example, to compile wsl2 kernel, switch `KERNEL_FORK` to wsl2 directly:\n\n $ make BOARD=x86_64/pc\n $ make config KERNEL_FORK=wsl2\n $ make kernel\n\nTo configure the wsl2 kernel version, configure it as following:\n\n $ make edit\n LINUX[KERNEL_FORK_wsl2] := linux-msft-wsl-5.10.74.3\n\nThe value should be one of the available tag in `git tag` list.\n\n## 4.2 Using U-Boot Bootloader\n\nChoose one of the tested boards: `versatilepb` and `vexpress-a9`.\n\n $ make BOARD=vexpress-a9\n\nDownload Uboot:\n\n $ make uboot-source\n\nCheckout the specified version:\n\n $ make uboot-checkout\n\nPatching with necessary changes, `BOOTDEV` and `ROOTDEV` available, use `flash` by default.\n\n $ make uboot-patch\n\nUse `tftp`, `sdcard` or `flash` explicitly, should run `make U-Boot-checkout` before a new `uboot-patch`:\n\n $ make uboot-patch BOOTDEV=tftp\n $ make uboot-patch BOOTDEV=sdcard\n $ make uboot-patch BOOTDEV=flash\n\n `BOOTDEV` is used to specify where to store and load the images for uboot, `ROOTDEV` is used to tell kernel where to load the rootfs.\n\nConfigure:\n\n $ make uboot-defconfig\n $ make uboot-menuconfig\n\nBuilding:\n\n $ make uboot\n\nBoot with `BOOTDEV` and `ROOTDEV`, use `flash` by default:\n\n $ make boot U=1\n\nUse `tftp`, `sdcard` or `flash` explicitly:\n\n $ make boot U=1 BOOTDEV=tftp\n $ make boot U=1 BOOTDEV=sdcard\n $ make boot U=1 BOOTDEV=flash\n\nWe can also change `ROOTDEV` during boot, for example:\n\n $ make boot U=1 BOOTDEV=flash ROOTDEV=nfs\n\nClean images if want to update ramdisk, dtb and uImage:\n\n $ make uboot-images-clean\n $ make uboot-clean\n\nSave U-Boot images and configs:\n\n $ make uboot-save\n $ make uboot-saveconfig\n\n## 4.3 Using QEMU Emulator\n\nBuiltin QEMU may not work with the newest Linux kernel, so, we need compile and\nadd external prebuilt qemu, this has been tested on vexpress-a9 and virt board.\n\nAt first, build qemu-system-ARCH:\n\n $ make B=vexpress-a9\n $ make qemu\n $ make qemu-save\n\nQEMU-ARCH-static and qemu-system-ARCH can not be compiled together. to build\nQEMU-ARCH-static, please enable `QEMU_US=1` in board specific Makefile and\nrebuild it.\n\nIf QEMU and QTOOL specified, the one in bsp submodule will be used in advance of\none installed in system, but the first used is the one just compiled if exists.\n\nWhile porting to newer kernel, Linux 5.0 hangs during boot on QEMU 2.5, after\ncompiling a newer QEMU 2.12.0, no hang exists. please take notice of such issue\nin the future kernel upgrade.\n\nIf already download QEMU and its submodules and don't want to upadte the submodules,\njust skip it:\n\n $ make qemu git_module_status=0\n\n## 4.4 Using Toolchains\n\nThe pace of Linux mainline is very fast, builtin toolchains can not keep up, to\nreduce the maintaining pressure, external toolchain feature is added. for\nexample, ARM64/virt, CCVER and CCPATH has been added for it.\n\nList available prebuilt toolchains:\n\n $ make gcc-list\n\nDownload, decompress and enable the external toolchain:\n\n $ make gcc\n\nSwitch compiler version if exists, for example:\n\n $ make gcc-switch CCORI=internal GCC=4.8\n\n $ make gcc-switch CCORI=linaro\n\nIf not external toolchain there, the builtin will be used back.\n\nIf no builtin toolchain exists, please must use this external toolchain feature, currently, aarch64, arm, riscv, mipsel, ppc, i386, x86_64 support such feature.\n\nGCC version can be configured in board specific Makefile for Linux, Uboot, Qemu and Root, for example:\n\n GCC[LINUX_v2.6.11.12] = 4.4\n\nWith this configuration, GCC will be switched automatically during defconfig and compiling of the specified Linux v2.6.11.12.\n\nTo build host tools, host gcc should be configured too(please specify `b=i386/pc` explicitly):\n\n $ make gcc-list b=i386/pc\n $ make gcc-switch CCORI=internal GCC=4.8 b=i386/pc\n\n## 4.5 Using Rootfs\n\nBuiltin rootfs is minimal, is not enough for complex application development,\nwhich requires modern Linux distributions.\n\nSuch a type of rootfs has been introduced and has been released as docker\nimage, ubuntu 18.04 is added for arm32v7 at first, more later.\n\nRun it via docker directly:\n\n $ docker run -it tinylab/arm32v7-ubuntu\n\nExtract it out and run in Linux Lab:\n\n (host)$ sudo apt-get install -y qemu-user-static\n\n ARM32/vexpress-a9 (user: root, password: root):\n\n (host)$ tools/root/docker/extract.sh tinylab/arm32v7-ubuntu arm\n (lab )$ make boot b=arm/vexpress-a9 U=0 V=1 MEM=1024M ROOTDEV=nfs ROOTFS=$PWD/prebuilt/fullroot/tmp/tinylab-arm32v7-ubuntu\n\n ARM64/raspi3 (user: root, password: root):\n\n (host)$ tools/root/docker/extract.sh tinylab/arm64v8-ubuntu arm\n (lab )$ make boot b=aarch64/virt V=1 ROOTDEV=nfs ROOTFS=$PWD/prebuilt/fullroot/tmp/tinylab-arm64v8-ubuntu\n\nMore rootfs from docker can be found:\n\n $ docker search arm64 | egrep \"ubuntu|debian\"\n arm64v8/ubuntu Ubuntu is a Debian-based Linux operating system 25\n arm64v8/debian Debian is a Linux distribution that's composed 20\n\n## 4.6 Debugging Linux and U-Boot\n\n### 4.6.1 Debugging Linux\n\nCompile the kernel with debugging options:\n\n $ make feature FEATURE=debug\n $ make kernel-olddefconfig\n $ make kernel\n\nCompile with one thread:\n\n $ make kernel JOBS=1\n\nAnd then debug it directly:\n\n $ make debug\n\nThe above command will use tmux to split into two terminals, each running QEMU and gdb respectively, and load the script from .gdb/kernel.default.\n\nTo switch tmux panes, use CTRL+b followed by the arrow key (e.g., ←).\n\nTo customize kernel gdbinit script, simply copy one and edit it manually:\n\n $ cp .gdb/kernel.default .gdb/kernel.user\n\nIt equals to:\n\n $ make debug linux\n\nto automate debug testing:\n\n $ make test-debug linux\n\nfind out the code line of a kernel panic address:\n\n $ make kernel-calltrace func+offset/length\n\nif the debug port has been used, please try to find out who used the port and kill it:\n\n $ sudo netstat -tlp | grep 1234\n tcp 0 0 0.0.0.0:1234 0.0.0.0:* LISTEN 3943/qemu-xxx\n $ sudo kill -9 3943\n\n### 4.6.2 Debugging U-Boot\n\nTo debug U-Boot with `.gdb/uboot.default`:\n\n $ make debug uboot\n\nThe above command will use tmux to split into two terminals, each running QEMU and gdb respectively.\n\nTo switch tmux panes, use CTRL+b followed by the arrow key (e.g., ←).\n\nTo automate U-Boot debug testing:\n\n $ make test-debug uboot\n\nThe same to kernel gdbinit script, customize one for uboot:\n\n $ cp .gdb/uboot.default .gdb/uboot.user\n\n## 4.7 Test Automation\n\nUse `aarch64/virt` as the demo board here.\n\n $ make BOARD=virt\n\nPrepare for testing, install necessary files/scripts in `src/system/`:\n\n $ make rootdir\n $ make root-rebuild\n\nSimply boot and poweroff (See [poweroff hang](#poweroff-hang)):\n\n $ make test\n\nDon't poweroff after testing:\n\n $ make test TEST_FINISH=echo\n\nRun guest test case:\n\n $ make test TEST_CASE=/tools/ftrace/trace.sh\n\nRun guest test cases (`COMMAND_LINE_SIZE` must be big enough, e.g. 4096, see `cmdline_size` feature below):\n\n $ make test TEST_BEGIN=date TEST_END=date TEST_CASE='ls /;echo hello world'\n\nReboot the guest system for several times:\n\n $ make test TEST_REBOOT=2\n\n NOTE: reboot may 1) hang, 2) continue; 3) timeout killed, TEST_TIMEOUT=30; 4) timeout continue, TIMEOUT_CONTINUE=1\n\nTest a feature of a specified Linux version on a specified board (`cmdline_size` feature is for increase `COMMAND_LINE_SIZE` to 4096):\n\n $ make test f=kft LINUX=v2.6.36 b=malta TEST_PREPARE=board-init,kernel-cleanup\n\n NOTE: `board-init` and `kernel-cleanup` make sure test run automatically, but `kernel-cleanup` is not safe, please save your code before use it!!\n\nTest a kernel module:\n\n $ make test m=hello\n\nTest multiple kernel modules:\n\n $ make test m=exception,hello\n\nTest modules with specified ROOTDEV, nfs boot is used by default, but some boards may not support network:\n\n $ make test m=hello,exception TEST_RD=ram0\n\nRun test cases while testing kernel modules (test cases run between insmod and rmmod):\n\n $ make test m=exception TEST_BEGIN=date TEST_END=date TEST_CASE='ls /root;echo hello world' TEST_PREPARE=board-init,kernel-cleanup f=cmdline_size\n\nRun test cases while testing internal kernel modules:\n\n $ make kernel-setconfig y=debug_fs\n $ make test m=lkdtm TEST_BEGIN='mount -t debugfs debugfs /mnt' TEST_CASE='echo EXCEPTION \u003e /mnt/provoke-crash/DIRECT'\n\nRun test cases while testing internal kernel modules, pass kernel arguments:\n\n $ make test m=lkdtm lkdtm_args='cpoint_name=DIRECT cpoint_type=EXCEPTION'\n\nRun test without feature-init (save time if not necessary):\n\n $ make test m=lkdtm lkdtm_args='cpoint_name=DIRECT cpoint_type=EXCEPTION' TEST_INIT=0\n Or\n $ make raw-test m=lkdtm lkdtm_args='cpoint_name=DIRECT cpoint_type=EXCEPTION'\n\nRun test with module and the module's necessary dependencies (check with `make kernel-menuconfig`):\n\n $ make test m=lkdtm y=runtime_testing_menu,debug_fs lkdtm_args='cpoint_name=DIRECT cpoint_type=EXCEPTION' LINUX=v5.1 TEST_PREPARE=kernel-cleanup\n\nRun test without feature-init, boot-init, boot-finish and no `TEST_PREPARE`:\n\n $ make boot-test m=lkdtm lkdtm_args='cpoint_name=DIRECT cpoint_type=EXCEPTION'\n\nTest a kernel module and make some targets before testing:\n\n $ make test m=exception TEST=kernel-checkout,kernel-patch,kernel-defconfig\n\nTest everything in one command (from download to poweroff, see [poweroff hang](#poweroff-hang)):\n\n $ make test TEST=kernel,root TEST_PREPARE=board-init,kernel-cleanup,root-cleanup\n\nTest everything in one command (with U-Boot while support, e.g. vexpress-a9):\n\n $ make test TEST=kernel,root,uboot TEST_PREPARE=board-init,kernel-cleanup,root-cleanup,uboot-cleanup\n\nTest kernel hang during boot, allow to specify a timeout, timeout must happen while system hang:\n\n $ make test TEST_TIMEOUT=30s\n\nTest kernel debug:\n\n $ make test DEBUG=1\n\n**Notes**: The above tests may fail on some boards with some Linux versions, please upgrade the kernel versions if necessary.\n\n## 4.8 File Sharing\n\nTo transfer files between QEMU Board and Host, three methods are supported by\ndefault:\n\n### 4.8.1 Install files to rootfs\n\nSimply put the files with a relative path in `src/system/`, install and rebuild the rootfs:\n\n $ mkdir src/system/root/\n $ touch src/system/root/new_file\n $ make root-rebuild\n $ make boot\n\n### 4.8.2 Share with NFS\n\nBoot the board with `ROOTDEV=nfs`:\n\n $ make boot ROOTDEV=nfs\n\nHost:\n\n $ make env-dump VAR=ROOTDIR\n ROOTDIR=\"/labs/linux-lab/boards/\u003cBOARD\u003e/bsp/root/\u003cBUILDROOT_VERSION\u003e/rootfs\"\n\n### 4.8.3 Transfer via tftp\n\nUsing tftp server of host from the QEMU board with the `tftp` command.\n\nHost:\n\n $ ifconfig br0\n inet addr:172.17.0.3 Bcast:172.17.255.255 Mask:255.255.0.0\n $ cd tftpboot/\n $ ls tftpboot\n kft.patch kft.log\n\nQEMU Board:\n\n $ ls\n kft_data.log\n $ tftp -g -r kft.patch 172.17.0.3\n $ tftp -p -r kft.log -l kft_data.log 172.17.0.3\n\n**Note**: while put file from QEMU board to host, must create an empty file in host firstly. Buggy?\n\n### 4.8.4 Share with 9p virtio\n\nTo enable 9p virtio for a new board, please refer to [qemu 9p setup][034]. qemu must be compiled with `--enable-virtfs`, and kernel must enable the necessary options.\n\nReconfigure the kernel with:\n\n CONFIG_NET_9P=y\n CONFIG_NET_9P_VIRTIO=y\n CONFIG_NET_9P_DEBUG=y (Optional)\n CONFIG_9P_FS=y\n CONFIG_9P_FS_POSIX_ACL=y\n CONFIG_PCI=y\n CONFIG_VIRTIO_PCI=y\n CONFIG_PCI_HOST_GENERIC=y (only needed for the QEMU Arm 'virt' board)\n\n If using `-virtfs` or `-device virtio-9p-pci` option for qemu, must enable the above PCI related options, otherwise will not work:\n\n 9pnet_virtio: no channels available for device hostshare\n mount: mounting hostshare on /hostshare failed: No such file or directory\n\n `-device virtio-9p-device` requires less kernel options.\n\n To enable the above options, please simply type:\n\n $ make feature FEATURE=9pnet\n $ make kernel-olddefconfig\n\nDocker host:\n\n $ modprobe 9pnet_virtio\n $ lsmod | grep 9p\n 9pnet_virtio 17519 0\n 9pnet 72068 1 9pnet_virtio\n\nHost:\n\n $ make BOARD=virt\n\n $ make root-rebuild\n\n $ touch hostshare/test # Create a file in host\n\n $ make boot U=0 ROOTDEV=ram0 PBR=1 SHARE=1\n\n $ make boot SHARE=1 SHARE_DIR=src/modules # for external modules development\n\n $ make boot SHARE=1 SHARE_DIR=build/aarch64/linux-v5.1-virt/ # for internal modules learning\n\n $ make boot SHARE=1 SHARE_DIR=src/examples # for c/assembly learning\n\nQEMU Board:\n\n $ ls /hostshare/ # Access the file in guest\n test\n $ touch /hostshare/guest-test # Create a file in guest\n\nVerified boards with Linux v5.1:\n\n| boards | Status |\n|-----------------|------------------------------------------------------------|\n| aarch64/virt | virtio-9p-device (virtio-9p-pci breaks nfsroot) |\n| arm/vexpress-a9 | only work with virtio-9p-device and without U-Boot booting |\n| arm/versatilepb | only work with virtio-9p-pci |\n| x86_64/pc | only work with virtio-9p-pci |\n| i386/pc | only work with virtio-9p-pci |\n| riscv64/virt | work with virtio-9p-pci and virtio-9p-dev |\n| riscv32/virt | work with virtio-9p-pci and virtio-9p-dev |\n\n## 4.9 Learning Assembly\n\nLinux Lab has added many assembly examples in `src/examples/assembly`:\n\n $ cd src/examples/assembly\n $ ls\n aarch64 arm mips64el mipsel powerpc powerpc64 riscv32 riscv64 x86 x86_64\n $ make -s -C aarch64/\n Hello, ARM64!\n\n## 4.10 Learning C\n\n### 4.10.1 Host build and Run\n\nUse hello as example:\n\n $ cd src/examples/c/hello\n $ make\n gcc -fno-stack-protector -fomit-frame-pointer -fno-asynchronous-unwind-tables -fno-pie -no-pie -m32 -Wall -Werror -g -o hello hello.c\n Hello, World!\n\n### 4.10.2 Cross build and Run\n\nUse X32 (Code for x86-64, int/long/pointer to 32bits), ARM, MIPS, PPC and RISC-V as example:\n\n $ sudo apt-get update -y\n\n $ sudo apt-get install -y libc6-x32 libc6-dev-x32 libx32gcc-8-dev\n $ gcc -mx32 -o hello hello.c\n $ ./hello\n Hello, World!\n\n $ sudo apt-get install -y libc6-dev-armel-cross libc6-armel-cross\n $ arm-linux-gnueabi-gcc -o hello hello.c\n $ qemu-arm -L /usr/arm-linux-gnueabi/ ./hello\n Hello, World!\n\n $ sudo apt-get install -y libc6-dev-mipsel-cross libc6-mipsel-cross\n $ mipsel-linux-gnu-gcc -o hello hello.c\n $ qemu-mipsel -L /usr/mipsel-linux-gnu/ ./hello\n Hello, World!\n\n $ sudo apt-get install -y libc6-dev-powerpc-cross libc6-powerpc-cross\n // Must use -static for Linux Lab v0.6, otherwise, there will be segmentation fault\n $ powerpc-linux-gnu-gcc -static -o hello hello.c\n $ qemu-ppc -L /usr/powerpc-linux-gnu/ ./hello\n Hello, World!\n\n $ sudo apt-get install -y libc6-riscv64-cross libc6-dev-riscv64-cross\n $ riscv64-linux-gnu-gcc -o hello hello.c\n $ qemu-riscv64 -L /usr/riscv64-linux-gnu/ ./hello\n Hello, World!\n\nAbove run through `qemu-user`, to run on target boards, please copy the binaries to target boards' rootfs with help from section 4.8.1.\n\nThe main packages are `libc6-dev`, `libc6` or `libgcc`, but x32 is an expection, it is libx32gcc. please list them via `apt-cache search`.\n\n## 4.11 Running any make goals\n\nLinux Lab allows to access Makefile goals of the APPS easily, for example:\n\n $ make kernel help\n $ make kernel menuconfig\n\n $ make root help\n $ make root busybox-menuconfig\n\n $ make uboot help\n $ make uboot menuconfig\n\n Or\n\n $ make kernel-help\n $ make kernel-menuconfig\n\n $ make root-help\n $ make root-busybox-menuconfig\n\n $ make uboot-help\n $ make uboot-menuconfig\n\nAllows to run sub-make goals of kernel, root and U-Boot directly without entering into their own building directory.\n\n## 4.12 Speed up kernel development\n\n### 4.12.1 Speed up compiling and save disk life\n\n**Notes**：This operation may lose data, please take care!\n\nThis feature aims to create a ram based temporary filesystem as the 'build' directory, to store the building data, **If not backup them, they will be lost after shutting down the machine**.\n\nCreate temporary building cache:\n\n $ make build cache\n\nCheck the status of building cache:\n\n $ make build status\n\nUse the cache for building speedup:\n\n $ time make kernel\n\nBackup the cache to a persistent file (If the building file are important to you):\n\n $ make build backup\n\nStop the building cache, revert back to use the build directory on the disk:\n\n $ make build uncache\n\nUse the backup as the build directory:\n\n $ sudo mount /path/to/backup-file /labs/linux-lab/build/\n\n### 4.12.2 ONESHOT Mode\n\nv0.9 adds a `ONESHOT` switch, it can be used to enable such functions:\n\n- Auto cache `build/` in memory\n- Auto cache `src/` in memory\n- Auto enable fast fetch, a.k.a git shallow fetch\n\nIt is good for:\n\n- Disposable, destroy after using\n - If want, please save kernel and its config with `kernel-save` and `kernel-saveconfig`\n\n- Better for big-memory, small-disk and slow-CPU host machines\n - Both `src/` and `build/` are put in memory, not in disk\n\n- Good for instant kernel downloading and building\n - If target host has no Linux kernel source code, and the network is slow\n\nTo use it, please simply run this before others:\n\n $ export ONESHOT=1\n\nIf want to make it persistent, just configure it in `.labinit`:\n\n ONESHOT := 1\n\n### 4.12.3 Nolibc Mode\n\nv1.2-rc2 adds Nolibc mode, allows to build ultra small kernel and application, and package them together via initrd, to achieve \"Kernel-only\" deployments.\n\nNolibc adds two types of files:\n\n- Small kernel config file: `boards/\u003cARCH\u003e/\u003cBOARD\u003e/bsp/configs/linux_v6.x_nolibc_defconfig`\n- Small nolibc application: `src/examples/nolibc/hello.c`\n\nJust similar to `ONESHOT`, before developing, just run this to enable `NOLIBC` mode:\n\n $ export NOLIBC=1\n\nOr, write to `.labinit` to let it always work:\n\n NOLIBC := 1\n\nTo change the target nolibc aplication, we can configure `NOLIBC_SRC`, otherwise, the above hello.c will be used by default:\n\n $ make nolibc-clean\n $ make kernel NOLIBC_SRC=$PWD/src/examples/nolibc/hello.c\n\nIt is very good for pure kernel development.\n\n### 4.12.4 Tiny Mode\n\nBased on Nolibc mode, v1.4-rc2 adds Tiny mode, allows to build ultra small kernel but with initrd boot support.\n\nUsage:\n\n $ export KCFG=linux.tiny.config\n $ make kernel\n $ make boot ROOTDEV=ram0\n\nCompare to defconfig, it only enables minimal config options and makes sure initrd boot with an interactive shell, so, the compiling speed is x10 faster.\n\nIt is very good for kernel features testing, development and research.\n\n## 4.13 More Usage\n\nRead more:\n\n* Why\n * [Why Using Linux Lab V1.0 (In Chinese)][041]\n * [Why Using Linux Lab V2.0 (In Chinese)][042]\n\n* User Manual\n * [Linux Lab v1.4 User Manual][056]\n * [Linux Lab v1.3 User Manual][054]\n * [Linux Lab v1.2 User Manual][053]\n * [Linux Lab v1.1 User Manual][050]\n * [Linux Lab v1.0 User Manual][033]\n * [Linux Lab v0.9 User Manual][032]\n * [Linux Lab v0.8 User Manual][031]\n * [Linux Lab Loongson Manual V0.2][030]\n\n* Linux Lab Videos\n * [CCTALK][022]\n * [Bilibili][025]\n\n* Video Courses use Linux Lab as experiment environment\n * [The Perspective Linux ELF][035]\n * [《Rust Language Quickstart》][006]\n * [《Software Reverse Engineering Quickstart》][037]\n * [《Linux Kernel Livepatch Introduction》][038]\n\n* The books or courses Linux Lab supported or plan to support\n * [books or courses list][014]\n\n* The boards Linux Lab supported or plan to support\n * [ARM IMX6ULL][024]\n * RISCV-64 D1\n\n* The hardwares developed by Linux Lab community\n * [Linux Lab Disk][024], pre-installed Linux Lab disk\n * Support Ubuntu 18.04-21.04, Deepin 20.2+, Fedora 34+, Mint 20.2+, Ezgo 14.04+, Kali, Manjaro\n * [Pocket Linux Disk][024], pre-installed Linux distribution disk\n * Support Ubuntu 18.04-21.04, Deepin 20.2+, Fedora 34+, Mint 20.2+, Ezgo 14.04+, Kali, Manjaro\n\n# 5. Linux Lab Development\n\nThis introduces how to add a new board for Linux Lab.\n\n## 5.1 Choose a board supported by QEMU\n\nlist the boards, use arm as an example:\n\n $ qemu-system-arm -M ?\n\n## 5.2 Create the board directory\n\nUse `vexpress-a9` as an example:\n\n $ mkdir boards/arm/vexpress-a9/\n\n## 5.3 Clone a Makefile from an existing board\n\nUse `versatilepb` as an example:\n\n $ cp boards/arm/versatilebp/Makefile boards/arm/vexpress-a9/Makefile\n\n## 5.4 Configure the variables from scratch\n\nComment everything, add minimal ones and then others.\n\nPlease refer to `doc/qemu/qemu-doc.html` or the online one \u003chttps://www.qemu.org/docs/master/\u003e.\n\n## 5.5 At the same time, prepare the configs\n\nWe need to prepare the configs for linux, buildroot and even uboot.\n\nBuildroot has provided many examples about buildroot and kernel configuration:\n\n buildroot: src/buildroot/configs/qemu_ARCH_BOARD_defconfig\n kernel: src/buildroot/board/qemu/ARCH-BOARD/linux-VERSION.config\n\nU-Boot has also provided many default configs:\n\n uboot: src/u-boot/configs/vexpress_ca9x4_defconfig\n\nKernel itself also:\n\n kernel: src/linux-stable/arch/arm/configs/vexpress_defconfig\n\nLinux Lab itself also provide many working configs too, the `xxx-clone` target is a\ngood helper to utilize existing configs:\n\n $ make list kernel\n v4.12 v5.0.10 v5.1\n $ make kernel-clone LINUX=v5.1 LINUX_NEW=v5.4\n $ make kernel-menuconfig\n $ make kernel-saveconfig\n\n $ make list root\n 2016.05 2019.02.2\n $ make root-clone BUILDROOT=2019.02.2 BUILDROOT_NEW=2019.11\n $ make root-menuconfig\n $ make root-saveconfig\n\nEdit the configs and Makefile until they match our requirements.\n\n $ make kernel-menuconfig\n $ make root-menuconfig\n $ make board-edit\n\nThe configuration must be put in `boards/\u003cBOARD\u003e/` and named with necessary\nversion info, use `raspi3` as an example:\n\n $ make kernel-saveconfig\n $ make root-saveconfig\n $ ls boards/aarch64/raspi3/bsp/configs/\n buildroot_2019.02.2_defconfig linux_v5.1_defconfig\n\n`2019.02.2` is the buildroot version, `v5.1` is the kernel version, both of these\nvariables should be configured in `boards/\u003cBOARD\u003e/Makefile`.\n\nMore usage about the `xxx-clone` commands:\n\n $ make qemu-clone QEMU=\u003cold_version\u003e QEMU_NEW=\u003cnew_version\u003e\n $ make uboot-clone UBOOT=\u003cold_version\u003e UBOOT_NEW=\u003cnew_version\u003e\n $ make kernel-clone LINUX=\u003cold_version\u003e LINUX_NEW=\u003cnew_version\u003e\n $ make root-clone BUILDROOT=\u003cold_version\u003e BUILDROOT_NEW=\u003cnew_version\u003e\n\n## 5.6 Choose the versions of kernel, rootfs and U-Boot\n\nPlease use `tag` instead of `branch`, use kernel as an example:\n\n $ cd src/linux-stable\n $ git tag\n ...\n v5.0\n ...\n v5.1\n ..\n v5.1.1\n v5.1.5\n ...\n\nIf want v5.1 kernel, just put a line \"LINUX = v5.1\" in `boards/\u003cBOARD\u003e/Makefile`.\n\nOr clone a kernel config from the old one or the official defconfig:\n\n $ make kernel-clone LINUX_NEW=v5.3 LINUX=v5.1\n\n Or\n\n $ make B=i386/pc\n $ pushd Linux-stable \u0026\u0026 git checkout v5.4 \u0026\u0026 popd\n $ make kernel-clone LINUX_NEW=v5.4 KCFG=i386_defconfig\n\nIf no tag existed, a virtual tag name with the real commmit number can be configured as following:\n\n LINUX = v2.6.11.12\n LINUX[LINUX_v2.6.11.12] = 8e63197f\n\nLinux version specific ROOTFS are also supported:\n\n ROOTFS[LINUX_v2.6.12.6] ?= $(BSP_ROOT)/$(BUILDROOT)/rootfs32.cpio.gz\n\n## 5.7 Configure, build and boot them\n\nUse kernel as an example:\n\n $ make kernel-defconfig\n $ make kernel-menuconfig\n $ make kernel\n $ make boot\n\nThe same to Rootfs, U-Boot and even QEMU.\n\n## 5.8 Save the images and configs\n\n $ make root-save\n $ make kernel-save\n $ make uboot-save\n\n $ make root-saveconfig\n $ make kernel-saveconfig\n $ make uboot-saveconfig\n\n## 5.9 Upload everything\n\nAt last, upload the images, defconfigs, patchset to board specific bsp submodule repository.\n\nFirstly, get the remote bsp repository address as following:\n\n $ git remote show origin\n * remote origin\n Fetch URL: https://gitee.com/tinylab/qemu-aarch64-raspi3/\n Push URL: https://gitee.com/tinylab/qemu-aarch64-raspi3/\n HEAD branch: master\n Remote branch:\n master tracked\n Local branch configured for 'git pull':\n master merges with remote master\n Local ref configured for 'git push':\n master pushes to master (local out of date)\n\nThen, fork this repository from gitee.com, upload your changes, and send your pull request.\n\n# 6. FAQs\n\n## 6.1 Docker Issues\n\n### 6.1.1 Speed up docker images downloading\n\nTo optimize docker images download speed, please refer to section 6.1.6.\n\n### 6.1.2 Docker network conflicts with LAN\n\nCloud Lab use a default `172.20.0.0/16` subnet, if this conflicts with another one, please change it like this:\n\n $ tools/docker/rm-all\n $ vim configs/linux-lab/docker/subnet\n $ cat configs/linux-lab/docker/subnet\n 172.23.0.0/16\n $ tools/docker/run linux-lab\n\nIf lab network still not work, please try another private network address and eventually to avoid conflicts with LAN address.\n\n### 6.1.3 Why not allow running Linux Lab in local host\n\nThe full function of Linux Lab depends on the full docker environment managed by [Cloud Lab][028], so, please really never try and therefore please don't complain about why there are lots of packages missing failures and even the other weird issues.\n\nLinux Lab is designed to use pre-installed environment with the docker technology and save our life by avoiding the packages installation issues in different systems, so, Linux Lab would never support local host using even in the future.\n\n### 6.1.4 Run tools without sudo\n\nTo use the tools under `tools` without sudo, please make sure add your account to the docker group and reboot your system to take effect:\n\n $ sudo usermod -aG docker \u003cUSER\u003e\n $ newgrp docker\n\nIf get error: \"newgrp: group 'docker' does not exist\", please add 'docker' group manually:\n\n $ sudo groupadd docker\n\n**Notes**: Currently, root user is not allowed to run Linux Lab。\n\n### 6.1.5 Network not work\n\nIf ping not work, please check one by one:\n\n* DNS issue\n\n if `ping 8.8.8.8` work, please check `/etc/resolv.conf` and make sure it is the same as your host configuration.\n\n* IP issue\n\n if ping not work, please refer to [network conflict issue](#docker-network-conflicts-with-lan) and change the ip range of docker containers.\n\n### 6.1.6 Client.Timeout exceeded while waiting headers\n\nThis means must configure one of the following docker mirror sites:\n\n* [Aliyun Docker Mirror Documentation][018]\n* [USTC Docker Mirror Documentation][020]\n\nPotential methods of configuration in Ubuntu, depends on docker and Ubuntu versions:\n\n`/etc/default/docker`:\n\n echo \"DOCKER_OPTS=\\\"\\$DOCKER_OPTS --registry-mirror=\u003cyour accelerate address\u003e\\\"\"\n\n`/lib/systemd/system/docker.service`:\n\n ExecStart=/usr/bin/dockerd -H fd:// --registry-mirror=\u003cyour accelerate address\u003e\n\n`/etc/docker/daemon.json`:\n\n {\n \"registry-mirrors\": [\"\u003cyour accelerate address\u003e\"]\n }\n\nPlease restart docker service after change the accelerate address:\n\n $ sudo service docker restart\n\nFor the other Linux systems, Windows and macOS System, please refer to [Aliyun Mirror Speedup Document][018].\n\nIF still slow, please check if the mirror site is configured normally and without typos:\n\n $ docker info | grep -A1 -i Mirrors\n Registry Mirrors:\n https://XXXXX.mirror.aliyuncs.com/\n\n### 6.1.7 Restart Linux Lab after host system shutdown or reboot\n\nIf want to restore the installed softwares and related configurations, please save the container manually:\n\n $ tools/docker/save linux-lab\n\nAfter host system (include virtual machine) shutdown or reboot, you can restart the lab via the \"Linux Lab\" icon on the desktop, or just like before, issue this command:\n\n $ tools/docker/run linux-lab\n\nCurrent implementation doesn't support the direct 'docker start' command, please learn it.\n\nIf the above methods still not restart the lab, please refer to the methods mentioned in the 6.3.9 section.\n\nIf resume from a suspended host system, the lab will restore automatically, no need to do anything to restart it, just use one of the 4 login methods mentioned in the 2.4 section, for example, start a web browser to connect it:\n\n $ tools/docker/webvnc\n\n### 6.1.8 the following directives are specified both as a flag and in the configuration file\n\nIf getting such error:\n\n unable to configure the Docker daemon with file /etc/docker/daemon.json: the\n following directives are specified both as a flag and in the configuration\n file: registry-mirrors: (from flag: [https://docker.mirrors.ustc.edu.cn/], from\n file: [https://xxx.mirror.aliyuncs.com])\n\nMeans both `/etc/docker/daemon.json` and `/etc/default/docker` configured `registry-mirrors`, please comment the late one and restart docker:\n\n $ sudo service docker restart\n\n### 6.1.9 pathspec FETCH_HEAD did not match any file known to git\n\nIf get such error while running `make boot`, it means network issue, please refer to section 6.1.5。\n\n Could not resolve host: gitee.com\n error: pathspec 'FETCH_HEAD' dit not match any file(s) known to git\n\n### 6.1.10 Docker not work in Ubuntu 20.04\n\nIf docker not work in Ubuntu 20.04, please use `doc/install/daemon.json` and clean up the arguments of dockerd, learn more from [docker daemon][008]:\n\n $ sudo cat /etc/systemd/system/docker.service.d/docker.conf\n [Service]\n ExecStart=\n ExecStart=/usr/bin/dockerd\n\n $ sudo cp /etc/docker/daemon.json /etc/docker/daemon.json.bak\n $ sudo cp doc/install/daemon.json /etc/docker/\n $ sudo service docker restart\n\nPlease make sure using the best `registry-mirrors` for better download speed.\n\n### 6.1.11 Error creating aufs mount\n\nIf not work with failure like \"error creating aufs mount to ... invalid arguments\", that means the storage driver used by docker is not supported by current system, please choose another one from [this page][009], and configure it in `/etc/docker/daemon.json`, for example:\n\n $ sudo vim /etc/docker/daemon.json\n {\n \"registry-mirrors\": [\"https://docker.mirrors.ustc.edu.cn\"],\n \"storage-driver\": \"devicemapper\"\n }\n\nThis issue is related to kernel version, the same system may upgrade kernel version and therefore support different storage driver.\n\n## 6.2 QEMU Issues\n\n### 6.2.1 Why kvm speedding up is disabled\n\nkvm only supports both of `qemu-system-i386` and `qemu-system-x86_64` currently, and it also requires the CPU and bios support, otherwise, you may get this error log:\n\n modprobe: ERROR: could not insert 'kvm_intel': Operation not supported\n\nCheck CPU virtualization support, if nothing output, then, cpu not support virtualization:\n\n $ cat /proc/cpuinfo | egrep --color=always \"vmx|svm\"\n\nIf CPU supports, we also need to make sure it is enabled in bios features, simply reboot your computer, press 'Delete' to enter bios, please make sure the 'Intel virtualization technology' feature is 'enabled'.\n\n### 6.2.2 Poweroff hang\n\nBoth of the `poweroff` and `reboot` commands not work on these boards currently (LINUX=v5.1):\n\n* mipsel/malta (exclude LINUX=v2.6.36)\n* mipsel/ls232\n* mipsel/ls1b\n* mips64el/ls2k\n* mips64el/ls3a7a\n* aarch64/raspi3\n* arm/versatilepb\n\nSystem will directly hang there while running `poweroff` or `reboot`, to exit qemu, please pressing `CTRL+a x` or using `pkill qemu`.\n\nTo test such boards automatically, please make sure setting `TEST_TIMEOUT`, e.g. `make test TEST_TIMEOUT=50`.\n\nWelcome to fix up them.\n\n### 6.2.3 How to exit QEMU\n\n| Where | How |\n|----------------------|---------------------------------------|\n| Serial Port Console | `CTRL+a x` |\n| Curses based Graphic | `ESC+2 quit` Or `ALT+2 quit` |\n| X based Graphic | `CTRL+ALT+2 quit` |\n| Generic Methods | `poweroff`, `reboot`, `kill`, `pkill` |\n\n### 6.2.4 Boot with missing sdl2 libraries failure\n\nThat's because the docker image is not updated, just enter into cloud-lab and rerun the lab (please must not use `tools/docker/restart` here for it not using the new docker image):\n\n $ tools/docker/rerun linux-lab\n\n## 6.3 Environment Issues\n\n### 6.3.1 NFS/tftpboot not work\n\nIf nfs or tftpboot not work, please run `modprobe nfsd` in host side and restart the net services via `/configs/tools/restart-net-servers.sh` in guest side and please make sure not use `tools/docker/trun`.\n\n### 6.3.2 How to switch Windows in VIM\n\n`CTRL+w` is used in both of browser and vim, to switch from one window to another, please use `CTRL+Left` or `CTRL+Right` key instead, Linux Lab has remapped `CTRL+Right` to `CTRL+w` and `CTRL+Left` to `CTRL+p`.\n\n### 6.3.3 How to delete typo in shell command line\n\nLong keypress not work in novnc client currently, so, long `Delete` not work, please use `alt+delete` or `alt+backspace` instead, more tips:\n\n| Function | VIM | Bash |\n|-----------------------|-------|---------------------------|\n| begin/end | `^/$` | `Ctrl + a/e` |\n| forward/backward | `w/b` | `Ctrl + Home/end` |\n| cut one word backword | `db` | `Alt + Delete/backspace` |\n| cut one word forward | `dw` | `Alt + d` |\n| cut all to begin | `d^` | `Ctrl + u` |\n| cut all to end | `d$` | `Ctrl + k` |\n| paste all cutted | `p` | `Ctrl + y` |\n\n### 6.3.4 Language input switch shortcuts\n\nIn order to switch English/Chinese input method, please use `CTRL+s` shortcuts, it is used instead of `CTRL+space` to avoid conflicts with local system.\n\n### 6.3.5 How to tune the screen size\n\nThere are tow methods to tune the screen size, one is auto scaling by noVNC, another is pre-setting during launching.\n\nThe first one is setting noVNC before connecting.\n\n* Press the left sidebar of noVNC web page\n* Disconnect\n* Enable 'Auto Scaling Mode' via 'Settings -\u003e Scaling Mode: -\u003e Local Scaling -\u003e Apply'\n* Connect\n\nThe second one is setting `SCREEN_SIZE` while running Linux Lab.\n\nThe screen size of lab is captured by xrandr, if not work, please check and set your own, for example:\n\nGet available screen size values:\n\n $ xrandr --current\n Screen 0: minimum 1 x 1, current 1916 x 891, maximum 16384 x 16384\n Virtual1 connected primary 1916x891+0+0 (normal left inverted right x axis y axis) 0mm x 0mm\n 1916x891 60.00*+\n 2560x1600 59.99\n 1920x1440 60.00\n 1856x1392 60.00\n 1792x1344 60.00\n 1920x1200 59.88\n 1600x1200 60.00\n 1680x1050 59.95\n 1400x1050 59.98\n 1280x1024 60.02\n 1440x900 59.89\n 1280x960 60.00\n 1360x768 60.02\n 1280x800 59.81\n 1152x864 75.00\n 1280x768 59.87\n 1024x768 60.00\n 800x600 60.32\n 640x480 59.94\n\nUpdate remote screen size:\n\n $ cd /path/to/cloud-lab\n $ tools/docker/resize 1280x1024 # Specifiy anyone above\n $ tools/docker/resize # If no argument, Sync with host system\n\nIf want fullscreen, follow these steps:\n\n1. If using virtual machine, fullscreen virtual machine at fist\n2. Run `tools/docker/resize` to resize remote lab screen size\n3. Enter into WebVNC Interface, Click the FullScreen button at the left sidebar\n\n### 6.3.6 How to work in fullscreen mode\n\nOpen the left sidebar, press the 'Fullscreen' button.\n\n### 6.3.7 How to record video\n\n* Enable recording\n\n Open the left sidebar, press the 'Settings' button, config 'File/Title/Author/Category/Tags/Description' and enable the 'Record Screen' option.\n\n* Start recording\n\n Press the 'Connect' button.\n\n* Stop recording\n\n Press the 'Disconnect' button.\n\n* Replay recorded video\n\n Press the 'Play' button.\n\n* Share it\n\n Videos are stored in 'cloud-lab/recordings', share it with help from [showdesk.io][019].\n\n### 6.3.8 Linux Lab not response\n\nThe VNC connection may hang for some unknown reasons and therefore Linux Lab may not response sometimes, to restore it, please press the flush button of web browser or re-connect after explicitly disconnect.\n\n### 6.3.9 VNC login with failures\n\nIf VNC login return \"Disconnect timeout\", wait a while and press the left 'Connect' button again, otherwise, check as following:\n\nAt first, check the containers' status (Up: Ok, Exit: Bad):\n\n $ docker ps -a\n CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES\n 19a61ba075b5 tinylab/linux-lab \"/tools/lab/run\" 4 days ago Up 4 days 22/tcp, 5900/tcp linux-lab-21575\n 75dae89984c9 tinylab/cloud-ubuntu-web \"/startup.sh\" 8 days ago Up 8 days ....443/tcp cloud-ubuntu-web\n\nIf the status is 'Exit', that means container may be shutdown or may never up, run it again to resume for the shutdown case:\n\n $ tools/docker/run linux-lab\n\nOtherwise, check the running logs:\n\n $ tools/docker/logs linux-lab\n\nIf normal, that means the login account and password may have been invalid for some exceptions, please regenerte new account and password with the coming steps:\n\n**Note**: The `clean` command will remove some containers and data, please do necessary backup before run it, for example, save the container:\n\n $ tools/docker/save linux-lab\n\nVNC login fails while using mismatched password, to fix up such issue, please clean up all and rerun it:\n\n $ tools/docker/clean linux-lab\n $ tools/docker/rerun linux-lab\n\nIf the above command not work, please try this one (**It will clean more data, please do necessary backup**)\n\n $ tools/docker/clean-all\n $ tools/docker/rerun linux-lab\n\n### 6.3.10 Ubuntu Snap Issues\n\nUsers report many snap issues, please use apt-get instead:\n\n* users can not be added to docker group and break non-root operation.\n* snap service exhausts the /dev/loop devices and break mount operation.\n\n### 6.3.11 How to exit fullscreen mode of vnc clients\n\nThe easiest method is kill the VNC server in Linux Lab:\n\n $ sudo pkill x11vnc\n\n## 6.4 Lab Issues\n\n### 6.4.1 No working init found\n\nThis means the rootfs.ext2 image may be broken, please remove it and try `make boot` again, for example:\n\n $ rm boards/aarch64/raspi3/bsp/root/2019.02.2/rootfs.ext2\n $ make boot\n\n`make boot` command can create this image automatically.\n\n### 6.4.2 linux/compiler-gcc7.h: No such file or directory\n\nThis means using a newer gcc than the one Linux kernel version supported, the solution is [switching to an older gcc version](#toolchain) via `make gcc-switch`, use `i386/pc` board as an example:\n\n $ make gcc-list\n $ make gcc-switch CCORI=internal GCC=4.4\n\n### 6.4.3 linux-lab/configs: Permission denied\n\nThis may happen at `make boot` while the repository is cloned with `root` user, please simply update the owner of `cloud-lab/` directory:\n\n $ cd /path/to/cloud-lab\n $ sudo chown \u003cUSER\u003e:\u003cUSER\u003e -R ./\n $ tools/docker/rerun linux-lab\n\n**Notes**: To make a consistent working environment, Linux Lab only support using as general user: 'ubuntu'.\n\n### 6.4.4 scripts/Makefile.headersinst: Missing UAPI file\n\nThis means MAC OSX not use Case sensitive filesystem, create one using `hdiutil` or `Disk Utility` yourself:\n\n $ hdiutil create -type SPARSE -size 60g -fs \"Case-sensitive Journaled HFS+\" -volname labspace labspace.dmg\n $ hdiutil attach -mountpoint ~/Develop/labspace -nobrowse labspace.dmg.sparseimage\n $ cd ~/Develop/labspace\n\n### 6.4.5 unable to create file: net/netfilter/xt_dscp.c\n\nThis means Windows not enable filesystem's case sensitive feature, just enable it:\n\n $ cd /path/to/cloud-lab\n $ fsutil file SetCaseSensitiveInfo ./ enable\n\n### 6.4.6 how to run as root\n\nBy default, no password required to run as root with:\n\n $ sudo -s\n\n**Notes**: Please don't use the 'su' command.\n\n### 6.4.7 not in supported list\n\nSuch information means the specified value is not supported currently:\n\n $ make boot ROOTDEV=vda\n ERR: /dev/vda not in supported ROOTDEV list: /dev/sda /dev/ram0 /dev/nfs, update may help: 'make bsp B=mips64el/ls3a7a'. Stop.\n\n $ make boot LINUX=v5.8\n Makefile:594: *** ERR: v5.8 not in supported Linux list: loongnix-release-1903 v5.7, clone one please: 'make kernel-clone KERNEL_NEW=v5.8'. Stop.\n\n $ make boot QEMU=loongson-v1.1\n Makefile:606: *** ERR: loongson-v1.1 not in supported QEMU list: loongson-v1.0, clone one please: 'make qemu-clone QEMU_NEW=loongson-v1.1'.\n\nThere are two main types:\n\n* One is the specified version is not there or has not been verified\n * Please clone one and verify it with the usage of `xxx-clone` from section 5.\n\n* Another is the specified value is invalid or simply not verified\n * For example, the above vda is not added in the `ROOTDEV_LIST`\n * This board may not support such type of device or just nobody verify and add it\n * This differs from board and kernel version\n\n### 6.4.8 is not a valid rootfs directory\n\nIf using prebuilt filesystem, this error means the rootfs dir, ramdisk or harddisk creating procedure has been interrupted by `CTRL+C` or similar operations and it means the filesystem is not complete. If no important changes in BSP repository, reset it may help:\n\n $ make bsp-cleanup\n\nIf using external filesystem, please make sure the filesystem architecture follows the Linux standards.\n\n# 7. Contact and Sponsor\n\n## 7.1 Contact Us\n\nWelcome to join our discuss group:\n\n* Wechat: **tinylab**\n* Email: contact /AT\\ tinylab /dot/ org\n\n## 7.2 Give me a star\n\nWelcome to mark our web site, star our git repositories:\n\n* Wechat Group: Tinylab-Org\n\n* Web site: \u003chttps://tinylab.org\u003e\n * Created near 10+ years\n * Focus on Linux kernel and embedded Linux system\n\n* Git Repositories\n * Gitee: \u003chttps://gitee.com/tinylab\u003e\n * Github：\u003chttps://github.com/tinyclub\u003e\n\n## 7.3 Buy our products\n\n* Store: \u003chttps://shop155917374.taobao.com\u003e\n * The taobao store of TinyLab.org community, mainly sell products around our open source projects\n * The products include Linux Lab Disk, Pocket Linux Disk and the Linux Lab boards\n * Welcome to buy some based on your requirement, you can search 'Linux Lab' in taobao application to find us\n\n* Circle: \u003chttps://t.zsxq.com/uB2vJyF\u003e\n * The VIP knowledge channel of TinyLab.org community\n * Create 3+ years, about 1000+ shares and 20+ Linux professionals\n\n* Courses: \u003chttps://m.cctalk.com/inst/sh8qtdag\u003e\n * TinyLab School -- The video channel of TinyLab.org community\n * Video Live, Video Courses are shared by active Linux professionals from TinyLab.org community\n\n## 7.4 Sponsor\n\n### 7.4.1 Sponsor via wechat\n\n![contact-sponsor](doc/images/contact-sponsor.png)\n\n### 7.4.2 Sponsor list\n\n* 2022\n * [Summer 2022][049]\n * Sponsored projects: [Microbench][045], [OpenHW Lab][046], [PWN Lab][047]\n\n * PLCT Lab\n * 2 D1 boards\n * Sponsored [RISC-V Linux Project][048]\n\n* 2021\n * [Lazyparser][015]\n * HelloGCC and HelloLLVM founder\n * 5000RMB\n\n * [Summer 2021][027]\n * Sponsored projects: Rust for Linux, openEuler Kernel for aarch64/virt and x86_64/pc\n\n * T-head\n * 1 D1 board\n\n * Allwinner\n * 3 D1 boards\n\n* 2020\n * [Loongson][005]\n * The famous Chinese Loongson CPU designer and manufacturer\n * Sponsored boards: mips64el/ls2k, mips64el/ls3a7a, mipsel/ls1b, mipsel/ls232\n\n * [Summer 2020][027]\n * Sponsored projects: Linux Lab docker image upgrade from Ubuntu 14.04 to Ubuntu 20.04\n\n * Embedfire\n * 6 imx6ull boards\n\n[001]: https://gitee.com/tinylab/linux-lab/blob/master/doc/install/arch-docker.md\n[002]: https://gitee.com/tinylab/linux-lab/blob/master/doc/install/manjaro-docker.md\n[003]: https://gitee.com/tinylab/linux-lab/blob/master/doc/install/ubuntu-docker.md\n[004]: http://gitee.com/tinylab/linux-0.11-lab\n[005]: http://loongson.cn/\n[006]: https://cctalk.com/m/group/89507527\n[007]: https://docs.docker.com\n[008]: https://docs.docker.com/config/daemon/\n[009]: https://docs.docker.com/storage/storagedriver/select-storage-driver/\n[010]: https://elinux.org/Work_on_Tiny_Linux_Kernel\n[011]: https://get.daocloud.io/toolbox/\n[012]: https://gitee.com/loongsonlab/loongson\n[013]: https://gitee.com/tinylab/csky\n[014]: https://gitee.com/tinylab/linux-lab/issues/I49VV9\n[015]: https://github.com/lazyparser\n[016]: https://github.com/tinyclub/linux-lab/issues/5\n[017]: https://git-scm.com/downloads\n[018]: https://help.aliyun.com/document_detail/60750.html\n[019]: http://showdesk.io/post\n[020]: https://lug.ustc.edu.cn/wiki/mirrors/help/docker\n[021]: https://lwn.net/images/conf/rtlws-2011/proc/Yong.pdf\n[022]: https://m.cctalk.com/inst/sh8qtdag\n[023]: https://shop155917374.taobao.com\n[024]: https://shop155917374.taobao.com/\n[025]: https://space.bilibili.com/687228362/channel/detail?cid=152574\n[026]: https://store.docker.com/search?type=edition\u0026offering=community\n[027]: https://summer.iscas.ac.cn\n[028]: https://tinylab.org/cloud-lab\n[029]: https://tinylab.org/linux-lab-disk\n[030]: https://tinylab.org/pdfs/linux-lab-loongson-manual-v0.2.pdf\n[031]: https://tinylab.org/pdfs/linux-lab-v0.8-manual-en.pdf\n[032]: https://tinylab.org/pdfs/linux-lab-v0.9-manual-en.pdf\n[033]: https://tinylab.org/pdfs/linux-lab-v1.0-manual-en.pdf\n[034]: https://wiki.qemu.org/Documentation/9psetup\n[035]: https://www.cctalk.com/m/group/88089283\n[036]: https://www.cctalk.com/m/group/88948325\n[037]: https://www.cctalk.com/m/group/89626746\n[038]: https://www.cctalk.com/m/group/89715946\n[040]: http://www.kernel.org\n[041]: https://tinylab.org/why-linux-lab\n[042]: https://tinylab.org/why-linux-lab-v2\n[043]: http://showdesk.io/2017-03-11-14-16-15-linux-lab-usage-00-01-02/\n[044]: https://tinylab.org\n[045]: https://gitee.com/tinylab/microbench\n[046]: https://gitee.com/tinylab/openhw-lab\n[047]: https://gitee.com/tinylab/pwn-lab\n[048]: https://tinylab.org/riscv-linux\n[049]: https://tinylab.org/summer2022\n[050]: https://tinylab.org/pdfs/linux-lab-v1.1-manual-en.pdf\n[051]: https://www.cctalk.com/m/group/90483396\n[052]: https://www.cctalk.com/m/group/90251209\n[053]: https://tinylab.org/pdfs/linux-lab-v1.2-manual-en.pdf\n[054]: https://tinylab.org/pdfs/linux-lab-v1.3-manual-en.pdf\n[055]: https://tinylab.org/tiny-riscv-box\n[056]: https://tinylab.org/pdfs/linux-lab-v1.4-manual-en.pdf\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinyclub%2Flinux-lab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinyclub%2Flinux-lab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinyclub%2Flinux-lab/lists"}