{"id":13813642,"url":"https://github.com/AkihiroSuda/lsf","last_synced_at":"2025-05-15T00:33:50.834Z","repository":{"id":58135668,"uuid":"530146931","full_name":"AkihiroSuda/lsf","owner":"AkihiroSuda","description":"Linux Subsystem for FreeBSD (😈 on 🐧)","archived":false,"fork":false,"pushed_at":"2022-08-29T09:17:47.000Z","size":66,"stargazers_count":160,"open_issues_count":2,"forks_count":3,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-04-19T21:03:49.076Z","etag":null,"topics":["elf","freebsd","ptrace","syscalls"],"latest_commit_sha":null,"homepage":"https://medium.com/nttlabs/linux-subsystem-for-freebsd-500b9a88fda4","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AkihiroSuda.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-29T09:17:29.000Z","updated_at":"2024-03-04T13:35:48.000Z","dependencies_parsed_at":"2022-09-09T22:40:56.763Z","dependency_job_id":null,"html_url":"https://github.com/AkihiroSuda/lsf","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AkihiroSuda%2Flsf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AkihiroSuda%2Flsf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AkihiroSuda%2Flsf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AkihiroSuda%2Flsf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AkihiroSuda","download_url":"https://codeload.github.com/AkihiroSuda/lsf/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225319293,"owners_count":17455743,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elf","freebsd","ptrace","syscalls"],"created_at":"2024-08-04T04:01:24.133Z","updated_at":"2024-11-19T08:30:51.772Z","avatar_url":"https://github.com/AkihiroSuda.png","language":"Go","readme":"# LSF: Linux Subsystem for FreeBSD\n\nEmulates FreeBSD on Linux.\nDesigned to be extensible to support other Unix-like OS personalities too.\n\n## Usage\nTested on Ubuntu 22.04 (kernel 5.15). Needs kernel 5.6 at least.\n\n### With Docker (easy)\n\n```console\n(linux)$ docker build -t lsf .\n\n(linux)$ docker run -it --rm --security-opt seccomp=unconfined lsf\n# file /bin/sh\n/bin/sh: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.1, FreeBSD-style, stripped\n# uname -a\nFreeBSD 177f2177ddab 13.1-RELEASE-p1 FreeBSD 13.1-RELEASE-p1 LSF  amd64\n```\n\n### Without Docker (hard, dangerous)\n\n\u003cdetails\u003e\n\n:warning: Running LSF outside a container is highly discouraged, and may result in breaking the host Linux.\n\n\u003cp\u003e\n\n```bash\nmake\ninstall _output/bin/lsf ~/bin/\n\nmkdir -p ~/freebsd/rootfs\ncurl -SL http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.1-RELEASE/base.txz | tar CxJ ~/freebsd/rootfs\n\ncd ~/freebsd/rootfs\nexport LD_LIBRARY_PATH=$(pwd)/lib\n\nlsf -- libexec/ld-elf.so.1 usr/bin/uname -a\n```\n\n\u003c/p\u003e\n\n\u003c/details\u003e\n\n## Status\nPOC.\n\n- Crashes very frequently.\n- Lots of syscalls are still unimplemented.\n- Only the x86\\_64 (amd64) architecture is supported.\n\n## Troubleshooting\n* Retry `docker run` several times if you see `Error: input/output error`.\n* Use `docker run -e LSF_DEBUG=1` to enable debug output.\n* Use `docker exec -it \u003cCONTAINERID\u003e /lsf -- /bin/sh` to open another shell.\n\n## How it works\n### Executable pages\nSurprisingly the Linux kernel does not validate the OSABI of the ELF binaries on `execve()`.\nSo, LSF can \"just\" load `ELFOSABI_FREEBSD` binaries without cooking up the `PROT_EXEC` pages by itself.\n\n### Syscalls\nSyscalls are trapped using the plain old [`PTRACE_SYSCALL`](https://man7.org/linux/man-pages/man2/ptrace.2.html).\n\nUnlike [UML](https://docs.kernel.org/virt/uml/user_mode_linux_howto_v2.html), `PTRACE_SYSEMU`, which reduces the ptrace overhead when the trapped syscall rarely needs to be executed, is not used. \nBecause in the case of LSF, most syscalls can be just passed through to the Linux kernel but with different register values such as the syscall number in the `RAX` register.\n\n[Syscall User Dispatch](https://docs.kernel.org/admin-guide/syscall-user-dispatch.html) is not used either.\n\n#### Syscall ABI\nThe syscall ABI is almost same across Linux and FreeBSD:\nThe syscall number is stored in the `RAX` register, and the syscall arguments are stored in the `RDI`, `RSI`, `RDX`, `R10`, `R8`, and `R9` registers.\n\nThis is similar to the [System V AMD64 ABI](https://en.wikipedia.org/wiki/X86_calling_conventions#System_V_AMD64_ABI) calling convention for the userspace (`RDI`, `RSI`, `RDX`, `RCX`, `R8`, `R9`).\nHowever, it should be noted that in the case of the syscalls, the fourth argument is stored in `R10`, not `RCX`,\nbecause [the `syscall` instruction (`0F 05`) clobbers `RCX`](https://www.felixcloutier.com/x86/syscall.html#operation).\n\nThe returned value is stored back in the `RAX` register. An errno is stored in the `RAX` register too, but as a negative value.\n\nIn addition, FreeBSD processes expect the `CF` flag of the `RFLAGS` register to be set on an error.\nLSF sets the `CF` flag using `PTRACE_SETREGS`.\n\n#### Syscall handlers\n\nSome syscalls can't be just passed through by changing the register values, when the corresponding syscall is missing in Linux, or the syscall has an incompatible argument such as a `struct` with\ndifferent struct members:\n```c\nint fstat(int fd, struct stat *buf);\n```\n\nIn such a case, LSF rewrites the syscall number in the `RAX` register to a \"NOP\" syscall number (`getpid()`), and handles the original syscall arguments in the userspace\nwhen the \"NOP\" syscall exits.\n\nThe userspace handler uses [`pidfd_getfd()`](https://man7.org/linux/man-pages/man2/pidfd_getfd.2.html) to fetch the file descriptors, translates the `struct` definitions, and calls Linux syscalls to emulate the requested FreeBSD syscall.\n\nThe `pidfd_getfd()` syscall has been available since Linux kernel 5.6, but disabled in [Docker's default seccomp profile](https://github.com/moby/moby/blob/v20.10.17/profiles/seccomp/default.json#L750).\nSo, running LSF inside Docker needs `--security-opt seccomp=unconfined`, or at least a custom seccomp profile to enable `pidfd_getfd()`.\nEnabling `pidfd_getfd()` does NOT require acquiring the `CAP_SYS_PTRACE` capability.\n\nInstead of using `pidfd_getfd()`, LSF could alternatively just use symlinks under `/proc/\u003cPID\u003e/fd/` and position information under `/proc/\u003cPID\u003e/fdinfo/` to create yet another descriptor with the similar internal state,\nbut this approach is not as robust as `pidfd_getfd()`, and very unlikely to work with descriptors of non-regular files.\n\n### Thread-local Storages\nFreeBSD processes expect the [TLS](https://wiki.osdev.org/Thread_Local_Storage) pointer (`FSBASE`) to be initialized by the kernel, while the Linux kernel does not provide it.\n\nLSF uses `PTRACE_PEEKTEXT` to inject [the `syscall` instruction (`0F 05`)](https://www.felixcloutier.com/x86/syscall.html) into the code of the FreeBSD process for allocating the TLS with `brk()`,\nand after single-stepping the `syscall` instruction, LSF restores the code and rewinds the instruction pointer to the original position.\n\nThe TLS is initialized with the the `.tdata` and `.tbss` sections of the ELF.\nAt the end of the TLS, there is the TLS pointer that points to itself.\nThe `FSBASE` register is set to this pointer.\n\n### Initial registers\nThe initial registers are different and modified using `PTRACE_SETREGS`.\n\n|         |        RSP      |  RDI  |   FSBASE   |\n|---------|-----------------|-------|------------|\n| Linux   | stack           |   -   |      -     |\n| FreeBSD | stack (aligned) | stack | end of TLS |\n\nThe stack layout is similar.\nThe stack begins with `argc`, `argv`, `envp`, and `auxv`, but `auxv` is slightly incompatible across Linux and FreeBSD.\n\n### Auxv\nFreeBSD processes expect the `AT_BASE` element in the [auxv](https://man7.org/linux/man-pages/man3/getauxval.3.html) to be always provided with a non-zero value,\nbut the Linux kernel sets `AT_BASE` to zero when the ELF interpreter (`/libexec/ld-elf.so.1`) is executed directly.\nIn such a case, LSF modifies the `AT_BASE` value on the stack to be the base address parsed from `/proc/\u003cPID\u003e/maps`.\n\nAlso, some of the auxv elements are incompatible and nullified.\n\n## Comparison with similar projects\n### Non-Linux on Linux\nFreeBSD (and others) on Linux:\n- [LilyVM](http://lilyvm.sourceforge.net/index.en.html) was a project in 2003-2013 to run the modified NetBSD/FreeBSD/Linux kernel using [ptrace](https://www.usenix.org/legacy/events/bsdcon03/tech/eiraku/eiraku.pdf),\n  while LSF only emulates syscalls without using the actual guest kernel code.\n  LilyVM also supported NetBSD and FreeBSD hosts, while LSF only supports Linux hosts.\n\nDarwin on Linux:\n- [Darling](https://github.com/darlinghq/darling) (until 2022) depended on a Linux kernel module, while LSF does not.\n- [Darling](https://github.com/darlinghq/darling) (since 2022) intercepts dylib calls, while LSF intercepts syscalls.\n- [Limbo](https://github.com/meme/limbo) uses Syscall User Dispatch for trapping syscalls, while LSF uses ptrace.\n\nSunOS and Solaris on Linux:\n- `CONFIG_SUNOS_EMUL` (for SunOS 4/Solaris 1) and `CONFIG_SOLARIS_EMUL` (for SunOS 5/Solaris 2) were natively present in the Linux kernel for the SPARC architecture until\n  [2008 (Linux 2.6.26)](https://github.com/torvalds/linux/commit/ec98c6b9b47df6df1c1fa6cf3d427414f8c2cf16). These were built in the Linux kernel, while LSF works as a user mode process.\n\nSystem V derivatives on Linux:\n- [iBCS2 compatibility layer](https://www.linuxjournal.com/article/2809) (c. 1994-1999?) was available for the Linux kernel (1.0-2.2) to support\n  the Intel Binary Compatibility Standard 2 for running binaries of SCO UNIX and System V Release 4 derivatives.\n  This was compiled in the Linux kernel, while LSF works as a user mode process.\n- [The Linux A.B.I. patch (aka ibcs-3 later)](http://linux-abi.sourceforge.net/) (2001-2011) was the kernel 2.4/2.6 port of the iBCS2 compatibility layer.\n  This was compiled in the Linux kernel, while LSF works as a user mode process.\n- [iBCS64](http://ibcs64.sourceforge.net/) (2014-2019) was a 64-bit fork of the Linux A.B.I. patch (ibcs-3).\n  This was compiled as a Linux kernel module, while LSF works as a user mode process.\n- [ibcs-us](https://ibcs-us.sourceforge.io/)(2019-) is a userspace reimplementation of iBCS64.\n  ibcs-us uses `SIGSEGV` handlers for trapping syscalls, while LSF uses ptrace. Also, ibcs-us needs `CAP_SYS_RAWIO` while LSF does not.\n\nWindows on Linux:\n- [Wine](https://www.winehq.org/) intercepts DLL calls, while LSF intercepts syscalls.\n\n### Linux on non-Linux\nLinux on FreeBSD:\n- [FreeBSD's Linux compatibility layer](https://docs.freebsd.org/en/books/handbook/linuxemu/) is built in the FreeBSD kernel, while LSF works as a usermode process.\n\nLinux on Darwin:\n- [Noah](https://github.com/linux-noah/noah) was a project in 2016-2020 to use VMM (but without using the actual Linux kernel) for trapping syscalls, while LSF uses ptrace.\n- [uKontainer](https://github.com/ukontainer) uses frankenlibc to intercept libc calls, and uses LKL to execute the Linux kernel in userspace, while LSF uses ptrace to trap syscalls without using the actual guest kernel.\n- [Lima](https://github.com/lima-vm/lima) runs the actual Linux kernel on VMM, while LSF only emulates syscalls.\n\nLinux on Solaris:\n- [Linux Branded Zone](https://docs.oracle.com/cd/E19455-01/817-1592/6mhahupcu/index.html) was built in the Solaris 10 kernel (removed in Solaris 11), while LSF works as a user mode process.\n\nLinux on System V derivatives:\n- [Lxrun](https://web.archive.org/web/20151025205205/http://www.ugcs.caltech.edu/~steven/lxrun/) (1997-2001) used `SIGSEGV` handlers for trapping Linux syscalls on SCO OpenServer, UnixWare, and Solaris,\n  while LSF uses ptrace.\n\nLinux on Windows:\n- [LINE](https://sourceforge.net/projects/line/) was a project in 2001 to emulate Linux by trapping syscalls using Win32 debug events or a Windows NT kernel driver `int80.sys`,\n  while LSF uses ptrace for trapping syscalls. Non-NT mode of LINE was very similar to LSF, although the target operating systems were different.\n- [Umlwin32](http://umlwin32.sourceforge.net/) was a project in 2002 to run the modified Linux kernel using LINE's `int80.sys` for trapping syscalls,\n  while LSF does not use the actual guest kernel code, and uses ptrace for trapping syscalls.\n- [coLinux](http://www.colinux.org/) was a project in 2004-2017 to run the modified Linux kernel as a Windows NT kernel driver, while LSF only emulates syscal and works as a usermode process.\n- WSL version 1 is built in the Windows kernel, while LSF works as a usermode process.\n- WSL version 2 runs the actual Linux kernel on VMM, while LSF only emulates syscalls.\n\n### Misc\n- [BSD on Windows](https://kotobank.jp/word/BSD%20on%20Windows-10738) (1995-1996) was a product to run 4.4BSD-Lite binaries on Windows 3.1 and 95 (but not on NT). Not much is known about this product today.\n","funding_links":[],"categories":["Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAkihiroSuda%2Flsf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAkihiroSuda%2Flsf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAkihiroSuda%2Flsf/lists"}