2.5. System Services in User Space

As with any microkernel-based operating system, a significant portion of kernel components in vortexOS are moved to user-space and adapted to function there. This design contrasts with monolithic kernels, which typically have hundreds of system calls due to the large number of integrated kernel components. These system calls serve as interfaces to the kernel components and often include numerous sub-calls through mechanisms like ioctl and procfs/sysfs. In contrast, microkernels like VortexOS have only a limited set of system calls—usually just a few dozen.

This reduction in system calls occurs because non-essential kernel components are shifted to user-space, relying instead on Inter-Process Communication (IPC), which will be detailed later.

The user-space bootstrap is the initial program launched by the kernel. It has a straightforward design: the kernel loads the initfs blob, which contains both the bootstrap executable and the initfs image that was provided by the boot loader. The kernel sets up an address space for this blob and jumps to a designated offset provided by the bootloader. The bootstrap process allocates a stack (using an Assembly stub), applies memory protection (mprotect), and completes the necessary steps to execute the init daemon. It also sets up the initfs scheme daemon.

For IPC, VortexOS primarily utilizes file-based system calls. The kernel needs to know which schemes to forward certain system calls to, which it determines through scheme prefixes or by tracking which scheme opened a file descriptor. File-based syscalls are tagged with either SYS_CLASS_PATH or SYS_CLASS_FILE. In the case of SYS_CLASS_PATH, the kernel matches the path prefix with the scheme name. For SYS_CLASS_FILE, the kernel remembers the scheme associated with the file descriptors. While most IPC operations use schemes, there are exceptions, such as standard pipes (similar to Linux's pipe2, read, write, close). Schemes can also establish custom pipe-like IPC mechanisms, like shm: and chan: from ipcd.

Schemes are implemented as Rust traits within the kernel, with built-in kernel schemes adhering to this trait. User-space schemes are provided via the UserScheme trait implementor, which involves messaging between the kernel and the scheme daemon. This communication channel is established when scheme daemons open a :SCHEME_NAME, which is parsed to the root scheme "" with path "SCHEME_NAME". Messages are sent by reading from and writing to the root scheme file descriptor.

Thus, file-based syscalls interacting with user-space files send messages to the respective scheme daemon. The results of these operations are sent back to the kernel, which then returns the outcome to the process that made the syscall.

Communication between user-space and the kernel is generally efficient, although the current syscall handler implementation may be somewhat unoptimized. Systems with Meltdown mitigations might experience exceptions, but such mitigations have not yet been implemented.