Notes on Darwin and the macOS architecture

In English

These notes are primarily based on two sources:

This is intended as an introductory-level overview, as I'm not a systems expert.

Still, I think it's sufficient to build a solid intuition for the macOS architecture.

Darwin

Darwin is the core operating system developed by Apple and shared by all Apple OS variants.

It's the name all *OS variants report via uname:

uname -a
uname -nvpm

macOS and all Apple OS variants are built on top of Darwin.

SystemVersion.plist is the source of the macOS version (the ProductBuildVersion is fed to the kernel by /sbin/launchd during system boot):

plutil -p /System/Library/CoreServices/SystemVersion.plist

Darwin is UNIX-certified and POSIX-compliant.

It includes a substantial amount of software from a variety of sources:

  • NeXTSTEP and OPENSTEP
  • Mach
  • various BSD flavors (largely FreeBSD)
  • the GNU software suite
  • the XFree86 project
  • and so on

Apple publishes parts of Darwin as open source, but the source code is often incomplete or selectively open:

  • some parts stay open because they come from BSD code and are hard to close, but Apple keeps some changes private (e.g., ARM support)
  • some projects have been closed after being open, others appear or disappear over time
  • tools like BSD's what command can sometimes trace a binary's origin, but it's unreliable (what /bin/ls)

macOS and its origins

Modern macOS is primarily derived from NeXTSTEP, rather than the classic Mac OS.

The classic Mac OS resembled systems like Windows 95:

  • kernel-less
  • cooperative multitasking without adequate memory isolation
  • a single misbehaving application or extension could crash the system (the "bomb" error window)

Apple's acquisition of NeXT marked a major turning point in its operating system strategy:

  • it brought Steve Jobs back to Apple, along with key NeXT engineers such as Avadis "Avie" Tevanian (one of the creator of the Mach microkernel at the heart of NeXTSTEP)
  • the integration of NeXT technology took four years
  • Apple's next-generation OS, Rhapsody, was based on NeXT's OPENSTEP
  • concepts like application "bundles", Mach-O binaries, and Objective-C are all inherited from NeXT
  • the system evolved through "Rhapsody", then became "OS X Server", and was released as "Mac OS X 10.0" in 2001
  • the OS matured three years later with version 10.4

Apple's macOS architecture overview

Apple's official layered view:

Layer Description

Application Layer (Cocoa)
  • components with a direct user interface (Dashboard, Spotlight, etc.)
  • main component = Cocoa framework

Media Frameworks
  • graphics and audio frameworks (CoreAudio, CoreImage, CoreAnimation, etc.)
  • underlying technologies (OpenAL, OpenGL, Quartz, etc.)

Core Services
  • support frameworks for applications and processes without a direct effect on UI
  • e.g., CoreFoundation (CF* APIs)

Core OS
  • lower-level system APIs
  • e.g., OpenDirectory, SystemConfiguration

Kernel and Device Drivers
  • XNU and kernel extensions

A more detailed view of macOS architecture

A more detailed layered view, including:

  • Objective-C and Swift, which are of significant importance across all architectures
  • a less vague classification of "Core Services" vs. "Application Layer"
  • more layers describing a complex code base (3rd party libraries + Apple libraries + XNU) vs. just two layers ("Core OS" and "Kernel")
Layer Description

Applications
  • or bundles
  • Finder, Siri, Spotlight, etc.

Frameworks
  • public frameworks: WebKit, CloudKit, etc.
  • private frameworks: CloudServices, AppleScript, CoreDuet, etc.

Runtime environments
  • Objective-C
  • Swift

Third Party Libraries
  • open source libraries integrated into the system
  • libSQLite, libXML, libiconv…

Darwin (Core) Libraries
  • libraries unique to Darwin (some are open source)
  • libXPC, libtrace, libMalloc, etc.

XNU
  • the kernel at the core of Darwin
  • which forms a complex subsystem in its own right

Below is a description of each part, preceded by a brief architectural aside about property lists.

Property lists

Property lists (plists) are a core user-space configuration format in Darwin (operating system) and a legacy of NeXTSTEP.

A property list is a structured key–value file used to store settings, service definitions, and application metadata.

Plists exist in XML or binary format (with a legacy ASCII/OpenStep format).

They are widely used across the system:

  • app settings (e.g., com.apple.*.plist)
  • launch services configuration (e.g., launchd uses plist files)
  • system configuration files
  • bundle metadata (like Info.plist inside apps)

plutil can be used to validate syntax (its default or -lint) and to convert between formats:

less /System/Library/LaunchDaemons/ssh.plist
plutil -lint /System/Library/LaunchDaemons/ssh.plist
plutil -p /System/Library/LaunchDaemons/ssh.plist
plutil -convert json /System/Library/LaunchDaemons/ssh.plist -o - | jq

# Find some plists in binary format.
file /System/Library/LaunchDaemons/*.plist | grep "Apple binary property list"

# Inspect a binary plist.
less /System/Library/LaunchDaemons/com.apple.syslogd.plist
plutil -convert xml1 /System/Library/LaunchDaemons/com.apple.syslogd.plist -o -
xxd /System/Library/LaunchDaemons/com.apple.syslogd.plist
hexdump -C /System/Library/LaunchDaemons/com.apple.syslogd.plist

Applications

The concept of "App" comes from NeXTSTEP.

Applications are bundles:

  • packages with a fixed directory structure
  • contain executable + resources (private dylibs, frameworks, icons, UI elements etc.)

Finder.app is located in /System/Library/CoreServices/:

ls -d1 /System/Library/CoreServices/*.app

Every app must contain an Info.plist.

Frameworks

Frameworks are a core macOS architecture concept inherited from NeXTSTEP.

A framework is:

  • a bundled shared library (.dylib)
  • plus optional resources in a Resources/ directory (images, configs, headers, etc.)

They can be versioned:

  • via a Versions subfolder with a letter encoding the version (usually A)
  • Current version is symlinked as Current

Locations:

/System/Library/Frameworks
/System/Library/PrivateFrameworks
/System/Library/Frameworks/CoreServices.framework/Versions/Current/Frameworks/

find /System/Library/Frameworks -name Current | xargs ls -l | grep -v A$

Frameworks can be:

  • private (Apple internal only, often closed source)
  • or public

"Umbrella frameworks" may contain sub-frameworks: Cocoa.framework, WebKit.framework…

Third Party Libraries

Darwin takes a similar approach to Unix-like systems in integrating existing open-source components rather than developing support for them from scratch.

Most libraries are in .dylib form (a dynamic library loaded at runtime instead of at compile time).

Location:

/usr/lib

Apple sometimes replaces some external libraries with its own implementations: encryption stack with CoreCrypto and CommonCrypto instead of OpenSSL's libcrypto.

Darwin (Core) Libraries

The most essential library in Darwin user space is libSystem.B.dylib, bundling fundamental components like:

  • the C standard library (libc)
  • threading (libpthread)
  • math (libm)
  • low-level system calls

Many libraries (like libSystem.B.dylib) used to exist as individual files in /usr/lib.

Since macOS Big Sur (2020), most are no longer present as standalone files but are instead bundled into the dyld shared cache.

Tools like otool still report the original install name (path), even if no file exists at that location:

otool -L /bin/ls | grep libSystem

XNU (kernel)

XNU (X is Not Unix) is the kernel of Darwin.

XNU = Mach kernel + components from FreeBSD + additional subsystems.

Mach was originally a microkernel, providing only task/thread management and interprocess communication (IPC).

This was not sufficient for a complete OS. Higher-level services (filesystems, networking, process model) had to be layered on top. The choice of top layer was FreeBSD.

Other components were also integrated into the kernel, resulting in a hybrid architecture where most services run in the same address space.

Components of XNU:

Component Description

Mach and BSD
  • Mach → scheduling, IPC, virtual memory
  • FreeBSD → processes, filesystems, networking, syscalls

libkern
  • basic runtime support
  • provides C utilities and C++ runtime used by kernel code and IOKit

IOKit
  • kernel driver framework (historically kext-based)
  • main client of libkern (uses its object model and runtime)
  • uses subclassing and overloading to simplify the driver creation

The Platform Expert (pexpert)
  • describes and initializes the underlying hardware ("what hardware exists and how it's wired")
  • behave similarly to a Hardware Abstraction Layer (HAL) for the kernel

ml_* APIs
  • machine-layer (ml) primitives for CPU/hardware-specific operations
  • not a "real" abstraction layer
  • e.g., ml_get_cpu_number()

Kernel Extensions (kext)
  • Mach-O binaries (the executable format on macOS) running in kernel space
  • extend kernel functionality (device drivers, security, filesystems, etc.)
  • locations: /System/Library/Extensions/, /Library/Extensions/
  • loaded kexts: kextstat, kmutil showloaded

Darwin supports distinct interfaces with the kernel:

  1. BSD-style system calls

    • POSIX + Apple-specific calls
    • each call is assigned a number
    • invoked via machine-specific instructions (necessary to switch the proc's flag from user-mode to kernel-mode)

    • list of sys calls (POSIX + Apple proprietary):

      less /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/syscall.h

  2. Mach traps

    • Mach system calls ("traps")
    • invoked in the same way as BSD syscalls
    • no public header file listing Mach traps, defined in XNU sources in osfmk/kern/syscall_sw.c
    • largely undocumented but important for Darwin

  3. machine dependent (machdep) calls

    • kernel interfaces used for hardware and architecture-specific operations
    • vary across architectures and variants (32 or 64-bit)

  4. diagnostic calls

    • small set of special kernel entry points
    • used for debugging and testing

Avant Notes on deep learning Après Shell fundamentals with bash

A Kemar Joint