The security monitor (SM) is part of MIT's Citadel: the first side-channel-resistant enclaves with secure shared memory on a speculative out-of-order processor.
The SM is a small (~9K LOC), trusted piece of software running at a higher privilege mode than the hypervisor or the OS.
Its role is to link low-level invariants exposed by the hardware (e.g., if a specific memory access is authorized), and the high-level security policies defined by the platform (e.g., which enclave is currently executing on which core and which memory region does it own).
This implementation targets a family of RISC-V (specifically, at least RV64IMA) implementations, and makes use of special hardware features described in Citadel and implemented on the open-source Citadel processor based on Riscy-OO.
Nevertheless, its implementation is generic and shoudl be easily portable to other architectures (mostly by editing code located in the platform folder).
Citadel is the latest of MIT's familly of secure architectures that support secure enclaves and build on previous work such as MI6 and Sanctum.
To build the SM, you will first need a functioning riscv toolchain.
Building the toolchain yourself will allow you to target the exact architecture rv64imafd (for instance, to avoid compiling with compressed instruction) and will ensure the generated binaries can run on the Riscy-OO processor.
The toolchain can be found on the official repo.
Clone the repository with the submodules and install the dependencies (see the repository's README for more detailed instructions).
Run the following instruction to target the right architecture
$ ./configure --prefix=/opt/riscv_linux/ --with-arch=rv64imafd
And then build both the newlib and linux target by running (with optional -j flag to build on multiprocessor machines)
$ make $ make linux
For easier debugging, add GDB_TARGET_FLAGS_EXTRA := --enable-tui to the Makefile.in file.
To debug the SM, tou might want to install QEMU with the special target that emulates the Riscy-OO processor. Building intructions can be found on the git repository.
Building the SM should be a simple matter of performing make sm.
This performs a series of steps:
- Uses the specific parameters to customize linker scripts
- Builds the enclave mini-SM (the SM code included within each enclave's domain).
- Builds a set of identity page tables for the SM. These are used to virtualize an OS's access to physical memory.
- Builds the SM.
make null_boot_loader builds a simple bootloader that will directly enter the SM.
make master_test builds the simple bootloader, the SM, sets up a simple enclave and test various functionality of the API including enclave creation, enclave entry, enclave exit, enclave deletion but also shared memory and secure mailboxes.
All elfs and binary files can be found in the build/ folder. The make disassemble-all and make source-all targets will disassemble and intermix the source code for every elfs files present in the build/ folder.
To run the SM, you will need QEMU (see instructions above).
Set up the global variable SANCTUM_QEMU to path leading to the qemu-system-riscv64 executable.
You can then run make run_master_test to run the test.
To debug the SM, simply run make debug_master_test. That will launch QEMU and pause waiting for a GDB instance. You can then launch a GDB intance by running riscv64-unknown-elf-gdb (that you should have built with the riscv toolchain earlier).
This should launch GDB in TUI mode, connect to QEMU, load the required symbols and have the emulator paused on the first instruction of the bootloader (address 0x1000).
All GDB instructions to set up the right symbols are can be found in .gdbinit.
Follow GDB instruction provided in the error message if the use of .gdbinit by GDB requires special authorization.
Interesting breakpoints to set up might be test_entry, enclave_entry but also sm_init, trap_vector_from_untrusted and trap_vector_from_enclave to break on SM entries.
To write your own untrusted application or enclave that interracts with the SM, refer to the encalve API and make sure to link the API headers located in the API folder. Example of enclaves and untrusted application interating together including skeleton-code to write your own can be foud on our organition github.
The api/ folder contains the SM API, includig the part exposed to untrusted code and the part exposed to enclaves.
The platform/ folder should contains most of the code specific to our hardware platform (in our case RISC-V and more specifically our fork of Riscy-OO).
First it countains the parameters.h file that details many static constants constants used to build the SM including size of structures and placement in memory.
This is where you will also find code related to memory protection but also core initialization and code used to clean and purge microarchitecture structures. Note that other parts of the SM still needs to be implemented in assembly and as a result are platform dependent.
The scripts/ folder contains all python scripts used when building the SM.
This includes the script to generate identity page tables.
The src/ folder contains the SM source code. In particular the initialization code run when booting the machine but also the api implementation, the code satisfying micro-kernel-like functionnalities (console interractions and illegal instructions handling for instance) ad the handlers for the enclave and untrusted code interractions.
The test/ folder contains code used to test the SM functionnalities including simple untrusted application and enclave code.