Porting XNU to BeagleBoard-xM (DM37x): Part 1

11 Jun 2013 in research

After a fairly long period of not doing anything due to personal issues, I've decided to finally get XNU running on the DM37x system on a chip developed by Texas Instruments. To assist early bringup I've decided to use a JTAG debugger and a serial port adapter.

Baby Steps

To start off, I needed to find a suitable way of booting the XNU kernel on BeagleBoard so I could test my code as soon as possible. My initial plan was to port my own bootloader called BootKit (which already has support for XNU) to that platform and then use it to boot up the kernel. About two days into working on the low level initialization code for the platform bits, it started to seem like a waste of time and I've abandoned this plan.

My next choice was the Das U-Boot bootloader which already had full support for BeagleBoard and was used both as the first level and the second level bootloader. Obviously, this bootloader lacked any support for the XNU kernel so I had to develop my own "extension" for it. To simplify the task, I've decided to recycle the code from the BootKit bootloader, integrating it into the u-boot tree and implementing a command that would compile the device tree definitions, map the mach-o kernel and execute it, which I've named bootx.

In essence, this command would load mach_kernel, map out the mach-o executable file at a certain address (0x80001000 for XNU) parse the XML device tree (yes, that's right, XML parser in a bootloader) specification, flatten it, populate the BootArgs structure and call the kernel entry point, passing the boot arguments to it.

Loading mach_kernel

In order to ease the pain in repetitively having to swap out the SD card, I've decided to use u-boot's built in TFTP support to download the kernel from my development machine (downloading it over UART was out of the question because the debug kernel image was just over 8MB in size). I ended up with a following command in my environment that would download and start the kernel:

/* Boot XNU from the network */ \
"xn=setenv ipaddr;" \
    "setenv serverip;" \
    "setenv usbethaddr ba:d0:4a:9c:4e:ce;" \
    "usb start;" \
    "echo downloading mach_kernel from tftp ...;" \
    "tftpboot 0x84000000 /mach_kernel;"\
    "echo loading devicetree.plist from flash ...;" \
    "fatload mmc ${mmcdev} 0x83000000 /devicetree.plist;"\
    "echo starting kernel ...;" \
    "bootx 0x80001000 0x84000000 0x83000000 0x0\0"

Platform Expert

The platform expert is a hardware abstraction module used by XNU to access machine specific components (not to be confused with architecture specific components like the MMU), which consists of early access to the serial port, interrupt controller initialization and handling of interrupts from the system timer. Unlike the x86 version, the platform expert routines are specific to each machine. For now, I have only decided to implement the serial output routine for BeagleBoard. Having done that, I've tweaked several kernel makefiles to add suport for OMAP3 as a machine config. I could now build the kernel using the following command:

make TARGET_CONFIGS="debug arm OMAP3" SDKROOT="/Darwin_ARM_SDK/"

Hello XNU

Of course, as expected, soon after starting, the kernel crashed very early without producing any visible output. I have anticipated that early printing might produce some useful information. I have tried to implement semihosting in the kernel in order to get some early output which unfortunately didn't quite work for an unknown reason. Instead of fiddling around with debugger configuration files, I have decided to do something simpler and make my early printing routine print to a preallocated buffer, which I could then dump using the debugger.

Early panic dump

From my early dump, I could see that the kernel was panicking out in arm_init due to a misaligned L1 table. I have quickly fixed that in the bootloader, and having recompiled it, I was able to get past early initialization where it called machine_startup to do general kernel initialization.

Odd crash

Shortly after being started, the kernel died due to a data abort happening before first VM map was created (full log here):

Fatal Exception: map is NULL, prob a fault during VM init, fault_addr is 0x8064bbb4

Quickly, I tried to read that address using a debugger, which seemed to work just fine, so I was sure it was not a virtual memory related problem. Just to be sure I dumped the translation tables and examined the entry for this page, which was perfectly valid. I have then looked at the data fault status register and noticed that it indicated that an external abort has occured (0x8).

Because I was confused as to what this actually meant, I did some quick searching and found an article on TI forums where someone has complained about similar behaviour. Apparently, this SoC asserts an external abort when an STREX or LDREX instruction is used on a page that is mapped with the Shared bit set. As this SoC is uniprocessor, I've simply disabled setting of that bit for pages, which resolved the issue.

Next Steps

Because getting all of the output via a serial console seemed boring, I've decided to implement video initialization in the platform expert so I could use XNU's video console. Usually, this task is left to the bootloader, but I've decided to implement it in the kernel. Because of the complexity of the OMAP DSS system, I took me a few hours to work out how to configure the display controller, after doing which I was able to populate PE_state.video and use that to set up the console. This was the result:

Video console

In order to advance further, I had to implement support routines for setting up the interrupt controller and the system timer in the Platform Expert, which I'm going to explain in my next post.