Magenta - Readme
__ __ _
| \/ | | |
| \ / | __ _ __ _ ___ _ __ | |_ __ _
| |\/| |/ _` |/ _` |/ _ \ '_ \| __|/ _` |
| | | | (_| | (_| | __/ | | | |_| (_| |
|_| |_|\__,_|\__, |\___|_| |_|\__|\__,_|
__/ |
|___/
##############################################################
# General
##############################################################
Magenta is an implementation of Darwin/BSD on top of the Linux kernel. It is made
up of a number of kernel and userland components that work together. It is fully binary
compatible with iPhone OS 5.0 (as in, it uses the same binary format).
So far, it includes the following libraries:
* CoreFoundation
* libstdc++
* libobjc
* libc++abi
* libicucore
* libncurses
* As part of libSystem
* libmath
* libunwind
* libsystem_blocks
* libC
All libraries are compiled for vanilla Darwin, so nothing is compiled for Linux. The only
exception is libC (which resides inside the dynamic linker) as it serves as the main bridge
between the userland and the kernel.
The final goal is probably recreating the iPhone OS 1.0 stack. I think this is a pretty feasible
goal, considering the fact that there are so many open source libs that can be used to replace
the proprietary libs used by Apple. Just as an example:
* CoreGraphics -> Cairo
-> FreeType
-> libpng/jpg
* Celestial -> Various open source media decoding libs.
* UIKit -> Chamelleon
This is a very weird project. You may ask, why am I doing this? The answer is: no fucking idea :)
This project is actually far more complicated than it sounds. *Far* more complicated.
##############################################################
# Q/A
##############################################################
* Which platforms does it run on?
* ARMv7 only. Nothing else is planned right now.
* Which SDK do I use?
* iPhoneOS5.0 and iPhoneOS2.0 (apple-arm-darwin)
* Why is libSystem built into the linker?
* I don't know, it just seemed more convinient.
* Is it open source?
* Nah. Not right now, at least.
* Why not use the FreeBSD kernel instead?
* I thought about using FreeBSD (or rather NetBSD) kernel instead of using Linux.
However, in this case, I prefer Linux due to the extensive features supported by
the kernel and the hardware compatibility (NetBSD offers limited support for ARM
hardware and it doesn't support my dev board (Arm RealView) and I can't be bothered
to make it myself).
* Will it run iPhone OS apps?
* No, because I'm not aiming to have compatible high level frameworks. Just think
about how much work is required to have a 100% compatible implementation of UIKit
or Celestial. HOWEVER, the CoreOS part should be 100% (or 99%) compatible. Just not
the higher level OS. If you're just interested in this because it will "run iOS apps"
please go away.
* Who are you?
* I'm Christina, a very obscure developer obsessed with embedded hardware. I really
like the ARM architecture.
##############################################################
# Notes
##############################################################
> Linking is slow. This is not a problem per se, due to the fact that linking
is meant to be slow. The libs are not prelinked so every time, the linker has
to resolve everything all over again. This can be resolved with prebinding,
fixed load addresses, shared caches and other things like that. Right now, these
things are not really trivial so let's leave it as is. Later on I can probably make
some sort of a shared cache with a splitseg and stick it on a shared page.
> Core exports a lot of garbage symbols that are not really needed. Not a problem but it
would be nice to limit the exported symbols via an exported file. Not trivial right now.
> Mach messaging is poorly implemented. I am aware of that and I will improve my crappy
implementation later. I did not forget about it, I've just been focusing on other things.
A large number of mach APIs and functionality still needs implementing. I will work on that
later.
> Linking mach object files into the kernel is a silly idea. It should not be done. Never the
less, I'm doing it as an experiement. This doesn't mean that it is a good idea or that anyone
should ever link object files that use incompatible ABI.
> I need some sort of an IOKit-compatible kernel interface. I have a few ideas which I will
work on in the meantime.
> ABI difference. Can be solved by using a custom compiler. The core difference is in 8-byte
type alignment. For a fix, might want to look at the ARM_DOUBLEWORD_ALIGN flag (see arm.h in gcc's src)
##############################################################
# Changelog
# > This changelog has only been maintained for
# a very short amount of time so it is missing earlier
# things.
##############################################################
* 14 May
* libmath
* Implemented 'ceil' and 'fmod'
* Implemented 'modf'
* Core/Linker
* Implemented 'atoi' and 'atol'
* Preliminary support for signals.
* Added more system calls.
* OSSpinLocks are now based on low level locks.
* ELF linker can now map files in.
* Fixed CRT externs to point to the right locations.
* Veronica
* Initial version of 'veronica', a simple startup daemon.
* 18 May
* Core/Linker
* Added 'stroull' and friends.
* Core/Linker
* libBoring stubs
* 19 May
* kld
* Detect if code is ARM or Thumb(2) and set the address bit accordingly.
* 20 May
* Core/Linker
* Support for TermIO ioctl calls.
* Fixed strcpy bug.
* The initial stack layout on initialization is now correct (Apple ABI).
* TermIO functions are now working correctly.
* Signal conversions.
* 'setsigaction' is now working correctly.
* Tidied up the structure of syscalls.
* Made OSLog/lnk::log/OSHalt more "colourful" (see 'OSTermSetTextColor' in OSLog.c)
* Implemented the 'clock()' function
* Silent mode flag to shut Core up if needed.
* Bash
* Fixed compilation issue.
* 23 May
* Core/Linker
* Can now read ELF dynamic symbols.
* tcsetattr now works as it should do on Darwin.
* 'getopt' support.
* OSHalt now exits instead of entering an infinite loop.
* 31 May
* TestSuite
* 'fts_tester' for testing the FTS API.
* 'list_directories' for testing readdir.
* FileCommands
* 'ls' command added.
* Kernel
* Darwin directory listing call (getdirentries)
* Argv/Argc passed on correctly
* Filesystem
* Moved libSystem's guts to /usr/lib/system/
* Core/Linker
* Trampolines for weird $NOCANCEL calls
* Added code to back up the argv stuff (stack).
* Reduced the amount of auxilary libs to the critical ones.
* UUID library (Libc/Extensions/UUID).
* Membership library (LibInfo).
* Stdout now newline buffered.
* Gdtoa library (Libc/Extensions/Gdtoa).
* ACL library (Libc/Extensions/ACL).
* PseudoTty library (Libc/Extensions/PseudoTty)
* Full 'opendir' family support
* Regex library (Libc/Extensions/Regex).
* Arc4random.
* 'rand' family.
* General
* 'fts' support (file traversing)
* Extended attribute stubs.
* 'stroffflags'
* String
* 'stpcpy' and 'strmode'
* Time
* 'strtftime' implemented
* Full support for tzcode.
* Fixed 'gettimeofday'
* PosixSpawin
* Added support for posix_spawn functions.
* No __posix_spawn syscall to back them yet.
* 1 June
* Core/Linker
* 'sigaction' can now take NULL arguments without crashing.
* 'atof' added.
* Tidied up LibSyscall's source code.
* 3 June
* zlib
* Added zlib 1.2.5
* Graphics
* Added 'libpng'
* Core/Linker
* Stdout now really newline buffered (__SWR | __SLBF).
* 'stack.s' thing.
* Stubs for mach thread API (not really needed).
* Kernel
* Tidied up the source tree.
* 'ke_array' support.
* Ports are now defined using an abstract union 'ke_port_t'.
* 'task' structure has new fields for its task port and a portright list.
* IPC now initalized as part of ke_runtime.
* Offloaded kmsg processing to 'mach_kmsg'.
* Different named port types (ipc & task).
* ke_runtime hooks for exit/fork/exec.
* Task ports assigned to each task.
* 4 June
* Core/Linker
* Thread ptr 0 is a synonym for main thread.
* Kernel
* Kmsg now requires MACH_MSG_SEND.
* CoreFoundation
* Fixed the 'HALT' screw up which made the compiler emit rubbish.
* Veronica
* Now uses a runloop source, eliminating a sched_yield() loop.
* 13 June
* Graphics
* Mesa3D for software EGL (GLESv1).
* Kernel
* Darwin system call personality (in 'entry-common.S').
* Mach
* Primitive support for port rights (slab).
* Port names are no longer global and are on per-right basis.
* Updated the mach stuff to support the new name model.
* Kernel IPC space to hold receive rights for task ports.
* Socket API
* Linux socket API no longer supported.
* Sockets are now BSD compatible on the API level.
* Core/Linker
* Search paths now include '/usr/lib'.
* Fixed the library search path function.
* Now uses the native socket API without translation.
* Bridge no longer supports Linux socket API.
* 'strsignal' and 'siglist' implemented.
* Native API for the signal stuff.
* Added some multibyte functions.
* More system calls supported.
* 16 Jun
* Kernel
* Mach
* Minor API changes.
* MachO Loader
* Added a link table to which the linker adds image bits.
* MachO Core Dumper
* Image list dumps and desliding using the link table.
* Support for arm-apple-darwin abi stack unwinding.
* Stack traces.
* Kld (Kext/Kernel Link Editor)
* Can resolve external symbols from the main kernel image.
* Support for nearly all types of relocations in objects.
* More error checks added.
* Proper section mapping, including S_ZEROED
* Mach object loading and execution actually works.
* Darwin system call personality (svc 0x80)
* Core/Linker
* lnk
* Added some minor functions.
* Two level symbol resolution.
* Images are now initialized based on their dependency order
(this is required as things like libobjc need to be inited
before the libs that export them)
* Circular symbol references (ie. CoreFoundation <-> Foundation).
* Advanced symbol lookups.
* Support for link tables.
* Improved support for 'dyld_image_state_bound'.
* Suppoer for re-exported symbols (needed by stdcxx).
* Resolution errors are now more informative.
* OSThread
* Bottom 10 TLS keys are now reserved.
* ObjC
* Fixed bug in objc-arm (see bottom of 'objc-msg-arm.s')
* libstdcxx
* Recompiled with a re-export from libcppabi.
* CoreFoundation
* NSObject type.
* NS-CF
* Most classes are bridged. '__bridge__' build flag to enable-disable.
The bridge is inspired on the PureFoundation bridging with a number of
additions to provide true bridging and add support for arbitrary
objc objects instead of limiting them to NSCF types.
* NSFoundation
* Forked from PureFoundation.
* Decent set of supported classes. Still a lot of stuff missing though.
(for example, things like NSURLConnection)
* 20-25 Jun
* Kernel
* Mach
* 'mmap_region_ex' implemented for mapping in foreign regions
* 'vm_allocate' implementation
* Support for 'task_for_pid' and 'pid_for_task'
* Mach ports are now concurrecny-safe.
* Implemented refcounting for port objects
* And port rights using 'r_kernel'
* Port rights now involve atomic counters
* 'mach_insert_right' implemented.
* Syscall stubs for some mach APIs.
* 'getdirentries' is now a system call.
* Reorganised the mach subsystem.
* Kld
* Better error handling and recovery.
* Loader
* Better support for thumb2.
* Core/Linker
* libsyscall
* Migrated some calls to 0x80.
* lnk
* Classic loader. Now the linker is able to load classic mach-o
binary files from older versions of iPhoneOS.
* Support for fixed load addresses for classic binaries.
* Rewrote the function that maps binaries from files.
* Support for bunny.dylib.
* CoreFoundation
* NSObject utilises '_root_objc' functions for basic operations
like 'retain' and 'release'.
* 30 Jun - 9 Jul
* Kernel
* Minor fixes that allow the kernel to be built using the 'arm-none-wtfabi'
toolchain. This toolchain is basically EABI but with 4-byte alignment for
large types which is compatbible with DarwinABI.
* Mach
* Port wait operations now migrated to interruptilbe wait queues.
* Support for ipc_port_set object creation.
* 'ipc_port_set' message receive operation.
* Added a trap for 'mach_port_insert_member'.
* Got rid of the 8-byte ABI trampolines.
* Port operations are now in separate structures.
* 20-29 Jul
* Kernel
* Mach
* 'vm_allocate'
* The size is now rounded to page boundaries. This will stop
mmap_ex from corrupting the PTEs on non-round sizes.
* Added more sanity checks for the arguments.
* Mach Threads
* Thread port class.
* Extended version of fork which allows forking of remote tasks ('mk_thread_fork').
* Creation of mach threads is done using an extended fork.
* Thread register states can be set.
* Limited support for suspension and resumption of threads.
* Getting thread registers.
* Termination with signals.
* Support for the 'thread_self' construct.
* Mach Hosts
* Host port class.
* Support for the 'host_self' construct.
* Tasks
* Iterating all threads in a task.
* Semaphores
* Basic API functions.
* Semaphore port class.
* BSD Threads
* Support for BSD thread:
* creation
* killing
* exiting
* cancellation
* Basic skeleton for the wq calls.
* Added new system call
* Core/Linker
* Switched over to using Apple's threading implementation which is now
possible due to the fact that the kernel now supplies the APIs
necessary for Mach and Apple BSD threads to exists.
* Reorganised and cleaned up 'syscall_compat'
* Added more methods from ctypes/runes.
* Added new system compatible syscalls.
* Added few new Darwin system call stubs to back the new calls
implemented in the kernel.
* Coloured logging now uses proper resets and VT_DEFAULT as the background
colour to avoid screwing things up on non-black terminals.
* Added stuff from '_simple.*'
* More advanced versions of environ functions.
* Support for 'malloc_zone_from_ptr' backed by an extension to dlmalloc.
* Other small additions and modifications.
* 30 Jul
* Core/Linker
* Fixed the 'dlopen' function so it does not force a reinitialization
af libraries that are already loaded.
* Added the 'dladdr' function so names of loaded images can be derived
from function addresses.
* libmath
* Renamed to 'libsystem_math.dylib'.
* Added a number of missing functions such as 'round'.