Overview of the Android system
As we can see from the architecture diagram, the architecture layers of Android include Application Framework, Android System Services, HAL, and Kernel. Binder IPC is used as a mechanism for inter-process communication. We will cover each of them in this section. Since recovery is also part of the system programming scope, we will also give an overview of recovery in this section.
Note
You can find more information about key porting layers and system architecture internals at the following Google website:http://source.android.com/devices/index.html
Kernel
As we know, Android uses the Linux kernel. Linux was developed by Linus Torvalds in 1991. The current Linux kernel is maintained by the Linux Kernel Organization Inc. The latest mainline kernel releases can be found at https://www.kernel.org.
Android uses a slightly customized Linux kernel. The following is a concise list of the changes to the Linux kernel:
- ashmem (Android Shared Memory): A file-based shared memory system to user space
- Binder: An interprocess communication (IPC) and remote procedure call (RPC) system
- logger: A high-speed in-kernel logging mechanism optimized for writes
- paranoid networking: A mechanism to restrict network I/O to certain processes
- pmem (physical memory): A driver for mapping large chunks of physical memory into user space
- Viking Killer: A replacement OOM killer that implements Android's "kill least recently used process" logic under low-memory conditions
- wakelocks: Android's unique power management solution, in which the default state of the device is sleep and explicit action is required (via a wakelock) to prevent that
Most of the changes were implemented as device drivers with little or no changes necessary to the core kernel code. The only significant subsystem-spanning change is wakelocks.
There are many Android patches accepted by the mainline Linux kernel today. The mainline kernel can even boot up Android directly. There is a blog from Linaro about how to boot Nexus 7 running a mainline kernel. If you want to try it, you can find the instructions at https://wiki.linaro.org/LMG/Kernel/FormFactorEnablement.
If a Linux device driver is available for a hardware device, it usually can work on Android as well. The development of device drivers is the same as the development of a typical Linux device driver. If you want to find out the merges on the mainline kernel related to Android, you can check the kernel release notes at https://kernelnewbies.org/LinuxVersions.
The Android kernel source code is usually provided by SoC vendors, such as Qualcomm or MTK. The kernel source code for Google devices can be found at https://android.googlesource.com/kernel/.
Google devices use SoC from various vendors so that you can find kernel source code from different vendors here. For example, the kernel source of QualComm SoC is under kernel/msm and the kernel source of Mediatek is under kernel/mediatek. The general Android kernel source code is in the folder kernel/common, which looks much like the Vanilla kernel.
The default build of AOSP is for various devices from Google, such as Nexus or Pixel. It started to include some reference boards from silicon vendors as well recently, such as hikey-linaro, and so on. If you need a vendor-specific Android kernel for your reference platform, you should get the kernel source code from your platform vendors.
There are also open source communities maintaining Android kernels. For example, the kernel for the ARM architecture can be found at Linaro for many reference boards. For Intel x86 architecture, you can find various versions of kernels in the Android-x86 project. As you can see from the following Linaro Linux Kernel status website, the linaro-android tree is a forward port of the out-of-tree AOSP patches. It provides a preview of what Google's next AOSP kernel/common.git tree "might" look like.
Note
The Linaro Android kernel tree can be found at https://android.git.linaro.org/gitweb/kernel/linaro-android.git.The status of this kernel tree can be seen at https://wiki.linaro.org/LMG/Kernel/Upstreaming.
HAL
HAL defines a standard interface for hardware vendors to implement and allows Android to be agnostic about lower-level driver implementations. HAL allows you to implement functionality without affecting or modifying the higher level system. HAL implementations are packaged into module (.so) files and loaded by the Android system at the appropriate time. This is one of the focuses for porting Android systems to a new platform. We will discover more about HAL in Chapter 3, Discovering Kernel, HAL, and Virtual Hardware. Throughout this book, I will give a very detailed analysis of the HAL layer for various hardware interfaces.
Android system services
Functionality exposed by application framework APIs communicates with system services to access the underlying hardware. There are two groups of services that application developers may interact mostly with. They are system (services such as window manager and notification manager) and media (services involved in playing and recording media). These are the services that provide application interfaces as part of the Android framework.
Besides these services, there are also native services supporting these system services, such as SurfaceFlinger, netd, logcatd, rild, and so on. Many of them are very similar to Linux daemons that you may find in a Linux distribution. In a complicated hardware module, such as graphic, both system services and native services need to access HAL in order to provide the framework API to the application layer. We will talk about system services when we debug the init process in Chapter 6, Enabling Wi-Fi on the Android Emulator to Chapter 9, Booting Up x86vbox Using PXE/NFS.
Binder IPC
The Binder IPC mechanism allows the application framework to cross process boundaries and call into the Android system services code. This enables high-level framework APIs to interact with Android system services. An Android application usually runs in its own process space. It doesn't have the ability to access system resources or the underlying hardware directly. It has to talk to system services through Binder IPC to access the system resource. Since applications and system services run in different processes, the Binder IPC provides a mechanism for this purpose.
The Binder IPC proxies are the channel by which the application framework can access system services in different process spaces. It does not mean it is a layer between the application framework and system services. Binder IPC is the inter-process communication mechanism that can be used by any process that wants to talk to another process. For example, system services can use Binder IPC to talk to each other as well.
Application framework
The application framework provides APIs to the applications. It is used most often by application developers. After an interface is invoked by the applications, application frameworks talk to the system services through the Binder IPC mechanism. An Android application framework is not just a set of libraries for the application developers to use. It provides much more than that.
The break-through technology that the Android application framework brought to the developer community is a very nice separation between application layers and system layers. As we know Android application development uses the Java language and Android applications run in an environment similar to the Java virtual machine. In this kind of setup, the application layer is separated from the system layer very clearly.
The Android application framework also provides a unique programming model together with a tight integration with the integrated development environment (IDE) from Google. With this programming model and related tools, Android developers can work on application development with great efficiency and productivity. All these are key reasons why Android has gained so much traction in the mobile device world.
I have given an overall introduction to all the layers in the previous Android system architecture diagram. As I mentioned about the scope of Android system programming before, we can consider all programming in Android systems as within the scope of system programming other than application programming. With this concept in mind, we actually missed one piece in the previous architecture diagram, which is recovery.
Recovery
In this chapter, we want to have a brief look at recovery as well, since we have three chapters about it in the second part of this book.
Recovery is a tool that can be used to upgrade or reinstall Android systems. It is part of the AOSP source code. The source code for recovery can be found at $AOSP/bootable/recovery.
The unique point about recovery compared to the other parts of Android is that it is a self-contained system by itself. We can look at recovery using the following diagram, and compare it to the Android and Brillo architectures that we talked about before:
Recovery is a separate system from Android that shares the same kernel with the Android system that it supports. We can treat it as a mini operating system or an embedded application that we can find in many embedded devices. It is a dedicated application running on top of the same Linux kernel as Android and it performs a single task, which is to update the current Android system.
When the system boots to recovery mode, it boots from a dedicated partition in the flash. This partition includes the recovery image that includes a Linux kernel and a special ramdisk image. If we look at Nexus 5 partitions, we will see the following list:
# parted /dev/block/mmcblk0 parted /dev/block/mmcblk0 GNU Parted 1.8.8.1.179-aef3 Using /dev/block/mmcblk0 Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) print print print Model: MMC SEM32G (sd/mmc) Disk /dev/block/mmcblk0: 31.3GB Sector size (logical/physical): 512B/512B Partition Table: gpt Number Start End Size File system Name Flags 1 524kB 67.6MB 67.1MB fat16 modem 2 67.6MB 68.7MB 1049kB sbl1 3 68.7MB 69.2MB 524kB rpm 4 69.2MB 69.7MB 524kB tz 5 69.7MB 70.3MB 524kB sdi 6 70.3MB 70.8MB 524kB aboot 7 70.8MB 72.9MB 2097kB pad 8 72.9MB 73.9MB 1049kB sbl1b 9 73.9MB 74.4MB 524kB tzb 10 74.4MB 75.0MB 524kB rpmb 11 75.0MB 75.5MB 524kB abootb 12 75.5MB 78.6MB 3146kB modemst1 13 78.6MB 81.8MB 3146kB modemst2 14 81.8MB 82.3MB 524kB metadata 15 82.3MB 99.1MB 16.8MB misc 16 99.1MB 116MB 16.8MB ext4 persist 17 116MB 119MB 3146kB imgdata 18 119MB 142MB 23.1MB laf 19 142MB 165MB 23.1MB boot 20 165MB 188MB 23.1MB recovery 21 188MB 191MB 3146kB fsg 22 191MB 192MB 524kB fsc 23 192MB 192MB 524kB ssd 24 192MB 193MB 524kB DDR 25 193MB 1267MB 1074MB ext4 system 26 1267MB 1298MB 31.5MB crypto 27 1298MB 2032MB 734MB ext4 cache 28 2032MB 31.3GB 29.2GB ext4 userdata 29 31.3GB 31.3GB 5632B grow
The list includes 29 partitions and recovery partition is one of them. The recovery ramdisk of recovery, it has a similar directory structure to the normal ramdisk. In the init script of recovery ramdisk, init starts the recovery program and it is the main process of the recovery mode. The recovery itself is the same as other native daemons in the Android system. The programming for recovery is part of the scope of Android system programming. The programming language and debug method for recovery is also the same as native Android applications. We will discuss this in more depth in the second part of this book.
