Wednesday, 25 July 2018

Timesys Getting Started Guide for Freescale i.MX28 EVK

THIS IS AVAILABLE IN TIMESYS

link: https://linuxlink.timesys.com/docs/gsg

Introduction

This document will describe in detail the procedures for booting a Linux kernel image and mounting a root file system over NFS on the Freescale i.MX28 EVK.

Prerequisites

Host Requirements

To properly boot a board using software from Timesys, your host machine must meet the following requirements:

Modern GNU/Linux Distribution. While you can use nearly any modern Linux distribution released in the last 24 months, Timesys recommends one of the following:
- Ubuntu (Most recent release or LTS)
- Fedora (Most recent release)
An internet connection on the Development Host.
Root or sudo permission on the Development Host.
A copy of the Linux Kernel (uImage) and Root File System (rootfs.tar.gz) for the Target Board downloaded from Factory. These are found in the output directory of your online build, or in the directory build_armv5l-timesys-linux-<libc>/images/ on the command line.
If you are booting your root file system over the network, you will need two network cards installed and configured in the Development Host. One to communicate normally with your LAN/WAN while installing host packages, the other to communicate solely with the target board.
An available serial port on your Development Host.

Target Requirements

To boot the Freescale i.MX28 EVK, you will need the following items:

Freescale i.MX28 EVK Board
Serial NULL Modem Cable
Ethernet Crossover Cable or Ethernet hub/switch and Ethernet Patch Cables

Once you have all of the necessary components, you should perform the following steps:

Connect the debug port of the board to the serial port of your workstation using the null modem cable.
If you are using a cross-over cable, connect the Ethernet port of the board to the second Ethernet port of your workstation.
If you are using an Ethernet hub or switch, connect the board to the hub with a straight-through Ethernet cable, then connect the hub to the second Ethernet port of your workstation.
Connect the power supply to your board.

Preparing the Target

Configuring Serial Communication

The i.MX28EVK uses a serial debug port to communicate with the host machine. The commands discussed in this section are meant to be performed by a privileged user account. This requires the root login or prepending each command with sudo.

Using Minicom

Start minicom on your host machine in configuration mode. As root:

# minicom -o -s -w
A menu of configuration should appear. Use the Down-arrow key to scroll down and select the Serial port setup option, and press Enter.
Verify that the listed serial port is the same one that is connected to the target board. If it is not, press A, and enter the correct device. This is /dev/ttyS0 on most Linux distributions.
Set the Bps/Par/Bits option by pressing the letter E and using the next menu to set the appropriate values. You press the key that corresponds to the value 115200, then press Enter.
Set Hardware flow control to No using the F key.
Set Software flow control to No using the G key.
Press Enter to return to the main configuration menu, and then press Esc to exit this menu.
Reset the board, and wait for a moment. If you do not see output from the board, press Enter several times until you see the prompt. If you do not see any output from the board, and have verified that the serial terminal connection is setup correctly, contact your board vendor.

TIP: If you experience an error similar to Device /dev/ttyS0 is locked when starting minicom, it usually means that another process is using the serial port (which is usually another instance of minicom). You can find the process that is currently using the serial port by executing the following:

# fuser /dev/ttyS0 /dev/ttyS0: 28358

# ps 28358 PID TTY STAT TIME COMMAND 28923 pts/0 S+ 0:00 minicom

This process can also be killed directly with fuser as root. Please use this command with caution:

# fuser -k /dev/ttyS0

Using GNU Screen

To quickly connect to a board using Gnu Screen, execute the following:

# screen /dev/ttyS0 115200

For more information about using screen, please consult the man page, or view the manual online at http://www.gnu.org/software/screen/manual/screen.html

Configuring the Network Interface

Finding and Changing the MAC Address

The MAC address on the i.MX28EVK is set by the ethaddr environment variable in U-Boot. If ethaddr is not set, it can be set using the setenv command.
Example

> setenv ethaddr 12:34:56:78:9a:bc

The MAC Address can be found using the printenv command in U-Boot.
Example

> printenv baudrate=115200 bootfile="uImage" stdin=serial stdout=serial stderr=serial ethaddr=12:34:56:78:9a:bc

NOTE Once the MAC address has been set, it cannot be changed without destroying the entire U-Boot environment.

Changing the IP Address

The IP address can be set manually by modifying the ipaddr environment variable, or automatically using the dhcp or bootp commands.

Preparing the Host

Setting up the network

The commands discussed in this section are meant to be performed by a privileged user account. This requires the root login or prepending each command with sudo.

Identify the network adapter connecting the Host to the Target

Timesys only supports direct Ethernet connections from the development Host to the Target board. Ideally, the development Host would have two network adapters; one adapter to connect to your LAN, and one Ethernet card to connect directly to the Target board with a crossover cable or Ethernet hub. If your development Host only has one network interface it must be directly connected to the Target board. The Ethernet adapter connected directly to the target board must be:

Configured with a proper static IP address and Subnet Mask.
Connected directly to the target board with either a crossover cable or its own Ethernet hub.

From a command prompt issue the command:

# /sbin/ifconfig

Each interface will report its IP address, Subnet Mask, and Default Gateway information:

eth0 Link encap:Ethernet HWaddr 00:19:bb:49:ff:0e inet addr:192.168.3.244 Bcast:192.168.3.255 Mask:255.255.254.0 inet6 addr: fe80::219:bbff:fe49:ff0e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:57214 errors:0 dropped:0 overruns:0 frame:0 TX packets:47272 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:43109083 (41.1 MB) TX bytes:6308206 (6.0 MB) Interrupt:16 eth1 Link encap:Ethernet HWaddr 00:10:b5:4a:c1:a9 inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.0.0.0 UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:21 Base address:0x1100 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:1974 errors:0 dropped:0 overruns:0 frame:0 TX packets:1974 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:226637 (221.3 KB) TX bytes:226637 (221.3 KB)

Note the IP address and Subnet Mask of the appropriate network connection. You will use this to configure the DHCP server.

Installing the server daemons on the development host

On Ubuntu 11.04 and newer:
# apt-get install xinetd tftp tftpd isc-dhcp-server \ nfs-kernel-server portmap
On Ubuntu 10.11 and older:
# apt-get install xinetd tftp tftpd dhcp3-server \ nfs-kernel-server portmap
NOTE: Older versions of Ubuntu use nfs-common and nfs-user-server in place of nfs-kernel-server
On Fedora Core:

# yum install xinetd tftp tftp-server dhcp nfs-utils

Important:
After installing these packages the DHCP server software may start automatically. Having the DHCP server running while you are connected to a LAN can interfere with the operation of other computers. After the DHCP service installs and starts issue these commands to stop the DHCP service and prevent it from starting automatically at boot:

To stop the dhcp service:
- On Ubuntu 11.04 and newer:
  
  # service isc-dhcp-server stop
- On Ubuntu 10.11 and older:
  
  # service dhcp3-server stop
- On Fedora Core:
  
  # /etc/init.d/dhcp stop
To prevent the service from starting automatically:
- On Ubuntu 11.04 and newer:
  
  # chmod 644 /etc/init.d/isc-dhcp-server
- On Ubuntu 10.11 and older:
  
  # chmod 644 /etc/init.d/dhcp3-server
- On Fedora Core:
  1. Click the System Menu
  2. Select Administration
  3. Select Services
  4. Select dhcpd
  5. Click the Customize button
  6. Uncheck Runlevel 2, 3, 4 and 5

Disable SELinux and Firewall on Fedora Core

On Fedora Core, SELinux and the firewall will interfere with many of the services that are needed to work with the target board. These should be disabled before continuing.
Generally Ubuntu does not have these services running by default.

Disable SELinux:
1. Click the System Menu
2. Select Administration
3. Select SELinux Management
4. Change System Default Enforcing Mode to Disabled
Disable Firewall:
1. Click the System Menu
2. Select Administration
3. Select Services
4. Select iptables
5. Click the Customize button
6. uncheck Runlevel 2, 3, 4 and 5

Setting up DHCP

Edit the dhcpd configuration file:
- On Ubuntu, edit /etc/dhcp/dhcpd.conf and include the following lines (note: on older versions of Ubuntu this file is at either /etc/dhcp3/dhcpd.conf or /etc/dhcpd.conf):
  
  subnet 10.0.0.0 netmask 255.0.0.0 { host targetboard { fixed-address 10.0.0.10; hardware ethernet 12:34:56:78:9a:bc; option root-path "/full/path/to/rfs"; filename "uImage"; } }
- On Fedora Core, edit /etc/dhcpd.conf and include the following lines:
  
  ddns-update-style ad-hoc; subnet 10.0.0.0 netmask 255.0.0.0 { host targetboard { fixed-address 10.0.0.10; hardware ethernet 12:34:56:78:9a:bc; option root-path "/full/path/to/rfs"; next-server 10.0.0.1; filename "uImage"; } }
Test the DHCP server on the network card that is connected to your development board. For this example assume eth1. This command will start the DHCP server in the foreground and output any status or error messages to the screen.
- On Ubuntu up to 12.04 LTS:
  
  # service dhcp3-server restart
- On Ubuntu 12.04 LTS and later:
  
  # service isc-dhcp-server restart
- On Fedora Core:
  
  # /usr/sbin/dhcpd -d eth1
- It is recommended to start the DHCP server in this manner each time you need to boot your Target board.

Setting up TFTP

Edit the xinetd.conf file
- On Ubuntu, edit /etc/xinetd.conf and add the following lines just above the line that reads includedir /etc/xinetd.d.
  
  service tftp { socket_type = dgram protocol = udp wait = yes user = root server = /usr/sbin/in.tftpd server_args = -s /tftpboot disable = no }
- On Fedora Core, the tftp-server package creates a /etc/xinetd.d/tftp file. Edit this file and change the disable line from yes to no. The contents of the file are:
  
  service tftp { socket_type = dgram protocol = udp wait = yes user = root server = /usr/sbin/in.tftpd server_args = -s /tftpboot disable = no per_source = 11 cps = 100 2 flags = IPv4 }
Create the /tftpboot folder if it does not exist:

# mkdir /tftpboot
Copy the kernel image to the /tftpboot directory:

# cp /path/to/kernel/image/uImage \ /tftpboot/uImage
NOTE Also copy other files that are required for booting, such as a device tree blob, to /tftpboot.
Restart the xinetd server with the following command:

# /etc/init.d/xinetd restart
Test the TFTP server with the following commands

# tftp localhost tftp> get uImage Received 1456898 bytes in 0.4 seconds tftp> quit
Set xinetd to start automatically on Fedora Core. Ubuntu users will skip this step.
1. Click the System Menu
2. Select Administration
3. Select Services
4. Select xinetd
5. Click the Customize button
6. Check Runlevel 2, 3, 4 and 5

Setting up NFS

As root, extract rootfs.tar.gz to a directory and note the path. This path will be referred to as /full/path/to/rfs in this document.

# mkdir /full/path/to/rfs # cd /full/path/to/rfs # sudo tar xvf rootfs.tar.gz
Export this path by editing /etc/exports to include a line similar to the following:

/full/path/to/rfs 10.0.0.10(rw,no_root_squash)
Restart the NFS services
- On Ubuntu issue the following commands in order:
  
  # service portmap stop # service nfs-kernel-server stop # service portmap start # service nfs-kernel-server start
  NOTE: Older versions of Ubuntu use nfs-common and nfs-user-server in place of nfs-kernel-server
- On Fedora Core:
  
  # /etc/init.d/nfs restart
Set nfsd to start automatically on Fedora Core. Ubuntu users will skip this step.
1. Click the System Menu
2. Select Administration
3. Select Services
4. Select NFS
5. Click the Customize button
6. Check Runlevel 2, 3, 4 and 5

Booting the Board

Set Environment Variables

You must set a few environment variables in order to boot the board over TFTP and DHCP. This is done with the setenv and saveenv commands in U-Boot. On the target, set the following environment variables:

Variable	Value
bootargs	console=ttyAM0,115200 ip=dhcp root=/dev/nfs rw
bootcmd	bootp\;bootm
loadaddr	42000000

If you are not using bootp to load the kernel, you must also specify the following values:

Variable	Value
serverip	10.0.0.1
ipaddr	10.0.0.10
bootfile	uImage
bootargs	console=ttyAM0,115200 ip=${ipaddr} root=/dev/nfs rw nfsroot=${serverip}:/full/path/to/rfs
bootcmd	tftp\;bootm 42000000

Example

> setenv bootargs console=ttyAM0,115200 ip=dhcp root=/dev/nfs rw > setenv bootcmd bootp\;bootm > setenv loadaddr 42000000 > saveenv

Load The Kernel

You can use bootp or tftp to load the kernel. Note that the DHCP server needs to be set up for bootp, and tftp is necessary for both.
Example

> bootp Speed: 100, full duplex BOOTP broadcast 1 Using eTSEC1 device TFTP from server 10.0.0.1; our IP address is 10.0.0.10 Filename 'uImage'. Load address: 42000000 Loading: ################################################################# ################################################################# ################################################## done Bytes transferred = 2632869 (282ca5 hex)

Boot the Kernel

The bootm command is used to boot the kernel. It loads the file that was previously uploaded using the bootp, dhcp, or tftp commands.
Example

> bootm ## Booting kernel from Legacy Image at 042000000 ... Image Name: Linux-2.6.31 Image Type: Linux Kernel Image (gzip compressed) Data Size: 2632805 Bytes = 2.5 MB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK Uncompressing Kernel Image ... OK

Additional Information

Factory Documentation

How To Modify and Use Custom Device Tree with Factory

Summary

This HOWTO covers a two-step process for both modifying the Factory-provided device tree and then integrating the (now) custom device tree with Factory for use in future builds.
NOTE: This HOWTO assumes that the Desktop Factory has been configured and built prior to attempting to modify the device tree. If these Factory configuration and build steps have not been completed yet, please do so before continuing with this HOWTO.
Custom Device Tree use with Factory can be broken up into the following two distinct tasks, each of which is covered within this HOWTO:

Note: To use the modified dts with new Factory installations (or to use an existing custom dts), simply follow the steps 2.1 and 2.2 prior to kicking off the BSP/SDK build.

Device Tree Introduction

The Device Tree is a generic term encompassing the components that make up a new method of describing the hardware layout of an embedded system that isn't compiled into the kernel proper. The device tree mechanism aims at replacing conventional board .c files that hard-code the hardware layout for each particular processor and/or board. This was adopted into the mainline kernel due to the growing number of hardware variants for a given processor (e.g. Freescale's i.MX6 processor) and the resulting need for processor/board support in the kernel proper. Using device trees, a single kernel image can be booted along with the appropriate Device Tree Blob (dtb) to adequately describe the hardware layout.
The Device Tree Blob (*.dtb) file is the file that is actually deployed to the hardware at boot time. It is typically loaded by the bootloader along with the kernel image. The machine-readable dtb is generated via compilation of human-readable Device Tree Source (*.dts) and Device Tree Source Include (*.dtsi) files by the Device Tree Compiler (dtc).

Device Tree Sources and Factory

The dts and dtsi files are located within the kernel source tree at arch/[arch]/boot/dts/. This is where the dtc looks for particular .dtsi files included in the .dts, and also where the device tree compilation takes place.
Desktop Factory comes packaged with dts files for board/kernel combinations which support device trees. These can be found in target/configs/kernel/[board]/ - typically named dts-[kernel-version]. It is from here that the appropriate device tree source is copied into the src/dl/k/kernel/kernel-[version]/ directory and then ultimately to the kernel working directory (during the kernel_devicetree-fetch stage of BSP/SDK build) as kernel_devicetree.dts. This renaming of the *.dts file is intentional to eliminate any conflicts (overwriting of or confusion) with existing *.dts files included in the kernel source tree.

Modify Existing dts Files

Modify Existing Device Tree Source (dts) Files

Device tree modifications should start with the existing kernel_devicetree.dts file in the downloaded source files for the kernel (found in the src/dl/k/kernel/kernel-[version]/ directory). For example:

vim src/dl/k/kernel/kernel-3.10/kernel_devicetree.dts

Factory compiles the kernel_devicetree.dts with the dtc built in the Linux kernel source tree near the end of the kernel-build step of the BSP/SDK generation. This action needs to be initiated next.

Rebuild Binary Device Tree Blob (dtb)

Once modifications have been made to the dts, it needs to be recompiled. There are a number of kernel_devicetree-* make targets available within Factory to manipulate the device tree, though to rebuild the dtb, we can simply call the following from the top-level Factory directory:

make kernel_devicetree-host-install

This will propagate modifications to the kernel_devicetree.dts located in src/dl/ to the kernel working directory and recompile the dtb. The resulting kernel_devicetree.dtb is then copied to the Factory images directory for ease of access (build_[arch]-timesys-linux-[libc]/images/). This process renames the dtb as [board].dtb during the copy to the Factory images directory.

Note: The above command will not rebuild the SDK Installer to include the new dtb file. The make installer command run from the top-level Factory directory is necessary to rebuild the SDK Installer and include the newly compiled dtb.

Deploy New dtb for Testing

Once created, the new *.dtb file needs to be copied to the boot media and loaded by the bootloader along with the kernel image. Please refer to the appropriate Board-specific Getting Started Guide for information regarding this step.

Integrate Custom dts with Factory

Once the device tree has been satisfactorily modified and tested, the modifications can be integrated into the Factory BSP/SDK build process. This is accomplished by storing the custom dts in an accessible location, setting Factory to fetch the custom dts, resetting the device tree portion of the build to a pristine state, and rebuilding the SDK Installer. This example illustrates the steps necessary for storing the custom dts in the src/local/ directory for use in the BSP/SDK build; however, it can indeed be located elsewhere on the local development host, on a network-accessible device, or in a SCM system.

Copy Modified dts to Local Sources Directory

Now that the dts has been modified and resulting dtb tested, integration of the custom dts starts by preserving the modifications in a safe location within Factory (source modifications made in the build_*/linux-[VER] directory will be lost during certain cleaning operations). Factory is designed to utilize a specific user-created source directory (src/local/ for custom files to be used during the BSP/SDK build process. Storing your custom dts in this src/local/ directory not only keeps custom files centrally located in Factory (good practice from a maintenance standpoint), but src/local/ is not effected by cleaning operations run within Factory.

The src/local/ directory structure is intended to closely mimic that of the src/dl/ directory after the Factory fetch stage of a BSP/SDK build. Keeping with this convention, the src/local/ directory, along with the expected kernel-related subdirectory structure can be created with one simple command run from the top-level factory directory (replacing [kernel version] with the kernel version specific to your selections):

mkdir -p src/local/k/kernel/kernel-[kernel version]/

and then copying the dts to this new location is as simple as (again, substituting in your build's specific values in place of variables indicated by [square brackets]):

cp build_[arch]-timesys-linux-[libc]/linux-[kernel version]/arch/[arch]/boot/dts/kernel_devicetree.dts src/local/k/kernel/kernel-[kernel version]/

Note: if using an existing custom dts file (not modifying the Factory-supplied dts), be sure to replace the path to the dts file in this command to reflect the location of your custom dts file.

Modify Workorder to Utilize Custom dts File

The dts file Factory uses during the BSP/SDK build is defined within the workorder. It is here that you specify which dts file to fetch and use in generating the dtb for the build. The menuconfig interface can be used to make the change to your custom dts file as such.

Run make menuconfig from the top-level Factory directory
Navigate to Target Software > Kernel > Create Device Tree Blob > Device Tree filename for your board
Modify value to reflect your custom dts file (in our case, this would be file://$(TSWO_SRC_LOCAL_LOCATION)/k/kernel/kernel-[kernel version]/kernel_devicetree.dts)
Exit menuconfig and save the workorder

Note: the $(TSWO_SRC_LOCAL_LOCATION) entered into the workorder in step 3 above evaluates to src/local/ in the top-level Factory directory.

Now that the Factory knows where to find your custom dts file, all that is left to do is clean the device tree sources and rebuild.

Clean Device Tree Sources

In order rebuild the dtb, while having Factory pull your custom dts from the src/local/ directory, the device tree portion of the kernel build stage needs to be reset within Factory. This can be done with the following, issued from the top-level Factory directory:

make kernel_devicetree-distclean

This action will remove the existing dts file stored in the src/dl/k/kernel/kernel-[kerenl version]/ directory (thereby requiring your modified dts to be fetched during the rebuild) and also remove the compiled *.dtb file from your kernel working directory and the BSP output (build_*/images/) directory.

Rebuild Device Tree and SDK Installer

With the device tree reset and everything set to build the dtb with the custom dts file, all that's left to do is rebuild the dtb and SDK Installer.

make kernel_devicetree-host-install
- this only regenerates the device tree in the kernel working directory and copies it to the build_*/images/ directory (dts modifications will not be reflected in the dtb included in the SDK Installer)
- this is automatically completed during the kernel build within Factory (make kernel-build from top-level Factory directory)
make installers
- this will run the above make kernel_devicetree-host-install and then rebuild the SDK Installer to include the custom dtb

How To Use a Custom Boot Logo / Splash Screen

If you would like to display a custom logo on the frame buffer of your board as the Linux kernel is booting, follow these instructions after you have a successful desktop factory build.

Prerequisites

Linux host with the GIMP Image Editor and netpbm utilities installed: On Ubuntu/Debian:
```
sudo apt-get install gimp netpbm
```
In Fedora/CentOS:
```
sudo yum install gimp netpbm netpbm-progs
```

Limitations

The Boot Logo is limited to 224 colors. You can start with an image with more, but the ppmquant step will reduce the image to 224 colors.
The logo will be displayed relative to the upper left hand corner of the screen.

Method

Run the GIMP image editor. It can be started on the command line with:
```
gimp
```
Open your logo image and scale it to fit your frame buffer. Some common frame buffer sizes are 320x240 and 480x272.
Save the image as logo.ppm, when prompted for a format, choose raw.

Run the following commands in the same directory you saved your logo.ppm:

ppmquant 224 logo.ppm > logo_224.ppm
pnmnoraw logo_224.ppm > logo_final.ppm

Copy the resulting logo_final.ppm to your kernel source directory replacing the default boot logo. If you are using the default Timesys kernel, this will be located at </path/to/your/factory>/build*/linux-<version>/drivers/video/logo/logo_linux_clut224.ppm
If you are using an external kernel, the path from drivers/ is the same.
Rebuild your kernel image by running
```
rm build_*/linux-<version>/.stamp_image_built && make
```
from your factory directory and boot the resulting kernel image on your board.

Troubleshooting

My logo doesn't display

Make sure framebuffer support and boot logos are enabled in your kernel configuration. These options can be found in Device Drivers -> Graphics Support

Make sure that the framebuffer console (Device Drivers -> Graphics Support -> Console display driver support -> Framebuffer Console support) is enabled in your kernel configuration.

Check dmesg for the following message: boot-logo bigger than screen. Try using a smaller image. You are limited by the size of your framebuffer- the kernel will not resize for you. The default logo size is 80x80, so if you are having difficulty sizing, try that first.

If your image is divisible by 8, but not by 16, then you need to disable the high resolution VGA console fonts (CONFIG_FONT_8x16) in your kernel configuration, and rebuild. In general, your image should be divisible by the VGA console font size of your system for best results.
1. Enable Device Drivers -> Graphics Support -> Console display driver support -> Select compiled-in fonts
2. Disable every option beneath that, except for Device Drivers -> Graphics Support -> Console display driver support -> VGA 8x8 font

The wrong logo is displayed

If your kernel shows the Timesys boot logo rather than the default penguin, you will need to de-select the Timesys boot logo option in the kernel-menuconfig interface at:
Device Drivers->Graphics Support->Bootup logo->224-color TimeSys Linux logo

What is Priority Inversion? Why You Care and What to Do About It

By far the most pervasive mechanism to control the sequencing of tasks in time-constrained systems is priority scheduling. Priority scheduling means that whenever a task or thread is ready to execute, its priority relative to other threads determines the actual order of execution. Ideally, priorities in a time-constrained system should always reflect the time constraints for the tasks. A useful priority assignment heuristic is to assign higher priorities to tasks that are part of tighter time constraints. Thus, a task that needs to respond in a shorter time will get higher priority than a task that needs to respond in a longer time. This intuitive heuristic, known as monotonic scheduling, works well when the system is not overloaded. The more commonly known rate-monotonic scheduling is a special case of this heuristic for periodic tasks. (For detailed explanations of these scheduling mechanisms, see the excellent textbook Real-Time Systems by Jane W. S. Liu (Prentice Hall; ISBN 0130996513; 1st edition).

What Is Priority Inversion?

The most desirable state of a priority scheduler is that the system should always be executing the task with the highest priority. However, it is impossible for an operating system to invariably achieve this state. When circumstances within the system force a higher-priority task to wait for a lower-priority task, priority inversion occurs. This can lead to unbounded priority inversion (discussed in the next section), where a high-priority thread misses its deadline.
Priority inversion is sometimes necessary, and is often outside the control of the operating system developer. At other times, priority inversion might be a deliberate consequence of design decisions made by system developers, as an attempt to balance priority enforcement with other desirable characteristics, such as maintaining integrity.
An example of priority inversion that is typically outside of an operating system developer’s control is the result of the interrupt architecture in most modern machines. When an interrupt occurs, most operating systems will handle the interrupt immediately, regardless of the time constraint of the tasks depending on it. At least some of the processing is done at high priority, to find out which task is waiting for the interrupt.
Another example is when a low-priority task has locked a resource (such as a device or data structure), and a higher-priority task attempts to lock it, the higher-priority task will remain blocked until the low-priority task releases the lock.

Because every operating system experiences incidents of priority inversion, the issue is not the presence of priority inversion, but rather the duration of each cause of priority inversion. The duration of priority inversion is very important in assessing whether an operating system will be suitable for a particular application or system.

What Is Unbounded Priority Inversion?

In most operating systems, an application might actually suffer unbounded priority inversion. An operating system that has large unbounded priority inversions makes a real-time application highly susceptible to timing failures.
Unbounded priority inversion is defined as any situation in which the duration of priority inversion is dependent not only on the time required to handle a shared resource, but on the (unpredictable) actions of other unrelated tasks as well. For example, in 1997, NASA landed a rover robot (Pathfinder) on Mars. The landing was very successful, using a highly innovative “bouncing ball” structure that opened up after landing to release the robot. A few days later, however, the lander software determined erroneously that its internal integrity had been lost and initiated a system reset sequence, aborting that day’s operations. The system reset repeated the next day. After a heroic effort by the Jet Propulsion Laboratory (JPL) team that built the Pathfinder, the problem was traced to priority inversion. Detailed examination of the priority inversion on MARS Pathfinder and the resolution using priority inheritance can be found at http://research.microsoft.com/~mbj/Mars_Pathfinder/Mars_Pathfinder.html.
The Pathfinder lander software contained a number of tasks. This document considers three of those tasks:

periodically checks the health of the spacecraft systems and software.
processes image data.
performs an occasional test on equipment status.

In each period,

resets a “Go/Nogo” hardware timer. It is assumed that the integrity of the lander software has been compromised if this timer ever times out. The processor is halted, all devices are reset, the software is completely reloaded, the spacecraft systems are tested, and the system starts over. This process does not complete until the next day.

Attempts to Lock

–Normal Execution	– Execution in Critical Section	– Unbounded Priority Inversion

The problem is that

and

share a common data structure, locking a mutex to control access to it. Thus, if

locks the mutex, and

tries to lock it,

must wait. Every now and then, when

has the mutex locked, and

is waiting for it,

runs because it has a higher priority than

. Thus,

must wait for both

and

to complete and fails to reset the timer before it expires.
Even though nothing was actually wrong, the JPL team (which built the Pathfinder) traced the system failure to an unbounded priority inversion after extensive analysis. The team solved the problem by enabling priority inheritance for the mutex.
Unbounded priority inversion happens in many circumstances. It can happen in a preemptible operating system kernel unless it is explicitly avoided (most operating systems are not protected). Device drivers frequently lock critical structures or disable interrupts for long time periods, resulting in unbounded priority inversion. IP packets are placed in FIFO (first in, first out) queues without regard to priority, resulting in unbounded priority inversion. Applications lock data structures using semaphores or mutexes, resulting in unbounded priority inversion.

Avoiding Unbounded Priority Inversion

While there are many known academic solutions to the problem of avoiding unbounded priority inversion, in practice two techniques are commonly used: priority ceiling emulation and disabling interrupts.
The priority ceiling emulation technique raises the priority of any locking thread to its priority ceiling, defined as the highest priority of any thread that ever uses that lock. This requires the developer to supply a priority ceiling for each lock. In contrast, priority inheritance raises the priority of a thread only when holding a lock causes it to block a higher-priority thread. When a higher-priority thread is blocked, the low-priority thread inherits the priority of the higher-priority thread it is blocking.
Kernel threads can also disable interrupts or preemption. Disabling interrupts or preemption is an effective locking technique that avoids unbounded priority inversion, but is of limited general applicability since it stops all other processing. It’s like turning all traffic lights red in a city whenever any car wishes to cross any intersection. However, because of its low overhead, it can work well for locks that are never held for more than a few instructions.

Priority Inheritance vs. Priority Ceiling Emulation

Both the priority inheritance protocol and the priority ceiling emulation protocol^* have strengths and weaknesses. When used correctly, each are useful components of a real-time designer’s toolbox.
Priority inheritance can be difficult in a complex environment; implementers not familiar with its nuances can easily make mistakes. However, there is ample evidence that it can also be used to great advantage. For example, in one jitter test (where jitter refers to the maximum deviation in the measured period of a periodic task), using an RTOS running on a heavily loaded 1.4GHz Pentium processor, the worst-case jitter dropped from 76,492 microseconds to 52 microseconds simply by enabling priority inheritance in the kernel. 1.4GHz is somewhat faster than is usually available in embedded systems, but even at this speed the worst-case jitter is still just over 70 milliseconds. Priority inheritance illustrates that faster hardware doesn’t make an application behave more predictably, but avoiding priority inversion does.
Priority ceiling emulation has certain desirable properties — it has good worst-case priority inversion control, it handles nested locks well, and can avoid deadlock in some cases. Priority inheritance can occasionally lead to poorer worst-case behavior when there are nested locks, and does not avoid deadlocks. However, most performance-critical applications and real-time operating systems minimize their use of nested locks, and there are other mechanisms to avoid deadlock when nesting cannot be avoided.
On the other hand, priority inheritance can be implemented so that there is no penalty when the locks are not contended, which covers the vast majority of time-constrained systems. This, in addition to the fact that many extra context switches are avoided, and medium-priority tasks are not preempted unnecessarily, leads to excellent average performance. In practical systems, including both hard and soft real-time systems, average performance is as important as worst-case response. In contrast, priority ceiling emulation will pay the cost of changing a thread’s priority twice regardless of whether there is contention for the lock or not, resulting in higher overhead and many unnecessary context switches and blocking in unrelated tasks.
Priority ceiling emulation is an attractive choice when the set of threads contending for the lock is known, when there may be nested locks, and when worst-case behavior is the only concern. On the other hand, priority inheritance is very effective when a lock is seldom part of a nested set, and when average performance is relevant in addition to worst-case performance.
Another important aspect to understand is that optimal use of priority ceiling emulation requires static analysis of the system to find the priority ceiling of each lock. While static analysis is highly desirable (and even necessary) for many applications with critical response requirements, it may be neither desirable nor cost-effective in many other applications in which only a few parts of the system are critical. Also, formal real-time analysis is primarily applicable to systems that are constructed according to a set of precise rules with limited dynamic operations. For systems that cannot be readily analyzed, priority inheritance is likely to be a more effective mechanism since it does not require static analysis.

Design and Analysis of Real-Time Systems

Priority inheritance and priority ceiling emulation are both effective and powerful techniques to prevent uncontrolled priority inversion when locks are used to protect critical sections in a real-time system. Real-time software designers must make intelligent decisions to use the appropriate technique, depending on their system requirements.
The solution to priority inversion problems starts with a sound architecture and design that must consider the decomposition of the system into tasks and shared resources, and how they impact the system’s ability to meet its timing constraints. Many performance problems can be solved by developing an architecture that avoids unnecessary coupling between tasks through shared resources.
Operating systems for performance-critical applications should provide a complete set of tools to manage priority inversion.

Protecting Resources

The foregoing discussion has concentrated on synchronization without creating unbounded priority inversion; it started with the supposition that locking is required to protect resources. Actually, it is important to remember that in a time-constrained application the best locking is no locking at all. There are four simple rules that can be applied for protecting resources:

Use non-locking atomic operations (such as a flip-buffer) to avoid locking. This approach is clearly optimal with respect to priority inversion.
Hold a lock (when it must be acquired) for as short a time as possible.
Ensure that you understand the limitations of each of the various methods for managing priority inversion when locks are being used.
Ensure that an operating system provides all the tools for managing priority inversion so you have the flexibility to meet a wide range of performance requirements.

Within the operating system, there are several things that can be done to handle priority inversion:

Make the OS kernel fully preemptible. When it needs to lock resources, the kernel’s locking should use mutexes that use priority inheritance to bound blocking within the kernel.
Prevent device drivers from disabling interrupts.
Run interrupt handlers at appropriate priorities, instead of arbitrarily high priorities (as most operating systems do).
Prioritize network packet processing according to the socket owner’s priority.

These features minimize or entirely prevent priority inversion, removing a major source of development delays for time-constrained software.

*Often incorrectly referred to as priority ceiling protocol. The original priority ceiling protocol, described in the paper “Priority Inheritance protocols: An approach to real-time synchronization,” by Sha, Lohoczky, and Rajkumar, is expensive to implement and not available in any commercial operating system. PCP emulation (also known as the highest locker protocol) is a simplified version of the original PCP protocol and is part of various standards such as POSIX (POSIX_PRIO_PROTECT), the Real-Time Specification for Java (RTSJ), OSEK, and the Ada 95 real-time specification. Priority inheritance is also part of standards such as POSIX (POSIX_PRIO_INHERIT) and the RTSJ.

This is copied from Timsys

Enabling Login on a Serial Port

To enable login on a serial port other than the default one, add this line to the /etc/inittab file:

 T0:12345:respawn:/sbin/agetty -L /dev/ttyS1 115200 vt100

where ttyS1 is the port, and 115200 is the baud rate. You can either edit this file on the target, or on the host as:

<rfs_dir>/etc/inittab

The Linux Startup Process

The core of the Linux operating system is known as the kernel, which is loaded into memory when an embedded Linux system boots. The kernel automatically probes, identifies, and initializes as much of your system’s hardware as possible, and then looks for an initial filesystem that it can access in order to continue the boot process.
The first filesystem mounted by Linux systems during the boot process is known as a root filesystem, and is automatically mounted at the Linux root directory /. Once mounted, the root filesystem provides the Linux system with a basic directory structure that it can use to map devices to Linux device nodes, access those devices, and locate, load, and execute subsequent code such as system code or your custom applications.
All Linux systems start in essentially the same way. After loading the kernel into memory and executing it, Linux systems execute a system application known as init, which is typically found in /sbin/init on Linux systems. The init process is process ID (PID) 1 on the system. It reads the file /etc/inittab to identify the way in which the system should boot and lists all other processes and programs that it should start.

Initial RAM Disks

An initial RAM disk is a compressed filesystem image that is bundled with a kernel. (For more information, refer to Initial RAM Disks.) If your system uses an initial RAM disk (initrd or initramfs), the boot sequence includes one extra step. Instead of initially executing the init process, the system uncompresses and mounts the initial RAM disk, and then executes the file /linuxrc. This file can be a shell script that lists other commands to execute, can be a multi-call binary such as BusyBox, or can be a symbolic link to a multi-call binary or to /sbin/init.
Because init is a user program, located in a filesystem, the Linux kernel must find and mount the first (or root) filesystem in order to boot successfully. Ordinarily, available filesystems are listed in the file /etc/fstab so the mount program can find them. But /etc/fstab is itself a file, stored in a filesystem. Finding the very first filesystem is a chicken-and-egg problem, and to solve it the kernel developers created the kernel command-line option root=, which specifies on which device the root filesystem exists.

initrd

When root= was first implemented, it would be either a floppy drive or a partition on a hard drive. Today, the root filesystem can be on dozens of different types of hardware, or spread across several locations in a RAID. Its location can move around between reboots, as with hot-pluggable USB devices on a system with multiple USB ports. The root filesystem might be compressed, encrypted, or loopback-mounted. It could even be located on a network server, requiring the kernel to acquire a DHCP address, perform a DNS lookup, and log in to a remote server (with username and password), all before the kernel can find and run the first userspace program.

Note:
On desktop Linux systems that use the GRUB (Grand Unified Boot Loader) boot loader, the system’s initial RAM disk is usually stored as a separate file external to the kernel. This file is typically located in the /boot directory and is identified in the GRUB configuration file (/etc/grub.conf). On most embedded systems, the initial RAM disk is created as a file external to the kernel, but is bundled with the kernel as a final step in the kernel build process. If you are using GRUB, this device is identified via one of your boot arguments, the root= parameter.

As a result, root= does not provide enough information to the kernel. Even including a great deal of special-case behavior in the kernel does not help with device enumeration, encryption keys, or network logins that vary from system to system. RAM disks such as initrd help to solve this problem.
For 2.4 and earlier kernels, initrd is still the only way to provide a RAM-based root filesystem. Initrd is a RAM-based block device — a fixed-size chunk of memory that can be formatted and mounted like a disk. Therefore, the contents of the RAM disk have to be formatted and prepared with special tools, such as mke2fs and losetup. Additionally, like all block devices, the RAM disk requires a filesystem driver to interpret the data at run time, which imposes an artificial size limit that either wastes space or limits capacity.
RAM disks waste even more memory due to caching. Linux is designed to cache all files and directory entries read from or written to block devices, so Linux copies data to and from the RAM disk into the “page cache” (for file data), and the “dentry cache” (for directory entries).
Initrd was designed as front end to the root= device detection code, not a replacement for it. When you boot a Linux system that uses an initial RAM disk, the system uncompresses and mounts the initial RAM disk, and then executes the file /linuxrc (which must therefore be executable) before running init. The linuxrc file performs functions such as logging onto the network, telling the kernel which block device contains the root filesystem (using root= and pivot_root), and then returns control to the kernel. Finally, the kernel mounts the root filesystem and executes init. This process assumes that the root filesystem was on a block device rather than a network share, and also assumes that initrd isn’t itself going to be the root filesystem.
Because of these limitations with initrd, the 2.6 kernel developers chose to implement a new mechanism for finding the initial filesystem – initramfs.

initramfs

Initramfs is a mechanism in which the files in the Linux cache are mounted like a filesystem, and retained until they’re deleted or the system reboots. Linux 2.6 kernels bundle a small RAM-based initial root filesystem into the kernel, containing a program called init that the kernel runs as its first program. Finding another filesystem containing the root filesystem is now the job of this new init program.
These RAM-based filesystems automatically grow or shrink to fit the size of the data they contain. Adding files to a ramfs (or extending existing files) automatically allocates more memory, and deleting or truncating files frees that memory. Because there is no block device, there is no duplication between the block device and the cache – the copy in the cache is the only copy of the data. In addition, a system using initramfs as its root filesystem doesn’t need a filesystem driver built into the kernel, because there are no block devices to interpret as filesystems; there are simply files located in memory. Best of all, this isn’t new code, but a new application for the existing Linux caching code, which means it adds almost no size, is very simple, and is based on extremely well-tested infrastructure.
The contents of an initramfs do not have to be general-purpose. For example, if a given system’s root filesystem is located on an encrypted network block device, and the network address, login, and decryption key are all to be found on a given USB device that requires a password to access, the system’s initramfs can have a special-purpose program that knows all about this process, and makes it happen. Even better, systems that don’t need a large root filesystem do not need to switch to another root filesystem.
With initramfs, the kernel can return to following orders after it launches init. Because the init program is run with PID 1, the real root filesystem is initramfs until further notice, and the exec() system call can be used to pass that process ID to another program, if needed.