Writing a Linux Kernel Module — Part 1: Introduction
Introduction
In
this series of articles I describe how you can write a Linux kernel
module for an embedded Linux device. I begin with a straightforward
“Hello World!” loadable kernel module (LKM) and work towards developing a
module that can control GPIOs on an embedded Linux device (such as the
BeagleBone) through the use of IRQs. I will add further follow-up
articles as I identify suitable applications.
This
is a complex topic that will take time to work through. Therefore, I
have broken the discussion up over a number of articles, each providing a
practical example and outcome. There are entire books written on this
topic, so it will be difficult to cover absolutely every aspect. There
are also other articles available on writing kernel modules; however,
the examples presented here are built and tested under the Linux kernel
3.8.X+, ensuring that the material is up to date and relevant, and I
have focused on interfacing to hardware on embedded systems. I have also
aligned the tasks performed against my book, Exploring BeagleBone, albeit the articles are self-contained and do not require that you own a copy of the book.
Figure 1: GPIO performance in kernel space
This
article is focused on the system configuration, tools and code required
to build and deploy a “Hello World!” kernel module. The second article
in this series examines the topic of writing character device drivers
and how to write C/C++ programs in user space that can communicate with
kernel space modules. The third article examines the use of the kernel
space GPIO library code — it combines the content of the first two
articles to develop interrupt-driven code that can be controlled from
Linux user space. For example, Figure 1 illustrates an oscilloscope
capture of an interrupt-driven kernel module that triggers an LED to
light when a button is pressed (click for a larger version). Under
regular embedded Linux (i.e., not a real-time variant), this code
demonstrates a response time of approximately 20 microseconds (±5μs),
with negligible CPU overhead.
What is a Kernel Module?
A
loadable kernel module (LKM) is a mechanism for adding code to, or
removing code from, the Linux kernel at run time. They are ideal for
device drivers, enabling the kernel to communicate with the hardware
without it having to know how the hardware works. The alternative to
LKMs would be to build the code for each and every driver into the Linux
kernel.
Figure2: Linux user space and kernel space without this modular capability, the Linux kernel would be very large, as it would have to support every driver that would ever be needed on the BBB. You would also have to rebuild the kernel every time you wanted to add new hardware or update a device driver. The downside of LKMs is that driver files have to be maintained for each device. LKMs are loaded at run time, but they do not execute in user space — they are essentially part of the kernel.
Figure2: Linux user space and kernel space without this modular capability, the Linux kernel would be very large, as it would have to support every driver that would ever be needed on the BBB. You would also have to rebuild the kernel every time you wanted to add new hardware or update a device driver. The downside of LKMs is that driver files have to be maintained for each device. LKMs are loaded at run time, but they do not execute in user space — they are essentially part of the kernel.
Kernel modules
run in kernel space and applications run in user space, as illustrated
in Figure 2. Both kernel space and user space have their own unique
memory address spaces that do not overlap. This approach ensures that
applications running in user space have a consistent view of the
hardware, regardless of the hardware platform. The kernel services are
then made available to the user space in a controlled way through the
use of system calls. The kernel also prevents individual user-space
applications from conflicting with each other or from accessing
restricted resources through the use of protection levels (e.g.,
superuser versus regular user permissions).
Why Write a Kernel Module?
When interfacing to electronics circuits under embedded Linux you are exposed to sysfs
and the use of low-level file operations for interfacing to electronics
circuits. This approach can appear to be inefficient (especially if you
have experience of traditional embedded systems); however, these file
entries are memory mapped and the performance is sufficient for many
applications. I have demonstrated in my book that it is possible to
achieve response times of about one third of a millisecond, with
negligible CPU overhead, from within Linux user space by using pthreads,
callback functions and sys/poll.h.
An
alternative approach is to use kernel code, which has support for
interrupts. However, kernel code is difficult to write and debug. My
advice is that you should always to try to accomplish your task in Linux
user space, unless you are certain that there is no other possible way!
Get Source Code
Source Code for this Discussion
All
of the code for this discussion is available in the GitHub repository
for the book Exploring BeagleBone. The code can be viewed publicly at:
the ExploringBB GitHub Kernel Project directory, and/or you can clone the repository on your BeagleBone (or other Linux device) by typing:
The /extras/kernel/hello
directory is the most important resource for this article. The
auto-generated Doxygen documentation for these code examples is
available in HTML format and PDF format.
Prepare the System for Building LKMs
The
system must be prepared to build kernel code, and to do this you must
have the Linux headers installed on your device. On a typical Linux
desktop machine you can use your package manager to locate the correct
package to install. For example, under 64-bit Debian you can use:
molloyd@DebianJessieVM:~$ sudo apt-get update
molloyd@DebianJessieVM:~$ apt-cache search linux-headers-$(uname -r)
linux-headers-3.16.0-4-amd64 - Header files for Linux 3.16.0-4-amd64
molloyd@DebianJessieVM:~$ sudo apt-get install linux-headers-3.16.0-4-amd64
molloyd@DebianJessieVM:~$ cd /usr/src/linux-headers-3.16.0-4-amd64/
molloyd@DebianJessieVM:/usr/src/linux-headers-3.16.0-4-amd64$ ls
arch include Makefile Module.symvers scripts
You
can complete the first two articles in this series using any flavor of
desktop Linux. However, in this series of articles I build the LKM on
the BeagleBone itself, which simplifies the process when compared to
cross-compiling. You must install the headers for the exact version of your kernel build. Similar to the desktop installation, use uname to identify the correct installation. For example:
molloyd@beaglebone:~$ uname -a
Linux beaglebone 3.8.13-bone70 #1 SMP Fri Jan 23 02:15:42 UTC 2015 armv7l GNU/Linux
You can download the Linux headers for the BeagleBone platform from Robert Nelson’s website. For example, at: http://rcn-ee.net/deb/precise-armhf/. Choose the exact kernel build, and download and install those Linux-headers on your BeagleBone. For example:
You can then check that the headers have installed correctly: molloyd@beaglebone:~/tmp$ wget http://rcn-ee.net/deb/precise-armhf/v3.8.13-bone70/linux-headers-3.8.13-bo
ne70_1precise_armhf.deb
100%[===========================>] 8,451,080 2.52M/s in 3.2s
2015-03-17 22:35:45 (2.52 MB/s) - 'linux-headers-3.8.13-bone70_1precise_armhf.deb' saved [8451080/8451080]
molloyd@beaglebone:~/tmp$ sudo dpkg -i ./linux-headers-3.8.13-bone70_1precise_armhf.deb
Selecting previously unselected package linux-headers-3.8.13-bone70
molloyd@beaglebone:~/tmp$ cd /usr/src/linux-headers-3.8.13-bone70/
molloyd@beaglebone:/usr/src/linux-headers-3.8.13-bone70$ ls
Documentation Module.symvers crypto fs ipc mm scripts tools
Kconfig arch drivers include kernel net security usr
Makefile block firmware init lib samples sound virt
Under
the 3.8.13-bone47 Debian distribution for the BeagleBone, you may have
to perform an unusual step of creating an empty file timex.h (i.e., touch timex.h) in the directory /usr/src/linux-headers-3.8.13-bone47/arch/arm/
include/mach. This step is not necessary under the bone70 build.
include/mach. This step is not necessary under the bone70 build.
A Warning!
It
is very easy to crash the system when you are writing and testing LKMs.
It is always possible that such a system crash could corrupt your file
system — it is unlikely, but it is possible. Please back up your data
and/or use an embedded system, such as the BeagleBone, which can easily
be re-flashed. Performing a sudo reboot, or pressing
the reset button on the BeagleBone will usually put everything back in
order. No BeagleBones were corrupted in the writing of these articles
despite many, many system crashes!
The Module Code
The run-time life cycle of a typical computer program is reasonably
straightforward. A loader allocates memory for the program, then loads
the program and any required shared libraries. Instruction execution
begins at some entry point (typically the
main()
point in
C/C++ programs), statements are executed, exceptions are thrown, dynamic
memory is allocated and deallocated, and the program eventually runs to
completion. On program exit, the operating system identifies any memory
leaks and frees lost memory to the pool.
A kernel module is not an application — for a start there is no
main()
function! Some of the key differences are that kernel modules:- do not execute sequentially— a kernel module registers itself to handle requests using its initialization function, which runs and then terminates. The type of requests that it can handle are defined within the module code. This is quite similar to the event-driven programming model that is commonly utilized in graphical-user interface (GUI) applications.
- do not have automatic cleanup — any resources that are allocated to the module must be manually released when the module is unloaded, or they may be unavailable until a system reboots.
- do not have
printf()
functions — kernel code cannot access libraries of code that is written for the Linux user space. The kernel module lives and runs in kernel space, which has its own memory address space. The interface between kernel space and user space is clearly defined and controlled. We do however have aprintk()
function that can output information, which can be viewed from within user space. - can be interrupted — one conceptually difficult aspect of kernel modules is that they can be used by several different programs/processes at the same time. We have to carefully construct our modules so that they have a consistent and valid behavior when they are interrupted. The BeagleBone has a single-core processor (for the moment) but we still have to consider the impact of multiple processes accessing the module simultaneously.
- have a higher level of execution privilege — typically, more CPU cycles are allocated to kernel modules than to user-space programs. This sounds like an advantage, however, you have to be very careful that your module does not adversely affect the overall performance of your system.
- do not have floating-point support — it is kernel code that uses traps to transition from integer to floating-point mode for your user space applications. However, it is very difficult to perform these traps in kernel space. The alternative is to manually save and restore floating point operations — a task that is best avoided and left to your user-space code.
The concepts
above are a lot to digest and it is important that they are all
addressed, but not all in the first article! Listing 1 provides the code
for a first example LKM. When no kernel argument is provided, the code
uses the
printk()
function to display “Hello world!…” in
the kernel logs. If the argument “Derek” is provided, then the logs will
display “Hello Derek!…” The comments in Listing 1, which are written
using a Doxygen style, describe the role of each statement. Further
description is available after the code listing below.
In addition to the points described by the comments in Listing 1, there are some additional points:
- Line 16: The statement
MODULE_LICENSE("GPL")
provides information (via modinfo) about the licensing terms of the module that you have developed, thus allowing users of your LKM to ensure that they are using free software. Since the kernel is released under the GPL, your license choice impacts upon the way that the kernel treats your module. You can choose"Proprietary"
for non-GPL code, but the kernel will be marked as “tainted” and a warning will appear. There are non-tainted alternatives to GPL, such as"GPL v2"
,"GPL and additional rights"
,"Dual BSD/GPL"
,"Dual MIT/GPL"
, and"Dual MPL/GPL"
. See linux/module.h for more information. - Line 21: The
name
(ptr to char) is declared asstatic
and is initialized to contain the string “hello”. You should avoid using global variables in kernel modules — it is even more important than in application programming, as global variables are shared kernel wide. You should use thestatic
keyword to restrict a variable’s scope to within the module. If you must use a global variable, add a prefix that is unique to the module that you are writing. - Line 22: The
module_param(name, type, permissions)
macro has three parameters:name
(the parameter name displayed to the user and the variable name in the module),type
(the type of the parameter — i.e., one ofbyte
,int
,uint
,long
,ulong
,short
,ushort
,bool
, an inverse Booleaninvbool
, or a char pointercharp
), andpermissions
(this is the access permissions to the the parameter when using sysfs and is covered below. A value of0
disables the entry, butS_IRUGO
allows read access for user/group/others — See the Mode Bits for Access Permissions Guide) - Line 31 and 40: The functions can have whatever names you like (e.g.,
helloBBB_init()
andhelloBBB_exit()
), however, the same names must be passed to the special macrosmodule_init()
andmodule_exit()
on lines 48 and 49. - Line 31: The
printk()
is very similar in usage to theprintf()
function that you should be familiar with, and you can call it from anywhere within the kernel module code. The only significant difference is that you should specify a log level when you call the function. The log levels are defined in linux/kern_levels.h as one ofKERN_EMERG
,KERN_ALERT
,KERN_CRIT
,KERN_ERR
,KERN_WARNING
,KERN_NOTICE
,KERN_INFO
,KERN_DEBUG
, andKERN_DEFAULT
. This header is included via thelinux/kernel.h
header file, which includes it vialinux/printk.h
.
Essentially, when this module is loaded the
helloBBB_init()
function will execute, and when the module is unloaded the helloBBB_exit()
function will execute.
The next step is to build this code into a kernel module.
Building the Module Code
A Makefile is required to build the kernel module — in fact, it is a special kbuild Makefile. The kbuild Makefile required to build the kernel module in this article can be viewed in Listing 2.
The first line of this Makefile is called a goal definition and it defines the module to be built (hello.o). The syntax is surprisingly intricate, for example
obj-m
defines a loadable module goal, whereas obj-y
indicates a built-in object goal. The syntax becomes more complex when a
module is to be built from multiple objects, but this is sufficient to
build this example LKM.
The reminder of the Makefile is similar to a regular Makefile. The
$(shell uname -r)
is a useful call to return the current kernel build version — this ensures a degree of portability for the Makefile. The -C
option switches the directory to the kernel directory before performing any make tasks. The M=$(PWD)
variable assignment tells the make command where the actual project files exist. The modules
target is the default target for external kernel modules. An alternative target is modules_install
which would install the module (the make command would have to be executed with superuser permissions and the module installation path is required).
All
going well, the process to build the kernel module should be
straightforward, provided that you have installed the Linux headers as
described earlier. The steps are as follows:
You can see that there is now a hello loadable kernel module in the build directory with the file extension .ko.
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ ls -l
total 8
-rw-r--r-- 1 molloyd molloyd 154 Mar 17 17:47 Makefile
-rw-r--r-- 1 molloyd molloyd 2288 Apr 4 23:26 hello.c
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ make
make -C /lib/modules/3.8.13-bone70/build/ M=/home/molloyd/exploringBB/extras/kernel/hello modules
make[1]: Entering directory '/usr/src/linux-headers-3.8.13-bone70'
CC [M] /home/molloyd/exploringBB/extras/kernel/hello/hello.o
Building modules, stage 2.
MODPOST 1 modules
CC /home/molloyd/exploringBB/extras/kernel/hello/hello.mod.o
LD [M] /home/molloyd/exploringBB/extras/kernel/hello/hello.ko
make[1]: Leaving directory '/usr/src/linux-headers-3.8.13-bone70'
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ ls
Makefile Module.symvers hello.c hello.ko hello.mod.c hello.mod.o hello.o modules.order
You can see that there is now a hello loadable kernel module in the build directory with the file extension .ko.
Testing the LKM
This module can now be loaded using the kernel module tools as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ ls -l *.ko
-rw-r--r-- 1 molloyd molloyd 4219 Apr 4 23:27 hello.ko
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ sudo insmod hello.ko
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ lsmod
Module Size Used by
hello 972 0
g_multi 50407 2
libcomposite 15028 1 g_multi
omap_rng 4062 0
mt7601Usta 639170 0
You can get information about the module using the modinfo command, which will identify the description, author and any module parameters that are defined:
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ modinfo hello.ko
filename: /home/molloyd/exploringBB/extras/kernel/hello/hello.ko
description: A simple Linux driver for the BBB.
author: Derek Molloy
license: GPL
srcversion: 9E3F5ECAB0272E3314BEF96
depends:
vermagic: 3.8.13-bone70 SMP mod_unload modversions ARMv7 thumb2 p2v8
parm: name:The name to display in /var/log/kernel.log. (charp)
The module can be unloaded using the rmmod command:
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ sudo rmmod hello.ko
You can repeat these steps and view the output in the kernel log that results from the use of the
printk()
function. I recommend that you use a second terminal window and view the output as your LKM is loaded and unloaded, as follows: molloyd@beaglebone:~$ sudo su -
[sudo] password for molloyd:
root@beaglebone:~# cd /var/log
root@beaglebone:/var/log# tail -f kern.log
...
Apr 4 23:34:32 beaglebone kernel: [21613.495523] EBB: Hello world from the BBB LKM!
Apr 4 23:35:17 beaglebone kernel: [21658.306647] EBB: Goodbye world from the BBB LKM!
^C
root@beaglebone:/var/log#
Testing the LKM Custom Parameter
The
code in Listing 1 also contains a custom parameter, which allows an
argument to be passed to the kernel module on initialization. This
feature can be tested as follows:
If you view /var/log/kern.log at this point then you will see “Hello Derek” in place of “Hello world”. However, it is worth having a look at /proc and /sys first.
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ sudo insmod hello.ko name=Derek
If you view /var/log/kern.log at this point then you will see “Hello Derek” in place of “Hello world”. However, it is worth having a look at /proc and /sys first.
Rather than using the lsmod command, you can also find out information about the kernel module that is loaded, as follows:
This is the same information that is provided by the lsmod command but it also provides the current kernel memory offset for the loaded module, which is useful for debugging.
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ cd /proc
molloyd@beaglebone:/proc$ cat modules|grep hello
hello 972 0 - Live 0xbf903000 (O)
This is the same information that is provided by the lsmod command but it also provides the current kernel memory offset for the loaded module, which is useful for debugging.
The LKM also has an entry under /sys/module, which provides you with direct access to the custom parameter state. For example:
The version value is 0.1 as per the
molloyd@beaglebone:/proc$ cd /sys/module
molloyd@beaglebone:/sys/module$ ls -l|grep hello
drwxr-xr-x 6 root root 0 Apr 5 00:02 hello
molloyd@beaglebone:/sys/module$ cd hello
molloyd@beaglebone:/sys/module/hello$ ls -l
total 0
-r--r--r-- 1 root root 4096 Apr 5 00:03 coresize
drwxr-xr-x 2 root root 0 Apr 5 00:03 holders
-r--r--r-- 1 root root 4096 Apr 5 00:03 initsize
-r--r--r-- 1 root root 4096 Apr 5 00:03 initstate
drwxr-xr-x 2 root root 0 Apr 5 00:03 notes
drwxr-xr-x 2 root root 0 Apr 5 00:03 parameters
-r--r--r-- 1 root root 4096 Apr 5 00:03 refcnt
drwxr-xr-x 2 root root 0 Apr 5 00:03 sections
-r--r--r-- 1 root root 4096 Apr 5 00:03 srcversion
-r--r--r-- 1 root root 4096 Apr 5 00:03 taint
--w------- 1 root root 4096 Apr 5 00:02 uevent
-r--r--r-- 1 root root 4096 Apr 5 00:02 version
molloyd@beaglebone:/sys/module/hello$ cat version
0.1
molloyd@beaglebone:/sys/module/hello$ cat taint
O
The version value is 0.1 as per the
MODULE_VERSION("0.1")
entry and the taint value is 0 as per the license that has been chosen, which is MODULE_LICENSE("GPL")
.
The custom parameter can be viewed as follows:
You can see that the state of the
molloyd@beaglebone:/sys/module/hello$ cd parameters/
molloyd@beaglebone:/sys/module/hello/parameters$ ls -l
total 0
-r--r--r-- 1 root root 4096 Apr 5 00:03 name
molloyd@beaglebone:/sys/module/hello/parameters$ cat name
Derek
You can see that the state of the
name
variable is displayed, and that superuser permissions where not required to read the value. The latter is due to the S_IRUGO
argument that was used in defining the module parameter. It is possible
to configure this value for write access but your module code will need
to detect such a state change and act accordingly. Finally, you can
remove the module and observe the output:molloyd@beaglebone:/sys/module/hello/parameters$ sudo rmmod hello.ko
As expected, this will result in the output message in the kernel logs:
root@beaglebone:/var/log# tail -f kern.log
…
Apr 5 00:02:20 beaglebone kernel: [23281.070193] EBB: Hello Derek from the BBB LKM!
Apr 5 00:08:18 beaglebone kernel: [23639.160009] EBB: Goodbye Derek from the BBB LKM!
Conclusions
Click for the HTML and PDF version of the auto-generated Doxygen code documentation
Hopefully
you have built your first loadable kernel module (LKM). Despite the
simplicity of the functionality of this module there was a lot of
material to cover — by the end of this article: you should have a broad
idea of how loadable kernel modules work; you should have your system
configured to build, load and unload such modules; and, you should be
able to define custom parameters for your LKMs.
The
next step is to build on this work to develop a kernel space LKM that
can communicate with a user space C/C++ program by developing a basic
character driver. See “Writing a Linux Kernel Module — Part 2: A Character Device“. Then we can move on to the more interesting task of interacting with GPIOs.
No comments:
Post a Comment