Debugging

Debugging Tools and Strategies

Debugging ordinary programs is hard enough; kernel debugging is harder still. The following few paragraphs describes some common debugging tools and strategies that you may find useful.

Tools

`nm`	Dump symbol address, linkage types, and names. Useful for setting break points in Bochs or Qemu.
`readelf`	Dump ELF file metadata
`objcopy`	Translate object file formats
`objdump`	Dump object file metadata and disassembly
`addr2line`	Map pc to source line
gdb	ptype	Dump type information
	p sizeof	Type size
	p &((Foo*)0)->x	Member offset
	disass 0xaddr	Disassemble code
genassym	Export constant C expression to assembly to avoid having to maintain them in two places at once.
printf	Printf and primitive CGA console driver for the x86; includes "long long" arithmetic support.

Hints

Typically, a fault at a small address indicates that code derefenced a null pointer to a structure. The address is the offset of the member accessed; this value is useful for tracking down the offending code.
Printf school of debugging: when in doubt, print. Together with an undefined opcode such as ud2 on the x86 to cause traps, print statements can be very effective.
When possible, test each function separately. It is often possible to test your code in the context of a user-level process and, needless to say, it is also much easier to debug it there. Generic code such as malloc is always a good candidate for this strategy.
If you avoid cranking the compiler's optimizer, it is usually a simple matter to write a function to walk the stack and produce stack traces for trap handlers and debugging code. Beware the omitted frame pointer, however. With the optimizer on, you may need a disassembler to interpret function prologues.
Premature optimization is the root of all evil.
One can often detect use-after-free bugs by poisioning freed data structures with a known bit pattern (e.g., 0xcafebabe, 0xba5eba11).

Other tricks of the trade:

Print machine code in a fault handler or debugging statement and copy that code in to an array in a dummy C program to disassemble it with gdb. The C code looks something like this:

	unsigned char code[] = { 0x90, 0x90, 0x90 };

Then in gdb disassemble code.

Add trap handlers for undefined opcodes and then drop ud2 instructions at various points. The trap handler can print out the faulting address, providing a cheap debugging print in places that printf cannot safely be called.