Kernel Debugging

Tools

man

RTFM (man is a fine manual)

bochs

x86 emulator. Can be built with either an internal debugger (assembly) or a gdb stub (source-level).

gdb

gdb debugger. Connect to bochs to facilitate source-level debugging. (Also works with many other emulators, virtual machines, and can run locally).

less

Many of the commands listed here output to the console. To scroll through the output and search for text, pipe it through less. You can find a string by typing /thetext, then press enter. Usage:

$ verbose-command | less

objdump

Outputs an object file (ELF). Example usage:

$ objdump -M i8086,att -d file_16bit $ objdump -M att -d file_32bit $ objdump -M att -D file_32bit

The -M att flag causes the output to be in AT&T assembly syntax. The first example disassembles a 16 bit program (-M i8086). The second line ommits that flag, so the disassemby has a default size of 32 bits. The -d flag generates the disassembly; -D includes the data segments of the program, too.

readelf

Output ELF file metadata. Useful when writing your createimage for Project 1.

Dump symbol addresses, linkage types, and names. Useful for setting break points in bochs or Qemu. Run this through awk to output symbols in a format that bochs can use...

hexdump

Dump a file in hex, decimal, octal, and ascii in various widths.

Dump a file in octal, decimal, hex, and ascii in various widths. Note: file offsets are in octal

Wisdom

Premature optimization is the root of all evil .
Taking care to write correct code takes much less time than than fixing incorrect code with a debugger.
- Use assertions
- Document, then implement
Test cases
Debugging is significantly more effective when you know exactly what your code is supposed to do.
When possible, test each function separately. Try to narrow down where a bug may be coming from.
Typically, a fault at a small address indicates that code dereferenced a null pointer to a structure. The address is the offset of the member accessed; this value is useful for tracking down the offending code.
Look at the call stack to navigate your code's execution. In gdb, use frame and stack.
You can often detect use-after-free bugs by poisoning freed data structures with a known bit pattern (e.g., 0xcafebabe, or 0xdeadbeef).
Breakpoints
Watches
printf

Debugging with Compiler Optimizations

-O2

Makefile

CFLAGS += -O2

Function inlining

Your callstack will omit the inlined function
Breakpoints at the function may not work

Storing variables in registers

print var-name won't work
Watching the variable won't work
You'll have to figure out what register it's stored in

Ommiting the frame pointer

Result: the %ebp register is not pushed onto the stack; the debugger won't be able to trace the call stack
Backtraces may not be accurate (press Ctrl+C if the debugger stops responding)
You may have to determine the call stack by hand
Use frame addr to specify the stack frame manually

Instruction reordering

Line numbers become mistmatched when stepping through code

Bochs vs Real Computers

Question: Why won't my code work on a real machine when it works perfectly in bochs?
Answer: Your code has bugs.

In general, testing your code on multiple computers is a good practice because each has a slightly different setup; there may be machine-specific settings which hide your bugs.

If you find that your code works in bochs, but not on other machines, consider these differences that are unique to bochs:

Bochs initializes memory to 0 on startup. Uninitialized data, if you expect it to be 0, will not always be so

The timer interrupt is triggered at exact intervals; it is deterministic (real machines have jitters)

Initial register values may differ (on bootup, %dl will, however, point to the boot device)

The boot device may differ

Other Peculiarities of Bochs

When debugging, the first breakpoint will always be before the BIOS has run.

Your bootloader will not be loaded yet
Your kernel will not be loaded until after the bootloader runs

If you set a breakpoint at the start of your kernel (b _start), it will not be hit.