LinuxKernelWorkshop
From Hackerspace Brussels
|
We will have a look into the kernel sources. Our guides: {Nico} and dragos.
Prerequisites
- Know your way around in C-code (tip: read the K&R C-handbook Ebook )
- Have git installed on yr laptop
- KVM installed & running (or plain QEMU if you can take the performance hit). VirtualBox should be ok, especially if your CPU doesn't support virtualization.
- At least 4G free disk space. Preferably more.
Interesting material
- Have a look at this fosdem2010 lecture: linux kernel patching 101
- Linux kernel walkthrough, a nice walk trough kernel sources: [1]
- Linux kernel boot process: [2]
- Kernel hacking page (more links there) from kernelnewbies: [3]
- Kernel networking wiki: [4]
- kernel crashcourse http://www.crashcourse.ca/introduction-linux-kernel-programming/introduction-linux-kernel-programming : ! only 5 lessons available without registration and fee !
- presentation summary in etherpad (slide placeholder) [5]
Things to start with
Some introductory material & exercises:
- get a vanilla kernel, configure, build and run it
- basic kernel api: printing, lists, mem allocation, locking/synchronization
- device driver stuff:
- registering a device driver
- types of device drivers
- basic debugging techniques (aka: kernel oops is your new friend)
- implementing operations on the device (open/close, read, write, ioctl, etc)
- interrupt handling (on virt. machine with emulated serial ports is easiest, but maybe we could try something fancier)
- generic or Linux specific OS concepts:
- processes
- interrupts
- deferring work: softirq, tasklet, timer, workqueue, kernel threads
- basic memory management stuff
At the end of this we should get a device driver that sends and receives data over serial port.
Add your topics of interest here
- The big picture, where to find the code of each subsystem
- How to submit a kernel patch
- Try to roll my own GPIO driver (for openwrt)
- GPIO for openwrt to be able to control the power button of a motherboard
- Fix bugs from mainline kernel bugzilla
- Networking stuff
- device driver for messing up with packets
- play with real serial interfaces - (cable built : [6] )
- play with specialised debugging/tracing toys: kgdb, ftrace, lltng, ksplice.
- ...
Setup used
We can try using these (increasing order of setup difficulty):
- virtual machine (HSB will have this covered)
- a real PC
- an Open Wrt router
Who?
we're +- full - i think more than 15 ppl will be difficult to host/mentor -- if you're interested anyway, add your name and send me email (pieter@L45.be) - i'll contact you and see if there's any space left.
- ptr_here
- zoobab
- askarel
- sandb
- Kris VDW
- helixblue
- U_go
- Kristof Provost (kristof@sigsegv.be)
- Chri
- tazo
- protoX29A
- juliane (juliane@bruxxel.org)
- Bert Vermeulen
- fs111
- Wouter VR
Comments (add your comments/concerns here)
- we can loopback a physical serial port [7]. For the VM, 2 virtual serial ports will be connected via a pipe.
- Pair programming is encouraged.
- We are working off http://piratepad.net/lkw and http://ietherpad.com/lkw
Notes (Saturday)
(taken with etherpad during the workshop)
accounts on vm's
start your vm (on 172.22.33.22 or 172.22.33.23)
-> use X forwarding to start qemu:
- ssh -X yourusername@172.22.33.22
- cd /mnt/data
- cp -r victim_kvm_orig/ ${USER}_kvm
- cd ${USER}_kvm/
USE LESS RAM ;-) -> -m 128
- kvm -m 128 -hda victim-i386.raw -hdb extra1.raw -serial stdio
=> icmp does not work, udp and tcp are ok ;-)
- account: hsbxl / hsbxl
copy things to yr v-machine
wget -r ftp://172.22.33.21/lkw/dummy
tip: start a http server on yr laptop python -m SimpleHTTPServer
Installation of practical stuff on the virtual machine (done already on 172.22.33.23 ) packages : screen, vim (to use -> keys, console-tools (for loadkeys you have to do as root sudo loadkeys azerty, mc )
vm details:
> sudo apt-get install kvm > apt-get install uml-utilities > cp qemu-ifup & qemu-ifdown to /etc/ #you need to do this! > cp *.raw files to somewhere on your disk >sudo kvm -m 512 -hda victim-i386.raw -hdb extra1.raw -serial stdio or > sudo kvm -m 512 -hda victim-i386.raw -hdb extra1.raw -serial stdio or > sudo qemu -m 512 -hda victim-i386.raw -hdb extra1.raw -serial stdio > boot the first kernel in grub > access vm on victim.local > user: hsbxl, pass: hsbxl (root has same passwd) > git checkout HEAD - position yourself to latest commit (not recommended) > apt-get install console-data > dpkg-reconfigure console-data
Reverse ssh:
On the vm:
ssh -R 2222:localhost:22 username@172.22.33.59
This lets you do
ssh -p 2222 localhost
on your laptop, and get an ssh session on the VM
program
- get kernel compiled by 12:30
- get kernel running by 13:00
- break 13:00 - 14:00
- finish decoding oops by 15:00
- do basic tx on serial by 16:00
- get tx/rx running by 19:00
get the sources
> git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git > extra1.raw has this tree, checkedout to version 2.6.35 in /mnt/src/linux-2.6/
- move to more stable branch: v2.6.35-rc6:
> git checkout v2.6.35-rc6
config the kernel
* make menuconfig/gconfig/xconfig * make oldconfig
- build the kernel:
* old school: make && make modules_install install --> slower, more knowledge
* distro specific: mkdebpkg, make deb-pkg, make rpm-pkg --> faster, easier
things you'll need compiled in your kernel to be able to boot: -ACPI -disk device -some filesystem
more info on how linux boots: linux documentation project : pocket linux guide
> after u have the .dep, scp it to the VM & dpkg -i mykernel.deb to install it
C99 & gcc extensions & stuff:
* types: uint16_t
* const, register, inline, volatile, asmlinkage, __aligned. __packed, __read_mostly, __user, __kernel, __deprecated
* macros : do { } while (0) or ({})
* offsetof, containerof
* typecheck
* OO programming in C
- useful tools to browse the code : cscope/ctags, lxr, http://lxr.linux.no
- OS generic stuff:
* processes & their view in userspace and kernelspace
** http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory
* memory
** TLB (translation of virtual to physical mem)
** lazy allocation
* process && interrupt context
* syscalls
* k & u space
- browsing the source tree
http://lxr.linux.no arch * configs are available, probably old
debugging:
* printk
* decoding oops
> OOPs codes, look in fault.c
int * p = 0x1234;
*p = 2;
> objdump, addr2line
# # get the offset from (if you're happy yr machine is still running) - -- otherwise try to get a clue from the oops msg
cat /proc/modules
# (mix source with asm if debug symbols are avail)
objdump --adjust-vma=0xe17f2000 -dS dummy.ko | vi -
> panic_on_oops switch
* kdb/kgdb? tracing? lltng?
- kernel api:
* printk
* kmalloc / kfree
* list_head
http://elf.cs.pub.ro/so2/wiki/laboratoare/lab03 (google translate is your friend)
* spinlock_t
for protecting variables between interrupt context / process context or in between process contexts
* struct semaphore
for instance 5 dining philosophers share 3 forks
* atomic_t
* rcu
(read copy update)
device drivers:
- char devices:
* types of device drivers * major, minor numbers * Makefiles * request_chrdev_region * skeleton for device driver * init: request_region, request_irq, cdev_init, cdev_add * uninit: cdev_del, free_irq, release_region * in/out * interrupts * softirq, timers, workqueue, k threads * serial port (specs below): http://www.beyondlogic.org/serial/serial.htm
#rmmod existing serial devices rmmod 8250_pnp rmmod 8250 rmmod serial
#create an inode, which we'll hook our module onto
mknod /dev/test_uart c 42 1
# UART_MAJOR = 42
# UART_MINOR = 1
Linux kernel device driver book is very good resource q
extra topics:
* networking:
> good & short explanation with nice image: http://www.linuxfoundation.org/collaborate/workgroups/networking/kernel_flow
* netfilter
> skb struct: http://pastebin.com/nB8n8EBk
> ip,tcp,udp headers & their usage: http://pastebin.com/1dwBsBfV
> netfilter hook example: http://pastebin.com/YjSvpYBH
> hooking spots: 
* prepare a patch for upstream kernel
- hack time! - some extras: network hook drivers, mem mapping, mem management
cscope command in the linux kernel tree:
cscope -Rqk
after doing it once do
cscope -d
alt: make cscope
BUG: unable to handle kernel paging request at 00001234
IP: [<c89c3016>] m_init+0x6/0x20 [oops] **************************
*pde = 00000000 * Decode this! *
Oops: 0000 [#1] PREEMPT DEBUG_PAGEALLOC * Linux Kernel Workshop*
last sysfs file: /sys/devices/virtual/net/lo/operstate * 28-29 August *
Modules linked in: oops(+) netconsole pcnet32 crc32 ide_cd_mod cdrom **************************
..ooo......................
Pid: 2754, comm: insmod Not tainted (2.6.28.4 #2) ...ooo............ ++..
EIP: 0060:[<c89c3016>] EFLAGS: 00010292 CPU: 0 ....oo........... +..
EIP is at my_oops_init+0x6/0x20 [oops] .....oo......... ++.
EAX: 00000000 EBX: fffffffc ECX: c89c3380 EDX: 00000001 ......oo....... ++++ +++.
ESI: c89c3010 EDI: 00000000 EBP: c57cbe24 ESP: c57cbe1c .......oo.......---- +++ - +.
DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068 .........oo..... ++++ +++.
Process insmod (pid: 2754, ti=c57cb000 task=c66ec780 task.ti=c57cb000) ...........o... + .
Stack: .............o. +++ +
c57cbe34 00000282 c57cbf8c c010102d c57b9280 0000000c c57cbe58 c01708e4 ................ ++.
00000124 00000000 c89c3380 c57cbe58 c5db1d38 00000001 c89c3380 c57cbe60 ................. || |.
c0170981 c57cbf8c c014b698 00000000 00000000 c57cbf78 c57cbf20 00000580 ................. | ||.
Call Trace: ................. ++.
[<c010102d>] ? _stext+0x2d/0x170 ............ .. ++.
[<c01708e4>] ? __vunmap+0xa4/0xf0 ........ .. .. . +++.
[<c0170981>] ? vfree+0x21/0x30 ..... + . .. +++.
[<c014b698>] ? load_module+0x19b8/0x1a40 .... + .. ++.
[<c035d083>] ? printk+0x0/0x1a + . +.
[<c035e965>] ? __mutex_unlock_slowpath+0xd5/0x140 + +.
[<c0140da6>] ? trace_hardirqs_on_caller+0x106/0x150 + HSBXL
[<c014b7aa>] ? sys_init_module+0x8a/0x1b0 http://hackerspace.be
[<c0140da6>] ? trace_hardirqs_on_caller+0x106/0x150 irc.freenode.net #hsbxl
[<c0240a08>] ? trace_hardirqs_on_thunk+0xc/0x10
[<c0103407>] ? sysenter_do_call+0x12/0x43
Code: <a1> 34 12 00 00 c7 04 24 54 30 9c c8 89 44 24 04 e8 58 a0 99 f7 31
EIP: [<c89c3016>] m_init +0x6/0x20 [oops] SS:ESP 0068:c57cbe1c
---[ end trace 45eeb3d6ea8ff1ed ]---
