Monthly Archives: October 2011

Writing your own bootloader for a toy operating system (2)

This article is part of a short series. Find part 1 here, or go straight through to part 3.

Now that we know the structure of the boot parameter block (BPB) and extended boot parameter block (EBPB), we can start writing our first code. (If you need a refresher, have a look at part 1 of this article).

First code in GNU assembler

We’ll be using the GNU assembler, since it’s free, comes with a boatload of options, supports AT&T and Intel assembly syntax and plays nice with gcc and ld later on. Some of the preprocessor directives used may need some explanation, but all code will be in straightforward Intel syntax.

Here’s some boilerplate code to get started:

.code16
.intel_syntax noprefix
.text
.org 0x0
 
LOAD_SEGMENT = 0x1000
 
.global main
main:
  jmp short start
  nop
 
  # BPB and EBPB here
 
start:
  # rest of code

The pile of preprocessor instructions at the top tell the assembler to assemble code for real mode. Since all (intel-based) computers start up in real mode with 16-bit instructions, we won’t be able to write 32-bit code here yet. We also instruct GNU assembler that we’ll be using Intel syntax (e.g. mov ax, 1 instead of movw $1, %ax – some prefer the latter, but most readers of this text will be familiar with Intel).  The origin of our code will be 0×0, i.e. all absolute addresses start at 0×0, which will be convenient.

Then there’s the main entry point of our code, which corresponds to the first byte of actual output when assembled. The code under “main” simply jumps over the BPB and EBPB located at offset 0×3, resuming execution at the label start. We’ll flesh out the BPB/EBPB in a bit, since it’ll have to have a very exact size.

We’ve also defined a constant LOAD_SEGMENT, which is the segment where we’ll be loading our second stage boot loader (more about that later).

The Boot Parameter Block

The structure of the boot parameter block can be coded like this:

bootsector:
 iOEM:          .ascii "DevOS   "    # OEM String
 iSectSize:     .word  0x200         # bytes per sector
 iClustSize:    .byte  1             # sectors per cluster
 iResSect:      .word  1             # #of reserved sectors
 iFatCnt:       .byte  2             # #of FAT copies
 iRootSize:     .word  224           # size of root directory
 iTotalSect:    .word  2880          # total # of sectors if over 32 MB
 iMedia:        .byte  0xF0          # media Descriptor
 iFatSize:      .word  9             # size of each FAT
 iTrackSect:    .word  9             # sectors per track
 iHeadCnt:      .word  2             # number of read-write heads
 iHiddenSect:   .int   0             # number of hidden sectors
 iSect32:       .int   0             # # sectors for over 32 MB
 iBootDrive:    .byte  0             # holds drive that the boot sector came from
 iReserved:     .byte  0             # reserved, empty
 iBootSign:     .byte  0x29          # extended boot sector signature
 iVolID:        .ascii "seri"        # disk serial
 acVolumeLabel: .ascii "MYVOLUME   " # volume label
 acFSType:      .ascii "FAT16   "    # file system type

The fields in this structure correspond to the specification in part 1 of this text, and since they’re nicely labelled, we’ll be able to refer to them later on.

Real-mode Segments

After the start label, we can write some actual code. Let’s start by defining our real mode data segments:

  cli
  mov  iBootDrive, dl  # save what drive we booted from (should be 0x0)
  mov  ax, cs          # CS = 0x0, since that's where boot sector is (0x07c00)
  mov  ds, ax          # DS = CS = 0x0
  mov  es, ax          # ES = CS = 0x0
  mov  ss, ax          # SS = CS = 0x0
  mov  sp, 0x7C00      # Stack grows down from offset 0x7C00 toward 0x0000.
  sti

Here, we mask interrupts so that interrupt calls don’t mess up our sector declarations. We set ES = DS = SS = CS = 0×0, and make the stack grow down from 0x7C00 (our boot loader was loaded at 0x7C00). When done, we turn the interrupts back on. It’s important to note that the BIOS places the number of the boot drive in the DL register. We store it in our BPB for later use.

Resetting the disk system

Next, we need to prepare the floppy drive for use. This is done through BIOS interrupt 0×13, subfunction 0. We call it with the boot drive in DL:

  mov  dl, iBootDrive   # drive to reset
  xor  ax, ax           # subfunction 0
  int  0x13             # call interrupt 13h
  jc   bootFailure      # display error message if carry set (error)

If the reset fails, the carry flag will be set and we jump to a label where we handle a boot failure by showing a message, waiting for a keypress and rebooting. Come to think of it, we’ll need a way to print a string to the screen.

Printing a string

We’ll add a short function that uses BIOS interrupt 0×10, sub-function 9 to print characters to the screen. The calling code must point DS:SI to the null-terminated string to be printed.

.func WriteString
 WriteString:
  lodsb                   # load byte at ds:si into al (advancing si)
  or     al, al           # test if character is 0 (end)
  jz     WriteString_done # jump to end if 0.
 
  mov    ah, 0xe          # Subfunction 0xe of int 10h (video teletype output).
  mov    bx, 9            # Set bh (page nr) to 0, and bl (attribute) to white (9).
  int    0x10             # call BIOS interrupt.
 
  jmp    WriteString      # Repeat for next character.
 
 WriteString_done:
  retw
.endfunc

We can now define the “bootFailure” label:

diskerror: .asciz "Disk error. "
bootFailure:
  lea si, diskerror
  call WriteString
  call Reboot

Great. We’ve got code to reset the floppy drive, and if it fails, there’s code that prints failure strings and reboots. Although, we still have to write a Reboot function.

Rebooting

Here is some code that prints a “Press any key to reboot” message, waits for a keystroke, and reboots the machine.

rebootmsg: .asciz "Press any key to reboot\r\n"
.func Reboot
 Reboot:
  lea    si, rebootmsg    # Load address of reboot message into si
  call   WriteString      # print the string
  xor    ax, ax           # subfuction 0
  int    0x16             # call bios to wait for key
 
  .byte  0xEA             # machine language to jump to FFFF:0000 (reboot)
  .word  0x0000
  .word  0xFFFF
.endfunc

Here, we use BIOS interrupt 0×16, sub-function 0 to read a key (any key). We then add a far jump to 0xffff:0000 we causes the machine to reboot.

Putting it all together

When we combine the functions above into a single source file, we end up with this:

.code16
.intel_syntax noprefix
.text
.org 0x0                                        
 
LOAD_SEGMENT = 0x1000             ; load the boot loader to segment 1000h
 
.global main
 
main:
  jmp short start                 # jump to beginning of code
  nop
 
bootsector:
 iOEM:          .ascii "DevOS   "    # OEM String
 iSectSize:     .word  0x200         # bytes per sector
 iClustSize:    .byte  1             # sectors per cluster
 iResSect:      .word  1             # #of reserved sectors
 iFatCnt:       .byte  2             # #of FAT copies
 iRootSize:     .word  224           # size of root directory
 iTotalSect:    .word  2880          # total # of sectors if over 32 MB
 iMedia:        .byte  0xF0          # media Descriptor
 iFatSize:      .word  9             # size of each FAT
 iTrackSect:    .word  9             # sectors per track
 iHeadCnt:      .word  2             # number of read-write heads
 iHiddenSect:   .int   0             # number of hidden sectors
 iSect32:       .int   0             # # sectors for over 32 MB
 iBootDrive:    .byte  0             # holds drive that the boot sector came from
 iReserved:     .byte  0             # reserved, empty
 iBootSign:     .byte  0x29          # extended boot sector signature
 iVolID:        .ascii "seri"        # disk serial
 acVolumeLabel: .ascii "MYVOLUME   " # volume label
 acFSType:      .ascii "FAT16   "    # file system type
 
.func WriteString
WriteString:
  lodsb                   # load byte at ds:si into al (advancing si)
  or     al, al           # test if character is 0 (end)
  jz     WriteString_done # jump to end if 0.
 
  mov    ah, 0xe          # Subfunction 0xe of int 10h (video teletype output)
  mov    bx, 9            # Set bh (page nr) to 0, and bl (attribute) to white (9)
  int    0x10             # call BIOS interrupt.
 
  jmp    WriteString      # Repeat for next character.
 
WriteString_done:
  retw
.endfunc
 
.func Reboot
 Reboot:
  lea    si, rebootmsg # Load address of reboot message into si
  call   WriteString   # print the string
  xor    ax, ax        # subfuction 0
  int    0x16          # call bios to wait for key
  .byte  0xEA          # machine language to jump to FFFF:0000 (reboot)
  .word  0x0000
  .word  0xFFFF
.endfunc
 
start:
  # Setup segments:
  cli
  mov  iBootDrive, dl  # save what drive we booted from (should be 0x0)
  mov  ax, cs          # CS = 0x0, since that's where boot sector is (0x07c00)
  mov  ds, ax          # DS = CS = 0x0
  mov  es, ax          # ES = CS = 0x0
  mov  ss, ax          # SS = CS = 0x0
  mov  sp, 0x7C00      # Stack grows down from offset 0x7C00 toward 0x0000.
  sti  
 
  # Display "loading" message:
  lea  si, loadmsg
  call WriteString
 
  # Reset disk system.
  # Jump to bootFailure on error.
  mov  dl, iBootDrive  # drive to reset
  xor  ax, ax          # subfunction 0
  int  0x13            # call interrupt 13h
  jc   bootFailure     # display error message if carry set (error)  
 
  # End of loader, for now. Reboot.
  call Reboot
 
bootFailure:
  lea  si, diskerror
  call WriteString
  call Reboot
 
# PROGRAM DATA
loadmsg:          .asciz "Loading OS...\r\n"
diskerror:        .asciz "Disk error. "
rebootmsg:        .asciz "Press any key to reboot.\r\n"
 
.fill (510-(.-main)), 1, 0  # Pad with nulls up to 510 bytes (excl. boot magic)
BootMagic:  .int 0xAA55     # magic word for BIOS

Points of note

  • The WriteString and Reboot sections are functions, that we’ll want to call various times in the other boot code that we’ll write soon. They are not part of the main body of code. That is why they are placed before the start label, so that execution will jump over them.
  • The loader will not print a “Loading OS…” message right after it sets the segments.
  • After having reset the disk system successfully, the loader will reboot. We’re doing that only because we haven’t written any more code yet that does interesting things.
  • The source code ends with a .fill preprocessor directive. This causes the assembler to fill up the output file with null bytes all the way to offset 510. The final two bytes contain the magic word required by some BIOSes. Compilation will now yield a file of exactly 512 bytes, which is what we need for our boot sector.

Summary

We’ve written assembler code that prepares data and stack segments and resets the floppy drive. We’ve also added functions for writing text to the screen, waiting for a keypress, and rebooting, which wraps up

I don’t know about you, but I’m just about ready to see this code in action! In the next section, we’ll see how we can actually compile and test this code.

On to section 3!

Writing your own boot loader for a toy operating system (1)

If you’re writing your own toy operating system, the first thing you’ll need is a boot sector. It’s a piece of code (the boot loader) that lives in the first sector of a (floppy) disk. This code gets called by the BIOS as soon as the computer starts up.

Note that you can actually start developing other components of your toy operating system before writing boot code, since you can use GRUB (GNU Grand Unified Boot Loader) or LILO to start your kernel. Using one of these tools brings advantages, since they’ll switch the processor to protected mode for you, and allow you to load kernels that are placed beyond cylinder 1024 of a hard disk.

However, writing your own boot code can be a very interesting exercise in assembly programming, and you’ll have full control over what your boot loader actually does. Plus, you get to try and do it better than the people who wrote the DOS/Win95 boot loaders (which isn’t saying a lot as you’ll see below).

Boot loader requirements

The boot code lives in the first sector of a floppy disk, which typically has a size of 512 bytes. However, 61 of those bytes are occupied by data, placed on the disk when it is formatted. This data includes the size of a disk sector, number of FAT tables, number of tracks per sector, volume ID, and more. This yields 451 bytes available for code, which is not a whole lot. That’s one reason we’ll use assembler to write our code.

The DOS/Windows bootloader and its limitations

Let’s consider the boot loader that most of us have used many times: the boot loader that comes with DOS or Windows (up to Windows 95). What does it do?

  • Reset the floppy disk system
  • Read the first sector of the root directory from the disk
  • Verify that the first file found there is IO.SYS (the kernel)
  • Load IO.SYS into memory
  • Transfer control to IO.SYS

Since the space available for actual code in the boot sector is limited, the author of the DOS boot loader introduced an important requirement: the file IO.SYS must be the first file in the root directory. The DOS code does not scan the entire root directory looking for the required file. If IO.SYS is not the first file found, then the boot code fails.

This is why DOS/Windows comes with the SYS.COM program, which is used to make a disk bootable. This program actually cleans the root directory of a floppy disk and copies IO.SYS into it as the first entry, effectively removing all the other files. It would have been much nicer if it had been possible to copy IO.SYS to the root directory of a disk, at any position. Then any disk could be make bootable without sacrificing the files on it. This can actually be done, but it requires more assembly code, something the DOS developers apparently did not find any space for – but we can do better.

At any rate, modern operating systems will switch the processor to protected mode, which allows us to address up to 4 GB of memory in a flat model (not segmented), and switch on paging to protect processes from one another. This wasn’t part of the DOS/Windows 95 boot loader, but we’ll need to do it.

How a boot loader gets called

When the computer starts up, it executes a power-on self test (POST). It then performs the following actions:

  • Determine which device (drive) to use for booting, using preferences stored in the CMOS.
  • Try to load the first sector (and only the first sector) from the boot drive into memory at address 0:0x7C00.
  • Verify that the the first sector is in fact bootable by checking for the presence of a magic number (see below).
  • Store the number of the drive used in register DL.
  • Point the CPU’s instruction pointer to 0:0x7C00, and start execution from there.

The computer knows how to do these things, and does them automatically, because the code for this is in its BIOS ROM. In other words, these procedures we get for free with any computer.

What a boot loader should do

Here’s a list of things that a modern boot loader should do in order to load and start your operating system’s kernel (we’ll cover concepts like the A20-line, IDT and GDT tables later):

  • Reset the floppy disk system
  • Write a “loading” message to the screen
  • Find the kernel in the root directory of the disk (at any position)
  • Read the kernel from disk into memory
  • Enable the A20-line
  • Setup the IDT and GDT tables
  • Switch to protected mode
  • Clear the processor prefetch queue
  • Run the kernel

Boot Sector Layout

The boot sector of a floppy disk has a very specific layout, because the BIOS requires access to certain data which it needs to find in the place it expects it to be. Also, an operating system will need to access this data to determine how large the disk is, what file system it uses, what its volume label is and so on. For this article, we’ll assume a floppy disk formatted with a FAT16 file system. The layout of the boot sector is then:

OffsetSizeContentsTypical value
00003CodeJump to rest of code
00038BPBOEM nameGreat-OS
00112Bytes per sector512
00131Number of sectors per cluster1
00142Number of reserved sectors1
00161Number of FAT tables2
00172Number of root directory entries (usually 224)224
00192Total number of sectors2880
00211Media descriptor0xf0
00222Number of sectors per FAT9
00242Number of sectors/track9
00262Number of heads2
00282Number of hidden sectors0
00302EBPBNumber of hidden sectors (high word)0
00324Total number of sectors in filesystem
00361Logical drive number0
00371Reserved
00381Extended signature0x29
00394Serial number
00438Volume labelMYVOLUME
00548Filesystem typeFAT16
0062448CodeBoot code
05102RequiredBoot signature0xaa55

A required element of the boot sector is the boot parameter block (BPB) and the extended boot parameter block (EBPB, for FAT16). This block must be placed at offset 3, size 59 bytes. Also, the boot sector must end with the magic number 0xaa55: (some) BIOSes will check whether this value is present at offset 510. If not, the BIOS will refuse to boot from the disk. All other bytes are available for us to fill in. We can calculate that that adds in fact up to 451 bytes. Also, the first three bytes are separated from the rest and should only be used to jump to the rest of the code, so that’s less 3 bytes for interesting code…

Here is a typical hex dump of a boot sector without any code. Colored in red are the parts in the BPB and EBPB as decribed above, and the magic number at the end. Everything else is available for code:

0x0000 00 00 00 47 72 65 61 74 2d 4f 53 00 02 01 01 00 ...Great-OS.....
0x0010 02 e0 00 40 0b f0 09 00 09 00 02 00 00 00 00 00 .à.@............
0x0020 00 00 00 00 00 00 29 73 65 72 69 00 00 00 00 00 ......)seri.....
0x0030 00 00 00 00 00 00 46 41 54 31 36 20 20 20 fa 88 ......FAT16   ú^
0x0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa ..............

Summary

This article described how a computer bootstraps. The Power-On Self Test (POST) causes the first sector of a (floppy) disk to be read into memory. This boot sector contains information about the disk and (at most) 451 bytes of code. In the next part of this guide, we’ll see how we can write assembly code to roll our own boot loader.

This article continues in part 2 of this series.

Jiskefet Lullo’s Gezinsverpakking: alle video’s!

Hier zijn ze dan:

Linking a flat binary from C with MinGW

If you’re trying to compile a kernel written in C for your own toy operating system, you may run into trouble compiling/linking your code. Assuming you’re using GRUB to load your kernel, or you’ve rolled your own boot sector, you’ll now want to compile your kernel code (written in C) to a flat binary. The toolchain provided by MinGW (gcc and ld) is well suited for this, as long as you know a few tricks.

Let’s start with a very simple kernel.c program just to see if we can get things working:

int main(void)
{
mylabel:
  goto mylabel;
}

We’ll compile this with gcc, switching on all warnings (the compiler is our friend):

gcc -Wall -pedantic-errors kernel.c -o kernel.exe

This will yield a working program that we can actually execute at the command prompt. It’ll pause indefinitely, as desired. However, there are a number of problems with the resulting binary:

First, the binary includes a PE header, which specifies how Windows must load and execute the program. We’re writing a kernel, so we don’t want any of this header data. We must find it way to remove it.

Second, the program is relocatable. The operating system (i.e. Windows) will load the code into memory where it wants, then use the information contained in the PE header to make sure that all references are correct. The references are provided relatively, that is, the can be relocated. For our kernel, this is not what we want: we want to load our kernel at a specific address (say 0×20000) and make all references work precisely (statically) there.

This can be illustrated by running objdump:

$ objdump -f kernel.exe
kernel.exe: file format pei-i386
architecture: i386, flags 0x00000132:
EXEC_P, HAS_SYMS, HAS_LOCALS, D_PAGED
start address 0x00401160

Objdump’s output shows that a PE header is present (pei-i386 file format) and that a default random start adress of 0×00401160 has been defined. Let’s see what we can do about the start address. Since we want our kernel to always run at 0×20000, we can instruct the linked to use that address to place the code. Linker options can be passed to gcc:

Hint: do not use gcc to compile but not link, then ld to do the linking separately. Strange error messages will ensue. It’s easier to simply pass the linking options to gcc and let gcc call ld for you.

$ gcc -Wall -pedantic-errors kernel.c -o kernel.exe -Wl,-Ttext=0x20000
$ objdump -f kernel.exe
kernel.exe: file format pei-i386
architecture: i386, flags 0x00000132:
EXEC_P, HAS_SYMS, HAS_LOCALS, D_PAGED
start address 0x00020160

Oh look: our start address is now 0×00020160. The excess 0×160 bytes are the space occupied by the header, which we don’t want. We can try to pass the option –oformat binary to the linker, which will make it link a flat binary for us. Unfortunately (under MinGW), we get this:

c:/mingw/bin/../lib/gcc/mingw32/4.5.2/../../../../mingw32/bin/ld.exe:
cannot perform PE operations on non PE output file 'kernel.exe'.
collect2: ld returned 1 exit status

This can be resolved though: let the linker create the kernel.exe executable, then pass it through objcopy to create the flat binary:

objcopy -O binary -j .text kernel.exe kernel.bin

This will yield, finally, an executable. Unfortunately, it’s 3376 bytes in size! About 10 bytes would be closer to the mark. Obviously, code is being included that we didn’t write: references to standard libraries. Since we don’t have any standard libraries in our fledgling operating system, we’ll need to remove this. This can be done by passing the -nostdlib argument to gcc:

$ gcc -Wall -pedantic-errors kernel.c -o kernel.exe -nostdlib -Wl,-Ttext=0x20000
C:\Users\Alex\AppData\Local\Temp\cc5nshHf.o:kernel.c:(.text+0x7):
  undefined reference to `__main'
collect2: ld returned 1 exit status

Foiled again! Now that we have no standard libraries, ld is looking for startup code that doesn’t exist. We did write a main function, but it’s actually looking for a wrapper to that main function normally supplied by the standard libraries. Let’s try a different approach: we’ll rename our main function.

int start(void)
{
mylabel:
  goto mylabel;
}

Now our code compiles, and we’re down to a flat binary of 2011 bytes. It turns out that we must also pass -nostdlib to the linker:

$ gcc -Wall -pedantic-errors kernel.c -o kernel.exe -nostdlib
  -Wl,-Ttext=0x20000,-nostdlib

Now we get an executable of 24 bytes. In fact, on my system I get:

00000000h: 55 89 e5 eb fe 90 90 90 ff ff ff ff 00 00 00 00
00000010h: ff ff ff ff 00 00 00 00

When disassembled, this yields:

push ebp
mov ebp, esp
jmp .-2

This corresponds exactly to the code we wrote: a stack frame is created for the start function (even though we are not interested in it – a C program must always start with a function), then an infinite loop is entered (which we wrote using a label and a goto statement).

Wait… this code only occupies 5 bytes. So why are there 24 bytes in the flat binary image? We can see that the first three unneeded bytes have a value of 0×90, which corresponds to NOP instructions. This is probably added to get at least an 8-byte boundary. However, why an additional 16 bytes are added, I actually don’t know. If anyone can explain, I’d be grateful.

Nevertheless, we have now produced a flat binary that can be launched by our boot sector or second stage boot loader. It can be placed at 0×20000 and includes no undesired headers. Just the code, please, ma’am.