Speed up install floppy boot

[boeckmann]

I made a few experiments on how to load the kernel faster from floppy disk. A quickly hacked together loader loads the kernel by multi-sector reading it from the reserved sectors of a modified FAT filesystem. This boots quite fast on an original IBM-PC.

Due to limited assembly skills I was not able to fit both multi-sector reads + patching the diskette parameters + directory and FAT traversal into the boot sector.

So question would be if it is worth the effort adjusting the 360k installation image creation process to benefit from some speed up or wait until @ecm-pushbx tells us that all this can indeed be done in a single sector :D

[ecm-pushbx]

So question would be if it is worth the effort adjusting the 360k installation image creation process to benefit from some speed up or wait until @ecm-pushbx tells us that all this can indeed be done in a single sector :D

Without having seen the code my guess is it's doubtful.

[mateuszviste]

I made a few experiments on how to load the kernel faster from floppy disk. A quickly hacked together loader loads the kernel by multi-sector reading it from the reserved sectors of a modified FAT filesystem. This boots quite fast on an original IBM-PC.

Due to limited assembly skills I was not able to fit both multi-sector reads + patching the diskette parameters + directory and FAT traversal into the boot sector.

Interesting experiment! How faster exactly is "faster"?
The fact that it needs a special FAT makes it less appealing, of course -- is there any way maybe to split the boot into more steps? I may talk nonsense as I did not look how it all works exactly, but I'd imagine something like:

1. Boot sector performs FAT traversal to locate the (single) sector containing the 2nd stage boot code (KERNBOOT.BIN?) and loads it.
2. KERNBOOT.BIN performs the multi-sector read of the KERNEL.SYS file.

The idea would be to make it possible to use multi-sector loading without FAT modification. This would perhaps also allow to use it on HDDs, albeit I suppose the speed advantage would be slim on a HDD.

[boeckmann]

Interesting experiment! How faster exactly is "faster"?

From bootsector to EDR-DOS kernel greeting from 29 seconds down to 5, round about. Of course does not take into account BIOS POST.

[ecm-pushbx]

You may be able to modify lDOS iniload to do multi-track reads. Then you could just use an lDOS boot sector loader (lDOS protocol doesn't need the entire file loaded), and perhaps modify it to load only as little as 1536 bytes by itself. After that do the handover to the iniload stage which will load the remaining data of the kernel file.

[boeckmann]

The idea would be to make it possible to use multi-sector loading without FAT modification. This would perhaps also allow to use it on HDDs, albeit I suppose the speed advantage would be slim on a HDD.

I do not think it makes huge difference when loading from HDD. It even becomes less important on faster systems and a 1.44 MB drive.

I attach the modified 360k floppy image in case one wants to "benchmark". You may use one of the official SvarDOS 360k images to compare.

boot.img.zip

[boeckmann]

Current highly unoptimized loader source:

defects:

• no DMA boundary checking (kernel has to be <= 64000 bytes)
• no fallback to single-sector reads

; SvarDOS FAT12 floppy boot loader
; under MIT license

;%define COM 1

%define LOAD_SEG 0x60

%ifndef COM
%define CODE_OFF 0x7c00
%define RELOC_SEG 0x2000 - ((CODE_OFF+(code_end-code_start)+(bss_end-bss_start) + 15) >> 4)
%else
%define CODE_OFF 0x100
%endif

%define BOOTSECT_SIZE 512

; relocate just below 128K

%define BPB(x) [bp + bpb.%+ x - loader_start]
%define BP_BASED(x) [bp + (x) - loader_start]

%define advance_to(o) times (o) - ($-loader_start) db 0x20

	org CODE_OFF

code_start:
loader_start:
	jmp short real_start
	nop
; BIOS parameter block
	db 'SVDOS3.3'
bpb:
.bps:		dw 512
.spc:		db 2
.resvd: 	dw 0x75
.nfat:		db 2
.rootdirent:	dw 112
.nsec:		dw 0
.mediatype:	db 0xf0
.fatsecs:	dw 1
.spt:		dw 9
.nheads:	dw 2
.hidden:	dd 0
.nsecbig:	dd 0
	advance_to(0x3e)

error: db 0

real_start:
	mov ax,RELOC_SEG
	mov ss,ax
	mov sp,loader_start - 0x80
	sti
	push cs
	pop ds
	mov es,ax
	mov bp,loader_start
	mov si,bp
	mov di,bp
	mov word es:[loadseg],LOAD_SEG
move_loader:
	mov cx,0x100
	cld
	rep movsw
	jmp RELOC_SEG:relocated
relocated:

patch_dpt:
	mov bx,4*0x1e
	lds si,[bx]
	mov di,dpt
	mov cx,8
	rep movsw
	mov ds,cx
	mov si,bx
	mov di,int1e_sav
	movsw
	movsw
	mov word [bx],dpt
	push cs
	pop ax
	mov [bx+2],ax
	push cs
	pop ds
	mov [drive],dl
	mov ax,BPB(spt)
	mov [dpt+4],ax
	call disk_reset

calc_values:
	mov ax,BPB(hidden)
	add ax,BPB(resvd)		; ax=start of FAT area
	mov [fatstart],ax
	mov bx,ax
	mov al,BPB(nfat)
	cbw				; clear ah
	mul word BPB(fatsecs)		; ax=nfat*fatsex
	add ax,bx			; ax=start of root dir
	mov [rootdirstart],ax
	mov bx,ax
	mov cl,4
	mov dx,BPB(bps)
	shr dx,cl			; dx=paras per sec
	mov [secparas],dx
	shr dx,1			; bps / 32
	mov ax,BPB(rootdirent)
	add ax,dx			; adjust to round up
	dec ax
	div dl				; al=root dir sectors
	cbw				; clear ah
	mov [rootdirsecs],ax
	add ax,bx			; ax=start of cluster area
	mov [datastart],ax

load_kernel:
	mov ax,[loadseg]
	mov es,ax
	mov ax,1
	mov cx,BPB(resvd)
	dec cx
	call read_chs_n
	jnc .ok
	inc byte [error]
.ok:

restore_dpt:
	xor ax,ax
	mov es,ax
	mov si,int1e_sav
	mov di,4*0x1e
	movsw
	movsw
	cmp byte [error],0
	jz boot_kernel
	call fatal
	db "ERR!",0


boot_kernel:
	mov dl,[drive]
	mov bl,dl
	jmp LOAD_SEG:0


; IN: ax=start sector, cx=sector count, es=destination
read_chs_n:
	push si
	push di
	push bx
	push dx
	mov si,cx			; si=sectors still to read
	mov di,ax			; di=first sector of current read
.next:	jcxz .r
	call n2chs			; calculate start CHS
	mov ax,cx
	and ax,0x3f			; restrict sec cnt to trk bounds
	sub ax,BPB(spt)
	dec ax
	neg ax
	cmp ax,si
	jbe .l1
	mov ax,si			; restrict sec to read to sec left
.l1:	mov dl,[drive]
	call dot
	call read
	jc .r
	sub si,ax			; adjust sectors left
	mov cx,si
	add di,ax			; adjust sector of next read
	mul word [secparas]
	mov dx,es
	add ax,dx
	mov es,ax			; adjust load address
	mov ax,di
	jmp .next
.r:	pop dx
	pop bx
	pop di
	pop si
	ret


; READ SECTORS VIA INT13,02
; IN:  AL=SECTORS TO READ, DL=DRIVE
;      DH,CX=CHS PREPARED BY N2CHS
;      ES:BX=BUFFER
; OUT: AL=NUMBER OF SECTORS READ, CY CLEAR
; ERR: CY SET ON ERROR
read:
	push si
	push bx
	mov si,4			; retry 4 times
.retry:	push ax
	mov ah,2			; INT13.02
	xor bx,bx
	stc
	int 0x13
	pop ax
	jnc .r
	dec si
	jz .err
	call disk_reset
	jmp .retry
.err:	stc
.r:	pop bx
	pop si
	ret


; convert logical sector (zero-based) to CHS value
; IN: ax=sector number
; OUT: dh=head, ch=cyl low, cl=ccssssss cyl high + sector
;      dl=destroyed
; ERR: CY set if cyl > 1023
n2chs:
	push ax
	push bx
	xor bx,bx
	xor dx,dx
	div word BPB(spt)		; ax=log trk no, dx=sector
	xchg dx,bx			; bx=sector, dx=0
	div word BPB(nheads)		; ax=cylinder, dx=head
	mov dh,dl			; dh=head
	cmp ax,1023
	ja .err
	xchg ah,al
	ror al,1
	ror al,1			; al=cyl hi << 6
	add ax,bx			; ax = ccssssss
	inc ax				; sector is one-based
	mov cx,ax
	jmp .r
.err:	stc
.r:	pop bx
	pop ax
	ret


disk_reset:
	push ax
	xor ax,ax
	int 0x13
	pop ax
	ret

debug_out:
	push bx
        mov bx,ax
        mov al,2
        out 0x7a,al
        mov al,bl
        out 0x7b,al
        mov al,4
        out 0x7a,al
        mov al,bh
        out 0x7b,al
        pop bx
       	ret

fatal:
	push cs
	pop ds
	pop si
.l:	lodsb
	test al,al
	jz .forever
	mov ah,0x0e
	mov bx,7
	int 0x10
	jmp .l
.forever:
	hlt
	jmp .forever

dot:
	push ax
	push bx
	mov ax,0x0e00 + '.'
	mov bx,7
	int 0x10
	pop bx
	pop ax
	ret

	advance_to(BOOTSECT_SIZE - 2)
bios_signature:
	db 0x55, 0xaa
code_end:

section .bss
bss_start:
drive: 		resb 1
secparas:	resb 2
int1e_sav:	resb 4
fatstart:	resw 1
rootdirstart:	resw 1
rootdirsecs:	resw 1
datastart:	resw 1
loadseg:	resw 1

dpt:		resb 0x10

loader_end:
bss_end:

[ecm-pushbx]

I'm considering how to add multi-track reading to lDOS iniload. We could disable LBA support, that'd free up a bunch of space. But then the file would be less generic.

The current sizes come out to:

../../ldosboot/iniload.asm:865: warning: 1 bytes in front of ms7_entry [-w+user]
../../ldosboot/iniload.asm:1275: warning: 18 bytes in front of ldos_entry [-w+user]
../../ldosboot/iniload.asm:1695: warning: 10 bytes in front of end [-w+user]
../../ldosboot/iniload.asm:1758: warning: 428 bytes in front of end2 [-w+user]

The end2 bit is particularly interesting. IBM-DOS and MS-DOS v6 only load 1536 bytes (up to end) so that amount should work. But MS-DOS v7 loads 2048 bytes (up to end2), and lDOS / RxDOS.3 loaders will load 8192 bytes easily:

Current loaders will load at least 8192 bytes if the file is as large or larger than that.

This could still be limited to 2048 or 4096 bytes to increase performance. In any case, it'll load up to end2 as well.

So we have 400 odd bytes to implement an optional(!) multi-track reading loader, which could always fall back to the original single-sector loader that's completely contained in the first 1536 bytes. For instance, if multi-track reading doesn't work, or if a 64 KiB DMA boundary is crossed, or LBA is available, or up to end2 isn't loaded, then the original loader could be used.

The new recommended load address for the lDOS / RxDOS.3 iniload stage at segment 200h means that without relocation and just loading subsequent sectors from the FS, we do not cross the 64 KiB ISA DMA boundary with any single sector. But perhaps it would require to split up a track read so that the multi-sector read also doesn't cross the boundary. I certainly want to support kernel files >= 64 KiB.

@boeckmann can you please add usage conditions to your example to make it proper FLOSS?

[boeckmann]

I'm considering how to add multi-track reading to lDOS iniload.

Do you indeed want to do multi-track reading? I deliberately did not read across track boundaries because I am not sure if this works under most systems, so I basically split the read into multiple multi-sector single-track reads.

@boeckmann can you please add usage conditions to your example to make it proper FLOSS?

Modified my code snipped to be under MIT

[ecm-pushbx]

I'm considering how to add multi-track reading to lDOS iniload.

Do you indeed want to do multi-track reading? I deliberately did not read across track boundaries because I am not sure if this works under most systems, so I basically split the read into multiple multi-sector single-track reads.

Yes, you caught me. I meant multi-sector, full-track reading. I mixed up these two and that's how we got "multi-track".

@boeckmann can you please add usage conditions to your example to make it proper FLOSS?

Modified my code snipped to be under MIT

Thanks! I'll probably get to experimenting at the latest on 2025-10-12 Sun.

[boeckmann]

The fact that it needs a special FAT makes it less appealing, of course -- is there any way maybe to split the boot into more steps? I may talk nonsense as I did not look how it all works exactly, but I'd imagine something like:

1. Boot sector performs FAT traversal to locate the (single) sector containing the 2nd stage boot code (KERNBOOT.BIN?) and loads it.

2. KERNBOOT.BIN performs the multi-sector read of the KERNEL.SYS file.

The idea would be to make it possible to use multi-sector loading without FAT modification. This would perhaps also allow to use it on HDDs, albeit I suppose the speed advantage would be slim on a HDD.

Yes, having the kernel.sys hidden in the reserved sectors is not that nice, albeit EDR-DOS and FreeDOS kernel should have zero problems with this. Has the benefit that the user cannot delete the kernel "by accident".

In addition to @ecm-pushbx "big" suggestion, some middle way would be to store a list of the sectors the kernel occupies with the boot loader. Then the kernel could be placed in the data area. Downside: if the kernel is moved on the disk booting will fail. However, this method in theory should optimally be as fast as loading the kernel from reserved sectors, as the directory and FAT do not have to be loaded from disk. Writing the sector list together with the loader would then be the job of the SYS command.

[mateuszviste]

Yes, having the kernel.sys hidden in the reserved sectors is not that nice, albeit EDR-DOS and FreeDOS kernel should have zero problems with this. Has the benefit that the user cannot delete the kernel "by accident".

some middle way would be to store a list of the sectors the kernel occupies with the boot loader. Then the kernel could be placed in the data area. (...)

That would work fine for floppies that aren't meant to be fiddled with, but would fail on hard disks as soon as the user runs pkg update kernledr.svp. But even for install floppies I am not sure it would not bite someone. I, for instance, often use the first SvarDOS install floppy as a generic boot disk and add stuff I need at the given moment. Should I want at some point to update the kernel on such bootdisk, it will fail. I am not worried about myself, but there might be other people with a similar workflow and they might be confused that replacing kernel.sys suddenly makes the floppy unbootable. All this to say - I am very impressed by a 6x speedup of kernel loading, but unsure how/where this cool hack could be applied without any usability risks.

[ecm-pushbx]

I did get around to doing it today after all. The link leads to the lDOS boot experimental repo.

I adapted Bernd's calculation for how many sectors are left within the same C and H coordinates (ie, the same track). I added code that checks for 64 KiB boundary crossings. The multi sector read code is used both for loading the entire FAT12 if not yet done and for loading the trailing file data. The latter walks the FAT to find how many consecutive data sectors can be read. (This assumes there will never be more than 64 Ki sectors per extent.)

The multi sector loader patches itself into two patch sites when it has been detected that it's usable (up to end2 loaded and no LBA to be used).

The new loader fits comfortably in the space before end2 with 81 bytes to spare.

I also added options to the FAT12 loader so that it may load only 4096 bytes from the load file (rather than > 20 KiB) and also make it ideally load only 2 FAT sectors, pessimally many more. Without this new option, the FAT12 boot sector loader would load the entire FAT (that is, 9 sectors on a 1440 KiB diskette) using single-sector reads, and pass it to iniload as part of the protocol. With the option _LOADFULLFAT12=0 every access to the FAT may read in a new sector if needed, and whenever one FAT sector is read the immediate next sector is read as well (to support straddling FAT entries). The two-sector FAT12 buffer is not expected by iniload so the boot sector invalidates the buffer address passed onwards.

Here's some scripting I prepared to test this:

nasm -D_LOAD_MIN_PARA=0 -D_LOADFULLFAT12=0 -D_FREE_PARA_LIMIT -I ~/proj/lmacros/ ~/proj/ldosboot/boot.asm -D_LOAD_NAME="'LDEBUG'" -o boot12.bin -l boot12.lst

nasm -D_BOOTPATCHFILE="'boot12.bin'" -I ~/proj/msdebug/ -I ~/proj/ldebug/bin/ ~/proj/bootimg/bootimg.asm -I ~/proj/lmacros/ -D_PAYLOADFILE=ldebug.com,lddebugu.com,extlib.eld,testmult.sld -o multi.img && qemu-system-i386 -curses -chardev serial,id=serial2,path=/tmp/vptty-dos -serial null -serial chardev:serial2 -fda multi.img -boot order=a

boot fda
e 0:7C3E "LDDEBUGU"

=== testmult.sld
boot protocol ldos fda/lddebugu.com
e ss:bp-6 as words 400          ; loadseg (say up to below here loaded)
e ss:bp-8 0 0                   ; FAT12 isn't passed
tp FFFFFF while ip < 600        ; go to multi patch area in end2
t 4

~/proj/patchini/patchqry lddebugu.com diskette 1 && mcopy -o -i disktest.img lddebugu.com ::LDDEBUGU.COM

Here's how to build a kernel file from the lDOS boot + scanptab + lmacros repos given a FreeDOS-compatible kernel file named fdkernel.sys:

nasm -I ../lmacros/ -I ../scanptab/ fdkernpl.asm -D_PAYLOAD_FILE=fdkernel.sys -o fd.pl && nasm -I ../lmacros/ -I ../scanptab/ iniload.asm -D_PAYLOAD_FILE=fd.pl -D_INILOAD_SIGNATURE="'FD'" -o fd.sys

[ecm-pushbx]

The latter walks the FAT to find how many consecutive data sectors can be read. (This assumes there will never be more than 64 Ki sectors per extent.)

Hmm. The kernel file must be < 1 MiB. To overflow a 16-bit word with sector counts you need 64 Ki sectors. At the smallest theoretical size (32 B/s) that's 2 MiB. So it cannot ever overflow, the code that checks whether all trailing data has been found will always activate its branch when cx reaches towards or beyond 07FFFh.

[ecm-pushbx]

Hmm. Multi-sector reads can fail for yet different reasons, right? How can I detect that? I want to fall back to single-sector reads then.