RISC-V Boot Flow
Table of Contents
1. RISC-V Boot Flow
1.1. opensbi
https://github.com/riscv-software-src/opensbi
qemu-system-riscv64 -M virt -bios fw_jump.elf -kernel Image -append "rootwait root=/dev/vda ro" -drive file=rootfs.ext2,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -netdev user,id=net0 -device virtio-net-device,netdev=net0 -nographic
1.1.1. FW_JUMP
其中 fw_jum.elf 的 entry 是 0x80000000, qemu 会根据约定跳转到这个地址, 启动 opensbi
当 opensbi 编译时指定 FW_JUMP 时, 会生成 fw_jump.elf, 它会在启动时跳转到下一级 bootloader, 这里的跳转地址是 0x80200000. 这个地址是 opensbi 中配置好的:
opensbi/platform/generic/config.mk
FW_TEXT_START=0x80000000 FW_DYNAMIC=y FW_JUMP=y ifeq ($(PLATFORM_RISCV_XLEN), 32) # This needs to be 4MB aligned for 32-bit system FW_JUMP_ADDR=$(shell printf "0x%X" $$(($(FW_TEXT_START) + 0x400000))) else # This needs to be 2MB aligned for 64-bit system FW_JUMP_ADDR=$(shell printf "0x%X" $$(($(FW_TEXT_START) + 0x200000))) endif
qemu 需要按 opensbi 的要求把 Image 加载到 FW_JUMP_ADDR 即 0x80200000. 比较特殊的是 `-kernel Image` 并没有指定这个地址, 因为 qemu 的 virt device 会自动把 kernel 加载到它需要的地址. (https://stackoverflow.com/questions/71363345/what-is-the-kernel-map-address-in-qemu-arm-when-bios-option-is-also-given)
通过 qemu generic loader (https://qemu-project.gitlab.io/qemu/system/generic-loader.html) 可以显式的加载 Image 到 0x80200000
qemu-system-riscv64 -M virt -bios fw_jump.elf -device
loader,file=Image,addr=0x80200000 -drive file=rootfs.ext2,format=raw,id=hd0
-device virtio-blk-device,drive=hd0 -netdev user,id=net0 -device
virtio-net-device,netdev=net0 -nographic ${EXTRA_ARGS}
但是使用 loader 时无法指定 kernel cmdline, 导致启动后会找不到 fs, 可以把 cmdline 直接编译到 kernel 或用 dts 指定解决这个问题
1.1.2. FW_PAYLOAD
通过 FW_PAYLOAD, 可以将 Image 打包到 fw_payload.elf 中, 这样就不需要再指定 Image 了, opensbi 会直接跳转过去.
qemu-system-riscv64 -M virt -bios fw_payload.elf -drive file=rootfs.ext2,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -netdev user,id=net0 -device virtio-net-device,netdev=net0 -nographic
1.2. u-boot
https://u-boot.readthedocs.io/en/latest/index.html
可以让 opensbi 先跳转到 u-boot, 然后由 u-boot 启动 kernel.
在 buildroot 中编译 u-boot, 指定 config 为 `BR2_TARGET_UBOOT_BOARD_DEFCONFIG="qemu-riscv64_smode"`
qemu-system-riscv64 -M virt -bios fw_jump.elf -kernel u-boot.bin -device loader,file=Image,addr=0x84000000 -drive file=rootfs.ext2,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -netdev user,id=net0 -device virtio-net-device,netdev=net0 -nographic
- u-boot 期望 Image 加载到 0x84000000 (kernel_addr_r)
- booti 命令需要指定 Image (kernel_addr_r)和 dtb (fdtaddr) 的地址
U-Boot 2022.04 (Oct 10 2022 - 18:20:16 +0800)
CPU: rv64imafdcsuh
Model: riscv-virtio,qemu
DRAM: 128 MiB
Core: 18 devices, 10 uclasses, devicetree: board
Flash: 32 MiB
Loading Environment from nowhere... OK
In: uart@10000000
Out: uart@10000000
Err: uart@10000000
Net: eth0: virtio-net#1
Hit any key to stop autoboot: 0
=> booti ${kernel_addr_r} - ${fdtaddr}
Moving Image from 0x84000000 to 0x80200000, end=81533000
## Flattened Device Tree blob at 8773baf0
Booting using the fdt blob at 0x8773baf0
Using Device Tree in place at 000000008773baf0, end 000000008773fe01
Starting kernel ...
[ 0.000000] Linux version 5.15.43 (user@user-SYS-4029GP-TRT) (riscv64-unknown-linux-gnu-gcc (g2ee5e430018) 12.2.0, GNU ld (GNU Binutils) 2.39) #1 SMP Mon Oct 10 17:38:43 CST 2022
1.3. dtb
dtb 由 qemu 提供, 然后一路传递给 opensbi, uboot, kernel. risc-v 的规范要求每次跳转时 (qemu->opensbi, opensbi->uboot, uboot->kernel) 都使用 a1 保存 dtb 的地址
另外, qemu, opensbi 和 uboot 都支持自己载入 dtb 文件而不使用上一级传递过来的 dtb. 例如 opensbi 编译时可以通过 FW_FDT_PATH 指定 dtb
qemu 使用的默认 dtb 可以用命令 dump 出来 (https://u-boot.readthedocs.io/en/latest/develop/devicetree/dt_qemu.html):
qemu-system-riscv64 -machine virt -machine dumpdtb=qemu.dtb
查看一个 qemu 传递给 opensbi 的 dtb:
~/source/buildroot-2022.08#[18:41:07]@user-SYS-4029GP-TRT> gdb-multiarch output/images/fw_jump.elf (gdb) b _start Breakpoint 1 at 0x80000000: file /backup/source/buildroot-2022.08/output/build/opensbi-custom/firmware/fw_base.S, line 50. (gdb) target remote localhost:1234 Remote debugging using localhost:1234 0x0000000000001000 in ?? () (gdb) c Continuing. Breakpoint 1, _start () at /backup/source/buildroot-2022.08/output/build/opensbi-custom/firmware/fw_base.S:50 50 MOV_3R s0, a0, s1, a1, s2, a2 (gdb) p /x $a1 $2 = 0x87000000 (gdb) x /10x $a1 // 0xedfe0dd0 是 dtb 文件的 magic number 0x87000000: 0xedfe0dd0 0xf20e0000 0x38000000 0x880d0000 0x87000010: 0x28000000 0x11000000 0x10000000 0x00000000 0x87000020: 0x6a010000 0x500d0000
