after trivial update, 15.0 ARM64 system no longer boots

[Lexi Winter]

hi lists,

i have a FreeBSD 15.0/arm64 system, an RPi4, which was previously
running 15.0 with pkgbase. i rebuilt main on my pkg server and updated
the RPi with 'pkg update', which only included ~2 commits neither of
which seemed like they had anything to do with booting, but after the
update, the system no longer boots.

the problem seems to be a hang during kernel initialisation:

Loading Image...

i am not really an expert on either ARM64 in general or on the RPi
hardware in particular. could anyone suggest how i could debug this
problem, e.g. to get more information about why the system won't finish
booting?

thanks, lexi.

[Ryan Stone]

Are you sure that the system isn't actually booting or are you saying
that because you get no console output after going to userland? It's
possible that /etc/ttys is not in sync with your console configuration
in loader.conf. If that happened, nothing from /etc/rc would print to
console during boot and you wouldn't get a login prompt, but
networking and the like would still work. If you need the console to
access it (e.g. no sshd is configured) you can try booting into
single-user and look at /etc/ttys from there. If you can't actually
reach single-user mode then the problem is something else.

Post by Lexi Winter
hi lists,
i have a FreeBSD 15.0/arm64 system, an RPi4, which was previously
running 15.0 with pkgbase. i rebuilt main on my pkg server and updated
the RPi with 'pkg update', which only included ~2 commits neither of
which seemed like they had anything to do with booting, but after the
update, the system no longer boots.
https://www.le-fay.org/tmp/30d/9fE0NG.jpeg
i am not really an expert on either ARM64 in general or on the RPi
hardware in particular. could anyone suggest how i could debug this
problem, e.g. to get more information about why the system won't finish
booting?
thanks, lexi.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Ronald Klop]

I have no clue. But can you boot in “verbose” mode?

And as the last message is usb, can you remove all usb devices and boot again?


Regards,
Ronald

Van: Lexi Winter <***@le-fay.org>
Datum: 15 maart 2024 16:57
Aan: freebsd-***@freebsd.org, freebsd-***@freebsd.org
Onderwerp: after trivial update, 15.0 ARM64 system no longer boots

Post by Lexi Winter
hi lists,
i have a FreeBSD 15.0/arm64 system, an RPi4, which was previously
running 15.0 with pkgbase. i rebuilt main on my pkg server and updated
the RPi with 'pkg update', which only included ~2 commits neither of
which seemed like they had anything to do with booting, but after the
update, the system no longer boots.
https://www.le-fay.org/tmp/30d/9fE0NG.jpeg
i am not really an expert on either ARM64 in general or on the RPi
hardware in particular. could anyone suggest how i could debug this
problem, e.g. to get more information about why the system won't finish
booting?
thanks, lexi.

[Lexi Winter]

Post by Lexi Winter
i am not really an expert on either ARM64 in general or on the RPi
hardware in particular. could anyone suggest how i could debug this
problem, e.g. to get more information about why the system won't finish
booting?

i dug into this a bit more, and to answer my own question:

- the boot failure that prompted the question appears to be a bug
related to mmc, i reported it as PR 277884 [0].

- part of the problem, that i realise i forgot to mention in my initial
post, was that my USB keyboard wouldn't work, so i couldn't interrupt
the loader to use boot -v / -s / etc. or to access kdb. this turned
out to be an issue with the keyboard itself - a different keyboard
worked fine. the keyboard *does* work fine in FreeBSD once it's
booted, which is odd; it's some $5 Amazon special, so i assume it just
implements USB badly.

- even with the working keyboard, ctrl+alt+esc doesn't seem to work to
break into kdb when the problem occurs. i'm not sure if i'm doing
something wrong here or that key sequence doesn't work over USB, so i
wanted to try it via a serial console instead, which led to...

- i played around with USB OTG a bit to see if i could get a serial
console that way. FreeBSD does support serial over the OTG port (via
the USB-C 'power' connector) and it works as a terminal in /etc/ttys,
but neither U-Boot nor FreeBSD seem to be able to use it as a system
console, which is a bit disappointing. it would be nice if there was
a way to get this working, because a serial console over USB would be
very handy for working with these devices.

- in the mean time, i ordered a DB-9 serial HAT that connects to the
UART to debug this and future issues. once that arrives i can
hopefully get some more info on the original problem.

regards, lexi.

[0] "RPi4: mmc broken with GENERIC-NODEBUG":
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277884

[Mark Millard]

Post by Lexi Winter

. . .

. . .
- even with the working keyboard, ctrl+alt+esc doesn't seem to work to
break into kdb when the problem occurs. i'm not sure if i'm doing
something wrong here or that key sequence doesn't work over USB, so i
wanted to try it via a serial console instead, which led to...
. . .

https://man.freebsd.org/cgi/man.cgi?ddb(4) reports:

QUOTE
Serial consoles can break to the debugger by sending a BREAK condition
on the serial line. This requires a kernel built with options
BREAK_TO_DEBUGGER is specified in the kernel. Most terminal emulation
programs can send a break sequence with a special key sequence or menu
selection. Sending the break can be difficult or even happen spuri-
ously in some setups. An alternative method is to build a kernel with
options ALT_BREAK_TO_DEBUGGER then the sequence of CR TILDE CTRL-B en-
ters the debugger; CR TILDE CTRL-P causes a panic; and CR TILDE CTRL-R
causes an immediate reboot. In all these sequences, CR represents Car-
riage Return and is usually sent by pressing the Enter or Return key.
TILDE is the ASCII tilde character (~). CTRL-x is Control x, sent by
pressing the Control key, then x, then releasing both.
END QUOTE

Note the lack of mention of the ctrl+alt+esc .

I expect that the:

https://docs.freebsd.org/en/books/developers-handbook/kerneldebug/

reference to "The default break-to-debugger sequence is Ctrl+Alt+ESC"
may be x86/i386 specific (historical tier 1) or have some other
specific type of context it applies to.

I've historically used ALT_BREAK_TO_DEBUGGER and its CR TILDE CTRL-B
on everything to get to the ddb> prompt via keyboards, including the
serial console. (But, thinking about it, I've not used that in some
time.)

===
Mark Millard
marklmi at yahoo.com



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de