KnowHow
Free Technical Resources

Debugging Embedded Rust: Article 2

The supporting code for this article lives in this GitHub repo.

The previous article laid out what a Rust build for the Cortex-M4 produces: an ELF executable holding the code and data bound for flash, plus DWARF debug information that lets a host-side tool map that machine state back onto our Rust source. That was all model and no hardware. This article puts a debugger on the board.

We'll connect to the NUCLEO-F446RE, break inside our small counter application, and follow the same running program through two toolchains: a conventional GDB/OpenOCD setup and the Rust-oriented probe-rs. Then we'll rebuild in release mode and watch what optimisation does, and doesn't, take away from a source-level session.

Two routes to the same processor

If you've debugged Cortex-M firmware in C or C++, your setup very likely involved GDB and OpenOCD:

GDB is the source-level environment (breakpoints, stack frames, variables, register and memory inspection, disassembly); OpenOCD is the bridge between GDB and the physical debug interface on the target.

Rust adds a second route through probe-rs:

probe-rs talks straight to supported debug probes: it controls the target, reads memory and registers, sets breakpoints, programs firmware. Its VS Code integration speaks the Debug Adapter Protocol, so there's no GDB sitting between the IDE and the target.

The difference is almost entirely on the host side. Both routes end up at the same Cortex-M debug hardware, because embedded Rust needs no special processor-side mechanism; the Cortex-M4 halts the core, exposes registers and memory, and provides hardware breakpoints and watchpoints exactly as it does for any language. The link carrying all of that is Serial Wire Debug (SWD), Arm's two-wire debug interface between the probe and the core. You don't need to know the SWD protocol to use what it exposes.

We'll use both routes across the series. probe-rs carries most of the Rust examples, especially once RTT and defmt arrive; GDB and OpenOCD earn their place whenever we want to poke at processor state directly from the command line, with commands like:

(gdb) info registers
(gdb) x/16wx 0x20000000
(gdb) x/i $pc
(gdb) disassemble

We pick whichever interface exposes the problem best, rather than treating one as a replacement for the other. The same flexibility extends past STM32: our NUCLEO carries an ST-LINK, but another Cortex-M board might use a J-Link or a CMSIS-DAP probe. The host tooling changes; the processor-level principles don't.

A first session

From the project root, build the application and start OpenOCD:

cargo build -p project1-baseline-counter
openocd -f openocd.cfg

Then, in a separate terminal, attach GDB:

gdb-multiarch -x .gdbinit target/thumbv7em-none-eabihf/debug/project1-baseline-counter

The firmware is now running on the NUCLEO-F446RE: the user LED toggles twice a second, and counter increments once per loop. Let's stop the processor and look at that same program from the debugger's side.

The counter update sits in its own function, on purpose:

#[inline(never)]
fn update_counter(counter: &mut u32) {
    *counter = counter.wrapping_add(1);
}

#[inline(never)] gives us a function with a stable boundary to break on. Set the breakpoint by its fully qualified path:

(gdb) break project1_baseline_counter::update_counter

You can list what's currently set with:

(gdb) info b

Then run to it:

(gdb) continue

GDB read the symbols and DWARF in the ELF to tie the processor's current program counter to a specific Rust function and source line. This first continue actually stops at the earliest breakpoint in the list, main (line 17, highlighted above). A next steps on into update_counter():

(gdb) next

Now the core is halted inside the function, and we can read the value it's working on:

(gdb) print *counter
$1 = 0

Note: print counter (without the *) gives you the address the reference holds, 0x2001ff5c, rather than the value behind it.

Step over the increment:

(gdb) next
14          *counter = counter.wrapping_add(1);
(gdb) print *counter
$2 = 1

The value is now 1. It's a small, unglamorous confirmation, and it's the whole point: counter is live state on the microcontroller, and we just watched it change while the Cortex-M was halted.

One level down is the processor itself:

(gdb) info registers
r0             0x1                 1
r1             0x2001ff5c          537001820
r2             0x1                 1
r3             0x501bd00           84000000
r4             0x280de80           42000000
r5             0x501bd00           84000000
r6             0x501bd00           84000000
r7             0x2001feb8          537001656
r8             0x0                 0
r9             0x0                 0
r10            0x0                 0
r11            0x0                 0
r12            0x501bd00           84000000
sp             0x2001feac          0x2001feac
lr             0x8000333           134218547
pc             0x80002aa           0x80002aa 
xPSR           0x1000000           16777216
fpscr          0x0                 0
msp            0x2001feac          0x2001feac
psp            0x0                 0x0
primask        0x0                 0
basepri        0x0                 0
faultmask      0x0                 0
control        0x0                 0

There's the program counter, stack pointer, and link register, alongside the general-purpose registers. Arm's calling convention passes a function's first argument in r0, so the reference to counter arrives there, yet it's r1 holding 0x2001ff5c by the time we've stopped. The disassembly explains why:

Dump of assembler code for function _ZN25project1_baseline_counter14update_counter17h94526bcf6fb81790E:
   0x08000294 <+0>:		push	{r7, lr}
   0x08000296 <+2>:		mov	r7, sp
   0x08000298 <+4>:		sub	sp, #12
   0x0800029a <+6>:		mov	r1, r0
   0x0800029c <+8>:		str	r1, [sp, #0]
   0x0800029e <+10>:	ldr	r0, [r1, #0]
   0x080002a0 <+12>:	str	r0, [sp, #4]
   0x080002a2 <+14>:	movs	r2, #1
   0x080002a4 <+16>:	str	r2, [sp, #8]
   0x080002a6 <+18>:	adds	r0, #1
   0x080002a8 <+20>:	str	r0, [r1, #0]
=> 0x080002aa <+22>:	add	sp, #12
   0x080002ac <+24>:	pop	{r7, pc}
End of assembler dump.

The compiler moved the argument from r0 into r1 at offset +6 (mov r1, r0), not because it had to but because that's the allocation this build happened to choose. Read straight down, the function loads the current value (ldr r0, [r1, #0]), adds one adds r0, #1), and stores it back (str r0, [r1, #0]): *counter = counter.wrapping_add(1), one instruction at a time. Note the demangled header, too, _ZN25project1_baseline_counter14update_counter17h...E; we'll come back to what that hash suffix is doing there, even on a function with no generics, in the reference piece on Rust's debugger representations.

None of these are separate facts. The source line, the register dump, and the raw instructions are three views of one running system.

The same walk works through probe-rs in an IDE. Instead of print, info registers, and x/i $pc, you use the variable, register, and disassembly panes; the presentation differs, but the underlying data is the same processor state read through the same ELF.

One thing to flag before moving on: while the core sits at a breakpoint, the program isn't advancing. For a blinking LED that costs nothing. For timing-sensitive firmware it can change the very behaviour you're trying to watch, and we walk straight into that in the next article.

Source intent, machine evidence

That session gives us a principle worth keeping for the whole series:

The source states intent: *counter = counter.wrapping_add(1) says the value should go up by one. The compiler turns that into instructions, and the debugger lets us inspect both the instructions and, in the end, the live value in memory. When everything works the two agree and the distinction never comes up. It's when they disagree that debugging gets interesting: if the source seems to increment a variable that never changes, you check the generated instructions; if they're present, you check the address being written; if memory changes correctly and the program still misbehaves, the fault is somewhere else entirely.

That's familiar from C and C++, and Rust doesn't change the principle. What Rust adds is more between the source and the machine, ownership, generics, typed peripheral APIs, iterators, closures, and most of the time that makes code easier and safer to reason about. When something surprises you, you can still slip underneath it to the processor state, which is exactly what we'll practise on a real bug next.

Debugging the release build

So far we've made life easy for the debugger. update_counter() has a stable boundary, and the debug build keeps a fairly direct line between Rust source and generated code.

Real firmware ships optimised, and that doesn't mean giving up source-level debugging. Cargo treats optimisation and debug information as independent knobs:

[profile.release]
opt-level = 3
debug = 2
strip = "none"

opt-level = 3 optimises hard; debug = 2 keeps full debug information in the ELF. You get an optimised image that GDB and probe-rs can still map back to source, which matters because an unoptimised build can differ from the shipped one in timing, stack usage, memory access, and call structure. Rebuild everything without optimisation to make a release-only bug easier to see, and you may quietly delete the conditions that caused it.

There's a limit to what the debug information buys you, though:

Take the counter function again with #[inline(never)] removed:

fn update_counter(counter: &mut u32) {
    *counter = counter.wrapping_add(1);
}

The reasonable guess is that release mode inlines update_counter() into main: the tidy one-function-to-one-block relationship disappears, but the increment still happens somewhere. Rather than assume, check it against the target:

(gdb) break project1_baseline_counter::update_counter
Function "project1_baseline_counter::update_counter" not defined.

Gone. Not folded into its caller, gone. Disassembling main confirms there's no increment, no counter, nothing of update_counter() left anywhere:

(gdb) disassemble _ZN25project1_baseline_counter18__cortex_m_rt_main17h078bf56349dca6e5E
Dump of assembler code for function _ZN25project1_baseline_counter18__cortex_m_rt_main17h078bf56349dca6e5E:
   0x0800021c <+0>:     push    {r7, lr}
   0x0800021e <+2>:     mov     r7, sp
   0x08000220 <+4>:     sub     sp, #72 @ 0x48
   0x08000222 <+6>:     movw    r1, #1
   0x08000226 <+10>:    mrs     r0, PRIMASK
   0x0800022a <+14>:    cpsid   i
   0x0800022c <+16>:    movt    r1, #8192       @ 0x2000
   0x08000230 <+20>:    ldrb    r2, [r1, #0]
   0x08000232 <+22>:    cbnz    r2, 0x80002a8 <_ZN25project1_baseline_counter18__cortex_m_rt_main17h078bf5634
   0x08000234 <+24>:    movs    r4, #1
   0x08000236 <+26>:    strb    r4, [r1, #0]
   0x08000238 <+28>:    msr     PRIMASK, r0
   0x0800023c <+32>:    bl      0x80017ce <_ZN96_$LT$cortex_m..critical_section..SingleCoreCriticalSection$u2
uire17he77f6b946ad61660E>
   0x08000240 <+36>:    movw    r1, #0
   0x08000244 <+40>:    movt    r1, #8192       @ 0x2000
   0x08000248 <+44>:    ldrb    r2, [r1, #0]
   0x0800024a <+46>:    cbnz    r2, 0x8000298 <_ZN25project1_baseline_counter18__cortex_m_rt_main17h078bf5634
   0x0800024c <+48>:    strb    r4, [r1, #0]
   0x0800024e <+50>:    bl      0x80017da <_ZN96_$LT$cortex_m..critical_section..SingleCoreCriticalSection$u2
ease17hc3d9845fa9afc280E>
   0x08000252 <+54>:    subs    r0, r7, #1
   0x08000254 <+56>:    bl      0x8000318 <_ZN13board_support8init_led17hf63c2a3ce8d74c64E>
   0x08000258 <+60>:    add     r4, sp, #4
   0x0800025a <+62>:    movs    r1, #84 @ 0x54
   0x0800025c <+64>:    mov     r0, r4
   0x0800025e <+66>:    bl      0x80002d2 <_ZN13board_support11init_clocks17hf2806278edb4f5b6E>
   0x08000262 <+70>:    ldr     r0, [sp, #44]   @ 0x2c
   0x08000264 <+72>:    movw    r1, #57360      @ 0xe010
   0x08000268 <+76>:    movt    r1, #57344      @ 0xe000
   0x0800026c <+80>:    movs    r5, #20
   0x0800026e <+82>:    ldr     r2, [r1, #0]
   0x08000270 <+84>:    movt    r5, #16386      @ 0x4002
   0x08000274 <+88>:    lsrs    r0, r0, #3
   0x08000276 <+90>:    bic.w   r2, r2, #4
   0x0800027a <+94>:    str     r2, [r1, #0]
   0x0800027c <+96>:    str     r0, [sp, #4]
   0x0800027e <+98>:    ldr     r0, [r5, #0]
   0x08000280 <+100>:   mov.w   r1, #500        @ 0x1f4
   0x08000284 <+104>:   lsls    r0, r0, #26
   0x08000286 <+106>:   mov.w   r0, #2097152    @ 0x200000
   0x0800028a <+110>:   it      pl
   0x0800028c <+112>:   movpl   r0, #32
   0x0800028e <+114>:   str     r0, [r5, #4]
   0x08000290 <+116>:   mov     r0, r4
   0x08000292 <+118>:   bl      0x8001748 <_ZN13stm32f4xx_hal5timer5hal_196_$LT$impl$u20$embedded_hal..delay..DelayNs$u20$for$u20$
er..delay..SysDelay$GT$8delay_ms17h987fbc279faab728E>
   0x08000296 <+122>:   b.n     0x800027e <_ZN25project1_baseline_counter18__cortex_m_rt_main17h078bf56349dca6e5E+98>
   0x08000298 <+124>:   bl      0x80002b8 <_ZN4core3ptr50drop_in_place$LT$critical_section..with..Guard$GT$17h1b6999d929935fd6E>
   0x0800029c <+128>:   movw    r0, #8432       @ 0x20f0
   0x080002a0 <+132>:   movt    r0, #2048       @ 0x800
   0x080002a4 <+136>:   bl      0x80019dc <_RNvNtCs2yasv005KOe_4core6option13unwrap_failed>
   0x080002a8 <+140>:   msr     PRIMASK, r0
   0x080002ac <+144>:   movw    r0, #8416       @ 0x20e0
   0x080002b0 <+148>:   movt    r0, #2048       @ 0x800
   0x080002b4 <+152>:   bl      0x80019dc <_RNvNtCs2yasv005KOe_4core6option13unwrap_failed>
End of assembler dump.

That's the whole loop: set up the LED, set up the clocks, then delay 500 ms and toggle, round and round. counter simply isn't there.

This is stronger than inlining, and the precise name matters: it's dead-code elimination. Nothing in the program ever reads counter back, no volatile access, no log, no side effect the compiler must preserve, so LLVM proves the increment has no observable effect and removes it outright, function and all. Inlining keeps the arithmetic and drops the function boundary; this dropped the arithmetic too.

That distinction changes how you debug it. Build a toy example specifically to watch a value move in a debugger, give that value no other effect, and the release build may decide there's nothing to compute at all; you end up hunting a variable that doesn't exist rather than one that's merely hard to reach. If you need a value to survive optimisation for inspection, give it an effect the compiler can't dismiss: a volatile write, a defmt log, or a genuine use of the result somewhere visible.

Variables that do survive can still shift under you in a gentler way. In a debug build the debugger has one clear place to recover a value from; in an optimised build it might live in a register for part of its life, move, fuse with another computation, or simply have no recoverable location at a given instruction. There, a debugger reports it as <optimized out> even with full debug information present:

Debug information can describe the generated program; it can't resurrect a runtime object optimisation deleted. Our counter took the extreme version of that road, never present in the compiled program rather than merely <optimized out> at one point.

The same forces make stepping look odd: several source statements collapsing into one instruction sequence, a function inlined into its caller, an operation moved because its original position stopped mattering. That's not the debugger failing; it's what representing optimised execution through the shape of the original source looks like. When the mapping gets hard to read, drop to the instruction view, where disassembly shows exactly what survived.

Our #[inline(never)] in the first session was a teaching aid, not general advice. Blocking inlining changes code size, timing, register allocation, and stack use, so if a failure only shows up in optimised firmware, disabling optimisation to make debugging easier can move the failure itself. That's the reasoning behind a rule we'll keep:

Turning optimisation off is still a fair experiment; just treat a failure that then vanishes as evidence, not proof, that optimisation is to blame. So we keep debug information in the builds we investigate, optimised ones included.

Not every failure is a crash

One distinction is worth naming before we finish, because the rest of the series leans on it. "Crash" is a single word for at least three different situations:

A panic is Rust deciding it can't safely continue, an unwrap() on None, a failed bounds check, and handing off to a panic handler. A processor fault is lower still: the Cortex-M itself judges an instruction unexecutable and raises an exception like HardFault or BusFault. The third case is usually the most maddening, because nothing is technically wrong: the CPU runs perfectly valid instructions, just to no useful end, stuck in a loop waiting on something that never comes, or an interrupt that re-fires forever. From outside, all three look the same, a dark LED, a silent serial port. A later article takes fault status registers apart to reconstruct exactly what a HardFault was; for now the habit is enough, ask where the program counter actually is before deciding which of the three you're facing.

A method that holds up

The tools and views changed through this article; the underlying process didn't:

Order is the whole game. An LED blinking too fast tempts you to tweak the delay constant, which hides the symptom without explaining it; a frozen board tempts you to assume a HardFault, which can point the investigation the wrong way. Start instead from what you can observe. Is the CPU running? Where's the program counter? What's actually in memory? Decide what the failure means only once those have produced evidence.

Source is usually the fastest starting point, because it states intent plainly. When source and observed behaviour part ways, you descend, only as far as the evidence demands:

Most bugs never need the bottom of that stack. The failures ahead get harder; the process stays the same.

So what actually changes from C?

If you can already debug embedded C or C++, most of that carries straight over. The Cortex-M hasn't changed: registers, a stack, a program counter, memory-mapped peripherals, exception state, hardware breakpoints and watchpoints, an SWD interface, and instructions you can disassemble and inspect. Where is the processor executing? What's in this register? Did this peripheral register get the write I expected? None of those are C questions or Rust questions; they're embedded-system questions.

What changes sits above the machine, in the source-level view. Rust brings representations that read oddly at first, generics monomorphised into specialised copies, functions that vanish into their callers, slices carrying a length beside their pointer, Option and Result in compact layouts, and it keeps panics as a failure path distinct from a processor fault. The tooling shifts too: GDB/OpenOCD and probe-rsdefmt are different doors into the same system, not rival models of it.

Rust hands you a much richer picture of the program, around ownership, types, and peripheral APIs especially, without replacing the machine beneath. Use the high-level view while it's enough. When it isn't, drop through the layers, which is precisely the move experienced embedded engineers already make the moment C source stops explaining what the processor is doing.

Where we go next

Across these two articles we've built the model of what a Rust build produces and put it to work live: a breakpoint, registers, memory, and a look at what optimisation removes.

Every one of those inspections, though, started by stopping the processor. That's fine for a counter going round a loop. It's much less fine the moment a bug depends on timing, where halting the CPU to look can make the symptom disappear on the spot.

That's exactly where the next article goes. We'll chase a real bug, the NUCLEO's LED blinking at the wrong rate, first the familiar way, by breaking in and checking state against what the source assumes. Then we'll hit a case where breaking in isn't good enough, and reach for RTT and defmt to watch the system while it keeps running instead of stopping it to look.

• Article 1: What Changes from C?

• Article 3: Two Ways to Watch a Running System

This article was written by Dr David Cabanis, principal member technical staff at Doulos and author of the book Embedded Software with Rust.