TamgaOS (yula)
RSS github

— Registers & Assembly Reference —

Quick Instruction Reference

On this post will be a little longer I believe — the Registers logic is finally okay — so I'm splitting it into sections:
1) LIFECYCLE OF RTOS,
2) REGISTERS,
3) ASSEMBLY,
4) DEBUG PROCESS.

RTOS Lifecycle & Stack Pointer Setup

When pushing context to the stack, the hardware saves eight 32-bit words: xPSR, ReturnAddress, LR (R14), R12, R3, R2, R1, and R0 — ref B1-643. If the processor implements the Floating-point extension (not implemented yet :) ), it can also push FP state or reserve space for it. See Stack alignment on exception entry, page B1-647.

MSP vs PSP

Normally MCU runs on MSP (Main Stack), but for real-time support every task needs its own special stack — so we MUST use PSP. This doesn't damage the MCU run cycle. All critical parts for MCU still run on MSP, so if any task breaks, only that task is affected; the MCU will not go HardFault etc.

For using PSP we need to set the SPSEL bit from the CONTROL Register. When MCU/board resets, SP_Main starts by default.

ModeStack
Handler mode (always)MSP — Reset_Handler, SysTick_Handler, PendSV_Handler, HardFault_Handler, other IRQs
TasksPSP :)

PSP isolation does not prevent HardFault completely, but gives the system a recovery chance. If a task corrupts its own stack, the HardFault handler still runs on MSP safely. Inside the handler we can read IPSR to identify which task caused the fault, then mark that task as TASK_PAUSED and trigger a context switch to continue running other tasks without a full system reset.

This is exactly why RTOS tasks must use PSP, not MSP. MSP is the last line of defense.

CONTROL Register

register layout
31      3  2      1      0
┌──────────┬──────┬──────┬──────┐
│ reserved │ FPCA │SPSEL │nPRIV │
│   0...0  │  0   │  1   │  0   │
└──────────┴──┴───┴──────┴──────┘
BitNameDescription
nPRIVPrivilegeIf 0, MCU runs on privileged mode. We don't want unprivileged — msr, cpsid etc. won't work without privilege.
SPSELStack SelectMSP or PSP. If 0, MSP — starts as MSP after reset.
FPCAFP ActiveFloating point bit. Must be 1 if using float numbers. (Currently 0, will implement later...)

PRIMASK / FAULTMASK / BASEPRI

PRIMASK

register layout
31                  1       0
┌───────────────────┬──────┐
│                   │  PM  │
│                   │      │
└───────────────────┴──────┘

The exception mask register, a 1-bit register. Setting PRIMASK to 1 raises the execution priority to 0.

CPSIE i is equivalent to writing 0 into PRIMASK. CPSID i is equivalent to writing 1 into PRIMASK. — ARM ref manual B5-802.

FAULTMASK

register layout
31                  1       0
┌───────────────────┬──────┐
│                   │  FM  │
│                   │      │
└───────────────────┴──────┘

FM=1 masks all exceptions and faults except NMI. Difference from PRIMASK: it also blocks fault handlers.

BASEPRI

register layout
31                  7       0
┌─────────────────────────────┐
│                  │ basepri  │
│                  │          │
└──────────────────┴──────────┘

Masks interrupts below a certain priority level. Entering 0 disables it; entering 1–255 blocks interrupts with priorities below that value.

xPSR — Combined Status Register

xPSR is actually the sum of 3 registers: APSR, IPSR, and EPSR — all together called xPSR. You can read it as a single 32-bit register.

xPSR layout
31 30 29 28 27    24    15       8  7      0
┌──┬──┬──┬──┬──┬────┬───────────┬──────────┐
│N │Z │C │V │Q │ T  │ reserved  │ ISR_NUM  │
└──┴──┴──┴──┴──┴────┴───────────┴──────────┘
 ▲  ▲  ▲  ▲  ▲   ▲                   ▲
 │  │  │  │  │   │                   │
 │  │  │  │  │   EPSR            IPSR
 └──┴──┴──┴──┘
    APSR
FieldRegisterDescription
N Negative FlagAPSR1 if result is negative
Z Zero FlagAPSR1 if result is zero
C Carry FlagAPSR1 if carry
V Overflow FlagAPSR1 if overflow
Q Saturation FlagAPSR1 if DSP overflow
T Thumb bitEPSROn ARM, always 1!
ISR_NUMIPSR0 in thread mode, exception number in handler mode

On reset, the processor is in Thread mode and IPSR Exception Number is cleared to 0. Value 1 (reset exception) is transitory — software cannot see it as a valid IPSR exception number. (Page B1-625)

For programming: to set initial xPSR we only need to set the Thumb bit → 0x01000000UL (bit 24). All else zero.

SCS — System Control Space

memory map 0xE000E000 – 0xE000EFFF
┌─────────────────────────────────────────┐
│  SCS (0xE000E000)                       │
│  ├── SysTick   (0xE000E010)             │
│  ├── NVIC      (0xE000E100)             │
│  ├── SCB       (0xE000ED00) <-  we use  │
│  │     ├── ICSR    0xE000ED04           │
│  │     ├── VTOR    0xE000ED08           │
│  │     ├── AIRCR   0xE000ED0C           │
│  │     └── SHPR3   0xE000ED20           │
│  └── MPU       (0xE000ED90)             │
└─────────────────────────────────────────┘

Memory-mapped registers. No MRS/MSR needed — direct pointer access.

SCB Registers — ICSR, VTOR, AIRCR, SHPR3

ICSR — Interrupt Control and State Register

Provides software control of NMI, PendSV, and SysTick exceptions, and provides interrupt status information.

ICSR layout
31      29 28 27      23 22      19 18    16 15    12 11     9 8       0
┌─────────┬──┬──────────┬──────────┬────────┬────────┬──────────┬─────────────┐
│         │PS│          │  VECTPENDING      │reserved│ RETTOBASE│ VECTACTIVE  │
│         │ET│          │  [20:12] │        │        │          │  [8:0]      │
└─────────┴──┴──────────┴──────────┴────────┴────────┴──────────┴─────────────┘
           ▲                  ▲                          ▲         ▲
           │                  │                          │         │
        PENDSVSET          waiting                  any            active
        (bit28)            exception                exception      exception
                           number
BitNameDescription
bit[28]PENDSVSETWrite 1 → make PendSV pending. Read → current pending state. Normally software writes 1 here to request a context switch.
bit[27]PENDSVCLRWrite 1 → remove PendSV pending status.
bit[26]PENDSTSETWrite 1 → make SysTick pending. Read → current SysTick pending state.
bit[25]PENDSTCLRWrite 1 → remove SysTick pending status.
bit[22:12]VECTPENDINGException number of highest priority pending exception. 0 = no pending exception.
bit[11]RETTOBASEIn Handler mode: 1 = no active exception other than current IPSR, 0 = other active exception exists.
bit[8:0]VECTACTIVEException number of current executing exception. 0 = Thread mode.

VTOR — Vector Table Offset Register

VTOR layout
31          7 6          0
┌────────────┬────────────┐
│   TBLOFF   │            │
│   [31:7]   │            │
└────────────┴────────────┘

Holds the vector table address. When an interrupt arrives, ICSR and VTOR work together: ICSR gives VECTPENDING (which exception?), VTOR gives the table base, and VTOR + (exception_no × 4) gives the handler address to jump to.

vector table offsets
VTOR + 0         → SP_main (reset value)
VTOR + (1  × 4)  → Reset_Handler
VTOR + (14 × 4)  → PendSV_Handler
VTOR + (15 × 4)  → SysTick_Handler

AIRCR — Application Interrupt and Reset Control Register

Not implemented yet. Working on software reset — might need this for a bootloader in a future issue.

AIRCR layout
31          16 15  11 10     8  7  3  2  1  0
┌─────────────┬──────┬────────┬──────┬────────────────────────────────────
│  VECTKEY    │rez   │PRIGROUP│rez   │SYSRESETREQ│VECTCLRACTIVE│VECTRESET│
│  [31:16]    │      │[10:8]  │      │           │             │         │
└─────────────┴──────┴────────┴──────┴───────────┴─────────────┴─────────┘
BitNameDescription
bit[31:16]VECTKEYMust write 0x05FA when writing!
bit[10:8]PRIGROUPPriority grouping
bit[2]SYSRESETREQWrite 1 → system reset
bit[1]VECTCLRACTIVEDebug mode — clear active exception
bit[0]VECTRESETMCU core reset only

SHPR3 — System Handler Priority Register 3

SHPR has 3 registers (SHPR1, SHPR2, SHPR3). We use SHPR3 for PendSV and SysTick: SHPR3[14] = PendSV priority, SHPR3[15] = SysTick priority.

SHPR3 layout
31      24 23      16 15       8 7        0
┌─────────┬──────────┬──────────┬──────────┐
│ PRI_15  │  PRI_14  │  rezerve │  rezerve │
│ SysTick │  PendSV  │          │          │
└─────────┴──────────┴──────────┴──────────┘
 0xE000ED23 0xE000ED22

We give both 0xFF — the minimum priority. If SysTick had higher priority than PendSV, SysTick could preempt PendSV during a context switch and corrupt it. Same priority prevents this.

Vector Table Alignment

The vector table must be naturally aligned to a power of two whose value is ≥ (Number of Exceptions × 4), with a minimum of 128 bytes. On power-on or reset, the processor uses entry at offset 0 as the initial value for SP_main. All other entries must have bit[0] set to 1, because this bit defines EPSR.T on exception entry. (Directly from reference manual :) )

From TamgaOs.elf biber output — we can see the align part:

biber — section headers
 Nr   Name                 Type         Address             Offset     Size       Flg    Lnk    Info   Align
  ----------------------------------------------------------------------------------------------------------
  0                         NULL         0x0000000000000000   0x000000  0x000000           0      0      0
  1    .vectors             PROGBITS     0x0000000000000000   0x010000  0x000190  A        0      0      4
  2    .flashconfig         PROGBITS     0x0000000000000400   0x010400  0x000010  A        0      0      4
  3    .text                PROGBITS     0x0000000000000410   0x010410  0x000A4C  AX       0      0      4
  4    .data                PROGBITS     0x0000000020000000   0x020000  0x000000  WA       0      0      1
  5    .bss                 NOBITS       0x0000000020000000   0x020000  0x0049C4  WA       0      0      4
  6    .noinit              NOBITS       0x00000000200049C4   0x000000  0x000000  WA       0      0      1
  7    .ARM.attributes      ARM_ATTR     0x0000000000000000   0x020000  0x00002F           0      0      1
  8    .comment             PROGBITS     0x0000000000000000   0x02002F  0x000049  MS       0      0      1
  9    .debug_line          PROGBITS     0x0000000000000000   0x020078  0x00013F           0      0      1
  10   .debug_info          PROGBITS     0x0000000000000000   0x0201B7  0x000072           0      0      1
  11   .debug_abbrev        PROGBITS     0x0000000000000000   0x020229  0x00003C           0      0      1
  12   .debug_aranges       PROGBITS     0x0000000000000000   0x020268  0x000060           0      0      8
  13   .debug_str           PROGBITS     0x0000000000000000   0x0202C8  0x00009F  MS       0      0      1
  14   .symtab              SYMTAB       0x0000000000000000   0x020368  0x000AE0           15     130    4
  15   .strtab              STRTAB       0x0000000000000000   0x020E48  0x0003A5           0      0      1
  16   .shstrtab            STRTAB       0x0000000000000000   0x0211ED  0x0000A3           0      0      1

.noinit section already added to linker.ld for holding debug/crash data. Still not fully implemented — only a starting point :)

Assembly Instructions — Detailed

ARM uses UAL (Unified Assembly Language). We define .syntax unified at the start. See PART A4.2.2 in the reference manual — IMPORTANT!

LDR — Load from memory to register

LDR syntax
LDR Rd, [Rn, #offset]
      ▲    ▲    ▲
      │    │    └── how many bytes forward
      │    └─────── base register
      └──────────── result register
example from my code
; r0 = task_t* pointer
ldr r1, [r0, #0]   ; r1 = task->sp  (offset 0 = sp field)
ldr r2, [r1, #24]  ; r2 = PC from stack frame (6 × 4 bytes forward)

; Equivalent C:
; r1 = *((uint32_t*)r0);
; r2 = *((uint32_t*)(r1 + 24));

LDMIA — Load Multiple Increment After

LDMIA syntax
LDMIA Rn!, {register list}
        ▲
        └── ! means Rn is automatically updated (writeback)
example
ldmia r1!, {r4-r11}

; starting from r1 address:
r4  ← [r1+0]
r5  ← [r1+4]
r6  ← [r1+8]
r7  ← [r1+12]
r8  ← [r1+16]
r9  ← [r1+20]
r10 ← [r1+24]
r11 ← [r1+28]
r1  = r1 + 32   ; ← because of ! automatically updated!

STR — Store Register to memory

STR syntax
STR Rd, [Rn, #offset]
     ▲    ▲    ▲
     │    │    └── how many bytes forward
     │    └─────── base address
     └──────────── value to write to memory

str r0, [r1, #-4]!  ; first decrement r1, then write!

MOV — Write constant to register

MOV syntax
MOV Rd, #value
     ▲    ▲
     │    └── constant (immediate)
     └─────── destination register

mov r0, #2  ; write 2 to r0

MRS / MSR — Move from/to Special Register

Note: These instructions will not work in Thread mode (they are ignored). They MUST run in Privileged mode. CONTROL[nPRIV] must be 0!

MRS / MSR examples
MRS Rd, special   ; read special → Rd
MSR special, Rd   ; write Rd → special

msr psp, r1       ; r1 → PSP  (set task's stack pointer)

mov r0, #2
msr control, r0   ; SPSEL=1 → switch to PSP

; Special register requires MRS/MSR — MOV won't work:
mov  r0, psp   ; ← WRONG
mrs  r0, psp   ; ← correct

mov  psp, r1   ; ← WRONG
msr  psp, r1   ; ← correct

ALIGNMENT — B1.5.7 Page B1-647 — see reference manual for full alignment rules on exception entry stack frames.

— Debug Log —

June 2026 TamgaOS Debug Log GDB Biber Cortex-M4

Part 6.1 — Debug Log
13 Bugs !!! Between Theory and a Blinking LED

Part 6 covered the theory — PendSV, stack frames, EXC_RETURN. This post covers what actually happened when I tried to run it on real hardware. Six distinct root causes, several hours of GDB sessions, and one blinking LED at the end.

The symptom was always the same: OpenSDA's red LED lit up, meaning the board had crashed. The causes were completely different each time. This is the log.

— Debug Timeline —

How it unfolded

Start: bare-metal blink works, scheduler crashes immediately
LED + UART worked perfectly without the scheduler. Adding scheduler.c to the build caused an instant HardFault before main() even printed anything.
Discovery: two separate RAM regions on K64F
The linker script had one contiguous RAM region. K64F actually has two: 0x1FFF0000 (64 KB) and 0x20000000 (192 KB). BSS spanning the gap caused the startup zero-loop to write into an invalid address range.
Discovery: vector table was 84 entries, should be 100
The hand-written startup had the wrong count of IRQ entries. PIT0 is IRQ48 — slot 64 in the table — but the table only had 84 words. PIT0's IRQ was landing on whatever garbage was at that offset.
Discovery: BSS not zeroed → g_current_task garbage on boot
Startup had a BSS zero-loop but it silently failed (signed vs unsigned address comparison bug). g_current_task started with a non-NULL garbage value, making the != NULL guard in sched_tick() useless.
Discovery: PendSV triggered with PSP = 0x0
PIT IRQ fired before sched_start_asm finished setting PSP. PendSV ran, executed mrs r0, psp, got 0x0, tried to stmdb r0!, {r4-r11} — immediate BusFault.
Discovery: sched_start assembly — bx lr with 0xFFFFFFFD from non-exception context
The assembly used bx lr with EXC_RETURN value 0xFFFFFFFD from a normal function context. Only valid inside an exception handler. CPU treated it as an invalid branch and faulted.
Discovery: CONTROL register read in IRQ always returns 0
Added a CONTROL & 0x02 guard in sched_tick() to only trigger PendSV when PSP is active. But PIT0_IRQHandler runs in Handler mode — CONTROL[1] is always 0 there. The guard permanently blocked PendSV.
Fix: interrupts masked during sched_start_asm, g_current_task zeroed in sched_init
cpsid i at the start of sched_start_asm prevents PIT from firing mid-transition. Explicitly zeroing g_current_task in sched_init() makes the guard reliable regardless of BSS state.
LED blinks. UART prints. Two tasks running. Scheduler works.

— Root Causes, One by One —

Bug 1 — Two RAM regions, one linker script

Bug #1BSS spans a physical gap in RAM

The K64F has two physically separate SRAM banks. Most linker examples you'll find online treat it as one contiguous block — which works for small programs that don't reach the boundary. Once s_tasks[TASK_MAX] pushed BSS past 64 KB, we were writing into the gap between the two banks.

The startup BSS zero-loop would appear to succeed (no fault, loop exits), but the data at addresses above 0x20000000 was never actually zeroed — it was reading back whatever the power-on reset left there.

Fix Split the MEMORY region in linker.ld into two entries matching the actual silicon:

m_data (rwx) : ORIGIN = 0x1FFF0000, LENGTH = 64K
m_data_2 (rwx) : ORIGIN = 0x20000000, LENGTH = 192K

Place BSS, data, and stack all in m_data_2. The gap disappears.
linker.ld — before vs after
/* BEFORE — one region, spans the physical gap */
MEMORY {
  RAM (rwx) : ORIGIN = 0x1FFF0000, LENGTH = 256K
}

/* AFTER — two regions matching the silicon */
MEMORY {
  FLASH    (rx)  : ORIGIN = 0x00000000, LENGTH = 1024K
  m_data   (rwx) : ORIGIN = 0x1FFF0000, LENGTH = 64K
  m_data_2 (rwx) : ORIGIN = 0x20000000, LENGTH = 192K
}

_stack_top = ORIGIN(m_data_2) + LENGTH(m_data_2);

Bug 2 — Vector table had 84 entries instead of 100

Bug #2PIT0 IRQ landed on garbage

The hand-written startup assembly vector table stopped at IRQ67. K64F's PIT0 is IRQ48, which is slot 64 in the table (16 core exceptions + 48 IRQs). With only 84 words in the table (16 core + 68 IRQs), the slot was there — but IRQ68 through IRQ83 were missing entirely.

The map file was the giveaway: .vectors 0x00000000 0x150. That's 336 bytes = 84 words. It should be 0x190 = 400 bytes = 100 words.

Fix Add the missing 16 .word Default_Handler entries for IRQ68–IRQ83. Verify in the map file: .vectors 0x00000000 0x190.

Bug 3 — BSS zero-loop silently failed

Bug #3Signed comparison with RAM addresses ≥ 0x20000000

The startup BSS loop used blt (Branch if Less Than — signed) to compare two RAM addresses. Addresses above 0x80000000 are negative in two's complement. 0x1FFF0000 and 0x20000000 are both positive, so the loop ran. But depending on the exact BSS layout, the loop would sometimes exit immediately or run indefinitely — impossible to reproduce reliably.

The real fix was moving BSS into m_data_2 (Bug 1), but additionally: g_current_task and g_next_task are now explicitly zeroed inside sched_init(), making the scheduler correct regardless of what the startup zero-loop does or doesn't do.

Fix Zero g_current_task and g_next_task explicitly in sched_init(). Never rely on BSS being implicitly zero on a bare-metal system without verifying the startup loop is correct.
scheduler.c — sched_init
void sched_init(void) {
    g_current_task = (task_t *)0U;   /* explicit zero — don't trust BSS */
    g_next_task    = (task_t *)0U;
    s_task_count   = 0U;
    SCB_SHP_PENDSV  = 0xFFU;
    SCB_SHP_SYSTICK = 0xFFU;
    // ... idle task init ...
}

Bug 4 — PendSV triggered before PSP was set

Bug #4PIT0 IRQ fires mid-transition, PendSV runs with PSP = 0x0

The sequence in sched_start_asm was: set PSP → set CONTROL → jump to task. But PIT0 was already running with a 1 ms period. Between setting CONTROL and the task actually starting, a PIT interrupt could fire. sched_tick() would call trigger_pendsv(). PendSV would run, read PSP (still partially set up or 0x0), and fault immediately.

Fix Wrap the PSP transition with cpsid i / cpsie i — disable interrupts for the duration of the critical setup, re-enable once the first task is safely running.
sched_start.s — interrupt-guarded PSP transition
sched_start_asm:
    cpsid   i               ; mask all interrupts

    ldr     r1, [r0, #0]    ; r1 = task->sp
    ldmia   r1!, {r4-r11}   ; restore software frame

    msr     psp, r1         ; PSP = hardware frame base
    isb

    mov     r0, #0x02
    msr     control, r0     ; Thread mode, PSP
    isb

    cpsie   i               ; interrupts back on — PSP is now valid
    isb

    ldr     r2, [r1, #24]   ; PC from hardware frame (offset 6*4)
    bx      r2              ; jump to task entry point

Bug 5 — EXC_RETURN from a non-exception context

Bug #5bx lr with 0xFFFFFFFD outside an exception handler

Early versions of sched_start_asm ended with movw lr, #0xFFFD / movt lr, #0xFFFF / bx lr. That EXC_RETURN value is only meaningful when executed from inside an exception handler. From a normal function call, the CPU doesn't recognize it as EXC_RETURN — it treats it as a branch to address 0xFFFFFFFD, which is in the reserved address space. Immediate fault.

Several workarounds were tried — SVC to get into exception context, naked functions — before landing on the simplest solution: jump directly to the task's PC and let the hardware frame do its job naturally when the task eventually returns to the OS (which it never does in a while(1), but the frame is still correct for PendSV context switching).

Fix Extract the PC from the hardware frame and jump to it directly with bx r2. PSP is already pointing at the frame, so PendSV can save/restore correctly on the next context switch.

Bug 6 — CONTROL register always reads 0 inside an IRQ

Bug #6PSP-mode guard permanently blocked PendSV

To prevent PendSV from firing when PSP wasn't set up yet, a guard was added to sched_tick():

the broken guard
uint32_t ctrl;
__asm volatile("mrs %0, control" : "=r"(ctrl));
if (ctrl & 0x02U) {   /* always 0 in Handler mode! */
    trigger_pendsv();
}

This looks reasonable — only trigger PendSV when PSP is active (CONTROL[1] = 1). But PIT0_IRQHandler runs in Handler mode. In Handler mode, the CPU always uses MSP and CONTROL[1] is always 0. The guard was permanently false. PendSV was never triggered after the first task started.

GDB caught this through a different symptom: both tasks showed state = TASK_READY and delay_ticks = 0, meaning they were ready to run but never getting scheduled. The idle task was the one running instead.

Fix Remove the CONTROL guard entirely. The real protection is the explicit g_current_task = NULL in sched_init() plus the cpsid/cpsie in sched_start_asm. Once the first task is running, PendSV can always be triggered safely.

— GDB Sessions —

Session 1 — Catching the HardFault location

The board crashed before printing anything. First step was confirming the crash was a HardFault and finding where it happened.

GDB — pyocd gdbserver -t k64f --persist
Remote debugging using localhost:3333 Default_Handler () at boards/k64f/startup_k64f.s:128 128 b Default_Handler (gdb) monitor reset halt Successfully halted device on reset (gdb) break HardFault_Handler (gdb) continue Breakpoint 1, HardFault_Handler ... (gdb) info registers lr 0xfffffff1 ← Handler mode EXC_RETURN (nested fault!) xpsr 0x1000003 ← EXC = 3 = HardFault psp 0x0 ← PSP was never set

The lr = 0xfffffff1 meant a fault-within-a-fault — the CPU tried to handle a fault and faulted again. Combined with psp = 0x0, the picture was clear: PendSV ran before PSP was initialized.

Session 2 — task_led never actually running

After fixing the vector table and linker script, the board stopped crashing immediately, but the LED never blinked. Breakpointing task_led to confirm it was being reached:

GDB — tracing sched_start → task_led
(gdb) break sched_start (gdb) continue Breakpoint 1, sched_start () at kernel/core/scheduler.c:139 (gdb) print g_current_task $1 = (task_t * volatile) 0x20000008 <s_tasks> (gdb) print g_current_task->func $2 = (void (*)(void)) 0xc65 <task_led> ← correct (gdb) break task_led (gdb) continue Breakpoint 2, task_led () at src/main.c:17 (gdb) stepi Program received signal SIGSEGV, Segmentation fault. Default_Handler () ← crashed on first instruction of task_led (gdb) x/1wx 0x400FF050 Cannot access memory at address 0x400FF050 ← GPIO clock not enabled

The memory fault on GPIO access was a red herring — the real cause was PendSV firing between sched_start_asm setting CONTROL and the task actually running, with PSP still partially initialized.

Session 3 — confirming PSP = 0x0

After adding the cpsid/cpsie guard in sched_start_asm and zeroing g_current_task in sched_init(), checking the register state at the moment of crash:

GDB — register state on crash
(gdb) info registers psp control psp 0x20000a54 ← s_idle_tcb+520 — correct, PSP is now valid control 0x2 ← Thread/PSP mode — correct (gdb) print s_tasks[0].sp $1 = (uint32_t *) 0x20000190 <s_tasks+392> (gdb) print s_tasks[0].state $2 = TASK_READY ; task is READY but never gets CPU time — Bug 6

Session 4 — idle task running instead of scheduled tasks

UART was printing, LED was not blinking. Connected GDB mid-run — caught the idle task running with both real tasks READY:

GDB — mid-run state inspection
0x0000046e in idle_task_func () at kernel/core/scheduler.c:19 19 while (1) { __asm volatile("wfi"); } (gdb) print g_current_task $1 = (task_t *) 0x2000084c <s_idle_tcb> ← idle is running (gdb) print s_tasks[0].state $2 = TASK_READY ← task_led is ready but never scheduled (gdb) print s_tasks[0].delay_ticks $3 = 0 ← no delay pending ; Root cause: CONTROL guard in sched_tick() always reads 0 in Handler mode ; PendSV was never being triggered — idle task ran forever

This was the last bug. Removing the CONTROL & 0x02U guard from sched_tick() let PendSV fire normally, and the scheduler started switching tasks correctly.

— Final State —

What works now

✓ Two tasks running concurrently — task_led (500 ms blink) and task_uart (1 s UART print)
✓ PendSV context switch — PSP-based, R4-R11 saved/restored by software
✓ sched_delay_ms — tasks block and yield correctly
✓ Idle task — runs when all tasks are blocked, WFI for power saving
✓ Priority scheduler — lowest number wins, round-robin among equal priorities

ComponentFileStatus
Linker scriptboards/k64f/linker.ldFixed — two RAM regions
Vector tableboards/k64f/startup_k64f.sFixed — 100 entries
Scheduler corekernel/core/scheduler.cWorking
Context switchkernel/arch/cortex_m4/pendsv_handler.sWorking
First task launchkernel/arch/cortex_m4/sched_start.sWorking — cpsid/cpsie guarded
PIT driverdrivers/k64f/pit.cWorking — calls sched_tick()
Mutex / semaphoreImplemented, testing in progress
MCG 120 MHzdrivers/k64f/mcg.cFixed — HGO0 removed, flash wait state added
PIT driverdrivers/k64f/pit.cFixed — 4x NOP clock gate barrier
Critical sectioninclude/sched_critical.hFixed — BASEPRI replaced with cpsid/cpsie
Idle TCB stackkernel/core/scheduler.cFixed — declaration order corrected

— Part 6.2: Getting sched_delay_ms to actually block —

Bug 7 — HGO0 flag: oscillator would not start

BUG 07MCG->C2 had HGO0_MASK set — crystal never locked

After the scheduler bugs were resolved, mcg_init_120mhz() was next. The MCG C2 register was written with MCG_C2_RANGE0(2) | MCG_C2_HGO0_MASK | MCG_C2_EREFS0_MASK. The HGO0 bit selects high-gain oscillator mode for crystals that need it. The 50 MHz clock source on this board does not — it drives the MCG external reference directly. With HGO0 set, the oscillator simply never started and the PLL lock loop spun forever, hanging the system before main() printed anything.

The symptom was identical to an earlier hang: board appeared dead, no UART output, LED did not respond. Isolated by removing mcg_init_120mhz() and testing with a bare GPIO toggle loop — which worked fine. Then re-added MCG piece by piece. The hang disappeared when HGO0 was removed.

FIXRemove MCG_C2_HGO0_MASK from the C2 write.
MCG->C2 = MCG_C2_RANGE0(2U) | MCG_C2_EREFS0_MASK;

Bug 8 — Flash wait state not set before clock switch

BUG 08SIM->CLKDIV1 written before FMC wait state — corrupt instruction fetch

mcg_init_120mhz() set SIM->CLKDIV1 (core /1, bus /2, flash /5) as its very first action. At that moment the CPU was still running on FEI (~20 MHz) with zero wait states. After CLKDIV1 was written the flash controller was immediately being asked to serve instructions at 120 MHz — but with zero wait states configured it can only reliably do ~25 MHz. The result was corrupt instruction fetches: random HardFaults, or code that appeared to run but produced wrong results.

K64F reference manual §3.8.2: flash requires RWSC = 5 for 120 MHz operation. FMC->PFB01CR bits [19:16] must be set before raising the clock.

FIXWrite FMC_PFB01CR RWSC = 5 before SIM->CLKDIV1.
FMC_PFB01CR = (FMC_PFB01CR & ~(0xFU<<16U)) | (5U<<16U);
Also increased crystal stabilisation delay from 10 000 to 50 000 NOPs (~2.5 ms).

Bug 9 — sched_start_asm: bx r0 instead of EXC_RETURN

BUG 09sched_start_asm used bx lr with EXC_RETURN from non-exception context

The original sched_start_asm called bx lr with lr = 0xFFFFFFFD (Thread/PSP EXC_RETURN). This is only valid inside an exception handler. Called from sched_start() as a normal C function, the CPU was in Thread mode. Executing bx 0xFFFFFFFD from Thread mode is an invalid branch to reserved memory: instant UsageFault.

The fix was to read PC directly from the task hardware frame at PSP+24 and branch with bx r0. CONTROL.SPSEL is set to 1 first so the CPU switches to PSP. The hardware frame sits untouched at PSP — when PendSV context-switches away and back, it restores the rest of the frame normally via EXC_RETURN.

FIXAfter ldmia r1!, {r4-r11} and msr psp, r1:
ldr r0, [r1, #24] // PC from hardware frame
cpsie i
bx r0 // branch directly, not EXC_RETURN

Bug 10 — LR initialised to 0xFFFFFFFE in task stack frame

BUG 10task_stack_init wrote 0xFFFFFFFE as LR — reserved EXC_RETURN value

Cortex-M4 EXC_RETURN values: 0xFFFFFFF1 (Handler/MSP), 0xFFFFFFF9 (Thread/MSP), 0xFFFFFFFD (Thread/PSP). Everything else in the 0xFFFFFFF_ range is reserved. task_stack_init wrote 0xFFFFFFFEUL as the initial LR. Caught in disassembly: mvn r2, #1 = NOT(1) = 0xFFFFFFFE. When PendSV returned via this LR, the CPU took a UsageFault.

FIXChange initial LR in task_stack_init:
*(--stack_top) = 0xFFFFFFFDUL; // Thread/PSP EXC_RETURN
Confirmed in disassembly: mvn r2, #2 = NOT(2) = 0xFFFFFFFD.

Bug 11 — idle TCB stack overwriting s_task_count

BUG 11s_idle_tcb declared before s_tasks — stack grew down into s_task_count

Static variable layout in BSS follows declaration order. scheduler.c had: s_idle_tcb at 0x20000008, then s_task_count at 0x20000818, then s_tasks at 0x2000081C. A full task_t is 2064 bytes (0x810), so &s_idle_tcb.stack[512] = 0x20000818 — exactly s_task_count. task_stack_init writes the initial frame downward from that address, directly clobbering s_task_count and the start of s_tasks.

GDB caught it: psp = 0x20000818 <s_task_count> — the CPU was running the idle task with its stack pointer sitting on top of the task count variable.

FIXReorder static declarations so idle TCB comes last:
static uint8_t s_task_count;
static task_t s_tasks[TASK_MAX];
static task_t s_idle_tcb; // stack grows into free space below

Bug 12 — SIM clock gating: PIT->MCR write silently dropped

BUG 12PIT->MCR written too soon after SIM->SCGC6 — peripheral not yet clocked

After SIM->SCGC6 |= SIM_SCGC6_PIT_MASK, pit_init immediately wrote PIT->MCR = 0x01. The write was silently discarded — the peripheral bus clock takes a few cycles to propagate after the clock gate opens. MCR remained at its reset value of 0x02 (MDIS = 1, module disabled). With MDIS set, PIT counters do not run and no interrupts fire.

This was invisible in debug builds: the debugger halt/resume overhead added enough cycles that the clock was always ready. In release builds at full speed the write was lost every time. Confirmed with pyocd: read32 0x40037000 → 0x00000002. After adding four NOPs: read32 0x40037000 → 0x00000001.

FIXAdd a short barrier after the clock gate write:
SIM->SCGC6 |= SIM_SCGC6_PIT_MASK;
__asm volatile("nop"); __asm volatile("nop");
__asm volatile("nop"); __asm volatile("nop");
PIT->MCR = 0x00000001UL;

Bug 13 — BASEPRI masking PendSV indefinitely

BUG 13SCHED_MAX_SYSCALL_PRIO blocked PendSV and sometimes PIT0

sched_critical_enter() wrote BASEPRI = 0x30 (priority level 3). BASEPRI blocks all exceptions with numeric priority ≥ that value. PIT0 was at priority 4 (0x40) and PendSV at 15 (0xFF) — both blocked. The critical section exited correctly, but if trigger_pendsv() fired just before sched_critical_exit(), the pending bit was set with BASEPRI still active. A subsequent re-entry into a critical section before PendSV got CPU time would block it again. In practice sched_delay_ms returned without a context switch ever happening.

FIXReplace BASEPRI with PRIMASK in sched_critical.h:
__asm volatile("mrs %0, primask" : "=r"(p));
__asm volatile("cpsid i");
Exit: __asm volatile("msr primask, %0" :: "r"(p));

— GDB Sessions: Part 5.2 —

Session 5 — sched_delay_ms: PendSV fires but task never resumes

After oscillator and flash wait-state fixes, the scheduler appeared to run but sched_delay_ms never blocked — UART printed at full speed, LED stayed solid. Broke on sched_delay_ms and inspected state after the critical section.

GDB — tracing sched_delay_ms
(gdb) break *0x4a2 (gdb) continue Breakpoint, 0x000004a2 in sched_delay_ms () (gdb) x/4xw 0x2000081c 0x2000081c: 0x20000fa8 0x00000102 0x000001f4 0x00000cbd sp=valid BLOCKED delay=500 func=task_led ✓ (gdb) break PendSV_Handler (gdb) continue ; PendSV_Handler breakpoint never fires ; BASEPRI = 0x30 was blocking PendSV (priority 0xFF) permanently ; replaced BASEPRI with cpsid/cpsie — PendSV fired immediately after

Session 6 — PSP pointing at s_task_count

With BASEPRI fixed, PendSV fired but caused an immediate HardFault. Connected GDB mid-run and interrupted:

GDB — idle task register state
0x00000502 in idle_task_func () (gdb) info registers sp psp msp control sp 0x20000818 <s_task_count> ← stack pointer on s_task_count! psp 0x20000818 <s_task_count> msp 0x2002fff0 ← MSP correct control 0x2 SPSEL=1 ✓ (gdb) print &s_idle_tcb $1 = 0x20000008 (gdb) print &s_task_count $2 = 0x20000818 ← s_idle_tcb + 0x810 = s_task_count exactly ; task_stack_init wrote initial frame into s_task_count and s_tasks[0] ; fix: move s_idle_tcb declaration to after s_tasks in scheduler.c

Session 7 — PIT->MCR = 0x02: module disabled in release build

After the stack overflow fix, the scheduler worked in debug builds but not release. LED stayed solid, UART printed only once. Used pyocd commander to inspect PIT registers while running:

pyocd commander — live register read
pyocd> read32 0x40037000 40037000: 00000002 ← PIT->MCR: MDIS=1, module disabled! pyocd> read32 0x40037108 40037108: 00000003 ← TCTRL: TEN=1 TIE=1 — written correctly pyocd> read32 0xE000E104 e000e104: 00010000 ← NVIC_ISER[1]: IRQ48 enabled ✓ ; TCTRL and NVIC correct but MCR never updated ; SIM clock gate not yet open when MCR was written ; debug build: halt overhead adds enough cycles — always works ; release build: back-to-back writes, MCR stays at reset 0x02 ; fix: 4x NOP after SCGC6 write — MCR reads 0x00000001 after

Lessons worth writing down

Check the silicon memory map

The K64F has two physically separate SRAM banks. Every tutorial I found treated it as one block. The reference manual (Chapter 4) is the only source that gets this right. Read the manual, not the tutorial.

CONTROL register behavior changes in Handler mode

In Handler mode (any exception/IRQ), the CPU always uses MSP and CONTROL[1] reads back 0, regardless of what Thread mode is using. A guard built on reading CONTROL inside an IRQ handler does nothing.

EXC_RETURN is only valid inside an exception handler

0xFFFFFFFD in LR means "return to Thread/PSP" only when the CPU is in exception context. From a regular function call, it's just an address in reserved memory — instant fault.

GDB + pyocd: always use --persist

pyocd gdbserver -t k64f --persist keeps the server alive across disconnects. Without it, every time GDB exits you need to restart pyocd before reconnecting.

map file is your first debugger

Before touching GDB, the map file told us: vector table size wrong (0x150 vs 0x190), sched_start_asm landing before Reset_Handler (link order bug), BSS at wrong address (linker region issue). Five minutes reading the map saved hours of stepping through assembly.

Clock gating requires a barrier

On K64F (and most Kinetis devices), writing a clock gate enable bit in SIM->SCGCx does not make the peripheral instantly accessible. The bus fabric needs a few cycles to propagate the enable. Writing a peripheral register in the very next instruction will be silently dropped. The manual documents this as "two bus clock cycles" but in practice four NOPs is the safe minimum. This only shows up in release builds — debug builds always have enough overhead to hide it.

pyocd commander is faster than GDB for register hunts

For questions like "is this peripheral register what I think it is?", pyocd commander with read32 is faster than setting a breakpoint and inspecting memory in GDB. No ELF required, works while the program is running, and the output is immediate. The PIT->MCR = 0x02 bug would have taken much longer without it.

Static variable declaration order matters for stacks

The C standard does not guarantee BSS layout, but in practice GCC lays out static variables in declaration order. If a task TCB with a large embedded stack array is declared before smaller variables, its stack can silently overwrite them. The fix is mechanical — declare large stack-bearing structs last — but the bug is invisible until you look at the actual addresses with GDB or a map file.

BASEPRI is a precision instrument — use cpsid for simplicity

BASEPRI lets you block interrupts above a specific priority threshold, which is useful for leaving high-priority ISRs (like a DMA completion handler) running during a critical section. But it also means PendSV — at the lowest priority — is always blocked whenever BASEPRI is non-zero. If your critical section accidentally stays active longer than expected (nested locks, early return paths), PendSV never fires. For a simple preemptive scheduler, cpsid i / cpsie i is the right tool: simpler semantics, impossible to accidentally block PendSV.

One thing that surprised me: the blt vs blo distinction in the BSS loop — signed vs unsigned branch — is the kind of bug that appears totally benign in simulation (small programs, addresses always positive) and only shows up on real hardware with a large enough BSS to cross the 0x80000000 boundary. It's not even a K64F-specific issue; any Cortex-M device with upper-bank RAM can hit this.

— Chapter V: ELF Analysis with biber —

Reading the binary with biber

After the scheduler was running, I used biber — a custom ELF analysis tool — to inspect the final binary. The goal was to verify that everything landed where the linker script said it should: correct RAM regions, correct section sizes, correct symbol addresses. It's the kind of check that catches subtle linker issues before they become runtime bugs.

biber -f tamgaos_k64f.elf -all
CLASS   : ELF32
DATA    : LittleEndian
TYPE    : ET_EXEC (executable)
MACHINE : .ARM
Entry   : 0x00000411    ← Reset_Handler + Thumb bit

PROGRAM HEADER
  Type    Flg  VAddr               PAddr               FileSize    MemSize
  LOAD    R-X  0x0000000000000000  0x0000000000000000  0x00000DE4  0x00000DE4
  LOAD    RW-  0x0000000020000000  0x0000000000000DE4  0x00000000  0x000049B8

Two load segments — exactly what we expect. The first is read/execute (flash), the second is read/write (RAM). The RAM segment has FileSize = 0 because it's all BSS — no initialized data to load from flash.

Section breakdown

The section header table is where the real layout lives. Each section maps to a specific region of the address space.

SectionAddressSizeNotes
.vectors0x000000000x190 = 400 B100 entries × 4 bytes — correct
.flashconfig0x000004000x010 = 16 BFlash security config at fixed offset
.text0x000004100x9D4 = 2516 BAll code — Reset_Handler starts here
.data0x200000000x000 = 0 BNo initialized globals — clean
.bss0x200000000x49B8 = 18.9 KBTask stacks, globals — all in m_data_2
.noinit0x200049B80x000 = 0 BReserved for crash log — empty for now

The .vectors size was the first thing I checked. Earlier in the debug process, the table was 0x150 (84 entries) instead of 0x190 (100 entries). PIT0 is IRQ48 — slot 64 in the table — so with only 84 entries, the PIT0 handler slot was missing entirely. biber made this immediately obvious.

The BSS at 18.9 KB is dominated by the task stacks: 8 tasks × 512 words × 4 bytes = 16 KB, plus globals, ring buffers, and TCB overhead. All of it sits in m_data_2 (0x20000000, 192 KB) — the correct bank.

Symbol table — what lives where

The symbol table shows every named object in the binary with its address and size. A few entries worth highlighting:

selected symbols from .symtab (163 total)
; Core scheduler globals
g_current_task   0x20000000  4B   GLOBAL  ← BSS start, first word in RAM
g_next_task      0x20000004  4B   GLOBAL

; Task storage
s_idle_tcb       0x20000008  2064B  LOCAL  ← 1 TCB = 512 stack + overhead
s_tasks          0x2000081C  16512B LOCAL  ← 8 × 2064 = 16512 bytes
s_task_count     0x20000818  1B   LOCAL

; UART ring buffer
s_tx_buf         0x200048A4  256B   LOCAL
s_tx_head        0x200049A4  4B   LOCAL
s_tx_tail        0x200049A8  4B   LOCAL

; Synchronization primitives
g_pit_sem        0x200049AC  8B   LOCAL   ← count(4) + max(4)
g_uart_mutex     0x200049B4  4B   LOCAL   ← owner pointer

; BSS boundary
_bss_end         0x200049B8         ← matches .noinit start exactly
_noinit_start    0x200049B8
_stack_top       0x20030000         ← 192KB above m_data_2 base

One thing that stands out: g_current_task sits at the very first address of BSS (0x20000000). This caused a subtle boot bug — because BSS wasn't being zeroed by startup, g_current_task contained garbage. The sched_init() call now explicitly zeroes it before anything else touches the scheduler.

Key findings and what they confirmed

Entry point thumb bit

The entry point is listed as 0x00000411, but Reset_Handler is at 0x00000410. The +1 is the Thumb bit — the CPU reads this from the reset vector and sets the T bit in EPSR before fetching the first instruction. Without it, a Cortex-M4 would take a UsageFault immediately on reset. The linker adds this automatically when you use .thumb_func or compile with -mthumb.

$t and $d markers

The symbol table is full of $t and $d entries with no name. These are assembler-generated markers: $t marks the start of a Thumb code region, $d marks a literal pool (data embedded in the code section). Debuggers and disassemblers use these to switch between Thumb disassembly and hex display — without them, a disassembler would try to decode literal pool data as instructions and produce nonsense.

Link order confirmed

Earlier, sched_start.o and pendsv_handler.o were landing in .text before startup_k64f.o, pushing Reset_Handler to a higher address. The fix was moving their code into a .kernel_asm section, placed after .text in the linker script. biber confirms it worked: Reset_Handler = 0x411 (start of .text), and sched_start_asm = 0xD81, PendSV_Handler = 0xDA9 — both well after the startup code.

.data is empty — good

A non-zero .data section means the linker has to copy initialized values from flash to RAM on startup. Every global like int x = 5 adds to it. TamgaOS has none — every global is either zero-initialized (BSS) or set explicitly at runtime. This keeps the startup path simple and eliminates one class of potential bugs.

Cross-checking with arm-none-eabi-size

After biber, a quick sanity check with the standard toolchain size utility:

arm-none-eabi-size tamgaos_k64f.elf
   text    data     bss     dec     hex filename
   2932       0   18872   21804    552c tamgaos_k64f.elf

The numbers match biber exactly — but the columns mean slightly different things. arm-none-eabi-size groups all flash-resident sections into text: that's .vectors (0x190) + .flashconfig (0x10) + .text (0x9D4) = 0xB74 = 2932. biber shows them separately. Both are correct — just different views of the same layout.

data = 0 confirms there are no initialized globals. bss = 18872 = 0x49B8 — matches _bss_end - _bss_start from the symbol table exactly. When these three numbers agree across two independent tools, the linker script is doing what you think it's doing.

The tool that pays for itself: reading the ELF directly — sections, symbols, program headers — takes five minutes and answers questions that GDB sessions alone never quite settle. Where did this symbol land? Is the section the right size? Did the linker script do what I thought? biber answers all three without needing the hardware.

References

Everything above traces back to these primary sources. When in doubt, read the spec — not a blog post. Including this one.

ARM architecture

NXP K64F

Project