Posts

Showing posts from June, 2024

Digging deep with heaptrack/massif

Debugging Memory Growth on a Ti-Sitara AArch64 ECU with Heaptrack A while back I had to track down a recurring heap growth issue on a customer’s automotive ECU based on a TI Sitara Cortex-A53. The system ran multiple ML models for ADAS feature extraction at different frequencies. Each model ran in its own POSIX preemptive real-time thread, pinned to cores with priorities like 120, 110, 80, 60 . When switching scheduling policies (for example SCHED_OTHER → SCHED_RR or SCHED_FIFO under PREEMPT_RT), the visible priority ranges changed — which is expected behavior on a real-time patched kernel. The Problem Over time, memory usage crept up until the system became unstable. The workload made it tricky — multiple ML models meant irregular allocation patterns with frequent buffer churn. I needed something lightweight that could run directly on the target to pinpoint the source. Why Heaptrack Heaptrack turned out to be the best fit. It hooks into memory allocations using a preload li...