| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
|
Comment on commit 1 fmu-v4 builds with CONFIG_STACK_COLORATION=y, so NuttX fills each new thread stack with a known pattern and anything the thread writes overwrites it. Reading back how much of the pattern survived gives the deepest the thread has ever reached, and that is the figure top and cat /proc/<pid>/stack report. I exercised the receive thread over everything I could think of, taking a reading after each step so an increase could be attributed: full parameter download and rewrite, a 30-item mission upload and readback, MAVFTP directory listing and a 400 kB log download, COMMAND_LONG, and gyro and baro calibration. The deepest was the MAVFTP log download at 1,908 bytes, against 7,432 reserved. For the replacement value I went with 4,000 rather than 2,840, which is what the code used before the refactor. The reason is that most of the depth turns out not to be in mavlink at all. Compiling the module with -fstack-usage accounts for only about 1,060 of those 1,908 bytes; the rest is NuttX file I/O below MAVFTP, reading the log off the SD card. I only ever exercised a read, never a write, and inside mavlink the read and write handlers are the same size, so whether a write goes deeper is decided by code I did not test. Writing to FAT does more work than reading, so it plausibly does. 2,840 would leave around 930 bytes spare where 4,000 leaves around 2,090, which is why I preferred the larger number. Being upfront: this investigation was done with Claude, and while I ran the measurements on my own board I can't personally vouch for every inference in the reasoning above, particularly around the FAT and MAVFTP call paths. So please treat the choice of 4,000 as a proposal rather than a conclusion. If someone with real context there thinks 2,840 is safe, I'm happy to change it, and if anyone can exercise a MAVFTP write or check the high-water on a board with more message handlers compiled in, that would settle it properly. |
Sorry, something went wrong.
There was a problem hiding this comment.
So this is not just about the Mavlink stack size but also v4 fixes to avoid running out of RAM, got it.
Sorry, something went wrong.
…eceiver) The receive thread stack is requested as sizeof(MavlinkReceiver) + 2840, but the object is not on that stack. It is a member of the heap-allocated Mavlink object (mavlink_main.h), and pthread_create is handed only (void *)this, which start_trampoline casts straight back to a pointer. Every MAVLink instance therefore reserves a second copy of the object as stack the thread can never touch. The term is a leftover from a refactor round trip: before 0a0c404 heap (new/delete in start_helper), stack 2840 0a0c404 object became a stack local, stack sizeof + 2840 4498509 object became a heap-resident member, term left behind so the pairing has been wrong since 4498509 (Apr 2021). On px4_fmu-v4 sizeof(MavlinkReceiver) is 4592 and sizeof(Mavlink) is 5568, so the receiver is 82% of its heap-allocated parent. PX4_STACK_ADJUSTED is the identity on NuttX and this board has no networking, so the request was 7432 B per instance. Deepest measured high-water over the receive paths is 1908 B, 26% of that. Replaced with a flat 4000 B, 2.1x the deepest measurement, returning 3432 B per MAVLink instance. Measured on px4_fmu-v4 with CONFIG_STACK_COLORATION=y, taking top once between each of: full parameter download and rewrite, 30-item mission upload and readback, MAVFTP listing and 400 kB log download, and COMMAND_LONG handling. MAVFTP was the deepest path. Effect on hardware: with a USB ground station attached the board goes from no MAVLink heartbeat at all in 110 s to a heartbeat in 1 s.
Work queue stacks are carved from the same allocator pool as the heap on NuttX, so an over-reserved queue stack costs the board free memory directly. This board runs out of heap before MAVLink can start (#28109). Measured at default sizes with CONFIG_STACK_COLORATION=y, reading StackUsed from top and /proc/<pid>/stack: queue default peak set to margin lp_default 3472 1412 2400 1.70x hp_default 2776 1104 2300 2.08x I2C1 2312 840 1900 2.26x Effect: 2040 B returned to the heap. Verified after trimming, on the resulting firmware rather than by calculation: queue trimmed peak margin lp_default 2376 1388 1.71x hp_default 2272 1104 2.06x I2C1 1872 840 2.23x lp_default's peak appears only after provoking the parameter autosave, which runs on that queue and writes through to storage. Idle it reads 1100 B, so the deep path has to be triggered deliberately: change a parameter to a genuinely different value, since PX4 only saves when the value changed, then re-read StackUsed. On a board too full to run top this is readable with cat /proc/<pid>/stack, which reports the same up_check_tcbstack() figure. The remaining queues are left alone because they have no slack: rate_ctrl 1.28x, SPI1 1.42x, INS0 1.56x, nav_and_controllers 1.69x. SPI2 looks generous at 3.02x but CONFIG_WQ_SPI_STACKSIZE sizes every SPI queue, so its headroom cannot be taken without starving SPI1. INS0 is why these are measured rather than estimated: its peak is 1180 B with ekf2 stopped and 3828 B once ekf2 runs in that queue.
Four of the seven ports carry hand-inflated buffers while every other port on the board uses 300. They are static allocations, so they come straight off the heap arena, on a board that runs out of heap before MAVLink can start (#28109). USART1 TX 2500 -> 600 ESP8266 header, usually unpopulated USART2 TX 1100 -> 600 TELEM1 USART3 RX 1200 -> 600 TELEM2 USART3 TX 900 -> 600 Effect: 3328 B of static RAM returned, flash unchanged. Measured with the NuttX submodule cleaned, since it builds in-tree and does not rebuild on a config change, and verified in the ELF with nm. No functionality is removed. What is given up is burst headroom: MAVLink defers when a TX buffer is full, so a stream is throttled rather than lost, and 600 B is still 83 ms at 57600 baud. Both telemetry ports have hardware flow control and RX DMA, so a full RX ring asserts RTS instead of dropping bytes. This cannot affect the failure in #28109, which is on USB, a CDCACM link with its own separate buffers.
CONFIG_FS_PROCFS_MAX_TASKS sizes system_load.tasks[], the per-task reporting table behind top, /proc and the cpuload accounting. NuttX defaults it to 128 and this board never overrode it, so 128 slots at 24 B each are reserved for the life of the board. It runs 25 tasks, counted with ls /proc. Effect: system_load 3096 -> 1560 B, so 1536 B of static RAM returned to the heap arena. The second effect matters more on this board. Every NSH builtin needs a 4096 B task spawn, and before this the largest free block with a ground station attached was 3760 B, so no console command would run at all: param and top both failed with "command not found", which is ENOMEM in disguise. With this the largest block reaches 4416 B and the console works. 64 leaves 2.56x headroom over the live count, enough for the transient tasks a calibration or parameter save adds. If the limit were ever exceeded, top and procfs would list fewer tasks and a task's CPU time would go unaccounted; nothing else is affected.
The verbose estimator uORB nodes cost heap this board does not have. Measured on hardware with EKF2 running and two MAVLink instances up: 7440 B free at 0 against 3696 B at 1. Effect: 3744 B returned. On this board that is not optional headroom. With the receive stack fix alone and this parameter at its upstream default, the board comes up and streams telemetry but answers no request at all, no PARAM_VALUE and no COMMAND_ACK, and a ground station hangs requesting capabilities. Cost: the default logging profile picks up estimator* topics by name prefix at 2 Hz via add_optional_topic_multi, which only logs advertised topics, so turning it off removes innovations, variances, test ratios, states, the aid_src_* set and the bias topics, and Flight Review's innovation plots go empty on this board. estimator_status still carries the filtered test ratios, filter fault flags, reset counters and pre_flt_fail_innov_*, and replay still works because SDLOG_PROFILE bit 1 logs EKF2's inputs. Applied with param set-default so an operator can opt back in. rcS sources rc.board_defaults at line 280 and starts ekf2 at 437, and EKF2 reads the parameter in its constructor, so the default lands in time. Note a board with this parameter explicitly saved keeps its saved value.
…mator and SIH This board is at 94.7% flash on main and out of heap before MAVLink starts (#28109), so it needs static RAM back, not only flash. Measured by building fmu-v4 with and without each: UAVCAN flash -147276 B static RAM -3520 B uxrce_dds_client flash -59508 B static RAM -64 B Effect: 3652 B of static RAM and 207 kB of flash returned. UAVCAN is the only module here that returns meaningful RAM, and it is paid even though the driver never starts, since UAVCAN_ENABLE defaults to 0. For contrast, dropping five other modules including attitude_estimator_q and local_position_estimator returned 128 B, so module removal is otherwise a flash lever only. The UAVCAN driver costing 3.5 kB of .bss wherever it is compiled in and not started is worth fixing properly for every constrained board. This is a trade and is offered as one. A Pixracer with a DroneCAN GPS or CAN ESCs loses them and must set CONFIG_DRIVERS_UAVCAN=y in a local build. mag_bias_estimator is likewise a deliberate feature removal: MBE_ENABLE defaults to 1 and rcS starts it, so pre-takeoff continuous magnetometer calibration goes away on this board. SIH never runs on hardware and uxrce_dds_client only starts when UXRCE_DDS_CFG points at a port, so neither holds heap today.
🔎 FLASH Analysispx4_fmu-v5x [Total VM Diff: -8 byte (-0 %)] FILE SIZE VM SIZE
-------------- --------------
+0.0% +55 [ = ] 0 .debug_abbrev
-0.0% -2 [ = ] 0 .debug_info
-0.0% -5 [ = ] 0 .debug_line
+67% +2 [ = ] 0 [Unmapped]
-0.0% -7 [ = ] 0 [section .debug_line]
+0.1% +8 [ = ] 0 [Unmapped]
-0.0% -8 -0.0% -8 .text
+0.0% +48 -0.0% -8 TOTAL
FILE SIZE VM SIZE
-------------- --------------
+0.0% +55 [ = ] 0 .debug_abbrev
-0.0% -2 [ = ] 0 .debug_info
-0.0% -5 [ = ] 0 .debug_line
+67% +2 [ = ] 0 [Unmapped]
-0.0% -7 [ = ] 0 [section .debug_line]
+0.0% +48 [ = ] 0 TOTAL
Updated: 2026-08-03T07:36:53 |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Solved Problem
Fixes #28109.
MavlinkReceiver::start() sizes the receive thread's stack as sizeof(MavlinkReceiver) + 2840, but the object is not on that stack: it is a member of the heap-allocated Mavlink (mavlink_main.h:517), and pthread_create is handed only (void *)this. On NuttX thread stacks come out of the heap pool, so every MAVLink instance has been reserving a second copy of the object as unreachable stack.
The term is a refactor leftover: it was added in 0a0c404a08 (Jun 2019) when the object became a stack local, which was correct, and 4498509426 (Apr 2021) moved it back to a heap member without removing it.
On px4_fmu-v4 that is a 7,432 B request per instance against a 1,908 B measured high-water (26%). The board runs out of heap before MAVLink starts and comes up with no connection at all, which is what the issue reports.
Both builds identical in flash and static RAM; one constant apart.
Solution
Commit 1 is shared code and applies to every NuttX board. Commits 2-6 are px4_fmu-v4 only, ordered so the free ones come first and the trades last.
Combined, clean builds: flash 94.73% → 82.61%, static RAM 39,032 → 30,516 B. On hardware, free with a GCS attached and EKF2 running: free heap 320 B → ~9-11 kB, largest free block 144 B → ~9-11 kB.
Test coverage
px4_fmu-v4, USB GCS attached, EKF2 running, never armed. Stock main vs commit 1 as above; full parameter download and rewrite; 30-item mission upload and readback; MAVFTP listing and 400 kB log download; COMMAND_LONG; gyro and baro calibration; EKF2_LOG_VERBOSE A/B with EKF2 liveness confirmed in both arms; work queue high-water at default sizes and again after trimming, with the parameter autosave path deliberately provoked.
Not covered: MAVFTP write (only read), and the margin is measured on px4_fmu-v4, which has nearly the smallest MavlinkReceiver of the boards above. A reading on a board with more handlers would be welcome.