feat: Q10 map layers, calibration, path & no-go zones (+ Q7 layer reuse)

[tubededentifrice]

Summary

Adds structured Q10 map layers, calibration, path placement, and vector overlays on top of #847. It also reuses the shared B01 grid-layer code for Q7 maps.

This PR is stacked on #847. Until #847 merges, GitHub will show those commits here too; the incremental work starts with feat: decompose Q10 map into separable layers.

Changes

• Split Q10 occupancy grids into separable background, wall, floor, and per-room layers. Each layer can be rendered as a transparent PNG for frontend compositing.
• Add GridCalibration and solve_calibration() to fit Q10 world coordinates to grid pixels from an active cleaning path.
• Populate MapData.path, vacuum_position, and charger position once calibration is available.
• Decode Q10 restricted-zone blobs into no-go, no-mop, and virtual-wall overlays in MapData.
• Decode and apply Q10 erase zones from the map packet tail so erased cells are removed from rendered maps and layers.
• Reuse the shared layer/calibration abstractions for Q7 SCMap payloads.
• Preserve existing overlays on partial status updates and keep path/position overlays after reparsing a calibrated map.

Notes

• Virtual walls, zoned-clean areas, and carpets remain best-effort until non-empty captures are available for each data point.
• Q10 calibration currently needs a sufficiently dense current-session path; no-clean calibration is left for a later PR.

Testing

• uv run pytest — 543 passed on the rebased stack
• uv run pre-commit run --all-files
• Focused coverage for Q10 layers, calibration, overlays, erase zones, parser edge cases, and Q7 layer reuse
• Live checked against Q10 captures for map layers, calibration fit, path placement, and no-go/no-mop zones

[allenporter]

If there are low level map parsing parts, you could send them out in parallel without blocking on the other changes if that would be helpful to parallelize review. otherwise, we can mark this in draft until the other blocking parts are reviewed and merged.

[tubededentifrice]

@allenporter ok, #850 is the standalone part

[andrewlyeats]

Really nice to see the Q10 calibration and layer work coming together here.

One thing that might help the no-clean calibration you've flagged as a follow-up: on our Q10 S5+ (ss07, fw 03.11.24) captures, the calibration data appears to be transmitted in the 0101 grid-frame header, so a GridCalibration can be derived without a session path (i.e. without solve_calibration's dense-path requirement). The header fields we see (absolute byte offsets in the 0101 frame, sub-type at 0–1; verified across our captures, scoped to ss07):

• bytes 11–12 x_min, 13–14 y_min (s16 BE) — the map origin, in 5 mm units (= 0.1 grid-px; ÷10 gives grid-pixel coordinates). This is the value solve_calibration is recovering by fitting.
• bytes 15–16 resolution (u16 BE) — reads 5 universally for us → 0.05 m/px (the 50 mm/px you use).
• bytes 17–22 charger x / y / phi — the dock position and orientation, as an alternative to inferring the charger from the path's first point.

On our decode, reading the origin straight from the header lands the path on-floor at the same rate as the auto-fit (they agree on ~29 of 31 captures; the two misses are null/keepalive frames where x_min == y_min == 0, which you'd just skip and fall back to a fit). So a header-derived calibration looks viable docked / pre-clean, which would drop the "needs a dense current-session path" requirement.

Happy to help — I can share scrubbed, annotated header captures, or sketch a PR against this branch if you'd like — and thanks for all the Q10 work. (Separately, the 0201 path header also carries a live SLAM heading field at bytes 10–11 if orientation is ever useful alongside vacuum_position — glad to detail that too.)

[tubededentifrice]

Thanks @andrewlyeats, this was really useful! I've implemented the header-origin part on this branch.

The 01 01 header now decodes into a Q10HeaderCalibration (x_min/y_min @11–14, resolution @15–16, charger @17–22), and origin_pixels() returns the grid-pixel origin (÷10, since the grid is 50 mm/px). Keep alive frames (x_min == y_min == 0) are detected and skipped > fall back to the path-fit, exactly as you described.

To use it, I added solve_calibration_with_origin(): with the origin fixed from the header, only the resolution + Y sign are fit, so it now confirms from a short path instead of needing a dense clean (the trait prefers the header origin at >=4 points, falls back to the full fit at ≥20).

One thing I couldn't pin down without your data: the header resolution reads 50 mm/px, but my path-fit lands at ~13-16 per pixel, i.e. the path coordinates don't look like millimeters, so I can't yet turn the header's 50 mm/px straight into the GridCalibration resolution. That's the only reason it isn't fully path-free yet (origin is exact; resolution still gets a quick on-floor validation against a short path).

So I'd love those scrubbed, annotated header captures, especially a few where you've noted the path/trace coordinate units alongside x_min/resolution. If I can confirm the path-unit <=> pixel relationship, I think the docked/pre-clean (zero-path) case falls out directly. And yes, please do detail the 0201 SLAM heading field too, orientation alongside vacuum_position would be nice to have.

Thanks again for the cross-capture validation here.

[andrewlyeats]

Glad the header origin helped. On your two questions:

1. Path-unit ↔ pixel. On our end res = 20 path-units/px lines the path up on the grid (~99.8% on floor), and our
byte-14 coords drop into your world_to_pixel as-is — per-axis scale + Y-flip, no transpose. So that's the resolution
that registers everything here, no fit needed.

The one open piece is the absolute scale: a path-unit isn't a millimeter, and we've only ever anchored 20 internally —
never ruler-measured it — so a constant scale factor could move the real-world number. Your 13–16/px isn't necessarily
wrong, it may just be that factor. Either way the divisor is a fixed constant, so once it's pinned the docked case
falls out; a single ruler-measured drive would settle the absolute. The captures let you check whether 20 registers your
path too.

2. I think that "stray leading point" is the robot's heading. Your 10-byte trace header starts the points at byte 10, so the
"first point" reads as bytes 10–13 = (heading, 0) — far enough from the real points (byte 14) that
_drop_stray_leading_point removes it. Bytes 10–11 are the 0201 SLAM heading (i16; 0 = +x / +90 = +y / ±180 = −x / −90 = −y); read it there, keep points from byte 14, and you recover orientation for free.

Two captures (ss07, fw 03.11.24). The code blocks are raw frame bytes; the two tables below decode every header byte
(A and B side by side).

Capture A — 2026-06-22.

0101 grid frame (header bytes 0–22; the rest of the frame is the LZ4-compressed grid):

01 01 6a 2b 59 80 01 00 db 00 fe 06 73 01 b2 00 05 03 82 03 3d 00 a7

0201 path frame (one stream — header, then (x,y) points from byte 14; first 110 of 5682 bytes):

02 01 00 a3 00 02 00 00 05 89 00 a8 00 00 00 05 00 4c f9 41 fc d8 f9 38 fc c7 f9 2c fc af f9 24 fc 9b f9 18 fc 87 f9 10 fc 77 f9 04 fc 63 f8 f8 fc 4f f8 f0 fc 3b f8 e8 fc 27 f8 dc fc 17 f8 d4 fc 03 f8 cc fb ef f8 cc fb db f8 c8 fb c3 f8 c8 fb af f8 c8 fb 97 f8 c8 fb 83 f8 c8 fb 6b f8 c4 fb 57 f8 c8 fb 3f f8 d4 fb 2f f8 e4 fb 23

Capture B — 2026-06-20.

0101 grid frame (header bytes 0–22):

01 01 6a 2b 59 80 01 00 dc 00 ff 06 73 01 bc 00 05 03 7f 03 3c 00 b1

0201 path frame (first 110 of 9862 bytes):

02 01 00 9b 00 02 00 00 09 9e 00 ac 00 00 ff fd 00 00 f9 55 fd 1c f9 68 fd 1b f9 7c fd 23 f9 94 fd 28 f9 b8 fd 28 f9 bc fd 3c f9 b8 fd 50 f9 a4 fd 58 f9 90 fd 58 f9 78 fd 58 f9 64 fd 58 f9 54 fd 64 f9 48 fd 78 f9 48 fd 88 f9 48 fd a0 f9 48 fd b4 f9 40 fd c4 f9 2c fd d0 f9 18 fd d0 f9 04 fd c8 f8 f0 fd c4 f8 f4 fd b0 f8 f0 fd 9c

0101 grid-frame header (all BE):

bytes	field	A	B	note
0–1	sub_type	0101	0101	grid-frame magic
2–5	map_id	6a2b5980	6a2b5980	per-map id (same map for both)
6	map_segmented	1	1	1 once finalized into rooms (0 while building)
7–8	width	219	220	grid px
9–10	height	254	255	grid px
11–12	x_min	1651	1651	origin; oy = −2·x_min = −3302
13–14	y_min	434	444	origin; ox = 2·y_min = 868 / 888
15–16	resolution	5	5	→ 50 mm/px
17–18	charge_x	898	895	dock X (same units as x_min)
19–20	charge_y	829	828	dock Y
21–22	charge_phi	−167°	−177°	dock heading (raw s16 negated)

0201 path-frame header (all BE; points then run from byte 14):

bytes	field	A	B	note
0–1	sub_type	0201	0201	path-frame magic
2–3	epoch	163	155	+1 per traversal (not per clean); power-cycle resets
4–7	const	00020000	00020000	constant; semantics unknown
8–9	point_count	1417	2462	may read one high
10–11	heading	168°	172°	i16; 0=+x / +90=+y / ±180=−x / −90=−y
12–13	const	0000	0000	constant
14 …	points	(5,76)* (−1727,−808) …	(−3,0)* (−1707,−740) …	BE int16 (x,y) path-units; *~origin sentinel; robot = last pt (A −1592,−753 → col 85,row 81 · B −1600,−761 → col 85,row 82)

3. One to glance at when the overlays render. On our ss07 a no-go (DP-55) and a wall (DP-57) only place right when
both keep their first wire coordinate on the same axis; your parse_virtual_wall_blob swaps DP-57 but parse_zone_blob
leaves DP-55, so if your device matches ours they'd come out transposed relative to each other. Easy to check once they
render — the #850 before/after is yours if it helps.

Thanks for folding the origin work so quickly!

[tubededentifrice]

Spot-on, confirmed it with your fixtures. The 0201 frame is a 14-byte header, not 10: bytes 10–11 are the SLAM heading, 12–13 a constant, points from byte 14. The old 10-byte header was folding the heading word into a phantom leading point (heading, 0), exactly the "stray point" the heuristic was papering over, and why the count read one high.

Verified byte-for-byte against both your captures and our binaries:

• Docked capture: count 0 + heading 00a9=169, no points (we were rendering (169, 0)).
• Corridor capture: count 14 + heading ffde=−34, with exactly 14 points from byte 14 (we were reading 15).
• Captures A/B: 1417 / 2462 == the byte-14 length math, so byte 8–9 is the exact point count, not "one high."

Pushed in 1d6956f: heading is now parsed onto the trace packet and surfaced as robot_heading, and I draw a facing tick on the robot marker (projected through world_to_pixel so the Y-flip stays consistent). The near-origin sentinel drop still runs, now correctly, since the heading isn't masquerading as point 0 anymore. Orientation for free, as you said. 🙏

On the other two:

1. Absolute scale, our fit already brackets ~13–16/px, so 20 and our number look like they differ only by that fixed divisor. I don't think it's resolvable from these captures; agreed a single ruler-measured drive settles it, and I'll wire that in once it's pinned.
2. DP-55 vs DP-57 transpose: good catch, and you're right that parse_virtual_wall_blob swaps while parse_zone_blob doesn't. Both decode to clean axis-aligned rectangles individually, so neither looks wrong in isolation, the question is purely whether they're consistent with each other. I'll render a no-go and a wall on the same map and check them against feat: B01 grid-layer decomposition + Q10 vector overlay decoding #850; if they land 90° apart I'll align parse_zone_blob to match. Thanks for the pointer.

[tubededentifrice]

Confirmed on our hardware, you were right. Captured a no-go zone (DP-55) and a wall (DP-57) on our devices and ground-truthed both against the app:

• DP-55: a deliberately wide no-go zone reads back wide (x-range >> y-range), matching the horizontal band as drawn, so our zone parser keeps the first wire word on x, no transpose.
• DP-57: a wall drawn horizontally below a room only decodes horizontal when we don't swap. Our parse_virtual_wall_blob was swapping to (y,x) (from an earlier reading of the feat: B01 grid-layer decomposition + Q10 vector overlay decoding #850 notes), placing every wall 90° off. Dropped the swap so DP-57 shares DP-55's order, exactly as you predicted. Fixed in cea9c74, with the real capture added as a regression fixture.

Also fixed a related rendering bug where 2-point wall segments were being skipped entirely. Thanks for the nudge on this one. Still owe you the ruler-measured drive to pin the absolute path-unit scale.

[tubededentifrice]

Pinned the scale on my robot: a dense path fits 20 path-units/px around the header origin, matching your internal "20". With the header's 50 mm/px grid that's 2.5 mm per path-unit (so not a millimeter). I corroborated it accross a ~8.8 m drive. Also widened our calibration range, which had been railing below 20. And while there, ground-truthed the trace heading (bytes 10–11) against a live drive, matches direction of travel exactly.

[tubededentifrice]

@allenporter Lots of fun experiments here, I cross checked everything with my hardware I believe, should be ready to review, thanks :)

[allenporter]

do we need to incorporate changes made form the other PR? this looks like its reverting something we iterated on in the last PR. if that is intentional then please update the PR description to describe why but im assuming its just a little stale and needs to be updated. Want to clairfy then i can take a look?

Thank you!

[andrewlyeats]

The header-calibration and erase-zone work here is great to see. Both match what we get on our ss07 — the erase section decodes identically (count, vertices-per, int16 rectangles), and reading the origin straight from the grid header is exactly the piece that places everything without a fit.

One thing that might be useful: on our ss07 the packet tail continues past the erase section with more structure before the trailing raster — it isn't opaque. After _parse_erase_zones consumes its bytes, the rest decodes byte-exact:

• carpet mask — a second compressed grid using the same LZ4 framing as the main grid ([u32 uncompressed_len][u16 compressed_len][block]). Empty on our current map (0 cells), but that's where carpet zones live.
• AI-obstacles (the "cones" the app shows) — [count: u8] then count × int16-BE (x, y), each coord raw / 50. Offset from your header pixel-origin (origin / 10) with the same Y-flip, all of them land on the floor footprint here (53/53). This is the layer that's otherwise cloud-only on camera models — on the lidar Q10 the marker positions are right in the frame.
• skip-clean zones — a short [count: u8] + int16 (x, y) list (/10), then on the full-map (03 01) frame a concatenated path package (a 17-byte header + int16 points) — probably the "+ signature" you noted.

Byte-exact on one 03 01 frame:

29 + 8963 (grid) + 82 (erase) + 253 (carpet) + 213 (obstacle) + 1 (skip) + 40925 (path) = 50466

The same erase/carpet/obstacle/skip layers also appear in the finalized 01 01 frames (same dims + origin), so they ride the live grid, not just the saved map.

Totally your call whether obstacles are in scope for this PR vs. a follow-up — happy to share fixtures (a frame with N obstacles, plus before/after) if it helps, same as the #850 ones. Thanks again for driving the Q10 map work forward.

[tubededentifrice]

@allenporter Good catch, you were right, this was a stale revert, not intentional. Fixed in 496474c

[tubededentifrice]

@andrewlyeats This was spot-on, I captured live 01 01 frames on my robots and your tail layout decodes byte-exact on our hardware too. Walking erase > carpet-mask > obstacles > skip consumes both frames to the last byte.

A couple of confirmations beyond what you had:

Carpet mask: exactly your framing ([u32 uncompressed][u16 compressed][LZ4 block], same as the main grid), and the uncompressed length equals width×height on the nose (325×283, 269×320), so it really is a full same-dims second grid. Unlike your map, ours aren't empty: R1 = 1047 carpet cells (kind 4), RDC = 2856 (kinds 3 and 4) ; looks like the value encodes the carpet kind. I'm using uncompressed == w*h as the locate-guard so a mis-framed tail just yields no carpet rather than garbage. This explains the empty dpCarpetUp: carpets ride in the map, not the DP. I've landed carpet on the branch (parse_map_packet().carpet_mask → MapData.carpet_map), so that part's done.

Obstacles / skip-clean: framing confirmed byte-exact ([u8 count] + N×int16), but here's where I'm stuck: both read count 0 on every frame our robots produce right now, including the finalized post-dock map. I ran a spot clean (table + chairs within a meter), but with lineLaserObstacleAvoidance it just reacts physically and never persists any "cone" markers, nothing gets written to the obstacle section. So I can confirm the shape but not the values (the /50 scaling, the on-floor placement you saw 53/53 on).

If that offer of a frame with N obstacles (plus a before/after) still stands, I'd love to take you up on it, that's the missing piece to validate the obstacle decode and wire it into MapData.obstacles. A 03 01 saved-map sample would be great too while we're at it: we only ever get 01 01 pushes here, so the trailing path package you mentioned is still unsampled, and our parser doesn't recognize 03 01 yet.

Thanks again, the byte-level cross-checks have been incredibly useful.

[tubededentifrice]

@allenporter i think we can also merge this one; for the obstacles, we'll simply make another PR

[allenporter]

Thank you, took me a little bit of time to gather my thoughts since this is a pretty large PR.

[allenporter]

This indicates to me the DPS fields on Status should probably live on the map trait? It seems like the update_from_dps API could potentially handle these types of calls.

[allenporter]

I'm trying to gather my thoughts here, but in the name of giving fast feedback i'll share a little unfiltered.

How many of these fields are needed for rendering the image bytes vs for use by consumers of the roborock API? I'm wondering if in a future PR we should be hiding some of these fields somewhere else as internal details of the trait rather than public fields? The fact that _render_packet updates all these members at once rather than returning an object makes me think we're missing an object holding these like ParsedMapData has MapData

Like should much of this be in the map module and not all exposed on the trait? (Even if some of the data objects updates come through multiple sources via the trait)

Maybe part of this is in my mind the map data modules all have low level details that i care less about readability (e.g. more vibe cody stuff in map is ok), but in the traits i care a lot more about state management, simplicity, readability, etc so i'm applying a higher bar here.