New memory domain logic by jsarha · Pull Request #5537 · thesofproject/linux

jsarha · 2025-09-17T20:19:44Z

Let's put this on hold until I get my experiment about implicit interim vs. lifetime heap division. E.g. Since the lifetime allocations are done at pipeline and module init phase and the interim allocations later - at runtime - it should be possible to create the interim heap when the first interim allocation arrives and use all memory that is left of lifetime heap. If this scheme works well, we can drop the interim vs. lifetime heap division.

The original description has been obsolete already for some time:

his is my first step towards implementing pipeline specific heaps. There is brand new create message payload definition and implementation. However, the target has already moved so that this payload is not enough anymore. 

There is also a problem that it looks like the FW receives some pipeline create messages with payload bit set, but the payload is not there. Firmware receives two messages where linux driver only logs about sending only one. This probably has something to do with forwarding the messages form core 0 to core 2. I have no idea how this can happen, but as I should move to work on kernel heap, I leave the debugging for now. 

The FW side code is here: https://github.com/thesofproject/sof/pull/10265

jsarha · 2025-09-17T20:20:41Z

FYI @ranj063 @lyakh @lgirdwood

lyakh · 2025-11-14T11:14:31Z

 	{SOF_TKN_COMP_STACK_BYTES_REQUIREMENT, SND_SOC_TPLG_TUPLE_TYPE_WORD, get_token_u32,
 		offsetof(struct snd_sof_widget, stack_bytes)},
+	{SOF_TKN_COMP_STATIC_BYTES_REQUIREMENT, SND_SOC_TPLG_TUPLE_TYPE_WORD, get_token_u32,
+		offsetof(struct snd_sof_widget, static_bytes)},


to the commit message: is it heap size for initialisation only or for the entire life-time of a single instance (estimated, possibly dependent on configuration or use-case)

jsarha · 2025-12-03T19:42:55Z

Updated to match and work with thesofproject/sof#10281 . I will address the review comments at later time.

jsarha · 2025-12-04T13:46:23Z

I'll mark this ready for review, but before merging it should be checked that thesofproject/sof#10265 is also going to go in without changes to IPC payload.

jsarha · 2025-12-11T18:25:55Z

The review comments are now addressed, but there is one loose end. The module init payload still uses the old DP memory data in the payload. Its easy change that too, but I need to work on the FW side for it to understand it and to be able to test the code.

jsarha · 2025-12-14T23:17:07Z

Change module init IPC payload as requested by @lgirdwood .

jsarha · 2026-01-12T22:14:19Z

@ujfalusi , @ranj063 , this thesofproject/sof#10265 is getting closer to be merged, could you review this too, hopefully before the merging.

lgirdwood

Just a few questions

jsarha · 2026-01-21T18:14:43Z

Missed some changes in include directory in the last commit, now the changes are included.

kv2019i · 2026-01-22T16:33:18Z

+	uint32_t stack_bytes;		/* required stack size in bytes */
+	uint32_t interim_heap_bytes;	/* required interim heap size in bytes */
+	uint32_t lifetime_heap_bytes;	/* required lifetime heap size in bytes */
+	uint32_t shared_bytes;		/* required shared memory size in bytes */


Reviewing this together with thesofproject/sof#10265 makes this a bit easier to follow. I still find relation of interim and lifetime a bit confusing, but additional notes in #10265 clarify this a bit (lifetime_heap_bytes memory needed in init and interim_heap_bytes stuff that may be allocated during runtime).

why change types? I think u32 is preferred in the kernel

jsarha · 2026-04-20T14:59:13Z

@kv2019i wrote in #5537 (comment) :

Not just the copilot :) I assume pipeline object is built from scratch, so this reset logic is needed, but good to
double-check. But I'm ok if you have checked for new pipelines we are starting from zero always (and we have
no code-paths to refresh same widgets again).

Yes, the pipeline widgets are always built from scratch. "Always" here means the only one time when SOF driver pulls itself up and builds its data structures based on the topology file. The code doing it is bit hard to follow. The benefit of that is that when you have spent couple of days to find that out, you never forget it 😅. Maybe it would deserve a comment some where describing how the driver actually works, but since its not entirely clear to me either, I have not done that.

Hmm, once I said that I'd better make sure if this is not one of those things that are made at runtime when pipelines are created in the frimware.

Oops. That is indeed the case. It was just the suggestiong where to reset the usages was quite awkward. I'll put it in better place.

Still, if I would know by heart all the widget opts branching from ipc topology ops struct (there is a lot of them), I would put this calculation in a place where its done only once at boot time, but unfortunately I do not where that would be, and this function seems to be used for similar purposes. So I just reset the values, but in more sensible place.

jsarha · 2026-04-20T16:21:56Z

Ok. Updated once more.

Copilot

Pull request overview

This PR is an initial step toward pipeline-specific heap/memory configuration support in the SOF IPC4 driver by extending topology tokens and IPC payload construction to carry per-module and per-pipeline memory requirements.

Changes:

Replace DP-specific dp_* widget memory fields with generalized per-instance memory requirement fields (domain_id, stack_bytes, interim_bytes, lifetime_bytes, shared_bytes).
Extend IPC4 topology token parsing to populate the new memory requirement fields and accumulate non-DP module requirements at the pipeline level.
Introduce a payload format + builder for SOF_IPC4_GLB_CREATE_PIPELINE to send pipeline memory configuration via an object array.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
sound/soc/sof/sof-audio.h	Renames/expands widget memory requirement fields to support new memory model.
sound/soc/sof/ipc4-topology.c	Parses new tokens, accumulates pipeline memory requirements, and adds IPC payload builders for module init and pipeline create.
include/uapi/sound/sof/tokens.h	Updates/extends topology token IDs for memory requirements.
include/sound/sof/ipc4/header.h	Adds IPC4 definitions/macros/structs for pipeline create payload object array and expands DP memory data struct.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+		if (pipe_widget->stack_bytes < swidget->stack_bytes)
+			pipe_widget->stack_bytes = swidget->stack_bytes;
+
+		dev_dbg(sdev->dev, "%s mem reqs to %s lifetime %u heap %u shared %u stack %u",


+	/* Put total payload size in words to the payload header */
+	payload_hdr->word0 |= SOF_IPC4_GLB_PIPE_PAYLOAD_WORDS(ext_pos);
+	*new_data = payload;
+
+	/* Update msg extension bits according to the payload changes */
+	msg->extension |= SOF_IPC4_GLB_PIPE_PAYLOAD_MASK;
+
+	dev_dbg(sdev->dev, "%s: payload word0 %#x", swidget->widget->name,
+		payload_hdr->word0);
+
+	return ext_pos * sizeof(int32_t);


+ * Macros for creating struct sof_ipc4_glb_pipe_payload payload with
+ * its associated data. ext_init payload should be the first piece of
+ * payload following SOF_IPC4_GLB_CREATE_PIPELINE msg, and its
+ * existence is indicated with SOF_IPC4_GLB_PIPE_PAYLOAD bit.


+#define SOF_TKN_COMP_STACK_BYTES_REQUIREMENT	420
+#define SOF_TKN_COMP_INTERIM_HEAP_BYTES_REQUIREMENT	421
+#define SOF_TKN_COMP_LIFETIME_HEAP_BYTES_REQUIREMENT	422
+#define SOF_TKN_COMP_SHARED_BYTES_REQUIREMENT	423


jsarha · 2026-05-08T06:14:39Z

SOFCI TEST

Remove dp-prefix from all module instance's memory attributes and related data structures. The attributes are not anymore exclusively for Data Processing module instances, but generic for all module instances. However, the module init payload is still only for DP module instances. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

The was inconsistency with SOF_TKN_COMP_STACK_BYTES_REQUIREMENT and SOF_TKN_COMP_HEAP_BYTES_REQUIREMENT token ids in the Linux driver code with SOF FW topology code. This commit fixes the Linux side to match tools/topology/topology2/include/common/tokens.conf See https://github.com/thesofproject/sof/blob/788861804ed08485496e979dd9c467c1a21b30c5/tools/topology/topology2/include/common/tokens.conf#L30 Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

jsarha · 2026-06-03T20:40:55Z

This version has taken couple of steps back. There is no more separate lifetime and interim heap sizes being specified. The FW manages the division by itself, that is if the approach in thesofproject/sof#10783 is accepted.

jsarha · 2026-06-09T20:39:12Z

FYI @lgirdwood , @ujfalusi , Now that thesofproject/sof#10842 is merged, there is no particular reason to hold this back.

jsarha · 2026-06-25T14:58:51Z

Drop the two commits that had no relevant changes.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

jsarha · 2026-06-25T18:30:40Z

+	 * requirement of all module instances belonging to the same
+	 * pipeline.
+	 */
+	if (swidget->comp_domain != SOF_COMP_DOMAIN_DP) {


Unset will never happen.

+	 * If this is not a Data Processing module instance, add the
+	 * required heap sizes to the sum of all modules instance's
+	 * belonging to same pipeline and find the maximum stack
+	 * requirement of all module instances belonging to the same
+	 * pipeline.


+ * Macros for creating struct sof_ipc4_glb_pipe_payload payload with
+ * its associated data. ext_init payload should be the first piece of
+ * payload following SOF_IPC4_GLB_CREATE_PIPELINE msg, and its
+ * existence is indicated with SOF_IPC4_GLB_PIPE_PAYLOAD bit.


+	SOF_IPC4_GLB_PIPE_DATA_ID_MAX = SOF_IPC4_GLB_PIPE_DATA_ID_MEM_DATA,
+};
+
+/* Pipeline memory configuration data object for ext_init object array */


Adds SOF_IPC4_GLB_PIPE_EXT_OBJ_ARRAY macros to set extension bit in SOF_IPC4_GLB_CREATE_PIPELINE indicating presence of the payload, and all necessary macros and structs to create the payload. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Start adding payloads to pipeline create messages. The payload contains information for payload specific memory configuration. All non DP module instances within the same pipeline share the same memory attributes and access the same resources. The new logic sums interim, lifetime, and shared heap memory requirements together and picks the highest stack requirement of all module instances belonging to a pipeline. These pipeline specific attributes are sent as struct sof_ipc4_glb_pipe_payload payload in pipeline's create message. The idea is to pass common memory configuration for all the Low Latency modules in the pipeline in pipeline create message payload. The Data Processing module instances will still have an individual memory configuration in struct sof_ipc4_mod_init_ext_dp_memory_data payloads as before. In their payload everything is as it was before, all attributes are copied directly from their topology attributes. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

…ry_data Change struct sof_ipc4_mod_init_ext_dp_memory_data to what is required for the SOF FW userspace DP processing. The earlier version of the firmware (v2.14) did not use the contents of the struct for anything, and if it receives a struct that is larger than the original, the extra words are simply ignored, so there should not be any problem in changing the struct. The following FW versions will expect larger struct and ignore anything that is smaller. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

…ad() Refactor sof_ipc4_widget_mod_init_msg_payload() to be easier to extend. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

…ad() Optimize sof_ipc4_widget_mod_init_msg_payload() so that it skips the payload allocation and rest of the function if no payload is needed. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

jsarha force-pushed the new_memory_domain_logic branch from 1541ecf to c39e6b2 Compare September 18, 2025 15:38

ranj063 reviewed Sep 22, 2025

View reviewed changes

Comment thread sound/soc/sof/ipc4-topology.c Outdated

Comment thread sound/soc/sof/ipc4-topology.c

jsarha force-pushed the new_memory_domain_logic branch from c39e6b2 to c9df7c7 Compare September 26, 2025 11:37

jsarha mentioned this pull request Sep 26, 2025

Code to decode pipeline create messages payload thesofproject/sof#10265

Merged

jsarha force-pushed the new_memory_domain_logic branch from c9df7c7 to cc4773a Compare October 7, 2025 21:36

jsarha force-pushed the new_memory_domain_logic branch from cc4773a to c503c3d Compare November 10, 2025 22:35

lyakh reviewed Nov 14, 2025

View reviewed changes

jsarha force-pushed the new_memory_domain_logic branch from c503c3d to ead2c46 Compare December 3, 2025 19:17

jsarha marked this pull request as ready for review December 4, 2025 13:44

jsarha requested review from bardliao, dbaluta, kv2019i, lgirdwood, plbossart and ujfalusi as code owners December 4, 2025 13:44

bardliao reviewed Dec 4, 2025

View reviewed changes

Comment thread sound/soc/sof/ipc4-topology.c Outdated

lgirdwood reviewed Dec 5, 2025

View reviewed changes

Comment thread include/uapi/sound/sof/tokens.h

Comment thread include/uapi/sound/sof/tokens.h Outdated

jsarha force-pushed the new_memory_domain_logic branch from ead2c46 to 91c9b6e Compare December 11, 2025 18:07

lgirdwood reviewed Jan 13, 2026

View reviewed changes

Comment thread include/uapi/sound/sof/tokens.h

Comment thread sound/soc/sof/ipc4-topology.c

Comment thread sound/soc/sof/ipc4-topology.c

lgirdwood previously approved these changes Jan 20, 2026

View reviewed changes

jsarha dismissed lgirdwood’s stale review via a3430c5 January 21, 2026 18:10

jsarha force-pushed the new_memory_domain_logic branch from b1e5288 to a3430c5 Compare January 21, 2026 18:10

kv2019i previously approved these changes Jan 22, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings April 20, 2026 16:16

jsarha dismissed kv2019i’s stale review via 37f470e April 20, 2026 16:16

jsarha force-pushed the new_memory_domain_logic branch from 7b5faa3 to 37f470e Compare April 20, 2026 16:16

Copilot started reviewing on behalf of jsarha April 20, 2026 16:17 View session

Copilot AI reviewed Apr 20, 2026

View reviewed changes

kv2019i previously approved these changes Apr 22, 2026

View reviewed changes

jsarha changed the title ~~New memory domain logic~~ [DNM] New memory domain logic May 12, 2026

jsarha mentioned this pull request May 28, 2026

[DNM] vregions phase II: Actually use the lifetime allocations and make the memory division between lifetime and interim dynamic thesofproject/sof#10783

Draft

Jyri Sarha added 2 commits June 3, 2026 00:47

jsarha mentioned this pull request Jun 3, 2026

Only heap size attribute thesofproject/sof#10842

Merged

jsarha dismissed kv2019i’s stale review via 2ccc28f June 3, 2026 20:39

jsarha force-pushed the new_memory_domain_logic branch from 37f470e to 2ccc28f Compare June 3, 2026 20:39

jsarha changed the title ~~[DNM] New memory domain logic~~ New memory domain logic Jun 9, 2026

jsarha force-pushed the new_memory_domain_logic branch from 2ccc28f to 669bd79 Compare June 25, 2026 14:56

Copilot AI review requested due to automatic review settings June 25, 2026 14:56

Copilot started reviewing on behalf of jsarha June 25, 2026 14:57 View session

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Jyri Sarha added 5 commits June 25, 2026 21:37

ASoC: SOF: ipc4-topology: Refactor sof_ipc4_widget_mod_init_msg_paylo…

b436b3e

…ad() Refactor sof_ipc4_widget_mod_init_msg_payload() to be easier to extend. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

ASoC: SOF: ipc4-topology: Optimize sof_ipc4_widget_mod_init_msg_paylo…

1404820

…ad() Optimize sof_ipc4_widget_mod_init_msg_payload() so that it skips the payload allocation and rest of the function if no payload is needed. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

jsarha force-pushed the new_memory_domain_logic branch from 669bd79 to 1404820 Compare June 25, 2026 18:44

Uh oh!

Conversation

jsarha commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jsarha commented Sep 17, 2025

Uh oh!

Uh oh!

Uh oh!

lyakh Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jsarha commented Dec 3, 2025

Uh oh!

jsarha commented Dec 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jsarha commented Dec 11, 2025

Uh oh!

jsarha commented Dec 14, 2025

Uh oh!

jsarha commented Jan 12, 2026

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jsarha commented Jan 21, 2026

Uh oh!

kv2019i Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

lyakh Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

jsarha commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jsarha commented Apr 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

jsarha commented May 8, 2026

Uh oh!

jsarha commented Jun 3, 2026

Uh oh!

jsarha commented Jun 9, 2026

Uh oh!

jsarha commented Jun 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

jsarha Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

jsarha commented Sep 17, 2025 •

edited

Loading

jsarha commented Apr 20, 2026 •

edited

Loading