Skip to content

Add bn::base::SegmentedVector#8289

Open
bdash wants to merge 1 commit into
devfrom
test_segmented_vector
Open

Add bn::base::SegmentedVector#8289
bdash wants to merge 1 commit into
devfrom
test_segmented_vector

Conversation

@bdash

@bdash bdash commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

A back-growing segmented array, consisting of a directory of pointers to geometrically sized chunks. It gives portable, controlled chunk sizing (unlike std::deque), O(1) random access, and a guarantee that growth never moves, copies, or reallocates existing elements.

Chunk capacities follow a geometric ramp (B, 2B, 4B, ..., C) and then a fixed plateau (C, C, C, ...), where B is InitialChunkElems and C is MaxChunkElems, both powers of two. A 0 means "use the byte-budget default for T", which gives three usage modes:

SegmentedVector<T> v;        // byte-budget defaults (small ramp to ~64 KiB)
SegmentedVector<T, N> v;     // pure fixed-size N-element chunks (no ramp)
SegmentedVector<T, B, C> v;  // explicit ramp from B to C

@rssor

rssor commented Jun 26, 2026

Copy link
Copy Markdown
Member

having benchmarked this on my branch, it immediately cuts about 40% of allocations related to instruction storage and will be useful in tons of other places, so in favor of merging and then pointing it towards those asap. I've only checked it with the fixed size 128 chunks, and it's probably worth exploring a few others (32->128 geometric might be a further optimization, but fixed-128 is already ~93% efficient).

@plafosse

Copy link
Copy Markdown
Member

@bdash did you say there were som eunit tests that went along with this?

@bdash

bdash commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Yes, on the branch of the same name in the internal repository.

@plafosse

Copy link
Copy Markdown
Member

Seems like there may be a potential inconsistency. bulk_construct ensures we're less than max_size but the other reserve, emplace_ back and push_ back do not.

Comment thread base/segmented_vector.h
// Total element slots across all currently allocated chunks.
size_type capacity() const noexcept { return chunk_base(m_directory.size()); }

void reserve(size_type n)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this check max_size similar to bulk_construct?

Comment thread base/segmented_vector.h
std::destroy_at(&m_directory);
std::construct_at(&m_directory, std::move(newDir));

append_range(other.begin(), other.end());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can append_range throw here after modifying *this? If so, you could possibly create a SegmentedVector tmp(other, other.m_alloc, CopyTag{}); at the top which moves the throwing bit before the *this modification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants