Shared-Memory Allocator

Module: cbase.allocator_protocol.c_shm_allocator (POSIX) Module: cbase.allocator_protocol.c_nt_shm_allocator (Windows)

A POSIX shared-memory backed page allocator that maps pages into a reserved virtual address region, enabling cross-process pointer stability — pointers within the region are valid across fork() and independent processes.

The allocator:

• Creates named shm_open objects and maps them into a reserved virtual address region.

• By default reserves 128 GiB of virtual address space for page mappings.

• Uses fixed-address mapping so pages land at stable addresses across processes.

• Is POSIX-only (Linux, macOS). On Windows, a limited NT compat layer is provided.

class cbase.allocator_protocol.c_shm_allocator.SharedMemoryAllocator(size_t region_size=AP_SHM_ALLOCATOR_DEFAULT_REGION_SIZE, str shm_prefix=PyUnicode_FromString(AP_SHM_ALLOCATOR_PREFIX))

Bases: object

__init__(*args, **kwargs)

active_page

Type:

SharedMemoryPage | None

allocated()

Return type:

Generator[SharedMemoryBlock, None, None]

autopage_alignment

Type:

int

autopage_capacity

Type:

int

autopage_capacity_max

Type:

int

calloc(size, with_lock=True)

Parameters:

• size (int)

• with_lock (bool)

Return type:

SharedMemoryBlock

cleanup_dangling(shm_prefix=None)

Parameters:

shm_prefix (str | None)

Return type:

None

dangling(shm_prefix=None)

Parameters:

shm_prefix (str | None)

Return type:

list[str]

dangling_pages(shm_prefix=None)

Parameters:

shm_prefix (str | None)

Return type:

list[str]

extend(capacity=0, with_lock=True)

Parameters:

• capacity (int)

• with_lock (bool)

Return type:

SharedMemoryPage

free(buffer, with_lock=True)

Parameters:

• buffer (SharedMemoryBlock)

• with_lock (bool)

Return type:

None

free_list()

Return type:

Generator[SharedMemoryBlock, None, None]

classmethod get_pid(cls, str shm_name)

Parameters:

shm_name (str)

Return type:

int

mapped_pages

Type:

int

mapped_size

Type:

int

name

str

Type:

SharedMemoryAllocator.name

owner

Type:

bool

pages()

Return type:

Generator[SharedMemoryPage, None, None]

pid

Type:

int

reclaim(with_lock=True)

Parameters:

with_lock (bool)

Return type:

None

region

Type:

int

region_addr

Type:

str | None

region_size

Type:

int

request(size, scan_all_pages=True, with_lock=True)

Parameters:

• size (int)

• scan_all_pages (bool)

• with_lock (bool)

Return type:

SharedMemoryBlock

shm_prefix

str

Type:

SharedMemoryAllocator.shm_prefix

class cbase.allocator_protocol.c_shm_allocator.SharedMemoryBlock

Bases: MemoryBlock

address

Type:

str | None

buffer

Type:

memoryview | None

capacity

Type:

int

next_allocated

Type:

SharedMemoryBlock | None

next_free

Type:

SharedMemoryBlock | None

page_address

Type:

str | None

size

Type:

int

class cbase.allocator_protocol.c_shm_allocator.SharedMemoryPage

Bases: object

address

Type:

str | None

allocated()

Return type:

Generator[SharedMemoryBlock, None, None]

capacity

size_t

Type:

SharedMemoryPage.capacity

classmethod from_buffer(buffer_addr)

Return type:

SharedMemoryPage

name

str

Type:

SharedMemoryPage.name

occupied

size_t

Type:

SharedMemoryPage.occupied

reclaim()

Return type:

None

cbase.allocator_protocol.c_shm_allocator.__reduce_cython__(self)

cbase.allocator_protocol.c_shm_allocator.__setstate_cython__(self, __pyx_state)

cbase.allocator_protocol.c_shm_allocator.cleanup()

Compile-Time Constants

Auto-page and SHM configuration:

from cbase.allocator_protocol.c_shm_allocator import (
    AP_SHM_AUTOPAGE_CAPACITY,             # 64 KiB — first/auto page size
    AP_SHM_AUTOPAGE_CAPACITY_MAX,         # 16 MiB — maximum page size
    AP_SHM_AUTOPAGE_ALIGNMENT,            # 4 KiB — page alignment
    AP_SHM_ALLOCATOR_PREFIX,              # "/c_cbase_shm" — default SHM prefix
    AP_SHM_NAME_LEN,                      # 256 — max SHM name length
    AP_SHM_PREFIX_MAX,                    # 64 — max custom prefix length
    AP_SHM_ALLOCATOR_DEFAULT_REGION_SIZE, # 128 GiB — default region size
)

Size-binned free-list tuning (added in v0.1.9):

from cbase.allocator_protocol.c_shm_allocator import (
    AP_SHM_EXACT_BIN_COUNT,              # 8192 — exact 8-byte-granular bins (≤ 64 KiB)
    AP_SHM_LARGE_BIN_COUNT,              # 9 — pow2-class bins (> 64 KiB)
    AP_SHM_BIN_COUNT,                    # 8202 — total bin count (EXACT + LARGE + 1)
    AP_SHM_PAGE_EXTEND_MAX,              # 128 MiB — max auto-page extension
    AP_SHM_PAGE_FIT_TO_REQUEST,          # 0 — when 1, page fits request instead of pow2 scaling
    AP_SHM_EXACT_BIN_PROBE_COUNT,        # 2 — exact bins to probe on miss before page path
)

Naming Convention

SHM objects follow a naming scheme based on the creator PID:

• Allocator SHM objects: /c_cbase_shm_ac_<pid>_<hash>

• Page SHM objects: /c_cbase_shm_pg_<pid>_<index>

Usage Examples

Creating and using a shared-memory allocator:

from cbase.allocator_protocol.c_shm_allocator import SharedMemoryAllocator

# Create with default 128 GiB region
alloc = SharedMemoryAllocator()

# Or with custom region size and SHM prefix
alloc = SharedMemoryAllocator(region_size=64 << 30, shm_prefix="/my_app_shm")

# Extend with a default-sized page
page = alloc.extend()

# Allocate zeroed memory
block = alloc.calloc(4096)
buf = block.buffer  # memoryview

# Free and reclaim
alloc.free(block)
alloc.reclaim()

Cross-Process Sharing

The allocator is designed for fork()-based multi-process setups:

import os
from cbase.allocator_protocol.c_shm_allocator import SharedMemoryAllocator

alloc = SharedMemoryAllocator()

# Parent allocates
block1 = alloc.calloc(1024)
buf1 = block1.buffer
buf1[0] = 42

pid = os.fork()
if pid == 0:
    # Child — same addresses, same data
    assert buf1[0] == 42
    buf1[0] = 99  # Visible to parent through shared memory
else:
    os.waitpid(pid, 0)
    assert buf1[0] == 99

Dangling SHM Cleanup

Orphaned SHM objects from crashed processes can be listed and cleaned up:

# List dangling allocator SHM names
dead = alloc.dangling()
print(dead)

# List dangling page SHM names
dead_pages = alloc.dangling_pages()

# Clean up all dangling objects
alloc.cleanup_dangling()

# Or target a specific prefix
alloc.cleanup_dangling(shm_prefix="/my_custom_prefix")

Extracting Creator PID from SHM Names

pid = SharedMemoryAllocator.get_pid("/c_cbase_shm_ac_12345_7f...")
print(pid)  # 12345

Note

The Windows implementation (NtSharedMemoryAllocator) provides a limited subset of functionality and does not support cross-process pointer stability.

See also

• Allocator Protocol — The pluggable protocol that selects backends

• Heap Allocator — In-process heap allocator