Allocator Protocol¶

The allocator protocol is the central abstraction of PyCyBase — a pluggable memory allocation layer that dispatches to heap, shared-memory (SHM), or raw malloc backends based on global environment settings.

Module: cbase.allocator_protocol

class cbase.allocator_protocol.MemoryBlock¶

Bases: object

Common base for heap/SHM block handles.

Tracks ownership of the underlying C buffer and holds a strong reference to the owning allocator (__allocator__), so an allocator can never be torn down while its blocks are still alive (e.g. interpreter-exit teardown order). Subclasses override _free_block to release the C buffer.

owner¶

Type:: bool

class cbase.allocator_protocol.HeapAllocator¶

Bases: object

__init__(*args, **kwargs)¶

active_page¶

Type:: HeapMemoryPage | None

allocated(self)¶

Return type:: Generator[HeapMemoryBlock]

autopage_alignment¶

Type:: int

autopage_capacity¶

Type:: int

autopage_capacity_max¶

Type:: int

calloc(self, size_t size, bool with_lock=True) → HeapMemoryBlock¶

Parameters:

size (int)
with_lock (bool)

Return type:

HeapMemoryBlock

extend(self, size_t capacity=0, bool with_lock=True) → HeapMemoryPage¶

Parameters:

capacity (int)
with_lock (bool)

Return type:

HeapMemoryPage

free(self, HeapMemoryBlock buffer, bool with_lock=True) → void¶

Parameters:

buffer (HeapMemoryBlock)
with_lock (bool)

Return type:

None

free_list(self)¶

Return type:: Generator[HeapMemoryBlock]

mapped_pages¶

Type:: int

owner¶

Type:: bool

pages(self)¶

Return type:: Generator[HeapMemoryPage]

reclaim(self, bool with_lock=True) → void¶

Parameters:: with_lock (bool)
Return type:: None

request(self, size_t size, bool with_lock=True, bool scan_all_pages=True) → HeapMemoryBlock¶

Parameters:

size (int)
with_lock (bool)
scan_all_pages (bool)

Return type:

HeapMemoryBlock

class cbase.allocator_protocol.HeapMemoryBlock¶

Bases: MemoryBlock

address¶

Type:: str | None

buffer¶

Type:: memoryview

capacity¶

Type:: int

next_allocated¶

Type:: HeapMemoryBlock | None

next_free¶

Type:: HeapMemoryBlock | None

parent_page¶

Type:: HeapMemoryPage | None

size¶

Type:: int

class cbase.allocator_protocol.HeapMemoryPage¶

Bases: object

address¶

Type:: str | None

allocated(self)¶

Return type:: Generator[HeapMemoryBlock]

allocator¶

Type:: HeapAllocator | None

capacity¶

Type:: int

classmethod from_buffer(buffer_addr)¶

Return type:: HeapMemoryPage

occupied¶

Type:: int

reclaim(self) → void¶

Return type:: None

class cbase.allocator_protocol.SharedMemoryAllocator(size_t region_size=AP_SHM_ALLOCATOR_DEFAULT_REGION_SIZE, str shm_prefix=PyUnicode_FromString(AP_SHM_ALLOCATOR_PREFIX))¶

Bases: object

__init__(*args, **kwargs)¶

active_page¶

Type:: SharedMemoryPage | None

allocated(self)¶

Return type:: Generator[SharedMemoryBlock, None, None]

autopage_alignment¶

Type:: int

autopage_capacity¶

Type:: int

autopage_capacity_max¶

Type:: int

calloc(self, size_t size, bool with_lock=True) → SharedMemoryBlock¶

Parameters:

size (int)
with_lock (bool)

Return type:

SharedMemoryBlock

cleanup_dangling(self, str shm_prefix=None) → void¶

Parameters:: shm_prefix (str | None)
Return type:: None

dangling(self, str shm_prefix=None) → list¶

Parameters:: shm_prefix (str | None)
Return type:: list[str]

dangling_pages(self, str shm_prefix=None) → list¶

Parameters:: shm_prefix (str | None)
Return type:: list[str]

extend(self, size_t capacity=0, bool with_lock=True) → SharedMemoryPage¶

Parameters:

capacity (int)
with_lock (bool)

Return type:

SharedMemoryPage

free(self, SharedMemoryBlock buffer, bool with_lock=True) → void¶

Parameters:

buffer (SharedMemoryBlock)
with_lock (bool)

Return type:

None

free_list(self)¶

Return type:: Generator[SharedMemoryBlock, None, None]

classmethod get_pid(cls, str shm_name)¶

Parameters:: shm_name (str)
Return type:: int

mapped_pages¶

Type:: int

mapped_size¶

Type:: int

name¶

str

Type:: SharedMemoryAllocator.name

owner¶

Type:: bool

pages(self)¶

Return type:: Generator[SharedMemoryPage, None, None]

pid¶

Type:: int

reclaim(self, bool with_lock=True) → void¶

Parameters:: with_lock (bool)
Return type:: None

region¶

Type:: int

region_addr¶

Type:: str | None

region_size¶

Type:: int

request(self, size_t size, bool scan_all_pages=True, bool with_lock=True) → SharedMemoryBlock¶

Parameters:

size (int)
scan_all_pages (bool)
with_lock (bool)

Return type:

SharedMemoryBlock

shm_prefix¶

str

Type:: SharedMemoryAllocator.shm_prefix

class cbase.allocator_protocol.SharedMemoryBlock¶

Bases: MemoryBlock

address¶

Type:: str | None

buffer¶

Type:: memoryview | None

capacity¶

Type:: int

next_allocated¶

Type:: SharedMemoryBlock | None

next_free¶

Type:: SharedMemoryBlock | None

page_address¶

Type:: str | None

size¶

Type:: int

class cbase.allocator_protocol.SharedMemoryPage¶

Bases: object

address¶

Type:: str | None

allocated(self)¶

Return type:: Generator[SharedMemoryBlock, None, None]

capacity¶

size_t

Type:: SharedMemoryPage.capacity

classmethod from_buffer(buffer_addr)¶

Return type:: SharedMemoryPage

name¶

str

Type:: SharedMemoryPage.name

occupied¶

size_t

Type:: SharedMemoryPage.occupied

reclaim(self) → void¶

Return type:: None

cbase.allocator_protocol.shm_cleanup()¶: cleanup()

class cbase.allocator_protocol.AllocatorProtocol¶

Bases: object

addr¶

Type:: int

buf¶

Type:: memoryview

size¶

Type:: int

with_freelist¶

Type:: bool

with_lock¶

Type:: bool

with_shm¶

Type:: bool

class cbase.allocator_protocol.AllocatorConfigContext(dict overrides=None, **kwargs)¶

Bases: EnvConfigContext

Parameters:: kwargs (Any)

__init__(*args, **kwargs)¶

Parameters:: kwargs (Any)

Configuration Contexts¶

The AllocatorConfigContext extends EnvConfigContext to dispatch configuration changes to the underlying heap and shared-memory allocators. Accepted keyword arguments:

Key	Type	Description
`locked`	`bool`	Enable/disable mutex locking for thread safety
`shared`	`bool`	Enable/disable shared memory allocation
`freelist`	`bool`	Enable/disable free-list reuse
`autopage_capacity`	`int`	Propagated to both heap and SHM allocators
`autopage_capacity_max`	`int`	Propagated to both heap and SHM allocators
`autopage_alignment`	`int`	Propagated to both heap and SHM allocators

Sentinel Contexts¶

The module exports pre-configured context instances for common use:

from cbase.allocator_protocol import AP_SHARED, AP_LOCKED, AP_LOCKFREE, AP_FREELIST

AP_SHARED — enable shared-memory allocation
AP_LOCKED — enable thread-safety locking
AP_LOCKFREE — disable thread-safety locking
AP_FREELIST — enable free-list reuse (no effect in SHM mode)

Usage Examples¶

Basic allocation with the default allocator (SHM-backed):

from cbase.allocator_protocol import AllocatorProtocol, AP_SHARED

with AP_SHARED:
    alloc = AllocatorProtocol(1024)
    alloc.buf[0] = b'x'
    print(f"size={alloc.size}, with_shm={alloc.with_shm}")

Composing contexts with |:

from cbase.allocator_protocol import AP_SHARED, AP_LOCKED, AP_FREELIST

# SHM + thread-safe + freelist
ctx = AP_SHARED | AP_LOCKED | AP_FREELIST

with ctx:
    alloc = AllocatorProtocol(4096)
    # ... use alloc ...

Inverting a context with ~ to disable a flag:

unlocked = ~AP_LOCKED  # disables locking

Using as a decorator:

@AP_SHARED
def allocate_shared():
    return AllocatorProtocol(2048)

Under the Hood¶

The module maintains three global allocator schematics (C structs) that define the default, SHM-only, and heap-only configurations:

AP_DEFAULT_ALLOCATOR — SHM-backed, locked, freelist-enabled
AP_SHM_ALLOCATOR — SHM-backed, locked, freelist-enabled, no heap
AP_HEAP_ALLOCATOR — heap-backed, locked, freelist-enabled, no SHM

AllocatorProtocol(size) checks AP_DEFAULT_ALLOCATOR.with_shm at construction time and selects the appropriate backend.