Architecture
aerospike-py uses PyO3 to expose the Aerospike Rust Client as a native Python extension. Applications get a typed, Python-friendly API while the Rust core handles Aerospike I/O.
Key properties:
- GIL-free I/O — the GIL is released during every database call, so other Python threads and coroutines run concurrently.
- Type-safe — ships with
.pyistubs for IDE autocompletion and type-checker support. - Zero dependencies — the base install has no external Python dependencies. NumPy and OpenTelemetry are optional extras.
Layers
┌─────────────────────────────────────────────┐
│ Your Python Application │
├─────────────────────────────────────────────┤
│ Python Wrapper Layer │
│ _client.py / _async_client.py │
│ NamedTuple wrapping, error decoration │
├─────────────────────────────────────────────┤
│ PyO3 Binding Layer │
│ client.rs / async_client.rs │
│ GIL management, Python ↔ Rust conversion │
├─────────────────────────────────────────────┤
│ Aerospike Rust Client │
│ aerospike-core crate (fully async) │
│ Aerospike wire protocol, connection pool │
├─────────────────────────────────────────────┤
│ Aerospike Server │
└─────────────────────────────────────────────┘
| Layer | Role |
|---|---|
| Python Wrapper | Thin layer that converts raw tuples from Rust into Record, ExistsResult, and other NamedTuples. Adds context-manager support and the @catch_unexpected decorator. |
| PyO3 Binding | #[pyclass] structs that bridge Python calls to the async Rust client. Handles GIL release/reacquire and type conversion between Python objects and Rust types. |
| Aerospike Rust Client | The aerospike-core crate — a fully async client that speaks the Aerospike binary wire protocol over TCP. Manages connection pooling, cluster discovery, and partition maps. |
| Aerospike Server | The Aerospike database (Community or Enterprise). |
Sync vs Async
Both clients expose the same API surface. The difference is how they schedule I/O on the internal Tokio runtime.
- Sync
- Async
import aerospike_py
client = aerospike_py.client({"hosts": [("localhost", 3000)]}).connect()
client.put(("test", "demo", "user1"), {"name": "Alice"})
record = client.get(("test", "demo", "user1"))
print(record.bins) # {"name": "Alice"}
client.close()
Each call releases the GIL and runs the Rust future on a Tokio runtime through block_on(). It reacquires the GIL only when returning the result, so other Python threads can run during the I/O wait.
import aerospike_py
client = aerospike_py.AsyncClient({"hosts": [("localhost", 3000)]})
await client.connect()
await client.put(("test", "demo", "user1"), {"name": "Alice"})
record = await client.get(("test", "demo", "user1"))
print(record.bins) # {"name": "Alice"}
await client.close()
Each call returns a Python awaitable backed by a Tokio future. The GIL is not held during I/O, so concurrent await calls overlap naturally with asyncio.gather() or task groups.
Performance comparison (vs official C client)
| Path | put | get | batch_read (NumPy) |
|---|---|---|---|
| Sync (sequential) | ~1.1x slower | ~1.1x slower | — |
| Async (concurrent) | 2.1x faster | 1.6x faster | 3.4x faster |
The sync path pays about 10% for block_on() on each call. The async path overlaps concurrent I/O and avoids that per-call scheduling cost.
Data Flow
Write path (put)
- Python dict
{"name": "Alice"}is converted to RustVec<Bin>. - Key tuple
("test", "demo", "user1")becomes an AerospikeKey(with RIPEMD-160 digest). - The GIL is released. The Rust client serializes bins into the Aerospike wire protocol and sends them over TCP.
- The server acknowledges. The GIL is re-acquired and
Noneis returned to Python.
Read path (get)
- The GIL is released. The Rust client sends a read request and receives the response.
- The Rust
Record(bins + generation + TTL) is converted to a Python tuple(key, meta, bins). - The Python wrapper layer wraps this into a
RecordNamedTuple:
record = client.get(("test", "demo", "user1"))
record.bins # {"name": "Alice"}
record.meta.gen # 1 (generation counter)
record.meta.ttl # 0 (seconds until expiration)
record.key.user_key # "user1"
Type conversion
| Python | Aerospike | Notes |
|---|---|---|
int | Integer | 64-bit signed |
float | Double | 64-bit IEEE 754 |
str | String | UTF-8 |
bytes | Blob | Raw bytes |
list | List | Nested types supported |
dict | Map | Nested types supported |
bool | Bool | |
None | Nil | Removes the bin on write |
Batch Operations
batch_read
Returns a LazyBatchRecords handle. Its mapping interface exposes successful
user-key reads as user_key → bins. Call to_dict() only when you need a plain
dictionary, or inspect batch_records / iter_records() when you need every
per-record result, including missing and failed reads.
- Sync
- Async
keys = [("test", "demo", f"user_{i}") for i in range(1000)]
batch = client.batch_read(keys, bins=["name", "age"])
for user_key, bins in batch.items():
print(user_key, bins["name"])
# Check which keys are missing
requested = {k[2] for k in keys}
missing = requested - set(batch.keys())
batch = await client.batch_read(keys, bins=["name", "age"])
for user_key, bins in batch.items():
print(user_key, bins["name"])
For high-throughput pipelines, pass a NumPy dtype to get a structured array with zero-copy columnar access. See the NumPy Batch Read guide for details.
import numpy as np
dtype = np.dtype([("score", "f8"), ("count", "i4")])
result = client.batch_read(keys).to_numpy(dtype)
print(result.batch_records["score"].mean()) # columnar access
batch_write
Each record is a (key, bins) tuple. Optionally add a third element for per-record metadata like TTL:
records = [
(("test", "demo", "user1"), {"name": "Alice", "age": 30}),
(("test", "demo", "user2"), {"name": "Bob"}, {"ttl": 3600}), # expires in 1 hour
]
results = client.batch_write(records, policy={"ttl": 86400}) # default: 1 day
TTL priority: per-record {"ttl": N} > batch-level policy={"ttl": N} > namespace default.
Retry: The default retry=0 does not retry failed records. Set retry to a
value greater than zero to retry eligible failures such as timeout, device
overload, or key busy with exponential backoff. Retries stop early when the
elapsed time approaches total_timeout.
results = client.batch_write(records, retry=3)
for br in results.batch_records:
if br.result != 0:
if br.in_doubt:
print(f"Key {br.key} may have succeeded — verify before retrying")
else:
print(f"Key {br.key} failed (code={br.result})")
Error Handling
Errors from the server are mapped to a Python exception hierarchy rooted at AerospikeError. Each exception carries the original error message and result code.
from aerospike_py.exception import RecordNotFound, AerospikeError
try:
record = client.get(("test", "demo", "missing"))
except RecordNotFound:
print("Record does not exist")
except AerospikeError as e:
print(f"Unexpected error: {e}")
Batch operations report failures per record instead of raising for the whole batch. Check br.result on each BatchRecord. See the Error Handling guide for the exception hierarchy and batch patterns.
Observability
aerospike-py has built-in support for tracing, metrics, and logging. All three are optional and have near-zero overhead when disabled.
OpenTelemetry Tracing
Every database operation emits an OTel span with db.system.name, db.namespace, db.operation.name, and other semantic attributes. Install aerospike-py[otel] and initialize:
from aerospike_py import init_tracing, shutdown_tracing
init_tracing() # uses OTEL_* env vars for exporter config
# ... use client ...
shutdown_tracing()
Prometheus Metrics
Operation durations are recorded as histograms. Expose them via the built-in HTTP server or read programmatically:
from aerospike_py import start_metrics_server, get_metrics
start_metrics_server(9090) # GET http://localhost:9090/metrics
print(get_metrics()) # text format
Logging
Rust internal logs are bridged to Python's logging module:
from aerospike_py import set_log_level, LOG_LEVEL_DEBUG
set_log_level(LOG_LEVEL_DEBUG)
See the Observability guides for detailed configuration.
Design Principles
- Rust-first — Core logic lives in Rust. Python is a thin wrapper for ergonomics (NamedTuples, context managers, factory functions).
- Zero Python dependencies — Base install has no external Python deps. NumPy and OpenTelemetry are optional extras (
pip install aerospike-py[numpy,otel]). - Type-safe —
.pyistubs provide full IDE support. All return types are NamedTuples with named fields, not raw dicts or tuples. - API compatibility — Method names, constants, and exceptions align with the official Aerospike Python client where practical.
- GIL-free I/O — Every database operation releases the GIL during the network call. Sync uses
py.detach()+ Tokioblock_on(); async usesfuture_into_py(). See Performance Tuning for runtime worker configuration.