Skip to main content

Bottleneck Analysis

TODO — pending re-measurement

The previous analysis relied on deleted benchmark/results/* artifacts (bottleneck-analysis.md and internal-stage-toggle-comparison.md). Those measurements found that spawn_blocking_delay and event_loop_resume_delay dominated the cost under the GIL. The Production Benchmark → Python 3.14t section preserves that result and shows both stages falling to nearly zero under free-threaded Python.

The page still needs a fresh per-stage breakdown from the new Isolated Benchmark harness (s1–s5, oha, Podman, and local Aerospike CE 8.1.2.1_3). The harness already supports stage profiling through AEROSPIKE_PY_INTERNAL_METRICS=1, which enables the db_client_internal_stage_seconds histograms with negligible disabled-path overhead. The full matrix has not yet been run and summarized.

Tracked work:

  • Re-run the per-stage profiling matrix on the isolated harness (s1 / s3 / s4_gather × py3.11 GIL × py3.14t free-threaded)
  • Re-validate the gather(N) → single batch_read(mixed keys) recipe on s4_gather vs s4_single
  • Document the stage-profiling activation surface (env var · runtime toggle · scoped context manager) once the new numbers exist

What this page will cover (when re-measured)​

  1. Per-stage breakdown of batch_read under load, with named stages (key_parse, limiter_wait, tokio_schedule_delay, io, into_pyobject, spawn_blocking_delay, event_loop_resume_delay, to_dict, to_numpy, merge_to_dict, future_into_py_setup).
  2. Stage-profiling activation — AEROSPIKE_PY_INTERNAL_METRICS=1 env var, aerospike_py.set_internal_stage_metrics_enabled(True) runtime toggle, scoped context manager. All three exist today; their overhead is roughly one AtomicBool load per stage entry when disabled.
  3. Recipes that measurably help. The known winners from the previous round were (a) switch to aerospike-py, (b) consolidate gather(N batch_reads) into a single batch_read(mixed keys), (c) move to Python 3.14t free-threaded. (a) and (c) are already quantified on the new harness in Isolated Benchmark and Production Benchmark; (b) is what s4_gather vs s4_single is set up to re-measure.

Until new measurements are available, use the Production Benchmark's 3.14t stage breakdown as the reference for where time is spent.