# FastAPI and p99 follow-up

The starting extension is `09140253d30af272785eadc1a1d2613a6e2066653a63dbe09a552dcc9d762705`,
archived with its source in `round4-before-source.tar.gz`. Final wheel identity,
qualification and benchmark totals are recorded below after verification.

## Implementation

The reactor services every returned event, with sixteen-event batches for cheap
work and at most 32 when final flushes are queued. Eager starts are bounded at
32, pipeline pumps at sixteen and ready operations at 128 per request. A
callback deadline that abandoned already-returned events caused stalls and was
rejected. The earlier adaptive 128-event candidate also had tail regressions;
its measurements remain explicitly an intermediate checkpoint.

Small final responses, below 64 KiB and after at least 20 microseconds of
connection activity, attempt a real nonblocking MSG_MORE send. Errors are checked
before send returns. Remaining bytes stay owned by the connection. The end of the bounded native
poll callback explicitly pushes kernel-held bytes by setting TCP_NODELAY,
then writes any remainder. Independently resumed applications retain a bounded
call_soon flush callback. Linux's TCP implementation explicitly pushes pending
data on this option, including when NODELAY is already enabled. Streaming sends
retain immediate writes; graceful drain accounts for pending kernel pushes.
This does not remove native/Python backpressure, request identities, validation,
Task isolation or the ASGI operation budget. No payload or response is cached.

Queued flushes do not arm EPOLLOUT before attempting the write. Early batching
prototypes armed and disarmed it around an already-scheduled flush, introducing
two epoll modifications per response. EPOLLOUT is now armed only for actual
remaining bytes after a flush attempt. Completion callbacks respect an existing
flush queue rather than forcing each eager Task's response out individually.

Request targets use spans into owned header storage; strict CRLF scanning still
rejects bare LF and embedded CR. Native head vectors are recycled only after
Python owns all scope values, with a 64 KiB storage capacity cap. Short normalized
header blocks (at most eight fields and 512 bytes) can share immutable tuples and
bytes across connections. Every request gets its own mutable list. Each
connection owns at most one snapshot; the worker hint is weak. Byte comparison,
not a hash or a route name, decides reuse. Large/unusual headers follow the
normal validated path. Header names retain the prior bounded intern cache.

The optional state namespace is omitted only with explicit lifespan off.
Auto/on still pass a fresh shallow copy. All benchmark matrices record this
setting, including an enabled comparison. Common scope values and Task keyword
names are interned; loop bound methods are captured once, while the task factory
itself is still queried for every dispatch. Status digits serialize directly.

Sender's vectorcall entry uses its existing PyO3 allocation and ownership. The
field offset is checked against the actual allocated layout before installing
type slots under the GIL. The type is immutable and non-subclassable; layout
changes fall back to the ordinary call path. Calls preserve PyO3 thread checks,
panic/error translation and dictionary type checks. Positional and keyword calls
are qualified at the socket boundary. There is no pointer registry or parallel
reference-counting system. This adds unsafe CPython integration and needs
qualification on other Python versions; it must not be described as no-new-unsafe.
The optimized call entry improved FastAPI throughput about 1.2% in the isolated
`round4-vectorcall2-isolated-c64` comparison, whose before version is the otherwise
identical metadata/batching candidate, not the user-turn starting version.

There are still three direct Rust dependencies and no added Python runtime
dependency. Native source is 2,474 lines, up from 2,102 at the start of this turn.

## Profiles and remaining overhead

The checkpoint user-cycle profiles `round4-retained-native` and
`round4-peregrine-native` recorded zero lost samples. Interpreter evaluation
accounted for 48.39% and 49.19% respectively; separately normalized user-cycle
shares cannot be treated as throughput headroom or entirely application work.
Python-only worker profiles contained 5,656 and 5,951 samples with no sampling
errors. Skirr's last Python frame was Starlette's send wrapper in 656/5,137
FastAPI-stack samples, versus 141/4,528 for Peregrine. These include native work
below the last frame. Peregrine's deferred I/O moves work outside the application
frame; this difference is not the cost of those Python statements alone.

Later candidate comparisons record user/system CPU seconds per request. In
`round4-poll-methods-c64`, FastAPI needed roughly 33.9 microseconds user time per
request in Skirr and 34.0 in Peregrine, with system time about 4.16 versus 3.90.
Those are noisy per-process accounting estimates for that candidate, not final
cycle measurements. A poll-based error probe did not improve them and was
replaced by the real MSG_MORE send. Routing, dependency handling, JSON encoding,
header construction and native call/allocation work remain visible; the workload
spends most of its time in Python. Application GC/allocator defaults are retained.

Final exact-wheel profiles and results are appended below. Profiles are
instrumented diagnostics, never throughput evidence. Saturation wrk p99 and
fixed-rate wrk2 corrected histograms are reported separately.

## Rejected and excluded evidence

The adaptive checkpoint is documented in `latency-round4-checkpoint-study.md`.
Other trials remain under `bench/results/round4-*`. Bound send methods, Python
Task helpers, the first unoptimized vectorcall path, exception-driven pending
Task detection, a global header-pair cache, a yielding flush acknowledgment and
64-event flush cohorts did not justify their cost or worsened tails. They are
not in the final implementation. The completed-awaitable implementation is
unchanged. An ordinary socket deadline test did not reproduce the discarded
event-abandonment stall and was removed; no misleading regression claim is made.

An initial perf JIT attempt omitted the required monotonic clock and failed.
Postprocessed intermediate JIT stacks whose native binary had been overwritten
are explicitly named invalid and excluded. Even valid JIT profiles have uneven
Python-frame coverage between eager and ordinary Tasks. Comparative Python
attribution uses Python-only worker samples, rather than those JIT percentages.

## Protocol qualification

The expanded socket cases protect preserved behavior: absolute-form path/query
handling, malformed CRLF, retained/mutated headers, same-length changed values,
large/duplicate headers, lifespan state isolation and send argument validation.
They are not claimed as old correctness bugs; the saved starting release is
also checked. Existing tests cover streaming visibility, reset error delivery,
partial writes, independently waiting apps, custom Task factories, context
isolation, Task failure/cancellation, ready-await fairness and pipelining.
No test-only runtime seam was introduced. This is an HTTP/1 ASGI prototype;
these checks do not establish complete protocol or production qualification.

References: [ASGI HTTP 2.5](https://asgi.readthedocs.io/en/latest/specs/www.html),
[Linux 6.17 TCP implementation](https://github.com/torvalds/linux/blob/v6.17/net/ipv4/tcp.c).

## Final-wheel qualification and profiles

The retained extension SHA256 is
`a126b0fbc2e541fda09d4d0e533d4dd4b7910891f4291f3ef9cf1837143ca73a`.
The wheel SHA256 is
`b2a01f65ba9b2c3ed2905a7786357d815d306bb03da6cce0912d30be7bc5fcbd`.
Strict Clippy, rustfmt and three parser tests pass. All 64 real-socket/ASGI tests
pass against the installed wheel on asyncio and uvloop with the debug allocator,
and against a separate Rust AddressSanitizer build on both loops. The starting
release also passes the expanded suite. Leak detection is disabled for the
uninstrumented Python embedding host. See `asan-round4-final-check.txt`.

`round4-inline-isolated-c64` compares the inline poll-boundary flush with the
otherwise identical queued-flush candidate: median FastAPI throughput improves
1.9% and p99 improves 2.3%. Raw throughput slips 1.2% in that trial. The selected
version additionally avoids entering the flush routine when no flush is queued;
its final controls are reported separately. Every already-returned poll event
is still processed. Errors leave the polling phase before pending data is
flushed; independently resumed apps retain their scheduled callback.

The prior qualified MSG_MORE-only extension (868aeefd...) and its processed
profiles remain in `round4-more-qualified-*`. Its complete c64 matrix and partial,
interrupted c256 matrix are intermediate evidence, not the final comparison.
All native reports were processed before replacing their binary.

The final exact-wheel delayed user-cycle profile records 4,047 samples, versus
3,966 in Peregrine's profile, with zero lost samples reported in either. The
interpreter evaluator accounts for 49.11% and 49.19% respectively. Allocation,
frame cleanup and Python attribute/API calls remain prominent. These are
separately normalized user cycles, not kernel costs or predicted headroom.

The final Python-only profiles contain 5,729 Skirr and 6,767 Peregrine samples,
with zero sampling errors. The last Python frame is Starlette's send wrapper
in 271/4,882 (5.6%) and 134/4,734 (2.8%) FastAPI-stack samples. Moving I/O out of
the application frame changes this attribution; it is not a direct measurement
of wrapper statements. Routing, dependency handling, JSON encoding and response
header construction are leading named application paths in both servers.

Tracing 1,001 sequential FastAPI responses shows 1,001 Skirr writes, no EAGAIN
read probes, no SO_ERROR probes and no EPOLLOUT modifications. There are also
1,003 TCP_NODELAY calls, including setup and about one pending-data push per
response. Peregrine has 1,002 writes and three NODELAY setup calls. This extra
push is a concrete remaining per-response cost. Sequential traces do not measure
the benefit of grouping client wakeups under concurrent load. All profiles and
traces were completed separately from uninstrumented benchmark timing.

## Interpretation of the saturation results

The completed five-repeat FastAPI matrices improve throughput over the starting
release by 3.4% at 64 connections and 3.1% at 256. Relative to Peregrine, Skirr is
0.3% ahead at 64 and 2.3% behind at 256. These are single-host medians, not a
statistical claim of superiority; sub-percent differences should be treated as
a tie given clock and run variation.

At 256 connections, FastAPI p99 falls from 15.35 to 14.38 ms, versus Peregrine's
16.34 ms. At 64, p99 is 3.93 ms versus the starting release's 3.61 and Peregrine's
3.73. That c64 result remains a regression in the median; it must not be omitted
or described as fixed. The per-run ranges overlap substantially: new Skirr
3.72–4.54, starting Skirr 3.01–4.89 and Peregrine 3.56–4.02 ms.

Tiny-response and 4 KiB upload throughput and median p99 beat Peregrine at both
connection counts. Relative to the starting release, tiny c256 p99 rises from
1.85 to 1.95 ms (new range 1.43–3.45), while tiny c64 is essentially unchanged (0.560 to 0.564 ms).
Uploads improve p99 from 0.79 to 0.61 ms at 64 and 2.67 to 2.49 ms at 256.
The optimization does not eliminate every tail regression or consistently beat
Peregrine on both FastAPI throughput and latency. Fixed-rate corrected latency
and lifespan-enabled results are reported separately below.

## Equal-rate, lifespan and final audit

At a fixed offered 20,000 requests/s, all servers sustain approximately 19,975
completed requests/s using wrk2. Corrected p99 is 9.21 ms in Skirr, 9.63 in the
starting release and 9.38 in Peregrine: 4.4% and 1.8% lower medians respectively.
The ranges overlap (Skirr 9.12–9.36, starting release 8.20–10.78, Peregrine
9.19–9.57 ms). These corrected histograms must not be pooled with the saturation
wrk percentiles. See [the fixed-rate report](../results/latency-round4-final-rate20k-report.txt).

With lifespan enabled at 64 connections, Skirr reaches 25,693 requests/s and
3.73 ms p99, versus the starting release's 25,226 / 3.60 and Peregrine's
25,850 / 3.98. Throughput improves 1.9% over the starting release and remains
0.6% below Peregrine. This setting preserves the state namespace, matching the
Skirr default auto/on path. The enabled p99 median also remains slightly above
the starting release; namespace omission is not presented as a default-mode gain.

The final audit covers 84 timed runs and 48,532,004 completed requests, with zero
socket/timeout or HTTP error counters. All exact source and binary hashes match
the qualified wheel, all observed serving-worker affinities are CPU 0, and every
raw run log is present. Server orders are rotated, CPU clocks recorded and
allocator/GC defaults retained. Exact status/body validation runs before and
after each timed workload, not on every wrk response. See
[the full report](../results/latency-round4-final-report.txt) and
[the audit](../results/latency-round4-final-audit.json).

Temporary performance holds restored the original power-saver profile. Perf
security remains at the user-enabled value 2; Skirr does not alter that setting.

The across-the-board FastAPI throughput-and-latency target is **not fully met**.
The c64 saturation p99 and tiny c256 p99 medians remain above the starting
release, and Peregrine retains the c256 FastAPI throughput lead. The retained
implementation improves throughput and matched-rate latency without weakening
the tested ASGI/HTTP contracts. Further work needs to reduce the per-response
TCP push/call/allocation costs while preserving closed-send errors and Task
semantics; broader protocol/fuzz/platform qualification remains separate work.