Prospective Spark concurrency profile — draft for root review

Status: plan and read-only grouping audit only. No production file, active case,
queued request, first benchmark, model or environment has been changed. The new
worktree is evaluation-exp8-throughput on branch exp8-throughput. The candidate
configuration is not an executable authorization or a completed implementation.

Scope

Keep p10c, p25c and g05b on the existing runtime for their entire conversations.
Their manifests bind current source hashes and their receipts identify completed
requests under that runtime. Migrating them would require changing those bindings
and continuation semantics. This proposal deliberately excludes that migration.
The existing three-editor kernel limit remains; this proposal grants no extra
concurrent kernel slot.

For untouched p50c, propose max_num_seqs=64 with GPU fraction 0.35. Qwen's actual
startup reported 97.18 full-context request capacity at the current memory budget,
and the 32-request pass peaked 30.6% KV usage. This supports considering 64; it does
not measure speedup or guarantee the new profile's post-profiling memory margin.

Untouched g10b/g25b/g50b are conditional candidates only. They may not start under
this profile until the existing legitimate Gemma request supplies actual startup
capacity telemetry and root separately reviews it. Qwen's capacity estimate is
not evidence of Gemma capacity. Do not run extra generation merely for timing.

All initial prompts, instructions, notice content/schedule, Python/forward budgets,
seeds, sampling, model revisions, protected tensors, tensor storage/compute types,
activation conventions and scoring remain unchanged. Each untouched case starts
from its unedited original base in a new conversation. Already prepared but never
started v1 case manifests remain archived; an external supersession index links
them to the new case manifest without changing either source record.

Isolation and source identity

Use a new remote runtime root TASK/reconstructed/exp8-s64-v2 and results root
TASK/results/reconstructed-exp8-s64-v2/runs. Local results go to
outputs/replication/reconstructed-exp8-s64-v2. Do not overwrite the existing
exp8 runtime, seven prepared v1 cases, startup review, scratch, receipts or results.
TASK is <gpu-project>.

All old/new editor and evaluator processes MUST use the same canonical lock:
TASK/reconstructed/exp8/gpu.lock. A new runtime must not derive a separate lock
from its own parent directory. This is an explicit implementation requirement.
New compile caches can live under the new runtime root. Reuse the exact existing
CPU dependency overlay read-only, existing CUDA environment read-only, base-model
paths, frozen Docker image ID and practice source; do not install new packages.
Mount the new harness into only new kernels. Old kernels retain their old mounts.

The candidate profile, all production source files, original prompt, base files,
practice files, dependency context and Docker identity receive fresh hashes.
Each new case needs its own root startup approval binding its manifest, profile
and source hashes. Keep that approval immutable per case. Gemma capacity approval
can therefore be added for a different case without modifying p50c's approval.

Implementation required after plan review

1. Add one centrally hashed execution-profile module/config to the fork. It owns
   the new namespaces, canonical shared lock and generation concurrency 64. The
   config must reject active v1 cases and unsupported model/case combinations.
2. Update forked controller routing, kernel harness path and per-case startup
   approval checks. Include execution-profile ID/hash in each initial manifest.
   The existing gpt-6-astra/max CLI bridge and actual context audit stay unchanged.
3. Have the forked backend pass --max-num-seqs 64 explicitly to generate.py; leave
   generation sample counts, seed function, max_tokens, dtype, eager mode,
   max_num_batched_tokens 4096, prefix-cache setting and all other controls intact.
   Existing generate.py already accepts the argument and reports its actual value.
4. Bind each new request signature and success receipt to canonical request,
   frozen edits, base hash, execution-profile hash and reviewed source identity.
   Both controller and backend must compute the same identity. Verify that
   identity, publications and result hashes on resume. A copied v1 receipt or a
   different profile must fail verification, never be silently reused or rerun.
   Existing v1 receipts keep their original semantics and remain untouched.
5. Record the profile/source identity in new frozen-final submission metadata;
   retain the existing immutable submission files and repeated-finalize behavior.
   Old and new submissions use the same tensor validation and checkpoint assembly.

The candidate configuration here describes these changes; the live code and this
worktree's production modules are still the approved v1 implementation. Code must
be implemented and independently reviewed before any deployment or launch.

Separate held-out proposal

Keep ALL Gemma final evaluations on the original 32-answer partitions: nine
MATH500 model variants at 63 shards each and the unedited GSM8K baseline at 83.
This preserves the queued first Gemma MATH500 shard 0/63 and keeps its entire
comparison family under one grouping/runtime. All 650 Gemma shards remain v1.
The existing first 32-record completion/independent audit gate still applies.

Only Qwen benchmark comparison families are candidates for 64-answer shards.
All seven Qwen model variants and all ten datasets are currently unstarted in
held-out scoring. Recheck this immediately before applying the plan; if any unit
has started, stop and seek a revised family-level plan rather than mixing jobs.
Use a separate exp8-bench-s64-v2 code/results namespace, a fresh startup review,
the same canonical GPU lock and the unchanged prepared dataset bytes.

The grouping is the same deterministic rule rows[index::stride], with
stride=ceil(number_of_problems / floor(64 / samples_per_problem)). Freeze the
mapping before any new result. Bind dataset SHA, ordered-ID SHA, grouping-manifest
SHA, exact per-shard ID/sample/seed tuple SHA and execution-profile SHA in every
job. Preserve each problem's n, sample indices 0..n-1 and SamplingParams(n=n,seed=0).
Require exactly n outputs; no resampling, filtering, truncation or changed caps.

verify_grouping.py was run read-only against all actual prepared JSONL files and
the frozen evaluator's SAMPLES declaration. grouping-verification.json proves
unique complete ID/sample coverage, original order reconstruction and unchanged
answer totals for both partitions. It stores hashes/counts, not question text,
answer keys or generated predictions. In set order AIME24/AIME25/AMC23/MATH500/
Minerva/Olympiad/GSM8K/MMLU-Pro/GPQA/IFEval, proposed Qwen counts are
8/8/10/32/17/43/42/16/13/17 = 206 shards per model, versus 406 currently.
Seven Qwen variants plus unchanged 650 Gemma shards yield 2092 total shards,
versus 3492 currently, with exactly 110910 total answers in either plan.

The new evaluator must require the frozen stride and per-shard tuple hash, include
all profile/grouping fields in its job identity, and reject cached results from a
different grouping or runtime. Aggregation requires every expected ID once with
the exact n samples and one common job/profile/grouping identity per model/set.
Do not blend a partly completed 32-answer partition with 64-answer shards. Existing
job 0/63 has SHA dc222e2f2a0dd0a4e4d554cbee975e3d2b6a357776d5df4330210ec0f3f18988
and remains untouched. No historical or fresh output is discarded or regenerated.

Review and validation before activation

* Prove source separation and shared-lock routing; compare every live production
  hash before/after staging. Test the same lock excludes an old/new worker pair.
* Reject v1 active cases before a CLI call/kernel start; verify untouched-case
  guards against any existing turn, CLI event, state or remote request.
* Re-run meaningful receipt/recovery/finalization tests under the new profile,
  including same-profile reuse and rejection of old/cross-profile receipts.
* Verify all sampling/token/pass/Python limits and item-seed outputs are unchanged;
  only the intended resource and Qwen final-grouping fields may differ.
* Validate Qwen's new startup capacity with the first legitimate future request;
  retain all telemetry and stop for diagnosis if the planned concurrency cannot
  fit. Gemma needs its separate existing-request telemetry review before any
  Gemma case approval. No automatic memory-fraction increase or token reduction.
* Inspect the first legitimately completed new-profile request/shard for exact
  output counts, prompt/token IDs, grades and provenance. Timing is observed from
  that work, not an extra generation run selected for a favorable result.
* Root reviews the implemented source/config and updated source/context hashes
  before deployment. Separate approval is required for the 64-answer Qwen shard
  plan; approving editor concurrency alone does not approve larger final shards.

Interpretation and scheduling risks

Only 32-concurrency data currently exist; 64 may or may not materially improve
throughput. Larger batching can change generated tokens despite identical item
seeds. Preserve that numerical/batching deviation in every result. Because old
editing cases remain 32 and new cases may use 64, comparisons across editing-pass
budgets also differ in resource profile; do not attribute every difference solely
to pass budget. Final Qwen comparisons would all use the same new scoring profile;
final Gemma comparisons would all retain the original scoring profile.

A 64-answer Qwen shard doubles the maximum generated-token work from 1,048,576 to
2,097,152 tokens, so one shard may hold the GPU lock longer and delay editor
requests. It still releases the lock after every shard. Runtime changes do not
speed up sequential Transformer activation replay or teacher forcing. Keep the
existing three-editor limit and monitor memory, errors, preemption and queue
latency. No unmeasured speedup or completion-time guarantee is claimed.
