Technical Specification: How bf16.ai Computes the 0–100 Behavioral Verification Index

Specification Summary

• Hard Activation Gate: Every upstream endpoint registered on bf16.ai must have a passing Behavioral Verification run on record before PostgreSQL constraints permit live traffic routing.
• Six Orthogonal Probes: Tokenizer boundary fingerprinting, temperature = 0 symbolic math & code invariants, 200K+ CSPRNG passkey retrieval, parallel JSON Schema tool-call conformance, benign dual-use low-refusal fidelity, and sustained throughput telemetry (time to first token and output speed).
• An Empirical Check, Not an Attestation: The index summarizes how an endpoint behaves on freshly seeded probes. It does not prove which weights, or which numeric precision, an upstream provider actually serves.

Open-weight model identifiers only mean something if the endpoint behind them behaves like the published checkpoint. In third-party inference markets, two failure modes routinely break autonomous agents:

1. Aggressive Sub-FP8 Quantization (Q3_K_M, uncalibrated NVFP4, or truncated KV-cache precision) that silently collapses multi-step symbolic reasoning and breaks nested JSON tool arguments while still passing casual chat prompts.
2. Crude Refusal Removal (naive SFT or over-broad layer scaling) that strips safety lectures at the cost of instruction-following coherence — colloquially known among ML engineers as "brain damage."

To catch both, bf16.ai runs an automated 6-Vector Behavioral Verification Runner before an endpoint can be activated, and records a 0–100 index with sub-scores for every run.

1. Vector Definitions & Pass Rules

The composite Behavioral Verification Index S ∈ [0, 100] is computed as a weighted floor-bounded aggregate of six orthogonal probe suites. Every prompt uses fresh CSPRNG-parameterized numbers, variable identifiers, and passkeys so upstream providers cannot pass via static response caching or benchmark memorization.

Vector ID	Probe Suite	Primary Failure Mode Caught
V1	Tokenizer & Special-Token Boundary Audit	Mismatched chat templates, missing <|tool_call|> tokens, wrong BPE vocab
V2	Deterministic Math & Symbolic Canary (T=0)	Sub-FP8 quantization drift, corrupted attention heads from sloppy abliteration
V3	Long-Context Passkey Retrieval (200K–1M)	Silent middle-out prompt truncation or broken RoPE scaling above 64K
V4	Parallel Tool-Call & Strict JSON Grammar	Malformed JSON arguments, dropped required fields, hallucinated tool names
V5	Dual-Use Developer Low-Refusal Fidelity	False-positive refusals on SAST SQLi diffs, staging DOM login forms, noir localization
V6	Time-to-First-Token & Sustained Output Throughput	Overloaded GPU replicas, thermal throttling, or misconfigured continuous batching

If any single vector drops below its own minimum — for example, if an endpoint is flawless on low-refusal execution (V5) but fails parallel JSON tool calling (V4) — the entire verification run is marked passed = false regardless of the weighted average.

2. What Each Verification Run Records

Every verification run stores the composite index and its sub-scores alongside the endpoint's Zero-Prompt Retention (ZDR) governance metadata.

Current Active Catalog

Model ID	Modality	Context	Retail Pricing (In / Cache / Out per 1M)
deepseek/deepseek-v4.1-flash-abliterated	Vision + Text + Tools	1,048,576	$0.30 / $0.05 / $1.20
zai-org/glm-5.3-flash-abliterated	Vision + Text	1,048,576	$0.24 / $0.06 / $1.00
zai-org/glm-5.3-abliterated	Text + Tools	1,048,576	$1.40 / $0.30 / $4.40

3. Database-Level Routing Enforcement & Zero-Charge Failover

Verification is not an advisory dashboard badge on bf16.ai — it is enforced directly in the data plane:

-- Simplified invariant enforced on model_upstream_endpoints before status = 'active'
CONSTRAINT model_upstream_endpoints_compliance_gate_chk CHECK (
  status <> 'active' OR (
    commercial_resale_permitted = true
    AND zdr_no_training_verified = true
    AND tokenizer_calibrated = true
    AND latest_verification_run_id IS NOT NULL
  )
)

Before routing any /v1/chat/completions, /v1/messages, or /v1/responses request, evaluateEndpointComplianceAndVerification() confirms that latest_verification_run_id references a passing run for that exact (model_id, endpoint_id) pair. Furthermore, if an upstream replica experiences transient network degradation before the first content or tool-call token is emitted, bf16.ai automatically fails over or releases the entire nano-USD credit hold with $0.00 charged to your wallet.

4. Programmatic Verification via /v1/models

Automated agent fleets can read each model's verification_index and pricing programmatically before dispatching batch jobs:

curl -sS https://api.bf16.ai/v1/models \
  -H "Authorization: Bearer $BF16_API_KEY" | jq '.data[] | {
    id,
    context_window,
    verification_index,
    pricing_usd_per_1m
  }'

By pinning X-BF16-Max-Price-Per-M-Output alongside model: "bf16/auto", your orchestrator delegates strictly to active endpoints that satisfy both our behavioral verification gate and your deterministic unit-cost ceiling.