TEE and Secure Inference
Confidential computing will become the standard way regulated PII reaches a hosted model, and the deciding variable is not whether attestation works — it does — but whether the performance tax falls below what a risk function will pay for a clean answer to 'who can see the prompt'.
A validation run. Researched position, no experiment.
Confidence
57%human-committedExpiry
16duntil review · 19 Sep 2026Lead time
—not yet mainstream · opened 17 Feb 2026Ownership
PRPriya Ramanfortnightly cadenceWhere it is
Emerging, with an experiment running. Trusted execution environments for inference — GPU-side enclaves with remote attestation — went from research to product in twelve months, and two hyperscalers now offer them in Sydney regions. The pitch is exact: the prompt and the model run inside hardware the provider cannot read, and the client gets a cryptographic attestation saying so. The cost is a performance tax we measure at 8–35% depending on model size and batch shape, plus a price premium of 20–50% on the enclave SKUs. For banking PII and health records that is a trade a risk function can evaluate, which is more than can be said for the current answer ('the provider promises'). What is not settled: whether attestation satisfies APRA and the OAIC as a control rather than a comfort, whether the tax shrinks with the next hardware generation, and whether AU-region enclave capacity is real or a roadmap slide. Our experiment on a banking PII workload is in the measuring stage.
Why a Quantium decision hinges on it
Every regulated engagement the firm delivers hits the same objection: the risk function will not let customer PII leave a controlled boundary to reach a hosted model. Today that objection is answered with contracts and de-identification, both of which limit what the model can do. If attestation is accepted as a control, the addressable workload for hosted inference in banking and health expands sharply, and the on-prem argument loses its last regulated-data leg. If it is not accepted, or the tax is too high, the firm should stop citing it in proposals. The economics gate is the firm's to measure; nobody else will do it on AU-region hardware with AU regulators in mind.
Field attributes
Position
What is demonstrated, what is hype, what would have to be true.
The shape every position request answers. Signal-tier fields carry a draft; assessed and tested fields carry a validated one.
- 01A 70B open-weight model inside a GPU enclave on AU-region hardware serves banking PII prompts with a 12% latency tax and a 9% throughput loss at our batch shape; attestation verified end to end (x-tee-inference, interim).
- 02Remote attestation can be checked by the client's own tooling, not the provider's; the chain of trust does not route through the party being distrusted.
- 03Two hyperscalers list enclave SKUs in Sydney regions; one has capacity we could actually provision, the other is waitlisted.
- 01'Zero-trust AI' as a marketing category. Attestation proves what code ran on what hardware; it does not prove the model will not leak the prompt in its output.
- 02TEEs as a substitute for de-identification. They change who can see the prompt, not what the model does with it.
- 03Claims of negligible overhead. The tax is real and varies fourfold with batch shape; the vendor number is the best case.
- 01APRA and the OAIC would have to accept attestation as a control in writing, which no AU regulator has done yet.
- 02The performance tax would have to hold under 15% at production batch shapes, or the economics revert to the on-prem argument.
- 03AU-region enclave capacity would have to be provisionable on demand; a waitlist is not a control.
- 01Conclude x-tee-inference and publish the tax and premium at three batch shapes; the number, not the concept, is what a risk function needs.
- 02Draft a regulator briefing on attestation as a control with the banking sector owner and put it in front of APRA through a client.
- 03If the tax holds under 15% and a regulator accepts the control, write the recommendation that TEE-backed hosted inference is the default for PII classes.
Signals · 9 in this cluster
What the cluster is made of.
Every item carries its source, tier and sightings. Detector-found signal sits beside human drops; downstream they are indistinguishable except by provenance.
Interim: 70B in a GPU enclave on Sydney hardware — 12% latency tax, 9% throughput loss, attestation verified
Measured on the banking PII workload at three batch shapes. Tax ranges 8–35%; production shape sits at 12%. Attestation checked with client-side tooling on every session. Not concluded; one hyperscaler, one model.
extracted claimThe enclave performance tax at production batch shape is about 12% for a 70B model on AU-region hardware.
'Confidential AI' track added to the AU cyber conference
Demand-band signal. First year the conference has a dedicated track; four of six talks are vendor. One talk from a health department describes a pilot.
AU bank hiring 'Confidential Computing Engineer — AI Platforms'
A major bank's first posting naming confidential computing for AI. Argus inference: at least one bank intends to build the capability in-house rather than wait for a regulator. Carried as inference.
APRA information paper on AI in regulated entities: silent on confidential computing
The paper discusses model risk, third-party risk and data handling. Attestation and enclaves do not appear. The absence is the finding: no regulator has taken a position for or against.
'If I show APRA an attestation report, will they accept it, or will they ask what it is?'
Asked by a bank's chief data officer after a TEE demo. The honest answer was 'they will ask what it is'. Became the regulator-briefing action in the position.
Confidential GPU Inference: Overhead Characterisation Across Model Sizes and Batch Shapes
Characterises enclave overhead on two GPU generations. Tax dominated by encrypted host-device transfer; shrinks with larger batches and larger models. Also notes that output leakage is out of scope for the enclave — the model can still emit what it was shown.
extracted claimEnclave overhead is transfer-bound and falls with batch size; it never reaches zero.
'Your enclave is airtight and your model just read the SSN back to the user'
Points out, with a demo, that enclaves protect the prompt in flight and at rest and do nothing about the model emitting it. Widely shared in security circles; it is why output controls stay in the pattern.
attest-verify — client-side remote attestation checker for GPU enclaves
Open-source verifier that checks the attestation report against vendor root keys without calling the provider. We use it in the experiment; it is the component that makes the trust story hold.
Two hyperscalers list GPU enclave SKUs in Sydney regions; one provisionable, one waitlisted
Confidential GPU inference goes on the AU price list. Premium 20–50% over standard SKUs. Both claim 'minimal overhead'; neither publishes a number. Capacity on one is a waitlist.
extracted claimEnclave inference is available in AU regions at a 20–50% SKU premium (overhead claim unverified).
Claims · 4 supporting, 1 refuting
The atoms.
A document cannot go stale; an assertion can. Claims are immutable and stamped with the extractor that produced them, so staleness, diffs and the graveyard operate at claim level.
Remote attestation can be verified by the client independently of the provider; the trust chain does not route through the party being distrusted.
TEEs do not address output leakage; a model inside an enclave can still emit PII in its response, so de-identification and output controls remain necessary.
No AU regulator has accepted attestation as a control for PII; until one does, TEEs are a comfort, not a compliance answer.
GPU enclave inference for a 70B model carries an 8–35% performance tax depending on batch shape; at production shapes it sits near 12%.
Enclave overhead is negligible for inference workloads.
Position history · the diff is the product
4 validation runs against a fixed brief. Confidence 42% → 57%.
Interim tax measured at about 12% at production batch shape; attestation verified client-side. Overhead-negligible claim retired. Regulator briefing drafted.
- GPU enclave inference for a 70B model carries an 8–35% performance tax depending on batch shape; at production shapes it sits near 12%.
- Remote attestation can be verified by the client independently of the provider; the trust chain does not route through the party being distrusted.
- c-tee-secure-inference-4 ↓ 0.35 → 0.15
Scoring · ordinal bands
Agents propose. A named human commits.
Uncommitted scores are visibly marked and never leave the building. Bands, not point estimates — false precision is the tell that a number was generated rather than derived.
Impact
committed · AWUnlocks hosted inference for the PII classes that currently block half of regulated engagements.
Timeline
committed · PRHardware and SKUs exist in Sydney; the regulator view is the long pole.
Cost
committed · PREnclave SKU premium plus a Type 3 with a banking PII corpus under a client agreement.
Cost of being wrong
committed · LFCiting attestation as a control before a regulator accepts it is a compliance representation the firm cannot back.
Demand
committed · CDEvery banking engagement with PII asks the 'who can see the prompt' question; two named TEEs unprompted.
TAM
agent-estimatedAgent-estimated from regulated-sector hosted inference spend in AU. Uncommitted.
Workforce readiness
agent-estimatedNobody in delivery has provisioned an enclave; attestation verification is a security-engineering skill. Agent-estimated.
Relevance · per vertical
Why it matters here, or explicitly does not.
Ranking is per vertical, not global. Sector owners commit notes against agent drafts.
The experiment runs on a banking PII workload because it is the case with the most engagements blocked on the prompt-visibility objection.
Mechanism · Enclave-hosted open-weight model in an AU region; client-side attestation check on every session; output PII filter retained.
Health records are the strictest PII class and the one where hosted inference is most often refused outright; attestation is the first argument the risk function has not already rejected.
Mechanism · Same pattern as banking with a state health department's data classification mapped to enclave tiers.
Claims triage on medical and financial detail is PII-dense and currently de-identified to the point of uselessness.
Mechanism · Enclave inference lets the triage model see the claim; de-identification moves to the output side.
Retail inference workloads rarely carry PII classes that a risk function blocks; the tax buys nothing.
Mechanism · None proposed.
Red team · the strongest case against
The strongest case against: TEEs answer a question regulators are not asking. APRA's concern is operational resilience and accountability; the OAIC's is purpose and disclosure. Neither has ever turned on whether a provider's engineers could technically read a prompt, and attestation may be an expensive answer to an objection that a contract and an audit already handle. Meanwhile the hardware has a history of side-channel breaks, and the first published enclave break on a GPU would remove the control overnight.
- —No AU regulator has asked for this. The field may be solving a security engineer's objection, not a compliance function's.
- —Enclave hardware has a record of side-channel vulnerabilities; a control that can be broken by a paper is a control a risk function will not rely on.
- —The tax plus the SKU premium can exceed the on-prem crossover for the very PII workloads where utilisation is highest, which would make on-prem the cheaper sovereign answer for exactly the cases TEEs target.
Source diversity
- Security research30%
- Vendor / provider20%
- Open-source infra10%
- Regulator10%
- Internal / Engel30%
A field supported by one epistemic community is a flag, not a finding.
Cross-pollination · typed joins
Connected, not merely similar.
Enabling, compounding, substituting, blocking. A satisfied dependency trigger is a far stronger signal than semantic proximity.
TEE-backed hosted inference is the alternative to owning hardware for regulated data; if the tax and premium exceed the on-prem crossover, on-prem wins those workloads back.
Attestation proves what ran; delegated authority proves who asked. Together they are the audit story for agentic inference on PII.
Enclave side-channel research is a security-arms-race field; a published break is the trigger that would reset this one.
Only an open weight can be loaded into a client-attested enclave; frontier closed models offer attestation on their own terms, which is a different trust model.
Share graph
Provenance running forward.
Discovery, not accountability. No counts, no rankings, no rollups to managers.
Convergence · who else is here
- LFLena Fischer · Red team & assurance2 drops
- PRPriya Raman · Research engineer · inference1 drop
- AWAdam Witanowski · Lab Director (acting)1 drop
- CDClaire Dubois · Sector owner · Banking1 drop
- HNHana Novak · Sector owner · Health1 drop
Several people’s drops meet here. An informal working group already exists and probably does not know it.
Lineage
What this field produced, and what it killed.
Experiments, recommendations and graveyard entries stay attached. The reasoning that killed a claim is the reusable asset.
Open questions · return to the pile
Every run leaves a record. Separately, its question either closes or returns to the pile with notes — which is what the next person proposing the same thing will see.
- 01What does APRA need to see to accept attestation as a control, and who is the right client to ask through?
- 02Does the tax shrink enough on the next GPU generation to make the SKU premium the only cost?
- 03What is the response plan when the first GPU-enclave side-channel break is published?