An IREE graph. Bare-metal kernels. Matrix execution.
The published Proton SDK connects a pretrained language model to the existing Proton NPU RTL through IREE, custom dispatches and independently checked INT8 kernels.
Verified in Verilator on the existing RTL. Batch one, bounded context, model-specific graph. These results do not establish silicon performance or general model support.
Start with a pinned checkpoint and an independent reference.
The importer validates the Stories260K checkpoint and tokenizer, generates an independent NumPy reference, and packs weights for the matrix tile layout. Linear operations use INT8 weights and dynamically quantized INT8 activations; other math and the KV cache remain FP32.
Make the graph and the kernel boundary inspectable.
The model-specific generator emits MLIR with explicit semantic dispatches. IREE compiles the graph to VMFB and static RISC-V wrappers. Those wrappers call the Proton operator interface; handwritten kernels implement the operator math.
Current input is a llama2.c checkpoint, not an ONNX or PyTorch graph. This is custom dispatch integration, not automatic matmul lowering or standard IREE ukernel selection.
Run the graph without a target operating system.
The bare-metal IREE VM, synchronous HAL, static dispatch code, weights and application are linked into one RISC-V executable. The application invokes each decoder step and manages the greedy token loop. A bounded state buffer holds scratch data and the KV cache.
Submit matrix tiles with explicit numeric semantics.
The linear kernel packs activation tiles, writes operands through MMIO, executes mzero and mmacc, and reads INT32 accumulators before FP32 rescaling. The scalar path uses the same quantization for a controlled comparison.
The matrix engine executes Q, K, V, attention output, FFN and vocabulary projections. Attention scores, softmax, value aggregation and nonlinear operations remain scalar FP32.
The current model uses no compiler-generated RVV. The matrix path is synchronous and CPU-fed, with no DMA queue.
Numerical results and hardware traces must agree.
Acceptance combines independent reference logits, exact kernel arithmetic, clean completion, positive cycle counts, retired matrix instructions and matching issue/done/response events. Wrong results, timeouts and corrupted evidence must be rejected.
Generalize the frontend while preserving the target contract.
PyTorch/ONNX import, general operator lowering, expanded shapes and new model families are future work. CNNs need convolution and associated operator support; ViTs need their embedding, normalization, activation and attention paths qualified.
Separately managed weights, larger memory platforms, RVV enablement and Linux support require their own validation. Future dialects or ukernels should preserve explicit layout, numeric and execution contracts.