From GGUF model to controlled accelerator execution.
The initial software direction begins with llama.cpp and GGML, then crosses a Proton backend, runtime, firmware, kernels, and measurable target execution.
Initial path is planned around GGUF and llama.cpp
Each boundary should remain observable.
The stack keeps model metadata, operator mapping, runtime commands, firmware behavior, kernel selection, and device evidence connected rather than hiding them behind a single opaque API.
Start from an established local-inference path.
GGUF supplies model and quantization metadata. llama.cpp and GGML provide the first host framework and reference path for integration.
Map supported operators without losing the reference path.
The planned backend selects Proton execution for supported work while keeping unsupported operations and numerical comparison explicit.
Commands, buffers, and synchronization form the device contract.
The runtime boundary is intended to define allocation, transfers, queues, command submission, synchronization, errors, and profiling records.
Device software coordinates vector, matrix, and memory engines.
Firmware sequences commands and data movement; kernels implement target-specific operator tiles and support routines.
Coverage grows from measurable, testable primitives.
Candidate work includes quantized linear layers, dot products, RMSNorm, RoPE, Softmax support, SiLU, elementwise operations, and layout conversion.
Performance data and output comparison travel together.
Timing, movement, utilization, operator coverage, and differential output checks are intended to produce one reviewable evidence record.
A future frontend after the first practical path is established.
IREE remains a planned compiler integration direction. It is not represented as implemented today.