Control, compute, and memory designed as one inference machine.
The candidate system joins a host runtime, RISC-V control, vector and quantized matrix execution, explicit local memory, DMA, and external model storage.
Interfaces and parameters remain workload-driven candidates
The host orchestrates; the tile executes explicit work.
Model orchestration and command submission begin on the host. Runtime queues, buffers, firmware, and DMA define the boundary into the accelerator tile.
A programmable control plane keeps the datapath manageable.
The control processor is intended to coordinate queues, firmware, command decoding, synchronization, and specialized execution without turning control logic into fixed-function complexity.
Vector execution covers the work around matrix multiplication.
Elementwise operators, reductions, normalization support, quantization helpers, and data-layout operations require a vector path designed alongside matrix compute.
The matrix engine is evaluated as a dataflow, not only a MAC count.
The proposed engine studies four-bit weight and eight-bit activation products with INT32 accumulation. Feeding, reduction, scaling, and utilization remain part of the decision.
Prefill and decode exercise the same hardware differently.
GEMM-oriented prefill exposes parallel work, while batch-one decode trends toward GEMV behavior and repeated weight movement. Both regimes must be modeled independently.
Local memory is explicitly managed working state.
Banking, capacity, access width, and software visibility determine whether vector and matrix engines stay fed without wasting area or bandwidth.
Movement must overlap execution where practical.
DMA scheduling is intended to move weights, activations, and intermediate data between external and local memory while compute proceeds on available tiles.
Bandwidth belongs in the product specification.
External DDR or LPDDR access is expected through host or SoC fabric. Sustainable bandwidth and transfer granularity must be tied to the target workload.
Context growth changes the memory contract.
KV-cache capacity and traffic scale with context and influence decode latency, external-memory demand, and the value of local buffering.
Low-bit multiplication is only one part of low-bit inference.
Unpacking, scales, zero-points, activation quantization, accumulation, conversion, and output handling must be covered end to end.
A visible baseline for measurement—not a frozen specification.
The current study uses an 8 × 8 physical matrix candidate, an approximate 0.5B–1.5B model envelope, and 128 KB, 256 KB, and 512 KB scratchpad points.
Evidence still needs to select the final interfaces and sizes.
Matrix shape, vector width, banking, DMA concurrency, queue model, supported quantization formats, and the prefill/decode balance remain open research questions.