Is EAGLE-3 Speculative Decoding Lossless?
Non-greedy acceptance, residual sampling, and why q matters.
Here are some of the things I've worked on in the past—projects, short papers, write-ups, etc.
From QK-clip to Muon clip
Clipping attention logits for MLA with shared rotary keys.
Parallel Transformer Blocks
Approximating the serial pre-norm block.
Token Embeddings Violate the Manifold Hypothesis
On singular token spaces.
Representative Sampling of High‑Dimensional Point Sets
Small but faithful subsets of large point sets.
Chess game
Play chess alone or with a friend.
Deficiencies in Networks
Communication deficiencies in optimization.