From QK-clip to Muon clip
Clipping attention logits for MLA with shared rotary keys.
Here are some of the things I've worked on in the past—projects, short papers, write-ups, etc.
Parallel Transformer Blocks
Approximating the serial pre-norm block.
Token Embeddings Violate the Manifold Hypothesis
On singular token spaces.
Representative Sampling of High‑Dimensional Point Sets
Small but faithful subsets of large point sets.
Chess game
Play chess alone or with a friend.
Deficiencies in Networks
Communication deficiencies in optimization.