# 17-Year-Old Attention Residuals Co-Author Joins Anthropic Claude Team Chen Guangyu, 17-year-old co-first author of Moonshot’s Attention Residuals paper, leaves Kimi and joins Anthropic’s Claude team for scaling RL work. Canonical URL: https://aiidelist.com/blog/chen-guangyu-anthropic-attention-residuals Language: en Published: 2026-10-15 Updated: 2026-10-15 ## Key Takeaways - **Chen Guangyu** (also known as Nathan Chen on X), who joined Moonshot AI at age 16, co-authored the Attention Residuals (AttnRes) technical report as equal first author alongside Yu Zhang and Jianlin Su. - The March 2026 paper replaces standard PreNorm residual accumulation with learned, input-dependent softmax attention over depth; Block AttnRes matches the loss of a baseline trained with 1.25× more compute and delivers gains on reasoning and code benchmarks. - Public screenshots show Anthropic researcher Sholto Douglas inviting him to the Bay Area team; Chen confirmed he is heading to San Francisco to join Anthropic’s Claude team on scaling RL work. - The move occurred within roughly 24 hours of his public departure announcement from Moonshot / Kimi, where he contributed to KDA kernel optimization, optimizer research, and K3-related development. - Community discussion highlights the speed of frontier talent movement and the concrete architectural output (depth-wise attention residuals) rather than age alone. ## Background and Trajectory Chen Guangyu entered machine learning around 2024. By late 2025 he had joined Moonshot AI (Kimi) at age 16. During his internship of less than a year he worked on low-level systems and model architecture, including GPU kernel optimization for Kimi Delta Attention (KDA), optimizer experiments, and support for Kimi K3 development. His most visible contribution is the Attention Residuals paper (arXiv:2603.15031, March 2026). He is listed as a co-first author with Yu Zhang and Jianlin Su on the Kimi Team technical report. The work received public recognition, including praise from Elon Musk. On 15 October 2026 he posted that he had left Moonshot AI, describing the year as the best of his life so far. Shortly afterward, screenshots circulated of a public exchange with Sholto Douglas (Scaling RL at Anthropic, formerly DeepMind). Douglas invited him to the San Francisco team; Chen replied confirming he was heading to the Bay Area. ## Attention Residuals: Technical Contribution Standard residual connections with PreNorm accumulate every preceding layer output with fixed unit weights. This produces uncontrolled hidden-state growth with depth and progressive dilution of earlier layers’ contributions. Attention Residuals replace that fixed sum with softmax attention over preceding layer outputs: \[ \mathbf{h}_l = \sum_{i=0}^{l-1} \alpha_{i \to l} \cdot \mathbf{v}_i \] Each layer uses a single learned pseudo-query vector to compute the depth-wise attention weights. The weights are input-dependent, so a layer can selectively emphasize different earlier representations according to content. To control memory and communication at scale, the paper introduces **Block AttnRes**. Layers are partitioned into blocks; attention is performed over block-level representations rather than every individual layer, reducing overhead from \(O(Ld)\) to \(O(Nd)\). Combined with cache-based pipeline communication and a two-phase computation strategy, the method is presented as a practical drop-in replacement with modest training and inference overhead. Reported results include: - Scaling-law experiments in which Block AttnRes matches the validation loss of a baseline trained with approximately 1.25× more compute. - Integration into the Kimi Linear architecture (48B total / 3B activated parameters) pretrained on 1.4T tokens. - Downstream gains versus the baseline, including GPQA-Diamond +7.5, HumanEval +3.1, MMLU +1.1, and improvements on math, code, and Chinese benchmarks. - More bounded output magnitudes and more uniform gradient distribution across depth, mitigating PreNorm dilution. Official code and the technical report are available on the MoonshotAI/Attention-Residuals GitHub repository. ## The Anthropic Move Screenshots from the public exchange show Sholto Douglas congratulating Chen, noting a recent meeting, and asking whether he still planned to attend CMU or would come to San Francisco to work with the prime/base10 team and others. Chen’s reply confirmed he was heading to the Bay Area. Sholto Douglas’s profile describes his focus as Scaling RL at Anthropic (ex-DeepMind). The timing—departure announcement followed within a day by public confirmation of the Anthropic move—has been widely discussed as an example of how quickly demonstrated architectural contributions can translate into offers at frontier labs. No official Anthropic announcement had been published at the time of the initial reports; the information rests on the public X exchange and Chen’s own departure post. ## Broader Context and Implications The episode illustrates several patterns visible in 2026 frontier AI talent markets: - **Demonstrated technical output over credentials**: Co-first authorship on a scaling-relevant architecture paper (with measurable benchmark and scaling-law gains) appears to have been the decisive signal, independent of traditional academic milestones or competition rankings. - **Speed of movement**: Public confirmation of a new role at another leading lab occurred within roughly 24 hours of the departure post. - **Cross-lab architecture transfer**: Techniques such as depth-wise selective residual aggregation that improve compute efficiency and reasoning are portable; labs compete for the researchers who can implement and extend them at scale. - **RL and scaling focus**: Chen’s next role is described in the context of Anthropic’s scaling RL efforts, aligning his prior systems and architecture work with current priorities around reinforcement learning and long-horizon capabilities. Independent commentary noted that the ability to modify core architectural components remains scarce and highly valued, regardless of the researcher’s age. ## Conclusion Chen Guangyu’s path—from entering machine learning in 2024, joining a major Chinese lab at 16, co-authoring a practical residual-connection replacement that improved scaling and downstream results, and moving to Anthropic’s Claude team within a day of leaving Moonshot—illustrates the current pace and criteria of frontier AI talent markets. Researchers and practitioners tracking architectural improvements can examine the Attention Residuals repository and technical report for the concrete design (pseudo-query depth-wise attention and Block AttnRes) and the reported efficiency and benchmark gains. The public exchange further shows how quickly such contributions are recognized across labs. Follow primary sources (the arXiv report, the GitHub repository, and the researchers’ own posts) for any subsequent technical updates or official role confirmations.