Moritz Haas

Moritz Haas

Applied Scientist

Amazon AGI Foundations

About Me

I am currently working as an Applied Scientist in the AGI Foundations team at Amazon.

My goal is developing a mechanistic understanding of large-scale neural networks that results in practical benefits. On the side, I am trying to improve statistical methods in climate science. Feel free to reach out, if you are interested in any of the two topics!

Previously, I was a PhD student in the ‘Theory of Machine Learning’ group at the University of Tübingen supervised by Ulrike von Luxburg and Bedartha Goswami.

Contact: {first-name}-ha@web.de

Interests

Deep Learning Theory
Scaling Theory
Statistics
Machine Learning in Climate Science

Education

PhD Theory of Machine Learning
University of Tübingen
MSc Mathematics
Ruprecht Karls Universität Heidelberg
BSc Mathematics
Ruprecht Karls Universität Heidelberg

Publications

How to Scale Mixture-of-Experts: From μP to the Maximally Scale-Stable Parameterization

How to Scale Mixture-of-Experts: From μP to the Maximally Scale-Stable Parameterization

* Equal first author, † Equal contributor. Despite the dominance of Mixture-of-Experts (MoE) architectures in frontier LLMs, there is no principled understanding of how hyperparameters should scale with width, expert width, number of experts, sparsity, and depth.

Leena Chennuru Vankadara*

• May 15, 2026

On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling

On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling

A primary goal of infinite-width theory has always been explaining neural networks as they are initialized and trained in practice (called ‘standard parameterization’, short SP). But there has always remained a fundamental gap: Existing theory for SP predicts a kernel regime with vanishing feature learning under small learning rates, and instability due to logit divergence under large learning rates.

Moritz Haas

• May 28, 2025

mup^2: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling

mup^2: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling

Naively scaling up standard neural network architectures and optimization algorithms loses desirable properties such as feature learning in large models (see the Tensor Program series by Greg Yang et al.). We show the same for sharpness aware minimization (SAM) algorithms: There exists a unique nontrivial width-dependent and layerwise perturbation scaling for SAM that effectively perturbs all layers and provides in width-independent perturbation dynamics.

Moritz Haas

• Oct 31, 2024

Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimension

Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimension

When can kernel or neural network models that overfit noisy data generalize nearly optimally? Previous literature had suggested that kernel methods can only exhibit such “benign overfitting”, also called “harmless interpolation”, if the input dimension grows with the number of data points.

Moritz Haas*

• May 23, 2023

Pitfalls of Climate Network Construction - A Statistical Perspective

Pitfalls of Climate Network Construction - A Statistical Perspective

In this paper, we explore in which ways estimation errors distort climate networks through false and missing edges. We find several mechanisms through which finite-sample noise can systematically distort climate networks.

Moritz Haas

• Apr 20, 2023

See all publications