Moritz Haas

Moritz Haas

PhD student

University of Tübingen

About Me

I am a PhD student in the ‘Theory of Machine Learning’ group at the University of Tübingen supervised by Ulrike von Luxburg and Bedartha Goswami.

My goal is developing a mechanistic understanding of deep learning that results in practical benefits. On the side, I am trying to improve statistical methods in climate science. Feel free to reach out, if you are interested in any of the two topics!

Contact: mo.{last_name}@uni-tuebingen.de

Interests

Deep Learning Theory
Scaling Theory
Statistics
Machine Learning in Climate Science

Education

PhD Theory of Machine Learning
University of Tübingen
MSc Mathematics
Ruprecht Karls Universität Heidelberg
BSc Mathematics
Ruprecht Karls Universität Heidelberg

Main Publications

mup^2: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling

mup^2: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling

Naively scaling up standard neural network architectures and optimization algorithms loses desirable properties such as feature learning in large models (see the Tensor Program series by Greg Yang et al.). We show the same for sharpness aware minimization (SAM) algorithms: There exists a unique nontrivial width-dependent and layerwise perturbation scaling for SAM that effectively perturbs all layers and provides in width-independent perturbation dynamics.

Oct 31, 2024

Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimension

Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimension

When can kernel or neural network models that overfit noisy data generalize nearly optimally? Previous literature had suggested that kernel methods can only exhibit such “benign overfitting”, also called “harmless interpolation”, if the input dimension grows with the number of data points. We show that, while overfitting leads to inconsistency with common estimators, adequately designed spiky-smooth estimators can achieve benign overfitting in arbitrary fixed dimension. For kernels, we show how you can mimick kernel ridge regression while interpolating the training data by adding a sharp spike to your kernel. For neural networks with NTK parametrization, you just have to add tiny fluctuations to the activation function. Remarkably, adding a Gaussian kernel with small bandwidth to the NTK or to the NNGP approximately translates into adding a high-frequency low-amplitude shifted sin-curve to the activation function.

May 23, 2023

Pitfalls of Climate Network Construction - A Statistical Perspective

Pitfalls of Climate Network Construction - A Statistical Perspective

In this paper, we explore in which ways estimation errors distort climate networks through false and missing edges. We find several mechanisms through which finite-sample noise can systematically distort climate networks. Most notably, spatially heterogenous estimation variance (for example caused by heterogenous autocorrelation patterns in different locations) causes large biases toward overestimating the importance of the particularly noisy locations. But even on isotropic data, many graph measures like betweenness or clustering coefficients can be heavily distorted, and spurious high-degree clusters and link-bundle teleconnections emerge.

Apr 20, 2023

Statistical Analysis of Wasserstein GANs with Applications to Time Series Forecasting

Statistical Analysis of Wasserstein GANs with Applications to Time Series Forecasting

We analyse Wasserstein GANs statistically. Interestingly, our excess risk bound for unconditional WGANS (from 2020!) captures a key advantage of generative models: Since we can generate as many samples as we want, the generalization error is only limited by the critic network and the dataset size. If we generate enough samples (and assume to find a global optimizer), the generator network may be arbitrarily large. We also show that large critic networks metrize weak convergence, that is they are able to distinguish arbitrary pairs of distributions and guide the generator to reproduce the data distribution.

Nov 5, 2020