DPhil Statistics candidate at the University of Oxford
MPhil, PhB. Mathematics at the Australian National University
I am a computational mathematician focused on designing and developing scalable algorithms for scientific applications. My current interests include generative modelling applied to inverse problems and scalable numerical methods for solving PDE. A key objective of my work is to create stable algorithms for high-performance computing, capable of modelling large-scale systems and solving inverse problems, even in non-traditional or challenging scenarios.
My goal is to develop rigorous mathematical frameworks for successful numerical methods, improving their stability and efficiency, and extending their application to a wider range of problems. Below are my current research interests, the frameworks I am developing, and their applications.
Our methodology matches state-of-the-art diffusion performance with no need for hyper-parameter search. We may also learn complicated fractal distributions, something infeasiable without our methodology.
Problem
Diffusion models progressively add Gaussian noise to data until it becomes normally distributed. The main challenge lies in learning how to reverse this process to estimate the original data’s density accurately.
Current Work
In our paper, we present a novel methodology for optimising the corruption schedule—the rate at which noise is added. We identify a geometric invariant that captures the complexity of the process and introduce an energy functional to measure the quality of the schedule. Our approach minimises this energy, leading to state-of-the-art performance without the need for hyperparameter tuning.
Future Work
We are extending this work for general Bayesian Inference problems in the natural sciences. Soon more to come!
Dispersion Relations
Plotted are dispersion relations for various discretisation schemes. The true continuous dispersion is in black, current methods in red and blue, and our new state-of-the-art method in green.
Problem
Climate models, geophysical modelling and fluid motions require accurate modelling of waves. However, propagating high frequency waves has always been a computational bottleneck, see Tam and Webb's 1992 paper for instance. This is due to the dispersion error of the discrete approximation of the continuous PDE problem being unbounded for the high-frequency components on the Fourier spectrum.
Current Work
We've created the first finite difference stencil that's accurate for high-frequency components of solutions. Our new scheme lets us compute more accurate solutions to certain hyperbolic PDE with magnitudes fewer computational resources. We have the first scheme that remains accurate for all frequencies. Our new scheme (DRP, green) achieves significant computational savings across different orders and does not support spurious wave modes that destroy numerical simulations (blue and red), as shown in the HPC earthquake simulation attached (the wild oscillations are errors). In this paper, we observe magnitude computational improvement in earthquake modelling with unseen accuracy.
Future Work
This new method can be applied to many more contexts, with the framework I developed in this paper and my MPhil thesis. These broader contexts include Climate models, gravity waves and Tsunami modelling.
Computational Savings
HPC Computational Results
Comparison of high frequency wave propagation between schemes. The occillations are numerical errors, our DRP scheme (green) does not suffer from such errors.
The U-Net Architecture
Problem
We often need to approximate a function or operator with limited recordings. To effectively do this, we need to impose a good inductive bias with our approximate. For a given problem, how do we impose an inductive bias on our neural network?
Current Work
In our initial paper (NeurIPS oral) we identified that a U-Net, a popular architecture, was implicitly working on Haar wavelet subspaces. Such a basis imputes a logical inductive bias on problems associated with diffusion models, imaging and PDE approximation. In our follow up paper, we identified a mathematical framework to generalise this technique to other bases and geometries, allowing for more general inductive biases in different geometries to be created.
Future work
We now have an established mathematical framework fo U-Net design and analysis. We are currently able to prove results relating to the stability of the U-Net and optimal tuning configurations without hyper-parameter search. We further formulated a more simple architecture, called a Multi-ResNet, for PDE surrogate modelling, with intended applications to large scale surrogate climate modelling tasks.
PDE Surrogate Modelling
We found that our Multi-ResNets (a simplified U-Net) achieve state-of-the-art performance on PDE surrogate modelling task, by careful choice of the basis functions used in the neural network design
MRI Imaging
The original U-Net was motivated for MRI imaging. Our analysis encompasses such applications and aims to improve the U-Net design in this area.
Fractal finite element used for our DG method. The finite element is made up of five smaller copies of itself.
Problem
Traditional calculus is designed for smooth geometries, but many real-world geometries—such as mountains, shorelines, trees, and clouds—are rough and irregular. These fractal-like structures may provide a better representation of natural phenomena. Can we adapt existing computational methods to work effectively on such rough, fractal geometries?
Current Work
The Discontinuous Galerkin (DG) method is widely used to solve partial differential equations (PDEs) governing physical models like heat flow and fluid dynamics. Implementing this method on fractal domains requires defining weak derivatives and constructing effective basis functions. In my thesis, I developed the first numerical method to solve PDEs on fractal domains using an adapted DG approach. This involved creating contractive operators on unconventional metric spaces that allow for the computation of both basis functions and derivatives.
Future Work
In geophysics, it has been observed that fault lines exhibit fractal structures. By using the fractal approximation methodology I developed, we can approximate these faults with fractal functions and apply the adapted DG method to solve PDEs, leading to improved models of such phenomena.
Orthogonal Polynomials on Fractals
The first 15 orthogonal basis functions of our mountain tile. We have a new methodology to compute orthogonal polynomials on such domains which can scale to thousands of basis functions.
Weak Derivative of Fractal Functions
We have developed a methodology to efficiently compute weak derivatives of fractal functions which are non-differentiable in the traditional sense. This is an example of the derivative (bottom) of the boundary of our fractal finite element (top).
Score-Optimal Diffusion Schedules
C. Williams, A. Campbell, A. Doucet, S. Syed
Advances in Neural Information Processing Systems (NeurIPS) 38, 2024
Full-Spectrum Dispersion Relation Preserving Summation-by-Parts Operators
C. Williams, K. Duru
SIAM Journal on Numerical Analysis 62 (4), 1565-1588, 2024
A Unified Framework for U-Net Design and Analysis
C. Williams*, F. Falck*, G. Deligiannidis, C.C. Holmes, A. Doucet, S. Syed
Advances in Neural Information Processing Systems (NeurIPS) 37, 27745-27782, 2023
Dispersion Relation Preserving FD Schemes and Self-Affine DG Elements
C.J. Williams
The Australian National University, 2021