Jonathan Richard Schwarz

I'm a Research Fellow at Harvard University, working on all forms of Efficient Machine Learning and its application to scientific and medical problems. Previously, I was a Senior Research Scientist at Google DeepMind and obtained my PhD from the joint DeepMind-University College London programme, advised by Yee Whye Teh and Peter Latham. My thesis focused on using sparse parameterisations and knowledge transfer for this purpose. Before that, I spent two years at the Gatsby Computational Neuroscience Unit and graduated top-of-the class from The University of Edinburgh.

My research focuses on the objective of building (i) efficient, (ii) general and (iii) robust Machine Learning systems. A central paradigm in my approach is the design of algorithms that can effectively abstract knowledge and skills present in related problems, enabling their utilisation for efficient learning on future tasks. In this way, agents gradually build diverse repertoires of skills, allowing transfer to future tasks using only a fraction of the otherwise required learning time and/or data.

To that end, most of my existing work falls within one or more of the following categories:

Meta-Learning: [ICML WS'18] [NeurIPS WS'18] [ICLR'19] [arXiv'19] [TMLR'22] [ICML'23] [NeurIPS'23a] [NeurIPS'23b] [arXiv'24]
Sparsity & Efficient Parameterizations: [ICLR'19] [ICLR'20] [NeurIPS'21] [TMLR'22] [JMLR'22] [ICML'23] [arXiv'24]
INRs / Neural Data Compression: [TMLR'22] [ICML'23] [ICLR WS'23] [NeurIPS'23] [CVPR'24]
Data Pruning / Online Curriculum Learning: [NeurIPS'23] [arXiv'23]
Continual Learning: [ICML'18] [NeurIPS'19] [ICLR'20] [NeurIPS'21] [arXiv'24]

I'm also the lead organiser of the Harvard Efficient Machine Learning Seminar Series. Join us here 📈.

Email / Google Scholar / Twitter / LinkedIn / Full CV: on request

News

Invited talk at the Hong Kong University of Science and Technology (HKUST) (April 2024).
Releasing our new perspective paper on Empowering Scientific Discovery with AI Agents (April 2024).
Releasing SMAT, our new state-of-the-art Meta-Learner (March 2024) !
Announcing the Harvard Efficient ML Seminar Series (March 2024) !
Announcing MAC, our new RAG/Online Adaptation method for LLMs (March 2024).
C3 accepted to CVPR (March 2024).
Invited talk at MIT on continual learning for LLMs (February 2024).
Invited talk at the Stanford Information Theory Forum on neural data compression with INRs (January 2024).
Invited talk at Mistral on Bad Students Make Great Teachers (January 2024).
Realeasing our new distributed curriculum learning framework (December 2023).
Introducing C3, our latest neural compression method (December 2023).
Invited talk at Stanford University (December 2023).
I've started a new position at Harvard, bringing advances in efficient ML to science and medicine !
Serving as area chair for ICLR'24.
Two papers accepted at NeurIPS 2023 !
Invited talk at The Royal Institution's Youth Summit on "AI for Scientific Progress" (September 2023).
Invited talk at Havard University on "Towards efficient and robust Machine Learning" (June 2023).
I'm visiting ETH Zurich to give a talk on Neural data compression with INRs (May 2023).
Join me and my co-organisers at the ICLR 2023 workshop on Neural Fields
VC-INR accepted to ICML 2023 !
Released our new paper on Modality-agnostic data compression ! (Feb 2023).
We have a new paper on Efficient Meta-Learning for large context sets. (Feb 2023).
New work on Spatial Functa ! (Feb 2023).
Had a great time visiting The University of Hong Kong and City University of Hong Kong (November 2022).
I'm giving a talk at The University of Cambridge on sparsity techniques for Meta & Continual Learning (October 2022).
Behavior priors for efficient reinforcement learning was accepted to JMLR (September 2022) !
Meta-Learning Sparse Compression Networks (MSCN) was accepted to TMLR (August 2022) !
I'm giving a talk at TU Eindhoven on sparsity techniques for Continual Learning (June 2022).
Honoured to be invited to the CVPR 2022 Workshop on CL for Vision to speak about sparsity techniques for Continual Learning and participate at the panel discussion. (June 2022)
I'm giving a talk at KAIST on our new paper "Meta-learning Sparse Compression Networks" (May 2022)
My new paper on Meta-learned Sparsity and compression with INRs is now available !

Research

Selected papers are highlighted.

Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts
Shengzhuang Chen, Jihoon Tack, Yunqiao Yang, Yee Whye Teh, Ying Wei°, Jonathan Richard Schwarz°

💻 Code
🔗 Project Website

arXiv 2024

° : Joint senior authorship

Empowering Biomedical Discovery with AI Agents
Shanghua Gao, Ada Fang, Yepeng Huang, Valentina Giunchiglia, Ayush Noori, Jonathan Richard Schwarz, Yasha Ektefaie, Jovana Kondic, Marinka Zitnik

arXiv 2024

Online Adaptation of Language Models with a Memory of Amortized Contexts
Jihoon Tack, Jaehyung Kim, Eric Mitchell, Jinwoo Shin, Yee Whye Teh, Jonathan Richard Schwarz

🗣️ Slides
🎞️ Video
💻 Code
🔗 Project Website

arXiv 2024

Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding
Talfan Evans, Shreya Pathak, Hamza Merzic, Jonathan Richard Schwarz, Ryutaro Tanno, Olivier J. Henaff

arXiv 2023

C3: High-performance and low-complexity neural compression from a single image or video
Hyunjik Kim, Matthias Bauer, Lucas Theis, Jonathan Richard Schwarz, Emilien Dupont

💻 Code
🔗 Project Website

CVPR 2024

Efficient Meta-Learning via Error-based Context Pruning for Implicit Neural Representations
Jihoon Tack, Subin Kim, Sihyun Yu, Jaeho Lee, Jinwoo Shin, Jonathan Richard Schwarz

🗣️ Slides
💻 Code

NeurIPS 2023

Secure Out-of-Distribution Task Generalization with Energy-Based Models
Shengzhuang Chen, Long-Kai Huang, Jonathan Richard Schwarz, Yilun Du, Ying Wei

NeurIPS 2023

Modality-Agnostic Variational Compression of Implicit Neural Representations (VC-INR)
Jonathan Richard Schwarz*, Jihoon Tack*, Yee Whye Teh, Jaeho Lee, Jinwoo Shin

ICML 2023

* : Joint first authorship

Spatial Functa: Scaling Functa to ImageNet Classification and Generation
Matthias Bauer*, Emilien Dupont, Andy Brock, Dan Rosenbaum, Jonathan Richard Schwarz, Hyunjik Kim*

arXiv 2023

Meta-Learning Sparse Compression Networks (MSCN)
Jonathan Richard Schwarz, Yee Whye Teh

Transactions on Machine Learning Research (TMLR) 2022

Behavior Priors for Efficient Reinforcement Learning
Dhruva Tirumala, Alexandre Galashov, Hyeonwoo Noh, Leonard Hasenclever, Razvan Pascanu, Jonathan Richard Schwarz, Guillaume Desjardins, Wojciech Marian Czarnecki, Arun Ahuja, Yee Whye Teh, Nicolas Heess

Journal of Machine Learning Research (JMLR) 2022

Powerpropagation: A sparsity inducing weight reparameterisation
Jonathan Richard Schwarz, Siddhant M. Jayakumar, Razvan Pascanu, Peter E. Latham, Yee Whye Teh

Neural Information Processing Systems (NeurIPS) 2021

💻 Code

Functional Regularisation for Continual Learning using Gaussian Processes
Jonathan Richard Schwarz*, Michalis K. Titsias*, Alexander G. de G. Matthews, Razvan Pascanu, Yee Whye Teh

International Conference on Learning Representations (ICLR) 2020

💻 Code

* : Joint first authorship

Multiplicative Interactions and Where to Find Them
Siddhant M. Jayakumar, Wojciech M. Czarnecki, Jacob Menick, Jonathan Richard Schwarz, Jack Rae, Simon Osindero, Yee Whye Teh, Tim Harley, Razvan Pascanu

International Conference on Learning Representations (ICLR) 2020

Meta-Learning surrogate models for sequential decision making
Jonathan Richard Schwarz*, Alexandre Galashov*, Hyunjik Kim, Marta Garnelo, David Saxton, Pushmeet Kohli, SM Ali Eslami°, Yee Whye Teh°

ICLR 2019 Workshop on Structure & Priors in Reinforcement Learning

*, ° : Joint first/senior authorship

	Experience replay for continual learning David Rolnick, Arun Ahuja, Jonathan Richard Schwarz, Timothy P. Lillicrap, Greg Wayne Neural Information Processing Systems (NeurIPS) 2019
	Information asymmetry in KL-regularized RL Alexandre Galashov, Siddhant M Jayakumar, Leonard Hasenclever, Dhruva Tirumala, Jonathan Richard Schwarz, Guillaume Desjardins, Wojciech M Czarnecki, Yee Whye Teh, Razvan Pascanu, Nicolas Heess International Conference on Learning Representations (ICLR) 2019
	Empirical Evaluation of Neural Process Objectives Tuan Anh Le, Hyunjik Kim, Marta Garnelo, Dan Rosenbaum, Jonathan Richard Schwarz, Yee Whye Teh NeurIPS 2018 workshop on Bayesian Deep Learning
	Attentive Neural Processes Hyunjik Kim, Andriy Mnih, Jonathan Richard Schwarz, Marta Garnelo, SM Ali Eslami, Dan Rosenbaum, Oriol Vinyals, Yee Whye Teh International Conference on Learning Representations (ICLR) 2019 💻 Code
	Neural Processes Marta Garnelo, Jonathan Richard Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, Yee Whye Teh ICML 2018 Workshop on Theoretical Foundations and Applications of Deep Generative Models (Spotlight talk) 🗣️ Talk (credit to Marta) 💻 Code
	Progress & Compress: A scalable framework for continual learning Jonathan Richard Schwarz, Jelena Luketina, Wojciech M. Czarnecki, Agnieszka Grabska-Barwinska, Yee Whye Teh, Raia Hadsell°, Razvan Pascanu° International Conference on Machine Learning (ICML) 2018 (Long oral) 🗣️ Talk 📊 Data (Sequential Omniglot) ° : Joint senior authorship
	The NarrativeQA Reading Comprehension Challenge Tomas Kocisky, Jonathan Richard Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gabor Melis, Edward Grefenstette 🗣️ Talk (credit to Tomas) 📊 Data Transactions of the Association for Computational Linguistics (TACL) 2018
	A Recurrent Variational Autoencoder for Human Motion Synthesis Ikhsanul Habibie, Daniel Holden, Jonathan Richard Schwarz, Joe Yearsley, Taku Komura 💻 Code 📊 Data British Machine Vision Conference (BMVC) 2017

Academic Workshops

(ICLR 2023) Neural Fields across Fields: Methods and Applications of Implicit Neural Representations

Jonathan Richard Schwarz, Hyunjik Kim, Emilien Dupont, Thu Nguyen-Phuoc, Vincent Sitzman, Srinath Sridhar

NeurIPS 2021 Workshop on Meta Learning

Jonathan Richard Schwarz, Fábio Ferreira, Erin Grant, Frank Hutter, Joaquin Vanschoren, Huaxiu Yao

NeurIPS 2020 Workshop on Meta Learning

Jonathan Richard Schwarz, Roberto Calandra, Jeff Clune, Erin Grant, Joaquin Vanschoren, Francesco Visin, Jane Wang

ICML 2020 Workshop on Continual Learning

Jonathan Richard Schwarz, Rahaf Aljundi, Eugene Belilovsky, Arslan Chaudhry, Puneet Dokania, Sayna Ebrahimi, Haytham Fayek, David Lopez-Paz , Marc Pickett

Based on Jon Barron's website.