Jonathan Richard Schwarz

I'm the Head of AI Research at Thomson Reuters (TR), leading TR's Foundational Research Team. In addition, I serve as an Expert Advisor to the UK's AI Safety Insitute. I joined TR through the acquisition of Safe Sign Technologies, for which I served as Co-Founder and Chief Scientific Officer (CSO).

Previously, I was also a Research Fellow at Harvard University and a Senior Research Scientist at Google DeepMind. I obtained my PhD from the joint DeepMind-University College London programme, advised by Yee Whye Teh and Peter Latham. My Thesis focused on sparse parameterisations and knowledge transfer for efficient Machine Learning. Before that, I spent two years at the Gatsby Computational Neuroscience Unit and graduated top-of-the class from The University of Edinburgh.

My research focuses on the objective of building (i) efficient, (ii) general and (iii) robust Machine Learning systems. A central paradigm in my approach is the design of algorithms that can effectively abstract knowledge and skills present in related problems, enabling their utilisation for efficient learning on future tasks. In this way, agents gradually build diverse repertoires of skills, allowing transfer to future tasks using only a fraction of the otherwise required learning time and/or data.

To that end, most of my existing work falls within one or more of the following categories:

Email  /  Google Scholar  /  Twitter  /  LinkedIn  /  Full CV: on request

News



Research

Selected papers are highlighted.

clean-usnob

Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts
Shengzhuang Chen, Jihoon Tack, Yunqiao Yang, Yee Whye Teh, Ying Wei°, Jonathan Richard Schwarz°

💻 Code
🔗 Project Website

ICML 2024

° : Joint senior authorship

clean-usnob

CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-training
David Brandfonbrener, Hanlin Zhang, Andreas Kirsch, Jonathan Richard Schwarz, Sham Kakade

NeurIPS 2024

clean-usnob

Empowering Biomedical Discovery with AI Agents
Shanghua Gao, Ada Fang, Yepeng Huang, Valentina Giunchiglia, Ayush Noori, Jonathan Richard Schwarz, Yasha Ektefaie, Jovana Kondic, Marinka Zitnik

Cell 2024

clean-usnob

Online Adaptation of Language Models with a Memory of Amortized Contexts
Jihoon Tack, Jaehyung Kim, Eric Mitchell, Jinwoo Shin, Yee Whye Teh, Jonathan Richard Schwarz

🗣️ Slides
🎞️ Video
💻 Code
🔗 Project Website

NeurIPS 2024

clean-usnob

Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding
Talfan Evans, Shreya Pathak, Hamza Merzic, Jonathan Richard Schwarz, Ryutaro Tanno, Olivier J. Henaff

ECCV 2024

clean-usnob

C3: High-performance and low-complexity neural compression from a single image or video
Hyunjik Kim, Matthias Bauer, Lucas Theis, Jonathan Richard Schwarz, Emilien Dupont

💻 Code
🔗 Project Website

CVPR 2024

clean-usnob

Efficient Meta-Learning via Error-based Context Pruning for Implicit Neural Representations
Jihoon Tack, Subin Kim, Sihyun Yu, Jaeho Lee, Jinwoo Shin, Jonathan Richard Schwarz

🗣️ Slides
💻 Code

NeurIPS 2023

clean-usnob

Secure Out-of-Distribution Task Generalization with Energy-Based Models
Shengzhuang Chen, Long-Kai Huang, Jonathan Richard Schwarz, Yilun Du, Ying Wei

NeurIPS 2023

clean-usnob

Modality-Agnostic Variational Compression of Implicit Neural Representations (VC-INR)
Jonathan Richard Schwarz*, Jihoon Tack*, Yee Whye Teh, Jaeho Lee, Jinwoo Shin

ICML 2023

* : Joint first authorship

clean-usnob

Spatial Functa: Scaling Functa to ImageNet Classification and Generation
Matthias Bauer*, Emilien Dupont, Andy Brock, Dan Rosenbaum, Jonathan Richard Schwarz, Hyunjik Kim*

arXiv 2023

clean-usnob

Meta-Learning Sparse Compression Networks (MSCN)
Jonathan Richard Schwarz, Yee Whye Teh

Transactions on Machine Learning Research (TMLR) 2022

clean-usnob

Behavior Priors for Efficient Reinforcement Learning
Dhruva Tirumala, Alexandre Galashov, Hyeonwoo Noh, Leonard Hasenclever, Razvan Pascanu, Jonathan Richard Schwarz, Guillaume Desjardins, Wojciech Marian Czarnecki, Arun Ahuja, Yee Whye Teh, Nicolas Heess

Journal of Machine Learning Research (JMLR) 2022

clean-usnob

Powerpropagation: A sparsity inducing weight reparameterisation
Jonathan Richard Schwarz, Siddhant M. Jayakumar, Razvan Pascanu, Peter E. Latham, Yee Whye Teh

Neural Information Processing Systems (NeurIPS) 2021

💻 Code

clean-usnob

Functional Regularisation for Continual Learning using Gaussian Processes
Jonathan Richard Schwarz*, Michalis K. Titsias*, Alexander G. de G. Matthews, Razvan Pascanu, Yee Whye Teh

International Conference on Learning Representations (ICLR) 2020

💻 Code

* : Joint first authorship

clean-usnob

Multiplicative Interactions and Where to Find Them
Siddhant M. Jayakumar, Wojciech M. Czarnecki, Jacob Menick, Jonathan Richard Schwarz, Jack Rae, Simon Osindero, Yee Whye Teh, Tim Harley, Razvan Pascanu

International Conference on Learning Representations (ICLR) 2020

clean-usnob

Meta-Learning surrogate models for sequential decision making
Jonathan Richard Schwarz*, Alexandre Galashov*, Hyunjik Kim, Marta Garnelo, David Saxton, Pushmeet Kohli, SM Ali Eslami°, Yee Whye Teh°

ICLR 2019 Workshop on Structure & Priors in Reinforcement Learning

*, ° : Joint first/senior authorship

clean-usnob

Experience replay for continual learning
David Rolnick, Arun Ahuja, Jonathan Richard Schwarz, Timothy P. Lillicrap, Greg Wayne

Neural Information Processing Systems (NeurIPS) 2019

clean-usnob

Information asymmetry in KL-regularized RL
Alexandre Galashov, Siddhant M Jayakumar, Leonard Hasenclever, Dhruva Tirumala, Jonathan Richard Schwarz, Guillaume Desjardins, Wojciech M Czarnecki, Yee Whye Teh, Razvan Pascanu, Nicolas Heess

International Conference on Learning Representations (ICLR) 2019

clean-usnob

Empirical Evaluation of Neural Process Objectives
Tuan Anh Le, Hyunjik Kim, Marta Garnelo, Dan Rosenbaum, Jonathan Richard Schwarz, Yee Whye Teh

NeurIPS 2018 workshop on Bayesian Deep Learning

clean-usnob

Attentive Neural Processes
Hyunjik Kim, Andriy Mnih, Jonathan Richard Schwarz, Marta Garnelo, SM Ali Eslami, Dan Rosenbaum, Oriol Vinyals, Yee Whye Teh

International Conference on Learning Representations (ICLR) 2019

💻 Code

clean-usnob

Neural Processes
Marta Garnelo, Jonathan Richard Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, Yee Whye Teh

ICML 2018 Workshop on Theoretical Foundations and Applications of Deep Generative Models (Spotlight talk)

🗣️ Talk (credit to Marta)
💻 Code

clean-usnob

Progress & Compress: A scalable framework for continual learning
Jonathan Richard Schwarz, Jelena Luketina, Wojciech M. Czarnecki, Agnieszka Grabska-Barwinska, Yee Whye Teh, Raia Hadsell°, Razvan Pascanu°

International Conference on Machine Learning (ICML) 2018 (Long oral)

🗣️ Talk
📊 Data (Sequential Omniglot)

° : Joint senior authorship

clean-usnob

The NarrativeQA Reading Comprehension Challenge
Tomas Kocisky, Jonathan Richard Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gabor Melis, Edward Grefenstette

🗣️ Talk (credit to Tomas)
📊 Data

Transactions of the Association for Computational Linguistics (TACL) 2018

clean-usnob

A Recurrent Variational Autoencoder for Human Motion Synthesis
Ikhsanul Habibie, Daniel Holden, Jonathan Richard Schwarz, Joe Yearsley, Taku Komura

💻 Code
📊 Data

British Machine Vision Conference (BMVC) 2017


Academic Workshops


clean-usnob

NeurIPS 2024 Compositional Learning: Perspectives, Methods, and Paths Forward

Jonathan Richard Schwarz, Ying Wei, Yilun Du, Laurent Charlin, Mengye Ren, Matthias Bethge

clean-usnob

ICLR 2023 Neural Fields across Fields: Methods and Applications of Implicit Neural Representations

Jonathan Richard Schwarz, Hyunjik Kim, Emilien Dupont, Thu Nguyen-Phuoc, Vincent Sitzman, Srinath Sridhar

clean-usnob

NeurIPS 2021 Workshop on Meta Learning

Jonathan Richard Schwarz, Fábio Ferreira, Erin Grant, Frank Hutter, Joaquin Vanschoren, Huaxiu Yao

clean-usnob

NeurIPS 2020 Workshop on Meta Learning

Jonathan Richard Schwarz, Roberto Calandra, Jeff Clune, Erin Grant, Joaquin Vanschoren, Francesco Visin, Jane Wang

clean-usnob

ICML 2020 Workshop on Continual Learning

Jonathan Richard Schwarz, Rahaf Aljundi, Eugene Belilovsky, Arslan Chaudhry, Puneet Dokania, Sayna Ebrahimi, Haytham Fayek, David Lopez-Paz , Marc Pickett

Based on Jon Barron's website.