Andrew Luo

I am a tenure-track Assistant Professor at the University of Hong Kong jointly appointed by Musketeers Foundation Institute of Data Science and the Psychology Department starting in 2024 October.

I received my joint PhD in Machine Learning & Neural Computation from Carnegie Mellon University (CMU) in 2024, where I worked with Michael Tarr and Leila Wehbe. Before that, I earned my undergraduate degree in Computer Science from the Massachusetts Institute of Technology (MIT) in 2019. I also have a Master of Science in Machine Learning Research from CMU.

My work focuses on understanding the computational principles underlying visual perception and how these principles can inform the development of improved generative models and intelligent machines. Ultimately, I aim to bridge the gap between human and machine reasoning, leading to both a deeper understanding of human cognition and advancements in artificial intelligence.

I have completed PhD student hiring for the 2025 Fall cycle. I welcome RAs (remote or in-person) or remote collaboration with PhDs, master's, and undergraduates.

Email  /  HKU Email  /  Google Scholar  /  Github /  WeChat / 中文

Research

I am interested in understanding human perception and building better generative models and machines that are capable of human-like reasoning.

We show that artifacts can be removed from pre-trained ViTs without any labeled data by introducing registers in post-training. Our method uses a model combined with test-time augmentation to distill itself, leading to significant improvements in open-vocabulary segmentation and dense prediction tasks.

We show how to construct higher visual cortex encoders that can generalize across subjects, scanners, voxel sizes, protocols, and images without any additional finetuning by using meta-learning across subjects and in-context learning across stimuli.

We propose image-conditioned decoding of perceived motion from fMRI data, we show that this can be used to animate images with an video diffusion model.

We propose an efficient gradient-free distillation module capable of extraction high quality dense CLIP embeddings, and utilize these embeddings to understand semantic selectivity in the visual cortex.

We investigate the problem of visual-acoustic navigation conditioned on a continuous acoustic field representation of audio.

We leverage ideas from information theory to understand the contributions of individual text tokens and their interactions when generating images.

We propose a way to leverage contrastive image-language models (CLIP) and fine-tuned language models to generate natural language descriptions of voxel-wise selectivity in the higher order visual areas.

We propose a way to generate images that activate regions of the brain by leveraging natural image priors from Diffusion models.

We examine the selectivity of the brain to real-world size in complex natural images.

We propose a learnable and compact implicit encoding for acoustic impulse responses. We find that our NAFs can achieve state-of-the-art performance at a tiny size footprint.

We show that having a prototype memory with attention mechanisms can improve image synthesis quality, and learn interpretable visual concept clusters.

We propose a surface based discriminator for implicit shape generation. Our discriminator uses differentiable ray-casting and marching cubes.

We propose contrained scene synthesis using graph neural networks, we show that generated scenes can be refined using differentiable rendering.

We propose a learnable decomposition of 3D shapes into symbolic programs that can be executed.