Junseok (June) Oh

Hi, I'm Junseok. I am a Software Engineer on the Search Ads AI team at Google and an MSE graduate in Data Science from the University of Pennsylvania. My research aims to push the frontiers of foundational multimodal intelligence—bridging multimodal foundation models with real-time embodied perception to build systems that seamlessly perceive, reason, and interact in the open world.

Specifically, I study multimodal social attention: how an agent infers who needs attention, what matters in the moment, and when assistance is appropriate using cues like gaze, head orientation, language, and turn-taking. At the MIT Media Lab Personal Robots Group (mentored by Dong Won Lee and Dr. Hae Won Park), I currently work on multimodal egocentric head-gaze prediction and real-time referential control. Previously, I conducted health-focused research spanning smartphone-based biosensing and single-cell analysis. I hold a B.S. in Computer Science with a minor in Mathematics from Purdue University.

Email  /  CV  /  Scholar  /  GitHub

News

Research

While listening, a robot must infer from streaming speech and visual context whether to shift gaze, what to look at, and where the referent is. We combine a multimodal foundation model and object segmentation with privileged information distillation to achieve low-latency, open-vocabulary referential gaze control.

We develop a smartphone-integrated, optomechanical dual-mode biosensing instrument that combines vision-based readout with frequency-shift measurements to enable low-cost, rapid detection of Salmonella Typhimurium for practical food-safety monitoring.

Research Projects

K-BioX - Stanford Cardiovascular Institute
Jan 2022 – Aug 2022 and Jan 2024 – May 2024
Advisors: Dr. Siyeon Rhee and Prof. Joseph C. Wu

Built an automated scRNA-seq pipeline over public GEO datasets to compare adult vs. P12 developmental stages, then used GSEA, cell-to-cell interaction profiling, and PCA/UMAP to surface adipogenesis/angiogenesis signals across epicardial adipose tissue (EAT) datasets.

Wittgen Biotechnology - UC Berkeley SkyDeck
May 2022 – Aug 2022
Research Intern

Worked on ML for high-resolution tumor classification and tailored drug recommendations by processing RNA-seq data from 40+ cancer patients, contributing to an AI platform for cancer heterogeneity profiling, and using Seurat for single-cell analysis and cell-type differentiation.

Teaching

Template based on Jon Barron's website; see Leonid Keselman's Jekyll fork.