GS Oh

I am a research engineer at Google DeepMind. My current interest includes AI agents, LLM post-training, and generative models. I am a core contributor of Gemini Deep Research and I have worked on reasoning, planning for tool/extensions for LLMs. Previously I worked on self/semi-supervised i18n models for automatic speech recognition and deliver them to the partners such as Pixel, Assistant, Youtube, and Google Cloud.

I earned my Ph.D. degree at the University of Michigan where I worked on generative AIs and decision-making under uncertainty. My PhD research focused on generative models (Normalizing flow, VAE, and Transformers) and learning-based planners (model-based RL, optimal control, as well as some model-free RLs), mainly in the context of autonomous driving (prediction & planning) and sequence prediction applications (e.g., NLP). In early 2020, I was a teaching assistant for the Machine Learning course (EECS545) at the University of Michigan.

During my PhD, I did an internship at Google where I developed a Transformer-based non-autoregressive neural sequence model for natural language processing. I also interned at Amazon AI (AWS AI) where I researched deep generative models for music generation using Transformers. In 2019, I interned at Uber ATG where I worked on autoregressive generative models and normalizing flow for autonomous driving.


Contact: gsoh@umich.edu



Publication

Improving Top-K Decoding for Non-Autoregressive Semantic Parsing Via Intent Conditioning
Geunseob (GS) Oh, Rahul Goel, Chris Hidey, Shachi Paul, Aditya Gupta, Pararth Shah, Rushin Shah
International Conference on Computational Linguistics (COLING), 2022.
Keywords: Transformers, Conversational Agents, Language Models, Semantic Parsing, NLP

CVAE-H: Conditionalizing Variational Autoencoders via Hypernetworks and Trajectory Forecasting for Autonomous Driving
Geunseob (GS) Oh, Huei Peng
Arxiv, 2022.
Keywords: Generative models, Autonomous driving, Variational autoencoder, Hypernetwork, Deep learning.

HCNAF: Hyper-Conditioned Neural Autoregressive Flow and its Application for Probabilistic Occupancy Map Forecasting, [Code]
Geunseob (GS) Oh, Jean-Sebastien Valois
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Keywords: Normalizing flow, Autoregressive models, Generative Models, Hyper-network, Autonomous driving, Density estimation

Control-theoretic Evaluation of Policy in Sequential Decision Making via Data-driven Differential Game
Geunseob (GS) Oh*, Nauman Sohani*, Huei Peng
Robust Autonomy Workshop at Robotics: Science and Systems (RSS), 2020.
Keywords: Robust control, Differential game, Reinforcement learning, Anomaly detection, Reachability

Impact of Traffic Lights on Trajectory Forecasting of Human-driven Vehicles Near Signalized Intersections
Geunseob (GS) Oh, Huei Peng
Workshop on Planning, Perception and Navigation for Intelligent Vehicles at IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.
Keywords: Deep learning, Generative Models, Mixture density network, Autonomous driving, Behaviour cloning

A Data-driven Model of Human Driver Behavior Using Falsification
Nauman Sohani*, Geunseob (GS) Oh*, Xinpeng Wang
American Control Conference (ACC), 2020.
Keywords: Formal verification, Deep learning, Optimal control, Behaviour modeling

Vehicle Energy Dataset (VED), A Large-scale Dataset for Vehicle Energy Consumption Research, [Code]
Geunseob (GS) Oh, David J. LeBlanc, Huei Peng
IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2020.
Keywords: Dataset, Autonomous driving

Eco-driving at Signalized Intersections: What is Possible in the Real-World?
Geunseob (GS) Oh, Huei Peng
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2018.
Keywords: Optimal control, Dynamic programming, Autonomous driving


Ph.D. Thesis

A Prediction and Planning Framework for Scalable Autonomous Driving in Urban Areas
Geunseob (GS) Oh, Huei Peng, Honglak Lee, Ram Vasudevan, Ilya Kolmanovsky
Available soon.
Keywords: Autonomous driving, Generative models, Model-based RL, Probabilistic ML, Optimal control.


Patents

Vehicle trajectory prediction near or at traffic signal
Geunseob (GS) Oh, Huei Peng
Published in 2021.
Keywords: Autonomous driving, Self-driving, Connected vehicles, Traffic lights, Deep learning.



Teaching

EECS545 - Machine Learning (with Prof. Honglak Lee): Teaching assistant (Jan - May 2020)



Professional Experience

  • 2022 - curr: Research Engineer, Google DeepMind
  • 2015 - 2021: PhD @ University of Michigan
  • Summer 2021: Research Intern, Google
  • Summer 2020: Applied Scientist Intern, Amazon AI
  • Summer 2019: Research Intern, Uber ATG
  • Summer 2014: Software engineer Intern, Harvard-MIT HST program



Favorite Quote

Quote from Interstellar (2014): “We’ve always defined ourselves by the ability to overcome the impossible. And we count these moments. These moments when we dare to aim higher, to break barriers, to reach for the stars, to make the unknown known. We count these moments as our proudest achievements. But we lost all that. Or perhaps we’ve just forgotten that we are still pioneers. And we’ve barely begun. And that our greatest accomplishments cannot be behind us, because our destiny lies above us.”