Jeonghwan Kim
Ph.D. Candidate, BLENDER Lab, UIUC
jk100 [AT] illinois.edu
About
Ph.D. candidate at BLENDER Lab at University of Illinois, Urbana-Champaign (UIUC), currently advised by Professor Heng Ji
I obtained my Master's from the School of Computing, KAIST, where I was advised by Prof. Sung-Hyon Myaeng.
My primary research interests lie in the intersection of Natural Language Processing (NLP) and Machine Learning (ML). I want to build next-generation language models that are capable of dealing with the interplay of general and domain-specific knowledge; the knowledge, either parametric or non-paramaeteric, could also manifest in various modalities, e.g., images, which has recently motivated me to look into multimodal large language models (MLLMs).
Currently, I'm interested in developing multimodal large language models (MLLMs) that can process both the visual and textual modalities, and understand them in the light of parametric and external knowledge.
Here are some keywords about my research interests:
- Multimodal Large Language Models (e.g., Large Vision Language Models)
- Retrieval-Augmented Generation (RAG) & Language Modeling
- Question Answering (QA) using LLMs
Research Experience
MetaRedmond, WA, USA
Research Scientist Intern May. 2025 - Aug. 2025
AmazonBellevue, WA, USA
Applied Scientist Intern May. 2024 - Aug. 2024
University of Illinois Urbana-Champaign (UIUC) Champaign, IL, USA
Graduate Research Assistant (Ph.D.) Aug. 2023 - Present
Advisor: Heng Ji
KAIST IR&NLP Lab Daejeon, Republic of Korea
Research Associate Mar. 2022 - July. 2023
Graduate Research Assistant (M.S.) Feb. 2020 - Feb. 20222
Advisor: Sung-Hyon Myaeng
Publications
* indicates equal contribution.
PARTONOMY: Large Multimodal Models with Part-Level Visual Understanding
Ansel Blume*, Jeonghwan Kim*, Hyeonjeong Ha, Elen Chatikyan, Xiaomeng Jin, Khanh Duy Nguyen, Nanyun Peng, Kai-Wei Chang, Derek Hoiem and Heng Ji
Preprint
Search and Detect: Training-Free Long Tail Object Detection via Web-Image Retrieval
Mankeerat Sidhu, Hetarth Chopra, Ansel Blume, Jeonghwan Kim, Revanth Gangi Reddy, Heng Ji
CVPR 2025
SYNTHIA: Novel Concept Design with Affordance Composition
Hyeonjeong Ha, Xiaomeng Jin, Jeonghwan Kim, Jiateng Liu, Zhenhailong Wang, Khanh Duy Nguyen, Ansel Blume, Nanyun Peng, Kai-Wei Chang, Heng Ji
ACL 2025
Infogent: An Agent-Based Framework for Web Information Aggregation
Revanth Gangi Reddy*, Sagnik Mukherjee*, Jeonghwan Kim*, Zhenhailong Wang*, Dilek Hakkani-Tur, Heng Ji
NAACL 2025, Findings
Aligning LLMs with Individual Preferences via Interaction
Shujin Wu, May Fung, Cheng Qian, Jeonghwan Kim, Dilek Hakkani-Tur and Heng Ji
COLING 2025
Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models
Jeonghwan Kim, Heng Ji
EMNLP 2024
Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise
Giwon Hong*, Jeonghwan Kim*, Junmo Kang*, Sung-Hyon Myaeng, Joyce Jiyoung Whang
NAACL 2024, Findings
ARMADA: Attribute-Based Multimodal Data Augmentation
Xiaomeng Jin, Jeonghwan Kim, Yu Zhou, Kuan-Hao Huang, Te-Lin Wu, Nanyun Peng, Heng Ji
EMNLP 2024 Workshop @ WikiNLP
FinePrompt: Unveiling the Role of Finetuned Inductive Bias on Compositional Reasoning in GPT-4
Jeonghwan Kim*, Giwon Hong*, Sung-Hyon Myaeng, Joyce Jiyoung Whang
EMNLP 2023, Findings
Graph-Induced Transformers for Efficient Multi-Hop Question Answering
Giwon Hong, Jeonghwan Kim, Junmo Kang, Sung-Hyon Myaeng
EMNLP 2022
Exploiting Numerical-Contextual Knowledge to Improve Numerical Reasoning in Question Answering
Jeonghwan Kim, Junmo Kang, Giwon Hong, Kyung-min Kim, Sung-Hyon Myaeng
NAACL 2022, Findings
Leveraging Order-Free Tag Relations for Context-Aware Recommendation
Junmo Kang, Jeonghwan Kim, Suwon Shin, Sung-Hyon Myaeng
EMNLP 2021
Have You Seen That Number? Investigating Extrapolation in Question Answering Models
Jeonghwan Kim, Giwon Hong, Kyung-min Kim, Junmo Kang, Sung-Hyon Myaeng
EMNLP 2021
Can You Distinguish Truthful from Fake Reviews? User Analysis and Assistance Tool for Fake Review Detection
Jeonghwan Kim*, Junmo Kang*, Suwon Shin*, Sung-Hyon Myaeng
HCI+NLP | EACL 2021
Vitæ
Full CV in PDF.
-
Meta May 2025 - PresentResearch Scientist (intern)
Multimodal RAG + MLLM -
Amazon May 2024 - Aug. 2024Applied Scientist (intern)
Multimodal Representation Learning -
UIUC Aug. 2023 - PresentPh.D. in Computer Science
BLENDER Lab -
IR&NLP Lab, KAIST Mar. 2022 - Jul. 2023Researcher
Working on QA and Graph Networks -
KAIST Feb. 2020 - Feb. 2022M.Sc. in School of Computing
IR&NLP Lab -
Republic of Korea Marine Corps Mar. 2015 - Dec. 2016Honorably Discharged
Mandatory Military Service -
Handong Global University Mar. 2014 - Feb. 2020B.Sc. in Computer Science & Electrical Engineering
Magna Cum Laude