Jeonghwan Kim

About

Ph.D. candidate at BLENDER Lab at University of Illinois, Urbana-Champaign (UIUC), currently advised by Professor Heng Ji

I obtained my Master's from the School of Computing, KAIST, where I was advised by Prof. Sung-Hyon Myaeng.

My primary research interests lie in the intersection of Natural Language Processing (NLP) and Machine Learning (ML). I want to build next-generation language models that are capable of dealing with the interplay of general and domain-specific knowledge; the knowledge, either parametric or non-paramaeteric, could also manifest in various modalities, e.g., images, which has recently motivated me to look into multimodal large language models (MLLMs).

Currently, I'm interested in developing multimodal large language models (MLLMs) that can process both the visual and textual modalities, and understand them in the light of parametric and external knowledge.

Here are some keywords about my research interests:

Multimodal Large Language Models (e.g., Large Vision Language Models)
Retrieval-Augmented Generation (RAG) & Language Modeling
Question Answering (QA) using LLMs

Research Experience

MetaRedmond, WA, USA

Research Scientist Intern May. 2025 - Aug. 2025

AmazonBellevue, WA, USA

Applied Scientist Intern May. 2024 - Aug. 2024

University of Illinois Urbana-Champaign (UIUC) Champaign, IL, USA

Graduate Research Assistant (Ph.D.) Aug. 2023 - Present

Advisor: Heng Ji

KAIST IR&NLP Lab Daejeon, Republic of Korea

Research Associate Mar. 2022 - July. 2023

Graduate Research Assistant (M.S.) Feb. 2020 - Feb. 20222

Advisor: Sung-Hyon Myaeng

Publications

^* indicates equal contribution.

Infogent: An Agent-Based Framework for Web Information Aggregation

Revanth Gangi Reddy*, Sagnik Mukherjee*, Jeonghwan Kim*, Zhenhailong Wang*, Dilek Hakkani-Tur, Heng Ji

NAACL 2025, Findings

Paper

Aligning LLMs with Individual Preferences via Interaction

Shujin Wu, May Fung, Cheng Qian, Jeonghwan Kim, Dilek Hakkani-Tur and Heng Ji

COLING 2025

Paper

Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models

Jeonghwan Kim, Heng Ji

EMNLP 2024

Paper

Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise

Giwon Hong*, Jeonghwan Kim*, Junmo Kang*, Sung-Hyon Myaeng, Joyce Jiyoung Whang

NAACL 2024, Findings

Paper

ARMADA: Attribute-Based Multimodal Data Augmentation

Xiaomeng Jin, Jeonghwan Kim, Yu Zhou, Kuan-Hao Huang, Te-Lin Wu, Nanyun Peng, Heng Ji

EMNLP 2024 Workshop @ WikiNLP

Paper

FinePrompt: Unveiling the Role of Finetuned Inductive Bias on Compositional Reasoning in GPT-4

Jeonghwan Kim*, Giwon Hong*, Sung-Hyon Myaeng, Joyce Jiyoung Whang

EMNLP 2023, Findings

Paper Poster

Graph-Induced Transformers for Efficient Multi-Hop Question Answering

Giwon Hong, Jeonghwan Kim, Junmo Kang, Sung-Hyon Myaeng

EMNLP 2022

Paper Poster

Exploiting Numerical-Contextual Knowledge to Improve Numerical Reasoning in Question Answering

Jeonghwan Kim, Junmo Kang, Giwon Hong, Kyung-min Kim, Sung-Hyon Myaeng

NAACL 2022, Findings

Paper Poster

Leveraging Order-Free Tag Relations for Context-Aware Recommendation

Junmo Kang, Jeonghwan Kim, Suwon Shin, Sung-Hyon Myaeng

EMNLP 2021

Paper Poster

Have You Seen That Number? Investigating Extrapolation in Question Answering Models

Jeonghwan Kim, Giwon Hong, Kyung-min Kim, Junmo Kang, Sung-Hyon Myaeng

EMNLP 2021

Paper Poster

Can You Distinguish Truthful from Fake Reviews? User Analysis and Assistance Tool for Fake Review Detection

Jeonghwan Kim^*, Junmo Kang^*, Suwon Shin^*, Sung-Hyon Myaeng

HCI+NLP | EACL 2021

Paper Poster

Vitæ

Full CV in PDF.

Meta May. 2025 - Aug. 2025

Research Scientist Intern
RAG + LLM
Amazon May. 2024 - Aug. 2024

Applied Scientist Intern
Multimodal Representation Learning
UIUC Aug. 2023 - Present

Ph.D. in Computer Science
BLENDER Lab
KAIST IR&NLP Lab Mar. 2022 - Present

Research Associate
Working on QA and Graph Networks
KAIST Feb. 2020 - Feb. 2022

M.Sc. in School of Computing
IR&NLP Lab
Republic of Korea Marine Corps Mar. 2015 - Dec. 2016

Honorably discharged as Sergeant
Mandatory Military Service
Handong Global University Mar. 2014 - Feb. 2020

B.Sc. in Computer Science & Electrical Engineering
Magna Cum Laude