HAE-RAE해례

Publications

Research

Benchmarks, evaluation methods, and studies of multilingual and multimodal language models. Everything below is publicly available.

2026
ICLR 2026
Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought
Guijin Son, Donghun Yang, Hitesh Laxmichand Patel, Amit Agarwal, Hyunwoo Ko, et al.
ACL 2026 Findings
What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models
Dasol Choi, Guijin Son, Hanwool Lee, Minhyuk Kim, Hyunwoo Ko, et al.
LREC 2026
Redefining Evaluation Standards: A Unified Framework for Evaluating the Korean Capabilities of Language Models
Hanwool Lee, Dasol Choi, Sooyong Kim, Ilgyun Jeong, Sangwon Baek, et al.
Preprint
KMMMU: Evaluation of Massive Multi-discipline Multimodal Understanding in Korean Language and Context
Nahyun Lee, Guijin Son, Hyunwoo Ko, Chanyoung Kim, et al.
Preprint
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
Nahyun Lee, Dongkeun Yoon, Guijin Son, et al.
2025
EMNLP 2025 Findings
From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for LLM Evaluation
Seokhee Hong, Sunkyoung Kim, Guijin Son, et al.
EMNLP 2025
Multi-LMentry: Can Multilingual LLMs Solve Elementary Tasks Across Languages?
Luca Moroni, Javier Aula-Blasco, Simone Conia, et al.
MRL 2025
Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gap
Hyunwoo Ko, Guijin Son, Dasol Choi
ACL 2025 SRW
Controlling Language Confusion in Multilingual LLMs
Nahyun Lee, Yeongseo Woo, Hyunwoo Ko, Guijin Son
Preprint
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, et al. (incl. Guijin Son) — multi-institution collaboration
Preprint
Ko-PIQA: A Korean Physical Commonsense Reasoning Dataset with Cultural Context
Dasol Choi, Jungwhan Kim, Guijin Son
2024
NAACL 2025
KMMLU: Measuring Massive Multitask Language Understanding in Korean
Guijin Son, Hanwool Lee, Sungdong Kim, Seungone Kim, Niklas Muennighoff, et al.
GoodData@AAAI 2025
LLM-as-a-Judge & Reward Model: What They Can and Cannot Do
Guijin Son, Hyunwoo Ko, Hoyoung Lee, Yewon Kim, Seunghyeok Hong
AI2ASE@AAAI 2025
Improving Fine-grained Visual Understanding in VLMs through Text-Only Training
Dasol Choi, Guijin Son, Soo Yong Kim, Gio Paik, Seunghyeok Hong
C3NLP@ACL 2024
K2EVAL: Harnessing the Evaluation of Linguistic Fluency and Ethnolinguistic Knowledge in Korean
Guijin Son, Hyunwoo Ko, Hoyoung Lee, Seunghyeok Hong, et al.
C3NLP@ACL 2024
Assessing LLMs' Ability to Navigate Cultural Knowledge Conflicts
Guijin Son, Hanwool Lee, Seonkyu Lim, Suzie Oh, et al.
2023
LREC-COLING 2024
HAE-RAE Bench: Evaluation of Korean Knowledge in Language Models
Guijin Son, Hanwool Lee, Suwan Kim, Huiseo Kim, Jae cheol Lee, et al.