Efficient LLM inference: KV cache quantization and pruning.
Undergraduate Researcher · SYMPLE Lab, GIST
I am a senior undergraduate in Electrical Engineering and Computer Science at GIST and a research intern at SYMPLE Lab, advised by Prof. Sangjin Kim.
My research focuses on KV cache optimization. As LLM context windows grow longer, the KV cache becomes a major memory and bandwidth bottleneck during inference, and I work on compressing it without losing long-context accuracy. Building on this, I am expanding my interests toward on-device LLM inference and NPU design.
I came to research from software engineering, building and operating web services used by students on campus. That work shaped how I approach systems problems: measure where the bottleneck is first, then change the structure.
Pairs non-orthogonal query and key transforms so that INT2 key quantization and query-channel pruning work together. Reaches 6.8× KV cache compression at 128K context and up to 3.75× decoding throughput in SGLang.
Efficient LLM inference: KV cache quantization and pruning.
Encodes GIST's curriculum rules, which change by admission year and major, into verifiable checks on uploaded transcripts. Used by about 51% of GIST undergraduates across four rebuilds.
Pipeline that scrapes National Assembly bills and restructures them with GPT, plus an embedding-based issue-cluster visualization.