Many Preferences, Few Policies: Compact Portfolios for Multi-Objective LLM Alignment

Under review at ICLR 2027

PALM, a provably near-optimal small portfolio of aligned LLMs that serves users with heterogeneous preferences across math reasoning, safety, and helpfulness.

arXiv

Recommended citation: Cheol Woo Kim*, Jai Moondra*, Roozbeh Nahavandi, Andrew Perrault, Milind Tambe, Swati Gupta. "Many Preferences, Few Policies: Compact Portfolios for Multi-Objective LLM Alignment." Under review at ICLR 2027. (* equal contribution)
Download Paper
Direct Link