COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space
Published in ICML 2026, 2026
Unsupervised skill discovery aims to learn diverse behaviors without reward functions, but uniform exploration often results in task-irrelevant or hazardous behaviors. Guided skill discovery addresses this by incorporating human intent, yet existing methods typically require training additional guidance models and rely on pre-defined rules or expert demonstrations. COLLIE leverages dense unsupervised data to construct a semantically coherent skill latent space, enabling reliable guidance from sparse online human feedback and a training-free construction of guidance signals, which eliminates the need for extra model training beyond skill learning. Theoretical analysis and experiments across state-based and pixel-based tasks show that COLLIE learns diverse, human-aligned skills, avoids hazardous behaviors, and achieves superior downstream performance with minimal human feedback.