An Integrated Framework for Controllable Text Generation

Natural language generation is an increasingly popular area for the application of deep learning techniques. In this talk, I will be discussing how we have tackled some of the longstanding challenges associated with this field, including the lack of interpretability and controllability in neural generation models, and the lack of task-specific training data and reliable evaluation methods. To achieve these goals, we have developed a new framework that consists of four major components: (1) high-quality data construction for real-world applications, such as text simplification and scientific writing; (2) controllable neural generation models; (3) interactive annotation interfaces; and (4) a redesigned and more reliable evaluation methodology. In particular, I will highlight our recent work on LENS, a learnable evaluation metric capable of differentiating between GPT-3.5 types of models that are highly competitive and perform at levels close to humans. LENS correlates better with human judgments than existing metrics, such as SARI and BERTScore, for evaluating text generation outputs. Moreover, LENS can serve as an effective learning objective in Minimum Bayes Risk (MBR) Decoding and Minimal Risk Training (MRT), which allows for the creation of state-of-the-art neural generation models that outperform fine-tuned large language models.