Bayesian Optimization in Language space: An Eval-Efficient AI Self-Improvement Framework

This paper discusses how to design an evaluation-efficient self-improving AI systems for societal and business problems like ad optimization where the cost of generating new content is low but evaluation is expensive. It argues that traditional human-driven optimization is slow and bottlenecked by content generation, but generative AI has shifted the bottleneck to efficient evaluation and prompt refinement. T-BoN BO addresses key challenges—lack of numerical gradients in language space, and the need to balance exploration and exploitation—by adapting classic **Bayesian Optimization (BO)** principles. Theoretically, the paper proves that T-BoN BO, which uses textual gradients and a Best-of-N selection rule, **emulates gradient-based Upper Confidence Bound (UCB) BO** and inherits its theoretical guarantees for evaluation efficiency. Empirical results in digital marketing scenarios demonstrate that T-BoN BO significantly **outperforms state-of-the-art baselines** in achieving performance gains under a fixed evaluation budget.