Grey-Box Bayesian Optimization for Human-in-the-loop Optimization

Bayesian optimization (BayesOpt) optimizes time-consuming-to-evaluate objective functions arising in materials design, drug discovery, neural architecture design, and other applications. It combines a Bayesian posterior distribution over the objective function with a decision-theoretic acquisition function that quantifies the value of objective function and constraint evaluations ("experiments").

While BayesOpt is a black-box optimization approach, we have recently shown that "peeking inside the box" can improve performance by several orders of magnitude. Key to this approach are statistical methods that incorporate additional information beyond the values of the objective function. For example, when optimizing quality in a manufacturing process, these methods incorporate observations of quality after each stage of the process, not just the quality of the final output.

This idea also offer a new way to interact with humans who have trouble choosing a single objective function. Rather than estimating a Pareto frontier like traditional multi-objective optimization methods, we can model the human as having a utility function drawn from a Bayesian prior. By iteratively updating a posterior on the human's utility function in response to questions ("which tradeoff between cost and quality do you like better?") and using this knowledge to prioritize experiments, we can identify a set of solutions whose maximum utility is likely to be large. This approach better leverages information about user preferences to provide much better efficiency than traditional mult-objective methods.

We describe the ideas behind these approaches and how they are being used to design novel energy materials in collaboration and optimize online platforms.