AI scientists and open-endedness

One of the most important questions in AI is how foundation models can be used as scientists that train AI agents, acting as an interface between our human needs and the open-ended space of possible skills and behaviors. This is a research direction I have been exploring for a while, starting with Motif, where we used LLMs as intrinsic motivators to drive open-ended exploration, and with MaestroMotif, where we used LLMs as skill designers to seed a library of skills that an agent can compose into complex behavior.

A way that I find beautiful of thinking about this research direction is that of AI-assisted Agent Design, a collaboration paradigm in which human and AI scientists work together for creating agents to the benefit of human users. Technically, this can mean a few things, and in my work it was instantiated as AI-assisted reward design and AI-assisted skill design.

research note · last updated 2026