How OpenAI Built its Groundbreaking Deep Research Product ft. Isa Fulford

The head of the Deep Research team at OpenAI on the reinforcement learning techniques and tools that power this new ChatGPT feature.

Post methodology: Claude 3.7 via custom Dust assistant @TDep-SubstackPost with the system prompt: Please read the text of the podcast transcript in the prompt and write a short post that summarizes the main points and incorporates any recent news articles, substack posts or X posts that provide helpful context for the interview. Please make the post as concise as possible and avoid academic language or footnotes. please put any linked articles or tweets inline in the text. Please refer to Podcast guests by their first names after the initial mention. Light editing and reformatting for the Substack editor.

Isa Fulford, who leads the deep research team at OpenAI, unveiled the capabilities and development journey of one of the company's most powerful AI tools. Deep Research represents a significant milestone in ChatGPT's ability to navigate the web, analyze data, and produce comprehensive reports on complex topics.

From Concept to Capability

Isa explained that Deep Research is an agentic capability in ChatGPT that conducts multi-step research online to solve complex tasks. When prompted, the system spends between 5 to 30 minutes searching through numerous web sources and reasoning about the content before delivering a fully cited, comprehensive report "at around the level of a research analyst." She added, "It's able to do in a few minutes what would take a human many hours."

The genesis of deep research came from OpenAI's internal progress with reinforcement learning and reasoning models. While the team had seen generalization from training on math, science, and coding tasks to other domains, they wondered what would happen if they trained directly on tasks that users perform in their daily lives.

Web browsing emerged as an ideal starting point for several reasons. First, online research is ubiquitous across professions and personal activities. Second, as Isa pointed out, "read-only agents" presented a more constrained environment with fewer safety considerations than agents that could take actions in the world.

Building the System

The development process began with creating a prototype to generate internal excitement. Isa and her colleagues, including Yash Patil and Thomas Dimson, hacked together a demo by prompting existing models to show what the Deep Research product might look like, without training any new models at that stage.

Once they had organizational buy-in, the team focused on training models specifically for web browsing and data analysis capabilities. This involved:

1. Creating reinforcement learning tasks to teach the model browsing capabilities

2. Developing tools for the model to use during training

3. Giving the model access to a browser for searching, clicking, and scrolling

4. Providing code execution capabilities for data analysis and visualization

The result is a version of OpenAI's o3 model that has been fine-tuned specifically for web browsing and data analysis.

Related: OpenAI Deep Research team on Sequoia’s Training Data podcast

Deep Research in Action

During her presentation, Isa demonstrated several use cases for Deep Research. One example involved analyzing venture capital investment trends in AI companies, complete with data visualization. Another showed how she used Deep Research while traveling in Korea to find night markets within 15 minutes of her location, drawing on both English and Korean sources to identify the best-rated food stalls at each market.

The system's design includes an initial clarification step where it asks questions to ensure it understands the user's request before beginning its research. This helps users be more specific about their needs and increases the likelihood of getting relevant results.

Isa also highlighted a biology use case, where Deep Research successfully identified gene therapies that have gained regulatory approval in the US for treating hemophilia, complete with citations and explanations.

Looking Ahead

While acknowledging that the system isn't perfect and can sometimes hallucinate information, Isa outlined several exciting directions for Deep Research:

1. Improving reliability to reduce hallucinations

2. Integrating Deep Research capabilities into OpenAI's main reasoning model

3. Bringing private context into Deep Research, including internal company knowledge and paywalled sources

4. Moving beyond information synthesis to enabling the system to take actions

The work on Deep Research has already improved other OpenAI models. As Isa noted, "o3 is good at searching because it's trained with the same tools and browsing datasets that we developed for Deep Research." This capability represents a significant step toward AI systems that can navigate the web with human-like research abilities, potentially transforming how professionals across industries gather and synthesize information.