This is the opening post in a two-part series. Read Part 2 here.
At roughly the same moment that Alan Turing was developing his ideas on machine intelligence in Manchester, another future heavyweight of computing, Douglas Engelbart, was building an alternative computing model more than 5,000 miles away in the Bay Area.
Engelbart held that computers, because they can combine and work with huge amounts of information, ought to assist people in solving problems instead of excluding them from the problem-solving process. This philosophy is now called augmented intelligence. Engelbart’s work in the field (first as a PhD student at UC Berkeley and later at SRI over the following decades) was perhaps most famously shown in “The Mother of All Demos” in 1968, when he introduced for the first time many of the computing capabilities we now regard as standard — the mouse, GUIs, hyperlinks, word processing, version control, and even video conferencing — all in one presentation.
Even though it’s tempting to imagine artificial intelligence crossing human-equivalence tests like Turing’s Imitation Game (or perhaps something more advanced for today’s general-purpose AI models), we really ought to be considering how Engelbart’s thinking maps onto our modern AI period. In other words, how do we create the next Mother of All Demos?
In practice, the next mother of all demos will be far more than a demo. We already possess the key ingredients — an entirely new collection of AI tools — so now we have to ask how those ingredients can help us rethink and redesign our current workflows and user experiences. By doing that, we can bring about a new category of AI-native experiences for search, scientific research, game design, and more.
The Mother of All Demos in the age of deep learning
If we’re creating a deep learning-era version of the Mother of All Demos, we could start with these models and tools. Although research labs seem to ship new models almost every week, these are an excellent place to start rethinking the way we engage with technology:
GPT-3
The newest model in OpenAI’s GPT line, GPT-3 is a 175-billion parameter language model trained on nearly all text available on the Internet. After training, GPT-3 can produce fluent text on virtually any subject (including in the voice of specific writers or authors), condense passages of text, and translate text into other languages. OpenAI also recently launched a successor, InstructGPT, which uses human feedback to lessen harmful or biased outputs.
Copilot
Github’s Copilot brings “translating text into different languages” into the world of programming. Built on OpenAI’s Codex — think GPT-3, but trained on words and code — Copilot is an “AI pair programmer” that can produce anything from single lines to entire functions of code, using a docstring that explains what the code should accomplish. More recently, DeepMind introduced a rival code-generation model, AlphaCode, which tackles coding problems from popular programming contests.
DALL-E
Where GPT-3 and Copilot create text from text, DALL-E is trained to create images from text. In fact, it uses the same model architecture as GPT, only trained on text-image pairs rather than text by itself. Although DALL-E’s 12-billion parameters are an order of magnitude fewer than GPT-3, the end result is still remarkably capable — DALL-E can generate anything from whimsical sketches to photorealistic images, and even produce drawings that contain ideas not present in DALL-E’s training data (or anywhere else, for that matter). OpenAI recently unveiled GLIDE as a lighter-weight (but equally performant) version of DALL-E.
AlphaFold
DeepMind’s AlphaFold is different from the earlier models in that it is not meant to be general purpose, but instead addresses one very specific issue: protein-folding. AlphaFold has learned to forecast the 3D structure of proteins from their component amino acid parts and, using about 170,000 known protein structures as training data, can now predict up to 98.5% of the proteins that exist in the human genome. Because proteins are so pervasive in biology and because protein structure is tied to protein function, accurately predicting protein structures gives AlphaFold huge potential across a broad set of industries and uses.
With that introduction to these well-known AI models, here are some ideas on how we might use them to improve and, more importantly, reimagine how we carry out certain tasks, processes, and even jobs.
A curated and informed search experience
“There is a growing mountain of research. But there is increased evidence that we are being bogged down today as specialization extends. The investigator is staggered by the findings and conclusions of thousands of other workers.”
— As We May Think, Vannevar Bush, 1945
When writing the lines above, Vannevar Bush, the scholar who became a wartime scientific adviser to Presidents Franklin Roosevelt and Harry Truman (and a major influence on Engelbart), worried that the “information explosion” set off by the rush of R&D during WWII would affect progress in the post-war scientific community. Bush argued that only by creating new tools to fight information overload could scientists be set free to pursue “objectives worthy of their best.”
The web pushes this to an entirely new scale. While scientists in Bush’s period shared their research through a small number of physically bound journals in each research field, the web has driven an exponential surge in scientific research that is published mainly online (open-access sites like Arxiv are a good example of this). A recent study indicates that the academic community produces more than 8 million new articles each year — a huge leap from the tens of thousands of articles published in the 1940s.
And keep in mind, that is only an estimate of the number of academic articles. As of 2016, the biggest newspapers in the United States were each publishing hundreds of pieces per day. Put academic articles and newspaper articles together, and you still have not come close to the full amount of what is on the web! If we’re going to keep creating this much, we’re going to need a better search experience.
As one example of the difficulties created by today’s information overload, let’s walk through a small thought experiment. Suppose I wanted help drafting this article and went to Google with the query “article on augmented intelligence and deep learning”:
Those 37.4 million results could include articles, op-eds, blog posts, social media and message board posts, videos, and research publications. The difficulty is choosing the most relevant (5? 10? 100?) results, summarizing them, and then combining them into suggestions, possible references, and maybe even candidate sentences/paragraphs for an article that readers of Future might find interesting.
Handling this firehose of information is a dimensionality-reduction problem — taking hundreds of thousands (or millions) of versions of a thing and distilling them into a small, usable set. Google has traditionally done this by surfacing the most relevant pages first, where “relevance” is set by PageRank — Larry Page’s well-known algorithm that blends semantic similarity between a page and a user’s query with the number of links pointing to a page.
Still, real dimensionality reduction calls for a search that reaches beyond counting references. For instance, you can picture several themes that might support an article on this subject, such as AGI vs. augmented intelligence through a strictly economic lens, or a technical exploration of how AI and augmented intelligence systems are constructed. You can also picture a range of sentiment, from strongly pro-AGI or anti-Engelbart, to pieces condemning AGI and putting dates on the next AI winter. This article sits somewhere in the middle.
To get the most out of the growing mountains of web content available to us, we need to be able to connect examples across all of these axes independently. In addition, we need to choose the right number of these buckets to show the user. Themes, opinions, and sentiment are a good beginning, but are there other ways of classifying content that we are overlooking? Too few buckets may narrow a user’s choices; too many buckets merely pushes the information-overload problem farther out.
Thankfully, GPT-3 can assist us here. GPT-3 is a general-purpose language model, so it can make an argument for any side, on any topic, in any number of styles. It can also condense any block of text and pull out key statistics. It is entirely plausible that we can use GPT-3 to make a first pass over content, scoring each item along multiple dimensions like those above and classifying it accordingly (presumably via a redesigned search interface). If we are researching an article, as in my earlier example, we may also be able to depend on GPT-3 to surface main points and other material to include, based on the author’s aims.
A GitHub for protein development
AlphaFold is another intriguing example here — but as a source of useful information that could genuinely benefit from a new search experience. The AlphaFold Protein Structure Database has roughly 360,000 predicted protein structures and DeepMind intends to add around 100 million (!!) more protein structures this calendar year. In addition, many experts have praised the promise of this database for de novo protein design — the process of algorithmically generating new proteins that do not occur naturally but follow natural folding laws. The uses of de novo proteins are far-reaching — new proteins will help us create better vaccines, fight cancer, and break down materials that damage the environment — but once we open the floodgates, brute-force methods for discovery and experimentation simply will not be enough.
To make this de novo landscape usable, and to operationalize AlphaFold more generally, we will need to search for and prioritize proteins by their structure and likely function. We will also need to refresh and redesign other basic computational tools such as version control and backtesting.
For instance, imagine you are aiming at a class of similarly structured de novo proteins that may work for breaking down carbon in the atmosphere. AlphaFold gives you 3D structure, but to move this (possibly large) set of protein candidates from simulation to reality you will have to synthesize each individual protein and gather experimental results to test whether it works. Lab experiments are slow and costly, though, so beginning from zero for every new experiment is not feasible.
In some situations, there may be experimental results for a similar, naturally occurring protein that was tested manually in a lab somewhere. You will need a way to simplify how the results for your new proteins could be compared with those existing results. Even better would be a GitHub-like service for protein design that lets experimentalists store the diffs of protein candidates and any experimental results available for them. This would unlock the advantages of open source and lower the barriers to entry for working with novel proteins.
Changing tone like we change fonts
Before windows and GUIs, there was the terminal. Before the terminal, there were punch cards. Generative AI models like GPT-3, AlphaFold, and DALL-E are now having their “terminal” moment. Working with them does not mean manually punching holes in cards, but it does require familiarity with programming, APIs, and at least a basic understanding of how the model operates under the hood. There is a huge opportunity to use these models to rethink how we interact with and visualize workflows for the problems we solve.
Take GPT-3, which produces coherent text but falls short on common sense and on its ability to invent truly new ideas from existing data sources. (The second issue was called by Turing “Lady Lovelace’s objection,” in reference to Ada Lovelace’s famous claim that computers have “no pretensions whatever to originate anything.”) We probably should not, then, redesign word processors so they look and behave like the Google homepage — all you need is a small text field for input and out comes your essay, letter, or poem! That technically simplifies the writing process, but it does not make writing (or communication) any better.
Instead, we should use the chance to redesign writing around the human strengths of ideation and revision. We can swap blank pages and blinking cursors for a canvas for building and rearranging webs of concepts, themes, and thought bubbles, and then let GPT take it from there. We can create tools for increasing or decreasing the sentiment of a sentence or adjusting the prosaic style of a paragraph with the same ease as changing fonts today.
A recent project out of Stanford has produced an exciting GPT-3-backed tool called CoAuthor that represents a first step toward some of the augmentation possibilities we’ve discussed. CoAuthor allows writers to request suggestions from GPT-3, displaying the small snippets of text in a similar way to how autocomplete suggestions are shown on smartphone keyboards. Here is an example of an interactive writing session with CoAuthor:
Interactive experimentation for complex science
You could imagine similar changes for AlphaFold. Although AlphaFold can generate 3D protein structures with unprecedented accuracy, it addresses only half of the large, messy problem that is understanding protein behavior. By knowing a protein’s structure, we gain a clue to its function. But protein behavior depends on both structure and context. Proteins fold into their structures under specific conditions, such as when they bind with other proteins/substrates, react to chemicals, or interact with surrounding molecules.
AlphaFold currently leaves out these conditions. Moreover, understanding information that is immediately actionable like, say, how a new drug is likely to interact with a particular protein (and at what binding sites) demands finer-grained mapping than AlphaFold can currently offer. (Future iterations of AlphaFold will aim at protein-protein complexes, which will help somewhat with this problem.)
It will be important to fill in this context in a streamlined way that is usable by both protein experts and people without PhDs in biology. Unity is a strong case study here. By supplying core infrastructure for rendering 3D graphics and simulating physical phenomena like gravity (and refining these ideas), Unity has played a central role in speeding progress for industries like gaming, self-driving vehicles, filmmaking, architecture/construction, and more.
Building plug-and-play, Unity-style scene creation with an encoding of what we understand about protein-protein and protein-drug interactions underneath might just let us produce large-scale simulations of how our target proteins might interact with, say, a mountain of plastic bottles or carbon-filled air currents over the Pacific Ocean. Such a tool would accelerate basic tests of validity for protein use cases, fill in missing context and, crucially, lower the bar for less technical users to iterate on new ideas.
In this part, we have taken a first look at how we can rethink existing workflows through the lens of recent advances in AI and deep learning. But there is a common thread in each of these use cases — they all take the model as is. In Part 2, we will look at a new set of tools and processes for interacting with these models, that make the models themselves more effective.