Predicting protein structures with a multiplayer online game

Abstract

People exert large amounts of problem-solving effort playing computer games. Simple image- and text-recognition tasks have been successfully 'crowd-sourced' through games, but it is not clear if more complex scientific problems can be solved with human-directed computing. Protein structure prediction is one such problem: locating the biologically relevant native conformation of a protein is a formidable computational challenge given the very large size of the search space. Here we describe Foldit, a multiplayer online game that engages non-scientists in solving hard prediction problems. Foldit players interact with protein structures using direct manipulation tools and user-friendly versions of algorithms from the Rosetta structure prediction methodology, while they compete and collaborate to optimize the computed energy. We show that top-ranked Foldit players excel at solving challenging structure refinement problems in which substantial backbone rearrangements are necessary to achieve the burial of hydrophobic residues. Players working collaboratively develop a rich assortment of new strategies and algorithms; unlike computational approaches, they explore not only the conformational space but also the space of possible search strategies. The integration of human visual problem-solving and strategy development capabilities with traditional computational algorithms through interactive multiplayer games is a powerful new approach to solving computationally-limited scientific problems.

Figure 1. Foldit screenshot illustrating tools and visualizations

The visualizations include a clash representing atoms that are too close (arrow 1); a hydrogen bond (arrow 2); a hydrophobic sidechain with a yellow blob because it is exposed (arrow 3); a hydrophilic sidechain (arrow 4); and a segment of the backbone that is red due to high residue energy (arrow 5). The players can make modifications including bands (arrow 6), which add constraints to guide automated tools and freezing (arrow 7), which prevents degrees of freedom from changing. The GUI includes information about the player’s current status, including score (arrow 8); a leaderboard (arrow 9), which shows the scores of other players and groups; toolbars for accessing tools and options (arrow 10); chat for interacting with other players (arrow 11); and a cookbook for making new automated tools or “recipes” (arrow 12).

Figure 2. Structure prediction problems solved by Foldit players

Examples of blind structure prediction problems in which players were successfully able to improve structures. Native structures are shown in blue, starting puzzles in red, and top scoring Foldit predictions in green. (a) The red starting puzzle had a register shift and the top scoring green Foldit prediction correctly flips and slides the beta strand. (b) On the same structure as above, Foldit players correctly buried an exposed Isoleucine in the loop on the bottom right by remodeling the loop backbone. (c) The top scoring Foldit prediction correctly rotated an entire helix that was misplaced in the starting puzzle. (d) The starting puzzle had an exposed Isoleucine and Phenylalanine on the top, as well as an exposed Valine on the bottom left. The top scoring Foldit prediction was able to correctly bury these exposed hydrophobic residues. (e) Another successful Foldit helix rotation that correctly buries an exposed Phenylalanine. Images were produced using PyMOL software.

Figure 3. Puzzles in which human predictors outperform the Rosetta rebuild and refine protocol

Panels a, b, and c show puzzle 986875. Panels d and e show puzzle 986698. (a) Comparison of Foldit player solutions (green) to the low energy structures sampled in Rosetta rebuild and refine trajectories (yellow) for blind Foldit puzzle 986875 based on the recently determined structure and sequence of 2kpo. The x-axis is the all-atom RMSD to 2kpo, and the y-axis is the Rosetta energy. The starting Foldit puzzle was 4.28 Å away from the native structure (shown by the black dot on the plot); Foldit players sampled many different conformations, with the top scoring submission (the lowest scoring Rosetta energy) 1.4 Å away from the native, while the automated Rosetta protocol did not sample below 2Å. The blue dots and lines correspond to the trajectory of a single Foldit player in c. (b) Superposition of the top-scoring Foldit prediction in green with the experimentally determined NMR model 1 in blue. The starting puzzle is in red, where the terminal strand is incorrectly swapped with its neighbor, 8% of all Foldit players were able to correctly swap these strands (Table S2). (c) A score trajectory with selected structures for the top scoring player in puzzle 986875 over a two hour window, showing how the player explores through high energy conformations to reach the native state. The y-axis is the Rosetta energy and the x-axis is the elapsed time in hours. The starting structure had a Rosetta energy of -243. Each point in the plot represents a solution produced by this player. The first structure (c1) is near the starting puzzle structure, shown as the black dot in a. The following structures (c2-6) are shown as blue dots in plot a. In structures c2-4 the player must explore higher energies to move the strand into place, shown by the blue lines. In structures c5-6 the player refines the strand pairing. (d) Comparison of Foldit player solutions (green) to the low energy structures sampled in Rosetta rebuild and refine trajectories (yellow) for blind Foldit puzzle 986698 based on the recently determined structure and sequence of 2kky. Foldit players were able to get the best Foldit score by correctly picking from multiple alternative starting Rosetta models (black) the model that was closest to the the native structure. (e) The native structure is shown in blue with the top scoring Foldit prediction shown in green. The top Rosetta rebuild and refine prediction given the same 10 starting models (shown in yellow) was unable to sample as close to the native as the Foldit players.

Figure 4. Player move preferences

(a) Different Foldit players take different approaches to solving the same problem. Each circle represents the move type frequencies used in the top solution produced by each player in different time frames: the inner denotes the first hour, the middle denotes the first day, and the outer denotes the puzzle’s entire duration. Each color represents a different type of move that can be made in the game. The left column reflects player move types for puzzles that start relatively close to the native topology. The right column reflects player move types for puzzles that start from a fully extended conformation. Each row represents a different Foldit player. Each player’s preferred move types across each puzzle class are distinct from one another, yet a player’s preferences are similar for both classes of puzzles. Also note that the move preferences change over the lifetime of a puzzle; local minimize is heavily preferred by the end of puzzles but not by all players at the beginning. The move types preferences are very different from Rosetta’s current best automated protocol, rebuild and refine, shown in b.