AlphaFold 2 - CASP

1 AlphaFold 2. John Jumper1* , Richard Evans1*, Alexander Pritzel1*, Tim Green1*, Michael Figurnov1*, Kathryn Tunyasuvunakool1*, Olaf Ronneberger1*, Russ Bates1*, Augustin dek1*, Alex Bridgland1*, Clemens Meyer1*, Simon A A Kohl1*, Anna Potapenko1*, Andrew J Ballard1*, Andrew Cowie1*, Bernardino Romera-Paredes1*, Stanislav Nikolov1*, Rishub Jain1*, Jonas Adler1, Trevor Back1, Stig Petersen1, David Reiman1, Martin Steinegger2, Michalina Pacholska1, David Silver1, Oriol Vinyals1, Andrew W Senior1, Koray Kavukcuoglu1.

2 Pushmeet Kohli1, Demis Hassabis1* . 1. DeepMind, London, UK, 2 Seoul National University, South Korea * Equal contribution Corresponding authors: John Jumper Demis Hassabis 2020 DeepMind Technologies Limited Protein folding at DeepMind 2020 DeepMind Technologies Limited DeepMind is on a long-term mission to advance scienti c progress We're interested in solving fundamental scienti c problems using AI. Protein folding is such an important fundamental problem that is well-suited for AI. We're thankful that CASP is providing such an ideal experimental setup to evaluate progress Presenting the work of the AlphaFold team 2020 DeepMind Technologies Limited Alex Bridgland Alexander Pritzel Andrew Cowie Andrew Senior Andy Ballard Anna Potapenko Augustin dek Bernardino Romera Paredes Clemens Meyer John Jumper Kathryn Tunyasuvunakool Michael Figurnov Olaf Ronneberger Richard Evans Rishub Jain Russ Bates Simon Kohl Stanislav Nikolov Tim Green + Jonas Adler, Trevor Back, Stig Petersen, David Reiman.

3 Martin Steinegger, Michalina Pacholska, David Silver, Oriol Vinyals, Koray Kavukcuoglu, Pushmeet Kohli, Demis Hassabis & with help from many others from across DeepMind Protein example: T1064 (ORF8) 2020 DeepMind Technologies Limited T1064 / 7jtl GDT. (ORF8, SARS-CoV-2). Ground truth Prediction 7 JTL: Flower, , et al. (2020) Structure of SARS-CoV-2 ORF8, a rapidly evolving coronavirus protein implicated in immune evasion. Biorxiv. Protein example: T1044 (RNA Polymerase) 2020 DeepMind Technologies Limited Folding as a single long chain Long-chain-trained model trained after the submission Individual domains T1041 T1042 T1043.

4 6VR4: Leiman, , et al. Virion-packaged DNA-dependent RNA polymerase of Ground truth crAss-like phage phi14:2 (CASP target). (To be published.). Prediction Inductive Bias for Deep Learning Models 2020 DeepMind Technologies Limited Convolutional Networks Recurrent Networks ( computer vision) ( language). data in regular grid data in ordered sequence information ow to local neighbours information ow sequentially Graph Networks ( recommender Attention Module ( language). systems or molecules) data in unordered set data in xed graph structure information ow dynamically controlled information ow along xed edges by the network (via keys and queries).

5 Putting our protein knowledge into the model 2020 DeepMind Technologies Limited Physical insights are built into the network structure, not just a process around it End-to-end system directly producing a structure instead of inter-residue distances Inductive biases re ect our knowledge of protein physics and geometry The positions of residues in the sequence are de-emphasized Instead residues that are close in the folded protein need to communicate The network iteratively learns a graph of which residues are close.

6 While reasoning over this implicit graph as it is being built residues residues system Design Inputs 2020 DeepMind Technologies Limited Sequence databases UniRef906 (JackHMMER3). BFD5 (HHblits4). MGnify clusters2 (JackHMMER3). HMMER. Structural databases PDB1 (training). PDB70 clustering (hhsearch4). All publicly available data. [1] Berman et al., Nature Structural Biology (2003) [2] Mitchell et al., Nucleic Acids Research (2019) [3] Potter et al., Nucleic Acids Research (2018) [4] Steinegger et al., BMC Bioinformatics (2019) [5] Steinegger et al.

7 , Nature Methods (2019) [6] Suzek et al., Bioinformatics (2015) Visualisations: The PyMOL Molecular Graphics system , Version Schr dinger, LLC. AS Rose, et al., Bioinformatics (2018) Embedding Trunk Heads 2020 DeepMind Technologies Limited MSA sequence-residue edges Con dence residues score High residues residues con dence Low sequences sequences Attention sequences .. con dence Genetic Update search pairs Update Structure seqs module pairing residues residues 3D structure Attention Pairwise residues residues.

8 Distances residue-residue edges MSA picture inspired by: Riesselman, , Ingraham, & Marks, , Nature Methods (2018) templates Template embedding 2020 DeepMind Technologies Limited 4 templates used (from PDB70 clusters, searched with HHsearch1,2). Input features are sequences, side chains, and distograms Templates are processed in the same way as the residue-residue representation Partial template: [1] Remmert, M., Biegert, A., Hauser, A., & S ding, J. (2012). HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment.

9 Nature Methods, 9(2), 173-175. [2] Steinegger, M. et al. (2019). HH-suite3 for fast remote homology detection and deep protein annotation. BMC Bioinformatics, 20(1), 1-15. Structure module 2020 DeepMind Technologies Limited End-to-end folding instead of gradient descent 3-D equivariant transformer architecture updates the rigid bodies / backbone Protein backbone = gas of 3-D rigid bodies Also builds the side chains (chain is learned!). Target: T1041. Image: Dcrjsr, vectorised Adam R dzikowski (CC BY , Wikipedia).

10 Structure module 2020 DeepMind Technologies Limited End-to-end folding instead of gradient descent 3-D equivariant transformer architecture updates the rigid bodies / backbone Protein backbone = gas of 3-D rigid bodies Also builds the side chains (chain is learned!). Target: T1041. Image: Dcrjsr, vectorised Adam R dzikowski (CC BY , Wikipedia). Structure module 2020 DeepMind Technologies Limited End-to-end folding instead of gradient descent 3-D equivariant transformer architecture updates the rigid bodies / backbone Protein backbone = gas of 3-D rigid bodies Also builds the side chains (chain is learned!)