Protein Structure Prediction

Or, "How to win the Nobel Prize"

10/19/2026

print view

Recall

• Network Architectures

  • Convolutional Neural Networks - learn functions on spatially local space
  • Graph Convolutional Neural Networks - learn functions on local neighborhood of nodes
  • Attention (Transformers) - learns query/key/value functions whose outputs are combined to identify what to pay attention to

• Molecular Representations

  • 2D: fingerprints, SMILES, graphs
  • 3D Graphs
    • Complete (full-connected), e.g. point cloud
    • Nearest neighbor
  • Grids (volumetric)

%%html
<script src="https://bits.csb.pitt.edu/preamble.js"></script>

The Problem

• We have resolved many fewer protein structures than we have sequenced
• Sequence (mostly) determines structure
• Can we computationally predict structure from sequence?

Rennie ML, Oliver MR. Emerging frontiers in protein structure prediction following the AlphaFold revolution. Journal of the Royal Society Interface. 2025 Apr 16;22(225):20240886.

Protein Folding

We can fold proteins with MD!

but...

CASP: Critical Assessment of protein Structure Prediction

• International contests since 1994

  • every 2 years
  • 200+ methods, 100 research centers, 80 targets

• Categories

  • Template Modeling
  • Free Modeling
  • Assessment

https://predictioncenter.org/

How do we know if protein structure predictions are good?

Comparing Structures: RMSD

Emphasizes correctness of the entire protein and heavily penalizes errors.

$$\begin{align*} \mathrm{RMSD}(\mathbf{v}, \mathbf{w}) &= \sqrt{\frac{1}{n} \sum_{i=1}^{n} \lVert \mathbf{v}_i - \mathbf{w}_i \rVert^2 } \\ &= \sqrt{ \frac{1}{n} \sum_{i=1}^{n} \Big[ (v_{ix} - w_{ix})^2 + (v_{iy} - w_{iy})^2 + (v_{iz} - w_{iz})^2 \Big] } \end{align*}$$

Requires aligned structures

Comparing Structures: Global Distance Test Total Score

$$ \mathrm{GDT\_TS} = \frac{1}{4} \cdot (P_{1} + P_{2} + P_{4} + P_{8}) $$

$P_i$ = fraction of predicted residues with distance from correct position equal to $d \le i$ Ångstroms

TM-score (Template Modeling Score)

TM-score measures the structural similarity between two protein structures. It emphasizes global fold similarity and is less sensitive to local outliers than RMSD.

$$\text{TM-score} = \frac{1}{L} \sum_{i=1}^{L} \frac{1}{1 + \left(\frac{d_i}{1.24\sqrt[3]{L-15}-1.8}\right)^2}$$

Where:

• $d_i$ = distance between the i-th pair of aligned residues
• $L$ = number of residues
• $1.24\sqrt[3]{L-15}-1.8$ = empirically derived length-dependent scaling factor

---

• Range: 0 to 1
• 1.0 → identical structures
• > 0.5 → typically same overall fold
• < 0.2 → likely unrelated structures
• Length-normalized → comparable across proteins of different sizes
• Downweights large distance errors → more fold-focused than RMSD

Comparing Structures: Local Comparisons

Compare pairwise distances, not positions

• dRMSD
• LDDT
  • 0.5 Å, 1 Å, 2 Å and 4 Å cutoffs
  • Only evaluates local distances (<15Å in reference structure)

Predicted Aligned Error (PAE)

How confident is AlphaFold in the relative placement of two residues?

AlphaFold: A practical guide       

$\mathrm{PAE}_{ij}$ estimates the positional error of residue $i$ when predicted and true structures are aligned using the frame of residue $j$.

• Low PAE: confident relative placement.
• High PAE: uncertain relative placement.
• Measured in Ångstroms; lower is better.
• Generally asymmetric: $\mathrm{PAE}_{ij} \ne \mathrm{PAE}_{ji}$.

Reading a PAE heatmap

• Low-error blocks along the diagonal often correspond to well-defined domains.
• High error between blocks indicates uncertain relative domain placement.

%%html
<div id="psp_local_global" style="width: 500px"></div>
<script>
$('head').append('<link rel="stylesheet" href="https://bits.csb.pitt.edu/asker.js/themes/asker.default.css" />');
var divid = '#psp_local_global';
jQuery(divid).asker({
    id: divid,
    question: "Both domains of a protein are predicted accurately, but their relative orientation is wrong. Which outcome is most plausible?",
    answers: [
        "Low RMSD and low lDDT",
        "High RMSD but high lDDT for most residues within each domain",
        "High RMSD and necessarily low lDDT everywhere",
        "Low RMSD and high lDDT"
    ],
    server: "https://bits.csb.pitt.edu/asker.js/example/asker.cgi",
    charter: chartmaker
});
$(".jp-InputArea .o:contains(html)").closest('.jp-InputArea').hide();
</script>

Spoilers!

2024 Nobel Prize in Chemistry

A Brief History of Protein Structure Prediction

Secondary Structure Prediction

Template-Based Structure Prediction

Use a known protein structure as a template for a target sequence.

• Homology modeling: identify an evolutionarily related protein using sequence or sequence-profile similarity.
• Threading / fold recognition: score how well the target sequence fits candidate structures, considering burial, secondary structure, and residue contacts. Can detect compatible folds when sequence similarity is weak.

How do we build the model?

1. Align the target sequence to the template.
2. Use template geometry to guide the backbone.
3. Model insertions, deletions, and side chains; refine and assess quality.

Both depend on choosing a suitable template and getting the alignment right.

I-TASSER

Replica Exchange Monte Carlo. Best webserver up until 2020.

Zhang Y. I-TASSER server for protein 3D structure prediction. BMC bioinformatics. 2008 Jan 23;9(1):40.

Classical Rosetta & Template Free Modeling

Key Idea: Combinatorial search in fragment space.

Advances in protein structure prediction and design. Kuhlman & Bradley. Nature Reviews, 2019.

Contact Prediction

• Predict if residues $(i,j)$ are within a certain threshold distance

• Use to constrain 3D structure prediction

%%html
<div id="psp_salt_bridge" style="width: 500px"></div>
<script>
$('head').append('<link rel="stylesheet" href="https://bits.csb.pitt.edu/asker.js/themes/asker.default.css" />');

var divid = '#psp_salt_bridge';
jQuery(divid).asker({
    id: divid,
    question: "In an MSA, two positions commonly contain Lys-Glu or Glu-Lys, but rarely Lys-Lys or Glu-Glu. Which interaction is most plausibly preserved by these correlated substitutions?",
    answers: [
        "A disulfide bond",
        "Hydrophobic packing",
        "A salt bridge",
        "A peptide bond"
    ],
    server: "https://bits.csb.pitt.edu/asker.js/example/asker.cgi",
    charter: chartmaker
});

$(".jp-InputArea:contains('%%html')").hide();
</script>

Evolutionary Covariance

Potts Model

Statistical mechanics model of sequence space with Hamiltonian:

$$\Large H = \sum_i h_i(s_i) + \sum_{i<j} J_{ij}(s_i,s_j)$$

GREMLIN

$$\Large P\left( x_i^{n} \,\middle|\, x_{-i}^{n}, \mathbf{v}, \mathbf{w} \right) = \frac{1}{Z_i} e^{v_i \!\left( x_i^{n} \right) + \sum_{\substack{j \neq i}}^{L} w_{i,j} \!\left( x_i^{n}, x_j^{n} \right)} $$

Markov Random Field method for learning Pott's Model from MSA

https://9kssnq.csb.app/ (Sergey Ovchinnikov)

Distance Prediction and ResNets

Distance-based protein folding powered by deep learning. Jinbo Xu, 2019.

AlQuraishi M. End-to-end differentiable learning of protein structure. Cell systems. 2019 Apr 24;8(4):292-301.

AlphaFold2

How good was AlphaFold2?

(Å RMSD95), median	AlphaFold	Next best
Backbone Accuracy	0.96	2.8
All-atom Accuracy	1.5	3.5

How good was AlphaFold2?

AlphaFold2 Model Overview

AlphaFold2 Deep Dive

Architecture Pt. 1

MSA and Pair Attention in the Evoformer

Attention

Jay Alammar - Illustrated Transformer

Model Overview

Evoformer Block

MSA Attention Example (row-wise)

Uses all of the residues in a single sequence to improve our understanding how residues interact.

MSA Attention Example (column-wise)

Uses all of the sequences in the MSA to improve our understanding of a single residue.

Evoformer Block

Triangle Update

Evoformer Block

Attention on the Pair Representation (Implicit △s)

The triangular self-attention updates the pair representation in the Evoformer block. The “starting node” version updates the edge $ij$ with values from all edges that share the same starting node $i$, (i.e. all edges $ik$). The decision whether edge $ij$ will receive an update from edge $ik$ is not only determined by their query-key similarity (standard attention), but also modulated by the information $b_{jk}$ derived from the third edge $jk$ of this triangle. Furthermore we extend the update with an additional gating $g_{ij}$ derived from edge $ij$.

Model Overview

MSA/Pair Communication

“Here’s what I got so far…”

“Hmm, I think these residues have an evolutionary signal.”

“Well, since you mentioned it, maybe take a look at these contacts too.”

Architecture Pt. 2

The Structure Module and “Residue Gas” Representation

Model Overview

Structure Module

Invariant Point Attention

Backbone Frames Make up the Residue Gas

$$ \Large T_i := (R_i,\vec{t}_i) \qquad \begin{aligned} \vec{x}_{\mathrm{global}} &= T_i\circ\vec{x}_{\mathrm{local}}\\ &=R_i\vec{x}_{\mathrm{local}}+\vec{t}_i \end{aligned} $$

Injecting Pair Representation Knowledge

Standard row-wise Sequence Attention

Attention on frames (3D-structure)

Structure Module Overview

Model Overview

Refinement in structure module

Improves both accuracy and stereochemical quality

© 2020 DeepMind Technologies Limited

Losses

Complete Loss Function

$$\small \mathcal{L}=\begin{cases} \begin{aligned}0.5\mathcal{L}_{\mathrm{FAPE}}+0.5\mathcal{L}_{\mathrm{aux}}+0.3\mathcal{L}_{\mathrm{dist}}+2.0\mathcal{L}_{\mathrm{msa}}+0.01\mathcal{L}_{\mathrm{conf}}\end{aligned} &\text{training}\\[12pt] \begin{aligned}0.5\mathcal{L}_{\mathrm{FAPE}}+0.5\mathcal{L}_{\mathrm{aux}}+0.3\mathcal{L}_{\mathrm{dist}}+2.0\mathcal{L}_{\mathrm{msa}}+0.01\mathcal{L}_{\mathrm{conf}}+0.01\mathcal{L}_{\mathrm{exp\ resolved}}+1.0\mathcal{L}_{\mathrm{viol}}\end{aligned} &\text{fine-tuning} \end{cases} $$

$$\tiny \mathcal{L}_{\mathrm{viol}}=\mathcal{L}_{\mathrm{bondlength}}+ \mathcal{L}_{\mathrm{bondangle}}+\mathcal{L}_{\mathrm{clash}} $$

$\mathcal{L}_{\mathrm{FAPE}}$	Alignment free point-wise comparison
$\mathcal{L}_{\mathrm{aux}}$	Angles; Difference and length;
$\mathcal{L}_{\mathrm{dist}}$	distogram
$\mathcal{L}_{\mathrm{conf}}$	confidence
$\mathcal{L}_{\mathrm{exp\ resolved}}$	“Is an atom experimentally resolved?”
$\mathcal{L}_{\mathrm{msa}}$	MSA Mask (a la BERT)

Frame aligned point error (FAPE) Loss

Predicted Accuracy (x-axis) vs True Accuracy (y-axis)

Pearson r=0.76

Computed with per-residue networks

%%html
<div id="psp_plddt_meaning" style="width: 500px"></div>
<script>
$('head').append('<link rel="stylesheet" href="https://bits.csb.pitt.edu/asker.js/themes/asker.default.css" />');
var divid = '#psp_plddt_meaning';
jQuery(divid).asker({
    id: divid,
    question: "A residue has pLDDT = 90. What does that mean?",
    answers: [
        "There is a 90% probability that its coordinates are exactly correct",
        "90% of the five models agree on its position",
        "Its position differs from experiment by 0.90 angstroms",
        "A learned head predicts a local lDDT score of about 90"
    ],
    server: "https://bits.csb.pitt.edu/asker.js/example/asker.cgi",
    charter: chartmaker
});
$(".jp-InputArea .o:contains(html)").closest('.jp-InputArea').hide();
</script>

Ablation Study - Which model components were most important?

Multiple Sequence Alignments Impact Predictions

Neff = normalized nonredundant sequences

Coverage = how much of the target does this template provide structural information for?

Iterative predictions are crucial

Model Overview

What’s new/interesting?

1. Joint embedding architecture of MSAs and pairwise features
2. Output structure representation (“residue gas”)
3. Equivariant attention (Invariant Point Attention)
4. Losses: Intermediate, Masked MSA, FAPE, others
5. Self-distillation (learning from unlabelled data)
6. Self-estimated accuracy

%%html
<div id="af2_det" style="width: 500px"></div>
<script>
$('head').append('<link rel="stylesheet" href="https://bits.csb.pitt.edu/asker.js/themes/asker.default.css" />');
var divid = '#af2_det';
jQuery(divid).asker({
    id: divid,
    question: "If you run AlphaFold2 multiple times on the same input, what do you get?",
    answers: [
        "The same structure",
        "Very similar structures",
        "The Boltzmann distribution",
    ],
    server: "https://bits.csb.pitt.edu/asker.js/example/asker.cgi",
    charter: chartmaker
});
$(".jp-InputArea .o:contains(html)").closest('.jp-InputArea').hide();
</script>

AF2 Inference: Five Predictions

• Five trained models in the original monomer release.

  • Models 1 and 2 use template inputs.
  • Models 3–5 do not use template inputs.

• Effectively deterministic for fixed weights and processed inputs with standard inference settings.

  • Different models or MSA samples can produce different predictions.

• Recycling refines a prediction using the same network weights. It does not simulate a folding trajectory.

Structure Prediction $\ne$ Protein Folding

Many proteins fold in vitro ... but it can get complicated

Molecular chaperones are a group of unrelated protein families whose roles include preventing protein aggregation, assisting in correct folding and assembly of proteins, or disassembling complexes or aggregates.

Chaperone Example: Heat Shock Protein 70

Chaperone Example: Heat Shock Protein 70

v= py3Dmol.view('1dkx',style='cartoon:color~magenta'); v.setStyle({'chain':'B'},{'stick':{'colorscheme':'whiteCarbon'}}); v.setStyle({'chain':'A','resi':'1-507'},{'cartoon':{'color':'lightgreen'}});v.zoomTo({'chain':'B'}); v.show()

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

Chaperones as Drug Targets

Geldanamycin HSP90 Inhibitor

• Extremely challenging

• Cancer: increased mutation burden and protein expression in cancer cells mean they are more dependent on chaperones for proper protein folding

  • Targeting heat shock proteins induces degradation of mutated proteins
  • Inhibitors commonly bind to ATP binding pocket
  • Toxicity issues

• Neurodegenerative Diseases: Alzheimer’s, Parkinson’s, and Huntington’s disease all involve misfolded protein aggregates.

  • Upregulating chaperones may reduce aggregation (Arimoclomol - Niemann–Pick disease)

Key Takeaways

• Evolution provides structural information. Templates provide known geometry; coevolving residues suggest interactions.

• AlphaFold2 integrates sequence, evolutionary, and geometric information. The Evoformer updates MSA and pair features; the structure module updates residue frames.

• Geometry matters in both the architecture and the loss. IPA uses spatial relationships; FAPE compares positions in residue-local frames.

• Assess local accuracy and global arrangement separately. pLDDT estimates local accuracy; PAE assesses confidence in relative placement; pTM is overall structure.

• Structure prediction is not a simulation of protein folding. A predicted structure does not reveal folding pathways or conformational populations.

Next Time

• AlphaFold successors
• AlphaFold3