“Sloppy” Systems Biology

Many systems models are strikingly vulnerable to even small changes in the variables

Systems biologists seek to model many complex biological interactions all at once. Typically, they input tens or even hundreds of variables to produce predic- tions about a system—for example, how a cell might react to an environmental signal, or how an animal might respond to a drug. But, researchers have now found, many systems models are strikingly vulnerable to even small changes in the variables, according to a recent analysis of 17 such simulations.


“This pattern we see is universal,” says Ryan Gutenkunst, PhD, who performed the research under James Sethna, PhD, a professor of physics at Cornell University. “It’s common among all these models.” The work was published in PLoS Computational Biology in October 2007.


In these diagrams of model input, the best predictions result when the input parameters (colored points) are within the central ellipse. The red points in (B) represent what happens when all parameters are measured accurately. In (C), the blue points show how the predictions can become inaccurate when even one  parameter is off. In (A), working backward from real data to  constrain parameters results in all of the yellow points staying within the ellipse, where predictions are most trustworthy. Courtesy of Ryan Gutenkunst.Typically, modelers scan the literature or perform experiments to define the parameters of a system. But, Gutenkunst notes, such experimental data might not reflect biological reality. For example, an enzyme may function differently in a test tube than it does in a cell. And although scientists knew some models were sensitive to parameter variation, the extent of the problem was elusive.


To test how well models deal with varying parameters, Gutenkunst and his colleagues collected 17 systems biology models, including the yeast cell cycle, circadian rhythms, and others, from the literature and an online database. All 17 examples were vulnerable to producing inaccurate predictions when parameters changed only a small amount. Gutenkunst and his co-authors say this means the models are “sloppy,” which doesn’t necessarily mean bad. “Sloppy’s a descriptive word for the fact that there’s all this wiggle room,” he says.


The traditional approach to modeling is akin to basic arithmetic: If every number on the left-hand side of an equation (i.e., the parameters) is known, then the answer (the prediction) is calculable. Gutenkunst and his co-authors support an alternative more like algebra: There are unknown variables on the left-hand side, but using a known answer on the right, it’s possible to work backwards to define them.


“You can still get good useful predictions out of these models,” Gutenkunst says. He suggests plugging in real-life information—the right-hand side of the equation—and searching for parameters that give the correct result. For example, modelers could use experimental data on how yeast grow to determine what parameters will work on the left-hand side of their cell-cycle equation. The researchers found that even if they can’t define the parameters precisely, they still get useful predictions.


While the algebra approach to modeling is not new, the notion that “sloppiness” pervades biological modeling will apply to many researchers, says Nathan Price, PhD, a systems biologist at the University of Illinois at Urbana-Champaign. “What they argue is that it’s not even very worthwhile to try to know all these parameters in advance,” Price says. “It’s a very broad message.”


All submitted comments are reviewed, so it may be a few days before your comment appears on the site.

Post new comment

The content of this field is kept private and will not be shown publicly.
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Enter the characters shown in the image.