“Sloppy” Systems Biology

Systems biologists seek to model many complex biological interactions all at once. Typically, they input tens or even hundreds of variables to produce predic- tions about a system—for example, how a cell might react to an environmental signal, or how an animal might respond to a drug. But, researchers have now found, many systems models are strikingly vulnerable to even small changes in the variables, according to a recent analysis of 17 such simulations.

“This pattern we see is universal,” says Ryan Gutenkunst, PhD, who performed the research under James Sethna, PhD, a professor of physics at Cornell University. “It’s common among all these models.” The work was published in PLoS Computational Biology in October 2007.

Typically, modelers scan the literature or perform experiments to define the parameters of a system. But, Gutenkunst notes, such experimental data might not reflect biological reality. For example, an enzyme may function differently in a test tube than it does in a cell. And although scientists knew some models were sensitive to parameter variation, the extent of the problem was elusive.

To test how well models deal with varying parameters, Gutenkunst and his colleagues collected 17 systems biology models, including the yeast cell cycle, circadian rhythms, and others, from the literature and an online database. All 17 examples were vulnerable to producing inaccurate predictions when parameters changed only a small amount. Gutenkunst and his co-authors say this means the models are “sloppy,” which doesn’t necessarily mean bad. “Sloppy’s a descriptive word for the fact that there’s all this wiggle room,” he says.

The traditional approach to modeling is akin to basic arithmetic: If every number on the left-hand side of an equation (i.e., the parameters) is known, then the answer (the prediction) is calculable. Gutenkunst and his co-authors support an alternative more like algebra: There are unknown variables on the left-hand side, but using a known answer on the right, it’s possible to work backwards to define them.

“You can still get good useful predictions out of these models,” Gutenkunst says. He suggests plugging in real-life information—the right-hand side of the equation—and searching for parameters that give the correct result. For example, modelers could use experimental data on how yeast grow to determine what parameters will work on the left-hand side of their cell-cycle equation. The researchers found that even if they can’t define the parameters precisely, they still get useful predictions.

While the algebra approach to modeling is not new, the notion that “sloppiness” pervades biological modeling will apply to many researchers, says Nathan Price, PhD, a systems biologist at the University of Illinois at Urbana-Champaign. “What they argue is that it’s not even very worthwhile to try to know all these parameters in advance,” Price says. “It’s a very broad message.”