First, a short introduction is in order. My name is Miller. I’m a physics student at UCLA, and a member of BASS. I’m not closeted or anything, I just prefer pseudonymity. I have my own active blog, “Skeptic’s Play“, but I will occasionally contribute to this one. As a blogger, I am probably self-absorbed, and will shamelessly plug my blog often. The following essay has been cross-posted on my blog.
There’s an article in the latest issue of Skeptic Magazine called “A Climate of Belief” by Patrick Frank. It says that the case for Global Warming being caused by CO2 is severely hurt by the fact that computer models of the climate are uncertain. At first, I thought it had raised a fairly good objection, at least good enough that I, mostly clueless about climate science, would have no idea how to refute it. But it turns out that the article fails at basic statistics.
The main argument of the article goes like this:
Computer models of climate show error bars in their results, but these error bars only show one kind of error: the variation between multiple runs of the simulation. What the error bars don’t show is the “physical uncertainty”, the measure of difference between the predicted and actual.
How do we estimate the physical uncertainty? We use the climate model to “retrodict” past climate, and then compare to the actual climate we had during that time. Frank shows that such retrodictions only calculated the total cloud cover with 10% accuracy. Of course, to show this, he uses retrodictions of the 1979-1988 period, and compares them to observations of 1983-1990. I have to wonder if it’s good practice to compare different decades.
He goes on to say that 10% cloud cover has a huge impact on global temperature. How big? 1.1°C a year. That means that after a hundred years, the uncertainty is 110°C! See the graph below of the uncertainty as it increases with time.
This graph is what really set my skeptical bells ringing. Yes it’s true that if the uncertainty is very large, we can draw no conclusions. But how can the error be so large? Intuitively, it does not make sense. If all your results are accurate within, say, 10°C, but the error bars are 100°C, that either means you’ve overestimated your error, or you got really, really lucky. Even global warming deniers will grant that the models are accurate within 10°C. Are they feeling lucky?
So where does his estimate of uncertainty go wrong? Frank’s problem is pure statistical innumeracy. Unfortunately, statistics is not common knowledge, so this sort of innumeracy can go right over some people’s heads. Allow me to explain.
Problem 1: Uncertainties do not add! If you have 1.1°C uncertainty in the first year, and 1.1°C uncertainty in the next year, what is the cumulative uncertainty? You might guess 2.2°C, but this assumes that both uncertainties are always in the same direction. Half of the time, they will be in opposite directions and partly cancel each other out. The result when you work out the math is a total uncertainty of 1.56°C after two years. Sure, it’s possible that it will be off by 2.2°C, but error bars are only supposed to cover the most likely data. The uncertainty does not increase in a straight line. It should be proportional to the square-root of time. That is, it will increase more slowly after a little while. I was extremely shocked at such an egregious error. Has Frank never taken a statistics class?
Problem 2: Uncertainties are reduced in a stable system. The environment is a mostly stable system. That is, it doesn’t swing wildly in temperature every century. If the temperature is a little higher than average one year, something will push it towards normal temperature. For instance, higher temperature might increase cloud cover, which reflects more of the sun’s light away from Earth. Therefore, a temperature uncertainty this year may not survive to the next year. When I said the uncertainty is proportional to the square-root of time, I assumed that the system has no stabilizing mechanisms. In fact, the uncertainty will increase much more slowly than that.
Problem 3: What’s the difference between Frank’s uncertainty and the already reported error bars? Frank asserts that they are different, but I’m not so sure. Frank bases his uncertainty estimate on the predictions of cloud cover. But is this uncertainty different from the uncertainty between different runs of the simulation? I imagine each time the simulation is run, it gives a slightly different prediction of cloud cover in the same way that it gives a slightly different prediction of temperature. So not only is Frank calculating the uncertainty incorrectly, it may have already been accounted for.
Frank seems incredulous that we can estimate the temperature decades from now when we can’t even estimate next year’s temperature accurately. But actually, this makes sense. We can’t predict the whether next week, but we can predict overall trends between seasons. Large, overall trends are easier to predict than year-to-year fluctuations!
I only spot the statistical errors, because that’s the part I know. Given the kinds of errors I see, I wouldn’t be surprised if the rest of it were also riddled with flaws.
Last 5 posts by Tristan Miller
- Presentation: Separation of Church and State - March 6th, 2010
- BASS protests the Westboro Baptist Church - January 17th, 2010
- BASS and friends protest the Phelps family - January 14th, 2010
- Snacks List - January 12th, 2010
- BASS Meeting I - January 6th, 2010
Miller, the mistake in your analysis is through assuming a random uncertainty, as in measurement error, that can go in either direction. In the SI to my article, you’ll find an analysis of the structure of cloud error indicating it stems from theory-bias, and is certainly not random. Theory-bias error produces a continuous divergence. I chose a linear model because that made the fewest assumptions. But depending on the nature of the theory-bias in the actual physics, the divergence could be non-linear. In either case, it would accumulate.
It’s true that climate is a stable system. However, the phase-space is extremely large. That means one can make predictions that have no information content, all the while remaining within the bounds of the system. This is the message of the wide error bars in the Figure you reproduced. The resolution of the models means that they produce no information about future climate, even though their output is bounded.
For an example, see reference 28 in the article and the discussion. The HadCM3 was used in a perfect model test, and it produced bounded solutions that contained no fidelity with respect to the target climate.
Finally, the published simulations I tested were already 10-averages. Random error would already have been reduced by a factor of about 3.2 from the scatter of individual runs.
If you look at reference 23, you’ll see a quote that average cloudiness doesn’t change much from year-to-year, and so averaged simulations can be compared to averaged data.
The cloud error of (+/-)2.8 W m^-2 is a real error that behaves like theory-bias and thus puts uncertainty into every calculated climate state that is coupled to cloud feedback. In a stepwise calculation, which is what future climate prediction is, such uncertainty accumulates. That uncertainty magnitude is also as large as the forcing of all the excess GHGs presently in the atmosphere. Certainty about the future state of Earth climate, including average surface temperature, is completely unjustifiable.
Leave aside the retrodictions, the plain old fashioned predictions are complete and total failures.
Not a single, I repeat, not a single IPCC climate model predicted the 20-30 years of cooling we face. Not a single, not a single model predicted this.
Now in the middle of the cooling they made a new model to include it post hoc, but you must remember that the ex cathedra consensus models failed to predict this.
Pat Frank,
Well, I humbly admit that I cannot answer your objections at this time. It is a combination of not being at all familiar with the field, and not understanding your responses (I’m working on it). All I can say for now is that your article did not by itself convince me that the error in cloud cover is well-correlated from year-to-year and simulation-to-simulation. Nor was I convinced that you can have such large error bars much larger than your errors unless the simulations were very lucky.
Perhaps you didn’t include the details of your reasoning in the article because you thought it would confuse readers? Obviously, I’m confused, so maybe you were right?
Miller,
I don’t pretend to understand all of Dr. Frank’s reply either, but I think I get the gist of it. In respect to your argument that uncertainties don’t add, this is only true if you have a random or near random distribution of uncertaintity. There is nothing that proves that the uncertainties of the computer models are truely random. This means that they need to be treated as additive until you can prove true randomness. There are many examples of estimations with very onesided biases. I work in the software industry and our industry is plagued by huge biases of underestimation. It many times more likely that a software will run over estimation than complete on the estimated date or before. This is because of an inherent bias most developers in my industry display.
I believe was such a bias that Dr. Frank was refering to in measuring the uncertainty with cloud cover. This couldn’t be covered in the multiple runs of the computer simulations if the operators were plugging in biased cloud cover estimates or cloud cover estimates based upon a theory with an inherent bias. The uncertainties that Dr. Frank mentions are an objective way to measure the potential for such bias.
Thanks for posting this and I regret not making it to last week’s meeting when this topic was discussed.
As Frank explains in his comment, I think you are incorrect in your calculation of the compound error in the models. Two year’s predictions are not two random variables being used in a single calculation. If this were the case, your analysis would be correct. However, as Frank explained in his article, the problem with the models is that the prediction for one year’s mean temperature is fed into the calculation for the following year’s prediction. Because of this, the error gets propagated in the fashion depicted in his graph.
I’m coming to tomorrow’s meeting and maybe we can talk some more about it then.
A few comments:
First of all, Pat Frank, contrary to what my tone might have been in the post, I am actually very pleased to have you drop by. Thanks for your time.
I am already familiar with the difference between random and systematic uncertainty. I did not use those words here because they were not in the article. The article simply asserted that the uncertainties accumulate linearly with time without much clear argument that the uncertainties are systematic. I looked at Figure 3, and they looked random to me.
At a second glance, I may have been mistaken about Problem 1. Perhaps the uncertainty in cloud cover is systematic over time.
However, I’m still wondering about Problem 3. Is the cloud cover uncertainty systematic between simulations? Table S2 in your supporting information document (http://tinyurl.com/6f3py6) suggests otherwise, since each simulation has a different error in cloud cover. Pat Frank, if you’re still around, could you explain this?
And if your analysis were accurate, wouldn’t you predict that the temperatures predicted by different simulations would correlate with their error in cloud cover? That is, simulations that predicted much higher cloud cover should also predict much lower temperatures compared to other simulations (with roughly a 1 degree difference per year). I don’t think such a correlation exists. What is wrong with my analysis?
#3 — Miller, please look on your own blog page. I posted a link there for the Supporting Information (SI) document, that you can freely download and examine at your leisure.
Please notice that references to the SI occur throughout my article. They were put there so that anyone would know where to look in the SI to find the analysis that supports the conclusions. Or doesn’t support them, if you find a fatal error.
I put the analysis in the SI to make the article itself accessible to the educated non-scientist reader. There also were space limitations. But my arrangement with Skeptic was, and is, that the SI be freely available for download from their website, ensuring unfettered examination. They have honored that agreement.
Please be assured that there is no attempt to befog, to obscure, to confuse, or to be coy about anything. It’s all there and in the open, with full references to the literature. I’m a professional scientist and I take my professional ethics very seriously indeed. I know that last is only my word, but there it is.
lrbinfrisco and Alan, thanks. You’ve got the gist of the message about uncertainty.
Miller, sorry I didn’t notice your second post until after posting my above reply. I’m happy to spend the time. Thanks for your interest. That applies to everyone else, too.
Anyway, if you look at the correlation matrix in Table S3 and the associated discussion, you’ll see that the cloudiness errors of the GCMs are indeed correlated one with another, very much more so than can be accounted by random errors. Sometimes the correlation is negative but still large.
Clouds turn out to be complex business. Very high ice clouds have an opposite effect on tropospheric temperature relative to low clouds. So, GCMs that are wrong on one of them may produce a different temperature bias relative to GCMs that are wrong on the other. On the other hand, depending on the bias, one GCM may produce excess warming or cooling because of systematic errors in one type of cloudiness, while another may produce the same excess by opposing errors on the other kind of cloudiness.
On the third hand, a GCM that is wrong on both types of clouds may produce compensating errors that give a reasonable-seeming instantaneous climate simulation, but which has no fidelity with the dynamics of the real climate. A GCM with compensating errors also will not be able to reliably predict future climates even if it happens to reproduce the temperature field of the current climate.
Regarding climate predictions, please understand that nothing in the article is about climate per se. The concern is to discern what the models are predicting as regards global average temperature, and a measure of the uncertainty in projected temperature from cloudiness error. That is, the article is an audit of GCM reliability, and not about climate.
Ok, I’ve finally gone through the supporting information document, or most of it (I may have missed some points). I am now convinced that I was mistaken about problem 1, and possibly problem 3 as well. I see that you’ve actually spent a lot of time trying to show the errors are systematic.
But I have a few more questions. First, could you give a fuller explanation of your answer to problem 2? What does it mean for the phase-space to be large, and how do we know it is so?
Second, won’t it decrease the error bars if different kinds of clouds can cancel each other’s effects out?
Third, you say that a GCM can produce compensating errors to give reasonable-seeming results. But isn’t it unlikely that every single GCM would do so in exactly the right way to produce similar results? Isn’t it more probable that your error analysis is mistaken somewhere?
Fourth, your objection to GCMs is based on retrodictions of cloud cover. Wouldn’t it be a lot simpler (and thus, less prone to error) to base it on retrodictions of temperature instead?
Hi Miller — Thanks for your continuing interest, and your forthright reappraisal.
Just to be clear: my intentions was to test the structure of the cloud error series. The conclusion was that they uniformly showed systematic behavior. I didn’t start out intending to show they are systematic. I tested them every way I could think of, and ’systematic’ was the outcome of the analysis.
About climate phase space (or “state-space”), I think of it like this. Every single independent variable of climate amounts to an orthogonal coordinate axis. All the orthogonal axes — the independent variables — produce a multi-dimensional phase space. There may be 10’s or thousands of coordinate axes. At any moment, the climate occupies one point in this space, and that point can be described on a coordinate system that includes all the independent variables as its axes.
The progression of climate through time is a vector in that complex multi-dimensional space.
That space is bounded in some way, which determines the volume within which climate can exist. For example, one boundary condition is the intensity of sunlight impinging Earth. Another is the blackbody temperature of that radiation. Other bounds include the rotational velocity of Earth, the size of Earth, the size and constitution of the atmosphere, and many other things, including the size of the oceans, their heat capacity, their salinity, and the sizes and profiles of the continents. All these things (and many others) combine to bound the phase space of our climate.
But with all the orthogonal (independent) axes available, the number of possible climate states becomes huge. We’ve seen some of them — ice ages, tropical ages, moderate ice-box ages (as we have now), etc. All of these have occurred without much apparent change in most of the climate boundary conditions. These are examples of Earth climate moving from region to region inside the space that bounds our climate. It’s apparent that these multiple regions are not very far apart within that complex space, in that Earth climate can transition among them fairly readily.
In fact the climate is chaotic in some sense, and can experience jumps without any particular change in the energy flux that drives it. We had a small jump like that in 1976, and the Younger Dryas event 12000 years ago may have been a large one. These amount to almost discontinuous transitions from one point in the climate phase space to another.
About clouds, I don’t see how compensating errors reduce error bars. If each opposing sort of error is acting simultaneously, they will reduce the apparent error in the outcome, i.e., in comparison with some known target state. But the uncertainty in the prediction remains a function of the inherent errors, and not in the difference between the result and the target.
Suppose you had an unrifled musket, for example, in which accidentally compensating errors in the manufacture of the barrel happened to produce musket-ball trajectories that were pretty straight over 50 yards, instead of flying off to hell-and-gone the way musket balls usually did. The existence of that one, modestly accurate, musket would not decrease the error inherent in musketry. Nor would a manufacturing accident mean that particular musket had an accuracy-producing design.
Next, GCMs have parameterization schemes that can be tuned to give reasonable results relative to a target climate, say, average Earth between 1980-1990. Usually, if these schemes are adjusted to reproduce the global temperature field, they get the global precipitation wrong, and vice versa. Typically the way it is done is that the parameters are adjusted to match some target climate, and then that particular scheme is used to try and retrodict other known climates. These might be called the normalization period and the verification period, respectively. If it passes those tests, then maybe the GCM can be used to project future climates. But since future climates will have different boundary conditions, e.g., higher CO2, then the parameterization schemes may not be applicable. This problem is discussed in Chapter 8 and Chapter 8 Supplemental of the IPCC 4th Assessment Report (the latest report).
Individual GCMs are individually tuned and they all include similar physics. In a test against a common target climate, multiple GCMs would all be tuned similarly. So, it’s not surprising that they’d all give similar results.
Demetris Koutsoyiannis has recently tested some climate predictions made by GCMs and published in the 2001 IPCC report (the TAR). He found that they had virtually no correspondence to reality. You can access his conference presentation abstract here: http://www.itia.ntua.gr/en/docinfo/850 with the “Presentation” link at the bottom of the page. This work is now submitted for publication.
Temperature is a scalar field that results from energy resident in the (gases of the) atmosphere. In order to test the accuracy of the projected temperature, one must know about the energy that produced it. The linear model that reproduced the trends of GCMs in Figure 2 in turn allows assessment of the forcings that produced the SRES temperature projections in Figure 1. Having that, one can proceed to assess forcing errors, and then compare the errors with the excess greenhouse gas forcing employed by the GCMs. The cloud retrodictions included in the CMIP tests provided the necessary cloud forcing data. Once the ratio of error to forcing is known, each in terms of W m^-2, the uncertainty in the calculated temperature is known. In this case, the uncertainty is restricted to a kind of minimal cloud error.
I make no claims that the cloud error I calculated is canonical, or the best one, or even complete. I only claim it’s a valid minimal estimate, even if partial. There are other errors in forcing that may also play into a global temperature calculation. These include, for example, the error in outgoing long wave radiation, and in reflected short wave radiation, at the top of the atmosphere, as calculated by GCMs, relative to reality. If these are wrong, then the net radiant energy in the atmosphere will be wrong and the calculated temperature will be wrong.
Regards
Pat Frank,
My physics intuition teaches me to think in terms of a potential well, rather than a flat phase space with boundaries. It is virtually guaranteed that there is a positive or negative feedback loop around every equilibrium point. Of course, that doesn’t necessarily mean the feedback loops have strong influence. It seems to me it is vital to your case to show that the feedback loops do not have a strong influence at +/- 100 degrees C. On the surface, this seems unbelievable to me, and I don’t recall you mentioning it in the article.
As to “compensating errors”, I meant it like this: Let’s say we flip N coins and take the number of heads minus the number of tails. The uncertainty of the result goes as sqrt(N), not N. Similarly, if you have a bunch of cloud errors, the fact some clouds cancel others out reduces the error. I think it is unlikely that they so perfectly cancel like my example with coins, but it can possibility you haven’t accounted for. Among other things, you need to show not just that the cloud error correlates over time and different GCMs but that the type of cloud error correlates as well.
I did not realize that GCMs had a “normalization” period. I feel like this is the first answer you’ve given as to why all the GCMs are consistent despite the fact that your model apparently predicts that they should all give different (and probably unreasonable) results. But it still seems like a remote possibility that this normalization process would so systematically create consistent but false predictions. Also, have you considered the possibility that most “theory bias” (from cloud cover or otherwise) has already been compensated for through slight adjustments of parameters? If the theory bias increases linearly with time (as your figure suggests) I think it’d be easy to correct for it.
I should explain more clearly my question about temperature retrodictions. It seems to me there are (at least) two ways to estimate the error of temperature predictions. One is to look at temperature retrodictions, while the other is to examine the uncertainty of the inputs (as you did). These two methods should give similar uncertainty estimates, unless something has gone wrong. Your method of examining the inputs gives a very large error, one that I can’t imagine exists in the temperature retrodictions. While I can’t pinpoint the source of this inconsistency (I have been trying), the error estimates based on temperature retrodictions strike me as more reliable. If you examine the inputs, you have to propagate the error through the entire GCM, and there are more opportunity for flaws in your error analysis.
Miller, think of the phase space as a kind of multi-dimensional potential energy landscape, with all sorts of hills and valleys, and peculiarly folded so that points that are well-removed along one coordinate can be proximate along another.
I never described the phase-space as “flat.” If you look at the post immediately above your last one, you’ll see I used the term, “multi-dimensional phase space” with many orthogonal axes. Such a space cannot be flat (2D).
Climate is not in equilibrium. It’s a quasi-stable state far from equilibrium, rather of a sort described by the non-equilibrium thermodynamics of Ilya Prigogine.
The (+/-)100 C uncertainty has nothing to do with climate as such. It is not a prediction that some future climate may be 100 C hotter or cooler than our own, and so worrying about feedback loops is beside the point. It has to do with the progressive, stepwise calculation (not climate) and the resulting accumulation of uncertainty in cloud forcing energy. That is, the accumulating uncertainty is not acting on climate but clouding one’s analytical view, showing that the mean climate that is calculated is really only one of many possible climates. The uncertainty becomes so large that no information remains about the distribution of clouds, about the magnitude and type of cloud forcing, and thus about the status of the projected future climate.
Your coin-toss example of uncertainty implies random error. Cloud error is not random. There’s no reason to think it will scale as sqrt(N). Compensating errors does not mean the computed system will correctly evolve dynamically, even if it evolves reasonably. I.e., there are many ‘reasonable-seeming’ predictive outcomes but only one right one.
In one of his papers, Carl Wunsch says that ocean models typically don’t converge to a unique solution. But when he asks modelers the meaning of a non-converged result, they brush him off because the results ‘look reasonable.’ This behavior is far from reassuring.
Your comment about adjusting out biases using parameters is prescient. Climate modelers must use a “hyperviscous” atmosphere in order to suppress unresolved turbulence. This non-physical aspect of GCMs is partly compensated by adjustments of the parameter schemes in order to produce reasonable results. But the parameter adjustments are rather ad hoc so as to get the desired normalization climate, and this means the parameters become unphysical, too.
What does it mean with respect to physical meaning to have a prediction from an unphysical model?
With respect to temperature predictions, you’re contrasting an estimate of inherent uncertainty with the actual error between observation and prediction. These need not necessarily be the same order of magnitude if the model is adjusted to give reasonable results, or if the ensemble of predicted temperatures is pruned of the ‘unreasonable’ outcomes.
That last is what the ClimatePrediction folks did in a Nature paper (v.433, p.403, 2005), predicting up to +11 C air temperature increase by 2100. The fact that they tendentiously pruned their results didn’t stop publication in Nature (as it should have done), and hasn’t stopped other intelligent people from crediting their “prediction,” anyway, though. I find this behavior far from reassuring, too.
Take a look here: http://tinyurl.com/2jxqgs at what Hendrik Tennekes wrote about modern climate modeling. Tennekes is a well-respected climate physicist.
Let me clarify my philosophical position a bit. I think climate physics is a magnificent endeavor. The climate physics I’ve seen in published work is beautiful and impressive. Working to model climate is extremely worthwhile, and I support the whole program unreservedly.
But in my opinion, climate physics has been hijacked by politics, and the proper dispassion of many climate scientists has been badly compromised. I don’t think this is good for physics, nor is it good for science. Science lives and dies on trust. When the science is consciously bent by scientists to serve even a sincere political end, the sine qua non of science is lost. I think this is being done.
GCMs are academic research tools that are properly the objects of study — how do they behave under varying conditions, how does one improve them in order to model climate, what do they reveal about our understanding of climate physics. But today they are being used as engineering models, as though they are fit to make physically reliable predictions. They are unable to do that, which is why the debate about them, having been infected by politics, has become so acrimonious. The science is lacking and so the debate has become ad hominem. I’m very worried that science is going to take a big hit because of this. These scientists have sacrificed their analytical trust on an altar of political feeling.
Really? If so, your article is very misleading. When I see error bars of 100 C, I usually assume that means that 68% of the time, the results are within 100 C, and 32% of the time, they’re not within 100 C. If that’s not what you meant, how can we say, “The error is much greater than the predicted temperature change, therefore we can draw no conclusions”? We can’t even compare the error with the predicted temperature change (not straightforwardly, anyways), since they’re two different things.
As it is, I focused this whole time on the ridiculously large error bars, because they intuitively don’t make sense. First, I thought you were practicing innumeracy, and then, you were ignoring important details such as negative feedback loops. If you had just propagated the errors normally, and found an actual error of say, 10 C, I would have had no objections aside from a few niggling doubts about things that I can’t quite place. And for the record, the rest of my objections are merely niggling doubts–I am not capable of serious criticism/validation of your work.
So, back to those other doubts…
I actually don’t have too many objections left. I’d like to thank you for the information about GCMs. It’s quite interesting to me, especially since I knew nearly nothing about them before I wrote this article.
I don’t think the citation of a few cases of pruning is enough to render all GCMs invalid. Obviously, pruning can bias the results, but there are plenty of circumstances when it is a valid scientific practice. To determine whether that is the case here requires looking at the details, which (sigh) I can’t do without some expertise.
Miller, you wrote, “Really? If so, your article is very misleading.
Under Figure 3, you’ll find this: “In terms of the actual behavior of Earth climate, this uncertainty does not mean the GCMs are predicting that the climate may possibly be 100 degrees warmer or cooler by 2100. It means that the limits of resolution of the GCMs—their pixel size—is huge compared to what they are trying to project.”
How is the article misleading?
You wrote, “When I see error bars of 100 C, I usually assume that means that 68% of the time, the results are within 100 C, and 32% of the time, they’re not within 100 C.”
But what I wrote about in the article concerned growing uncertainty from an intermediate result that is re-input in every step of a multi-step calculation, not about GCM outputs as such. The GCMs are made to propagate their mean result, not the limits of their uncertainties. In fact, those who publish GCM results virtually never show a true physical uncertainty. They only show the numerical variation in the mean projection from each of multiple runs. Those statistical error bars have no direct physical meaning. The IPCC reports are rife with such pseudo uncertainty limits.
Concerning your intuitions, most of science doesn’t make intuitive sense. It’s not intuitively sensible that Earth is a rotating spheroid travelling around the sun, instead of flat and stationary. It’s not intuitively sensible that we large complex beings evolved from microscopic bacteria, or that life self-organized from chemicals and energy. After repeated expsure, it’s easy to take all of that, and more, for granted and no longer contemplate the wild non-intuitiveness of most of the science we know.
If the large uncertainty bars are ridiculous — and in some sense they are — then maybe what should bother you is how anyone can claim that GCMs produce reliable climate forecasts, or that an anthropo-CO2 signature on climate has been detected.
Finally, thank-you for your interest. You should know that yours have been among the more substantive comments I’ve received since publication of the article.
Ah, but when I speak of intuition, I don’t mean some inborn knowledge, but a learned intuition after getting marked down in my lab reports for overestimating errors.
To take your own example, let’s say a bunch of models show that 2+2=5±0.1, where the error bar indicates the variance between models. Obviously, there is some systematic error on the order of 1. But then you say the systematic error is 10? If so, how did our answer even get close? You say that this is because this systematic error is only in an intermediate result of the model, not the output as such. Well, then what’s the uncertainty of the output as such? It had better be on the order of 1.
As for your above quote, of course your analysis doesn’t predict the climate will go ±100 C in 100 years. What it predicts is that the GCMs will go ±100 C in 100 years. Even if the different GCMs had perfectly correlated errors, we would expect to see the 100 C error by comparing the mean GCM results to the “reasonable” range of 0 to 10 C. That is, unless the 100 C error is not the kind of error I originally thought it was. By those error bars, did you really mean that GCMs had a 32% chance of giving results that were more than 100 C away from the “reasonable” range?
The article is misleading, because you use your calculated error as if it were straightforward systematic error in the output. How can you produce numbers like 0.44±15° C unless the numbers 0.44 and 15 are both talking about the same thing: the final result of the GCMs? So I assumed that’s what it was. But in order to calculate the error in the final result, you would have had to worry about things like feedback loops, uncorrelated/compensating errors, and some other things I brought up.
Miller, I’m not sure what you mean in your paragraph 2, but it certainly doesn’t represent anything I wrote or intended to convey in the article.
You wrote, “What it predicts is that the GCMs will go ±100 C in 100 years.”
No, it doesn’t. It estimates uncertainty in a prediction, not the prediction. We’ve been over this a few times, now, and reiterating your view using different words won’t make it right.
The cloud error is an error in energy, not in temperature. But the error in energy reflects the systematic uncertainty in cloudiness as it is calculated using GCMs. The growing uncertainty limits means that there is less and less information about cloudiness at greater and greater distances from the baseline climate (To). One ends up with a plausible-seeming future climate, but one that has no real information content.
Look at Collins’ 2002 paper (ref. 28) for an example of this sort of outcome. Collins discusses the initial value problem, rather than theory-bias, but the outcome is analogous. The projected climate was bounded but had zero fidelity with respect to the target climate. And that, using a perfect climate model.
To get an accurate output error (as opposed to a projection uncertainty), which is what you’re asking to see, someone would have to calculate an actual climate prediction factoring in the effect of the systematic cloudiness error during every time-step of the calculation. The compare the results with the target.
I’d like to see that, too. Good luck getting anyone to do it.
The cloud error estimated in my article, in terms of energy, is about 1.5 times the total forcing of all the excess CO2 plus water vapor enhancement, accumulated since year 1900. As this error increases linearly (as theory-bias), while forcing increases only with the log of CO2 concentration, the uncertainly limits expand faster than the projected temperature. You may find that intuitively incredible, but really incredulity is not a valid criticism.
Frank,
I’d actually like to wind this up now, as I have finals, and I can’t go on arguing all through summer. I feel I’ve advanced my argument as far as is worthwhile for now, and hopefully you feel the same way. We didn’t quite come to an agreement, but that’s to be expected.
So it was a pleasure to talk. Thanks for your time, explanations, clarifications, and corrections. And thanks for being polite throughout, despite my arguably less-polite tone in the original post. I tip my hat to you.
Thanks, Miller, and thanks for your continued interest. You have been far more polite than others on other blogs, and I’ve appreciated that. Good luck with your finals.
Hmm, so somehow P. Frank’s error doesn’t predict what he thinks the GCMs will output. He admits their predictions will be in a much smaller, reasonable range, he just doesn’t trust them. If the GCM predictions don’t walk off by 100C in 100 years, then P. Frank’s prediction of error is a pretty useless definition of error. I suspect his modeling of the error is grossly inadequate, precisely because he has tried to make a simple “model” that is not physically realistic.
No Gator you haven’t actually understood what he has said.
To simplify it for you. The further you go out the less likely you are to make a correct prediction. That’s just a characteristic of these types of models. They are about as accurate out 200 years as me saying it’s going to be colde in the winter and hot in the summer.
They are not truly models but are instead simulations and poor ones at that.
The idea of taking a bunch of faulty computer simulations, tuning them to presumed levels of forcing, averaging their results, then declaring that more accurate than each model is hilarious. Not only hilarious but circular.
It would be, and is more honest to just plot the the straigh line assumption already built in to your model. Of course if you have a bunch of models that each generate random fluctuations but have an underlying baseline trend that was assumed at the beginning when you average them it will show the trend.
Why bother doing that? If I write a program that generates the plot y = mx + b – ((rand(time) % 100) + 50) it will generate a squiggly line each time I run it. Yet if I take thousands of runs and generate the average I will get the line y = mx + b. That should be obvious.
It is also obvious to any computer programmer worth his salt that you can take that random value that is subtracted and store it in a variable to be added back later. Thus simulating heat being sequestered in the oceans being later released. Again, do it right and your underlying assumption of y = mx + b, will be hidden by random fluctuations.
It will look “weathery”, and you can even tune the amount you subtract and lag off so that it fits whatever slope you like.
Thus you can have an underlying assumption of y = mx + b but can generate averaged slopes of n, as in y = nx + b, where m >> n. For any m and n you choose.
You can then claim that the temperture change we are experiencing n is much much less than what the actual forcing is. This allows one to scream the sky is falling, plus claim that the slope n, was “predicted” by your model. So your model has been “tested by empirical evidence”.
What a parlor trick.
Now take that parlor trick and use some bad statistics on it to claim that it’s “statistically significant” and you’ve got a full fledged magic show.
Problem is when they catch your mathematical mistakes. Then you need a coverup.
I have just some very simple questions:
What will be the Global Avaerage Temperture for the year 2020 predicted by the GCM’s?
What’s the error on that?
What was the Global Average Temperture predicted for 2008 by GCM’s and what was the Error that was given in the year 1996?
I just want to compare the values and see by how much they are off.
I am just a high school student from austria participating in a physic contest, where we alo have to estímate errors and usually do this in the manner descibed above. I know this doesn’t account for “the models getting better” but rather gives you a measure of credability. I would expect that the models published in 1996 should at least be somewhat accurate when compared to the temperture observable today. It would be also interesting to know what the models from 1996 predict for the year 2020.
I do have a basic knowledge of statistics and think, that I could follow your arguments at least partially. As I am also a Skeptic I can’t relate to statements from the IPCC like “very accurately” or “showing good skill in predicting” something. However you also have to be a Skeptic regarding Skepicism and 100° does look a bit too much to me. I say this, knowing that “looks to much” is also something I would be skeptic about.
Therfore I think, that the values i requested would help me make my judgment on that topic much easier.
Thank you in advance for your reply,
Angel Usunov