Climate Audit

by Steve McIntyre

Cowtan and Way 2013

There has been some discussion of Cowtan and Way 2013 take on HadCRUT4 at Lucia’s, Judy Curry’s, Nick Stokes and elsewhere. HadCRUt4 has run cooler than other datasets (including UAH satellite) in recent years. Cowtan and Way observe that HadCRU does not estimate temperature in many Arctic gridcells. Because Arctic temperatures have risen more than low-latitude temperatures, they state that recent HadCRU temperatures are biased low. (Since GISS extrapolates into the Arctic, it is less affected by this bias.)
In the context of IPCC SOD FIgure 1.5 (or similar comparison of models and observations), CW13 is slightly warmer than HadCRUT4 but the difference is small relative to the discrepancy between models and observations; the CW13 variation is also outside the Figure 1.5 envelope.
cowtanway2013 vs ipcc ar5sod figure 1_5
Figure 1. Cowtan and Way 2013 hybrid plotted onto IPCC AR5SOD Figure 1.5
Next, here is a simple plot showing the difference between the CW13 hybrid and HadCRUT 4. Up to the end of 2005, there was a zero trend between the two; the difference has arisen entirely since 2005.
cowtanway2013_difference between hadcru4 and cw2013
Figure 2. Delta between CW Hybrid (basis 1961-1990) and HadCRUT4.
In their online commentary, Cowtan and Way praise Hansen for being the first person to report the effect of missing Arctic data on global temperature. However, no material discrepancy had arisen between their index and the HadCRUT4 index as of 2005 so that Hansen was, according to Cowtan and Way’s own data, observing a discrepancy that had not yet arisen, making their following praise to Hansen seem somewhat premature:
Probably the first mention of an underestimation of recent warming due to poor Arctic coverage comes from Hansen in 2006, who sought to explain why the NASA temperature data showed 2005 as being a record breaking warm year, in contrast to the Met Office temperature record.
That there are continuing defects in HadCRU methodology should hardly come as a surprise to CA readers. Attempts to reconcile and/or explain discrepancies between HadCRU and GISS also seem worthwhile to me.
Nor do efforts to apply kriging seem misplaced to me in principle. On the contrary, for someone with experience in ore reserves, it seems entirely natural e.g. see for example, some of Jeff Id’s discussion of Antarctica. I notice that their methodology results in changes to the Central England gridcell. While I don’t object to the use of kriging or similar methods to estimate values in missing gridcells, I don’t see any benefit to altering values in known gridcells, if that’s what’s happening here. (I haven’t parsed their methods and don’t plan to do so at this time.)
Co-author Way was an active participant at the secret SKS forum, where he actively fomented conspiracy theory allegations. Uniquely among participants in the secret SKS forum, he conceded that Climate Audit was frequently correct in its observations (“The fact of the matter is that a lot of the points he [McIntyre] brings up are valid”) and urged care in contradicting Climate Audit (“I wouldn’t want to go up against that group, between them there is a lot of statistical power to manipulate and make the data say what it needs to say.”)
Update: Here is annotation of IPCC AR5 SOD Figure 11.12 comparing observations to CMIP5 4.5, with both HadCRU4 outside envelope.
cw13 versus sod figure 11_12 cmip5
Bart Verheggen compared CMIP5 RCP8.5 to observations, saying that “recent observations are at the low side of the CMIP5 model range”.
However, my own calculations using RCP8.5 show that observations are outside the envelope. Verheggen’s calculations are not consistent with similar calculations by others (including IPCC) and I presume that he’s made an error somewhere.
cw2013 cmip5 rcp85 _baseline 1980_1999

Like this:

Like Loading...
This entry was written by Steve McIntyre, posted on Nov 18, 2013 at 12:56 PM, filed under Uncategorized and tagged cotwan, sks, way. Bookmark the permalink. Follow any comments here with the RSS feed for this post. Post a comment or leave a trackback: Trackback URL.

174 Comments

  1. Posted Nov 18, 2013 at 1:17 PM | Permalink | Reply
    Hi, Steve. Interesting remarks from Way: “Between them [Climate Audit denizens] there is a lot of statistical power to manipulate and make the data say what it needs to say.” I’m not familiar with Robert Way, but this quotation potentially exposes a biased view of statistics from this crowd: that statistics is good for manipulating results. Perhaps this is why certain methods are favored (and misused) so often in climate science?
    Steve: you can’t extrapolate from SKS to climate science. Another comment from an SKS participant at the secret SKS forum: “Real data and statistics are a subtle subject. I try to stay away from both, as far as possible.”
    • Posted Nov 18, 2013 at 3:02 PM | Permalink | Reply
      Yes, apologies … I meant “SKS” when I said “this crowd,” and should have referred to ‘climate science’ (TM) only to the extent that the like-minded participate.
      • Posted Nov 19, 2013 at 10:46 AM | Permalink | Reply
        Good answer. Interesting the host giving both you and Robert Way a hard time, here and three comments down, despite his admirable attitude to younger researchers in general:
        Over the years, I’ve tried to stay away from criticizing young scientists. Lewandowsky’s use of this stuff is worse than Wood’s error.
        I think Steve’s right on all three. It’s a pity Way hasn’t apologised for “conspiracy wackjob” – the kind of background noise that has contaminated climate debate for too long. Not all climate debate, I hasten to add, but too much.
  2. Lance Wallace
    Posted Nov 18, 2013 at 1:18 PM | Permalink | Reply
    headline misspells Cowtan
  3. David S
    Posted Nov 18, 2013 at 1:51 PM | Permalink | Reply
    Funny that someone who is criticising HADCRUT is happy to take a ride in Phil Jones’s time machine, the one that enables him to be upset by FOIA applications before they come in.
  4. Robert Way
    Posted Nov 18, 2013 at 1:59 PM | Permalink | Reply
    “I notice that their methodology results in changes to the Central England gridcell. While I don’t object to the use of kriging or similar methods to estimate values in missing gridcells, I don’t see any benefit to altering values in known gridcells, if that’s what’s happening here.”
    I am not sure where this assumption is based from but no we do not change values where there are observations in HadCRUTv4. The hybrid and kriging approaches are only used to estimate for missing values in the final reconstruction. We do however provide rigorous cross-validation where we take out blocks of cells and reconstruct them. Perhaps this is where the confusion arose.
    More information on the paper can be found on the website:
    http://www-users.york.ac.uk/~kdc3/papers/coverage2013/background.html
    As for the personal remarks I think it is unfair to characterize me as a person who “actively fomented conspiracy theory allegations.” I’m sure there will be instances over the course of the few years I participated at the forum where I expressed things that you (Mc) might describe that way but to characterize the breadth of my discussion there in such a manner is misleading.
    Secondly, to be taking what I said when I was 20, 21, 22 years old in a private forum and then publicizing it with that sort of comment attached really seems like an attempt to embarrass a young researcher trying to formulate opinions on things. I wonder what the world would be like if most young people in their undergraduate college years had to worry about their private discussions being hacked and then published on the internet to try and embarrass them. I have been, and will continue to be a constant defender of good practice in science and in the course of my experiences I have vigorously opposed both in fairly equal measure both denialism and alarmism.
    Steve: I will respond separately showing my issue with respect to the central England gridcell. If you don’t adjust actual data, then reconciliation of the differences should be fairly easy. It might relate to taking the anomaly. As to my remarks on your comments in the SKS forum: over the years, I’ve gotten tired of people privately conceding the validity of my criticisms of paleoclimate practices, but failing to do so publicly. In your case, your SKS forum comments show that you agreed with many of my criticisms, but, instead of saying so at SKS, you called me a “conspiracy wackjob” – an offensive and untrue allegation. instead of apologizing when I took issue in my above remarks – as you ought to have done – you complained that your remarks had become public. I understand that you were young at the time and I would be quite happy to accept your withdrawal of these offensive and untrue remarks and move on. But first you have to withdraw the allegations, rather than complaining about how they became public.
    • Nicholas
      Posted Nov 18, 2013 at 9:00 PM | Permalink | Reply
      Hi Robert, thanks for weighing in. It’s always nice to get feedback from one of the authors of the paper under discussion.
      Your comment that “I have been, and will continue to be a constant defender of good practice in science” is a good sign and I suggest you will probably get along quite well with Mr. McIntyre in the long run.
    • Nathan Kurz
      Posted Nov 18, 2013 at 9:24 PM | Permalink | Reply
      Hi Robert — Just wanted to say thanks for coming to discuss your paper, and I applaud your gracious response to Steve’s characterization. As an onlooker, I encourage you overlook the rough introduction and stick around to discuss the science.
    • kim
      Posted Nov 19, 2013 at 1:00 AM | Permalink | Reply
      So, may we presume that the ‘conspiracy wackjobs’ at sks are still young and foolish?
      ==========
      • thisisnotgoodtogo
        Posted Nov 19, 2013 at 8:40 AM | Permalink | Reply
        Kim, Mr Robert Way’s point in this is that he’s older and wiser now.
        That’s why he’s publically withdrawing his youthful excesses that were stolen and not highly redacted.
        At least that’s the way I read it. :)
      • michael hart
        Posted Nov 19, 2013 at 10:04 AM | Permalink | Reply
        The Arctic is haunted by the ghosts of Essex, McKitrick and Andresen (2007).
        The fourth owns a horse called albedo.
        • kim
          Posted Nov 19, 2013 at 10:13 AM | Permalink
          There’s an alternative abstract for this study: They crossed a white horse with a desert of no name.
          =============
        • michael hart
          Posted Nov 21, 2013 at 12:48 AM | Permalink
          Well, I guess if you can krig moving ice as it is being blown through the Fram Strait, then that should then be fairly trivial.
  5. Hansen?
    Posted Nov 18, 2013 at 3:18 PM | Permalink | Reply
    Actually there wasn’t the Hadcrut4 at the time hansen wrote that paper…
  6. Green Sand
    Posted Nov 18, 2013 at 3:57 PM | Permalink | Reply
    The shape of Figure 2 looks rather familiar
  7. Matt Skaggs
    Posted Nov 18, 2013 at 4:03 PM | Permalink | Reply
    Paywalled. The online commentary indicates that Thorne et al 2005 (Revisiting radiosonde upper air temperatures from 1958 to 2002) contains “supporting evidence not used in this work.” Since that one is not paywalled I figured I would have a look. It is a long paper mostly about data adjustments, so I did not read every word, but I was unable to find “supporting evidence.” First of all, the coverage of the polar regions was extremely low, the same problem supposedly being resolved by C&W13. There is little mention of high latitudes, appropriately enough, but Thorne05 has this to say about the Northern Hemisphere:
    “Zonal mean trends over the full period 1958–2002 exhibit warming throughout the troposphere, and cooling in the stratosphere (Figure 11). There is a relative minimum in the warming in the Northern Hemisphere around 60N.”
    Since they only have a few readings north of 60N (reaching to perhaps 70N), and I cannot find much specifically about those, I am having trouble seeing the support here. Their statement about the Southern Hemisphere data spread at high latitudes seems to apply equally well to their arctic data:
    “Given the sparsity of the network here, zonal mean trends are likely to be unreliable and may reflect the inadequacies of a neighbor based homogenization approach in datasparse high-latitude regions where correlation distances are small. We caution against an overinterpretation of these data.”
    Looks like the caution was not heeded.
  8. phi
    Posted Nov 18, 2013 at 4:08 PM | Permalink | Reply
    If we are looking for coolants bias, we will still find many others. But this is ridiculous as we continues to ignore the most important which is a warming bias of one or two orders of magnitude higher.
    This cuisine has nothing to do with science.
  9. Ronan
    Posted Nov 18, 2013 at 5:14 PM | Permalink | Reply
    Some readers might find this January 2010 post on Tilo Reber’s blog interesting and relevant.
    Note, though, that in that post, Tilo Reber was comparing HadCRUT3 to the 2010 GISTEMP version (GISTEMP= NASA GISS’s temperature estimate). As “Hansen?” alludes to above, HadCRUT4 was only introduced in March 2012.
    Also, NASA GISS switched from using the unadjusted version of GHCN 2 as their main dataset to the homogenized version of GHCN 3 in December 2011. The homogenized version of GHCN 3 shows more warming for the last few decades than the the unadjusted version. So, these post-2011 changes somewhat reduced “the pause” in GISTEMP.
    In other words, Tilo Reber’s comparisons of the two datasets in 2010 are not directly equivalent to Cowtan & Way’s 2013 comparison. Nonetheless, his criticisms might still be valid…
  10. Posted Nov 18, 2013 at 5:49 PM | Permalink | Reply
    Darn. Steve, beat to the funky breakpoint at 2005 in the difference between C&W2013 and HADCRUT4. My post should be done tomorrow, discussing why that’s odd–among other things.
  11. Posted Nov 18, 2013 at 5:56 PM | Permalink | Reply
    Note that surface temperatures were above the projection in 1998, because of the El Nino (positive ENSO index event) that year.
    http://www.esrl.noaa.gov/psd/enso/mei/
    Notice how the ENSO index has been much smaller lately, with a lot of negative La Ninas which store heat deep in the Pacific ocean where ARGO and the central limit theorem have measured it. It’s worth pondering what will happen when we have another El Nino like the one in 1998. All that heat hasn’t vanished, and we’ll have to deal with it sooner or later. Luckily for WUWT and Climate Audit, that future temperature spike (whatever its human cost) could serve as the basis for articles like “Global warming stopped in 201X!”
    Apparently, going down the up escalator never gets old:
    http://skepticalscience.com/still-going-down-the-up-escalator.html
    Steve: Can you provide a link to any Climate Audit post proposing or endorsing an escalator-type argument? I’ve discussed inconsistency between models and observations, but that’s a very different analysis.
    • ironargonaut
      Posted Nov 18, 2013 at 10:51 PM | Permalink | Reply
      I’ll bite on your flamebait.
      snip
      Steve: blog editorial policy discourages attempts to prove or disprove CAGW in single blogposts on other topics, on the grounds that all threads then become the same.
    • Posted Nov 19, 2013 at 1:04 AM | Permalink | Reply
      Steve: Can you provide a link to any Climate Audit post proposing or endorsing an escalator-type argument? I’ve discussed inconsistency between models and observations, but that’s a very different analysis.
      ====================
      I first saw this article on WUWT and mistakenly included you in that sentence. I’m sorry for lumping you together with WUWT like that.
      Steve: you seem to be somewhat antagonistic – can you identify any CA posts which you believe to be incorrect or to contain errors?
      • Posted Nov 19, 2013 at 2:05 AM | Permalink | Reply
        Steve: you seem to be somewhat antagonistic – can you identify any CA posts which you believe to be incorrect or to contain errors?
        ===================
        I lost interest around the 11th “etc.” here:
        http://dumbscientist.com/archives/climate-destabilization#comment-16500
        Steve: You rely entirely on disinformation from opponents.
        our criticisms of MBH98 were valid. In our discussion (see MM05 EE), we observed that you got different results depending on the number of PCs retained – a point misrepresented by opponents. The shape was determined by whether bristlecones were retained. The issue was then whether the MBH reconstruction was robust to bristlecones and whether bristlecones were a magic thermometer. I do not believe that they are and even if they are, this needs to be shown. Further, Mann withheld adverse verification statistics for his reconstruction.
        The Yamal incident has been widely discussed. I did not randomly pick a series from the internet as Gavin Schmidt falsely claimed. I used a Schweingruber series from the Yamal peninsula. My surmise that East Anglia had calculated (and withheld) a regional chronology with a high medieval period later turned out to be true.
        • Carrick
          Posted Nov 19, 2013 at 3:37 AM | Permalink
          Your moniker is well chosen.
        • Posted Nov 19, 2013 at 2:14 PM | Permalink
          Steve: You rely entirely on disinformation from opponents. our criticisms of MBH98 were valid.
          =======================================
          Presumably your opponents include the National Academy of Sciences:
          “The basic conclusion of Mann et al. (1998, 1999) was that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years. This conclusion has subsequently been supported by an array of evidence that includes both additional large-scale surface temperature reconstructions and pronounced changes in a variety of local proxy indicators, such as melting on ice caps and the retreat of glaciers around the world, which in many cases appear to be unprecedented during at least the last 2,000 years. …”
          http://books.nap.edu/openbook.php?record_id=11676&page=3
          If your criticisms were valid, subsequent independent reconstructions would have been significantly different, and the NAS wouldn’t have said this conclusion “has subsequently been supported by an array of evidence”.

          Steve: our criticisms were directed against misrepresentations and errors in MBH98. We did not offer an alternative view of the world or preclude the possibility that someone else might establish a similar conclusion. The NAS panel did not disagree with a single one of our criticisms. They did not support Mann’s bristlecones, but said that they should be avoided in temperature reconstructions. Most of the other reconstructions have their own problems: many use bristlecones despite the NAS recommendation. Some use contaminated data and upside down data. it is entirely possible that the Modern Warm Period is warmer than the Medieval Warm Period – there’s evidence both ways – I’ve never taken a position on it. Our position with respect to Mann et al 1998 was that they hadn’t established what they claimed to have established. This point remains unchanged even if someone subsequently accomplishes what Mann failed to do. In addition, Mann ought to have to disclosed adverse verification statistics.
        • Posted Nov 19, 2013 at 2:34 PM | Permalink
          Steve: our criticisms were directed against misrepresentations and errors in MBH98. … The NAS panel did not disagree with a single one of our criticisms.
          ======================================
          If the NAS panel didn’t disagree with your criticisms, we can all agree that the basic conclusion of Mann et al. (1998, 1999) that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years has subsequently been supported by an array of evidence.
          Steve: I’ve looked closely at the evidence and the “array of evidence” is not as convincing as one would think. Many of the studies use the same old data e.g. bristlecones and therefore do not provide independent support. I think that it’s really too bad that specialists have failed to ensure that subsequent studies used independent data, given the criticisms. What is the “array of evidence” that you have in mind? Again, I do not take the position that good arguments are impossible.
        • Posted Nov 19, 2013 at 3:35 PM | Permalink
          Steve: I’ve looked closely at the evidence and the “array of evidence” is not as convincing as one would think.
          =====================================
          Since that NAS report quote seems to disagree with that criticism, it seems like the NAS report does disagree with your criticisms.
        • Nicholas
          Posted Nov 19, 2013 at 4:02 PM | Permalink
          “If the NAS panel didn’t disagree with your criticisms, we can all agree that the basic conclusion of Mann et al. (1998, 1999) that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years has subsequently been supported by an array of evidence.”
          In other words: they got the right results using the wrong methods.
          Sorry, that isn’t how science works.
        • Howard
          Posted Nov 19, 2013 at 4:07 PM | Permalink
          He should be called OCD Scientist… check out his website and the voluminous comments he posts to his posts. Obviously he is very bright and passionate but does not go out into the real world, therefore, he prefers the monologue style of conversation.
        • Bruce Cunningham
          Posted Nov 19, 2013 at 4:09 PM | Permalink
          As Carrick said, your handle is well chosen. You and the NAS are incorrect that the results of MBH98,&99 have been supported by an array of evidence. As Steve pointed out to you, one of the main points of MBH99 was that there was no Medieval warm period. That there was a Medieval warm period that was probably as warm or warmer than today has been validated by a large array of studies (see McShane & Wyner). That one fact makes MBH99 utter rubbish (as Von Storch called MBH99). The fact that using red noise for input data in the MBH procedure, gives the same hockey stick output proves that it is rubbish. The hockey stick had become an Icon of the warming movement, and was used as the primary study to prove runaway warming for several years. Steve and Ross first published there critique in 2003, 2 years after the third assessment report was published. That it was so easily shown to be incorrect is a black eye for climate science and its peer review process by allowing such rubbish to be published and used to completely change the course of modern society. That you continue tout its validity shows just how unscientific you and anyone that agrees with you are.
        • kim
          Posted Nov 19, 2013 at 4:56 PM | Permalink
          Heh, DS must have missed Gavin Schmidt giving up MHB ’98, ’99 prior to 1500 AD over on a memorable thread at Keith’s one fine day.
          ============
        • John M
          Posted Nov 19, 2013 at 6:49 PM | Permalink
          For those wondering what DS is hot-and-bothered about…
        • Robert Way
          Posted Nov 20, 2013 at 1:51 AM | Permalink
          “What is the “array of evidence” that you have in mind? Again, I do not take the position that good arguments are impossible.”
          I think the melt rates observed on many ice caps (in particular in the Canadian Arctic) provide fairly strong evidence that the warmth in that portion of the Arctic was much lower during the MWP – as does some of the work by Miller. The paper by Shi et al (2013) is fairly convincing as to at least the last 1,000 years in the Northern Hemisphere. I am actually surprised that paper has not been discussed here since it aims at dealing with many of the criticisms of paleoclimate research.
          They use 45 annual proxies which are all greater than 1,000 years in length and all have a “demonstrated” temperature relationship based on the initial authors interpretations. Furthermore they use three different reconstruction methods (CPS, EIV and PC10+AR2). They also test for the impact of dendro versus non-dendro and compare their results with climate model simulations.
          Shi et al (2013). Northern Hemisphere temperature reconstruction during the last millennium using multiple annual proxies. Climate Research, 56: 231-244.
          Steve: In the past e.g. here, I’ve commented favorably on Miller’s work and observed that was “a line of argument that, in my opinion, might well be used to argue that modern warmth has surpassed MWP warmth. Assembling such facts would be far more persuasive to me than multiproxy varvology with upside down data.” This seems to offer a better prospect of convincing evidence than a multiproxy reconstruction using Miller’s data upside-down, as PAGES2K Arctic did. The incorporation of lake sediment data into multiproxy composites seems fraught with problems to me. Having said that, I thought that there is much very sensible discussion of lake sediment in Baffin Island, which I read recently. Shi et al 2013 is a relatively recent study and I haven’t parsed it. Perhaps it is more convincing than its predecessors. I certainly don’t preclude the possibility that the Modern Warm Period is warmer than the Medieval Warm Period.
        • Posted Nov 20, 2013 at 2:28 AM | Permalink
          kim, Gavin Schmidt did not give up MBH prior to 1500 AD. To my knowledge, he hasn’t given up anything about MBH. What you’re referring to is Mann 2008.
          Of course, if the work Mann published a decade later, widely promoted as an improvement over the original, was that flawed, one can’t hold much hope for the earlier work.
        • Jeff Norman
          Posted Nov 20, 2013 at 11:05 AM | Permalink
          ” I think the melt rates observed on many ice caps (in particular in the Canadian Arctic) provide fairly strong evidence that the warmth in that portion of the Arctic was much lower during the MWP”
          This “observation” runs counter to the anthropological evidence of both the Viking culture in Greenland and the Thule culture across the Canadian archipelago. Both cultures grew and thrived as a result of apparent warmer climates and disappeared as a result of apparent colder climates.
          Disappearing ice revealing prehistoric mosses does not automatically lead to a conclusion that it is warmer now than at any time since the ice first formed.
      • Robert Way
        Posted Nov 20, 2013 at 2:05 AM | Permalink | Reply
        Per Shi et al (2013) mentioned above.
        http://img11.imageshack.us/img11/109/t5od.jpg
        • Craig Loehle
          Posted Nov 20, 2013 at 10:04 AM | Permalink
          Re: Shi et al. the “no dendro” graph shows a current peak just barely above that of 1000 yrs ago. Not quite “unprecedented”. Also, 1000 yrs ago is not a magic number. My own and several other studies put the peak of the MWP at 1100 years ago, which is cut off by stopping at 1000AD. Going back farther, the Roman warm period may have been even warmer, and 2000 yrs ago isn’t that long ago to consider it irrelevant.
        • Robert Way
          Posted Nov 20, 2013 at 10:27 PM | Permalink
          The no dendro graph is unsurprisingly characterized by greater variance because it has significantly fewer series. I would agree that the peak of the MWP is probably around the early 900s (AD) and the Roman Warm period may have been warmer than today in regions but then you get into orbital related cooling as well on such a long timescale as 2,000 years. I’m not really convinced about your study. I do not think it was appropriate to use century scale pollen data in the way you did and I believe that this, in addition to other issues, led to a paper that was far less robust than other papers often criticized in the contrarian community.
    • Duster
      Posted Nov 19, 2013 at 5:43 PM | Permalink | Reply
      “…negative La Ninas …”
      What is a negative La Nina? It sound like it ought to be an El Nino.
      • Posted Nov 19, 2013 at 7:01 PM | Permalink | Reply
        Sorry for the confusion; I just meant to briefly note that negative values in the ENSO index plot correspond to La Nina events.
        • seanbrady
          Posted Nov 20, 2013 at 4:05 PM | Permalink
          “If the NAS panel didn’t disagree with your criticisms, we can all agree that the basic conclusion of Mann et al. (1998, 1999) that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years has subsequently been supported by an array of evidence.”
          Can you perhaps re-state that sentence? As written it is a simple non-sequitur:
          If the NAS panel agreed with McIntyre, then therfore we can all agree that current warming is unprecedented??
        • Posted Nov 20, 2013 at 6:00 PM | Permalink
          Sean, if the NAS panel didn’t disagree with McIntyre’s criticisms then he, the NAS, and I can all agree that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years has subsequently been supported by an array of evidence.
          Hopefully this answers your question, because in his response to Robert Way’s comment on Nov 20, 2013 at 1:51 AM, McIntyre wisely suggests that we should carefully examine papers like Shi et al. 2013 and ponder their implications. So I intend to do that, whenever I finish with what Howard calls my “monologue style of conversation”:
        • Posted Nov 20, 2013 at 6:06 PM | Permalink
          Oops! I need to rephrase this:
          “that the late 20th century warmth in the Northern Hemisphere was unprecedented”
          to:
          “that the basic conclusion the late 20th century warmth in the Northern Hemisphere was unprecedented”
          Sorry for the confusion. Those WUWT “conversations” have me a little rattled…
        • Posted Nov 20, 2013 at 6:10 PM | Permalink
          Oops again! I need to rephrase this:
          “that the basic conclusion the late 20th century warmth in the Northern Hemisphere was unprecedented”
          to:
          “that the basic conclusion that the late 20th century warmth in the Northern Hemisphere was unprecedented”
          Again, sorry for the confusion and the spam, but I want to be as clear as possible.
        • Posted Nov 20, 2013 at 6:25 PM | Permalink
          Again, I’m very sorry for the spam. But I just realized my first comment at 6:00 PM (~10 minutes ago) is quite understandably still awaiting moderation (undoubtedly because of the links). Ordinarily I’d just wait, but in the meantime my next two responses add confusion rather than subtract, so here’s my first comment without the WUWT links. You have my blessing to delete this clutter if you decide to clear the original. Again, sorry for the mess…
          ======================
          Sean, if the NAS panel didn’t disagree with McIntyre’s criticisms then he, the NAS, and I can all agree that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years has subsequently been supported by an array of evidence.
          Hopefully this answers your question, because in his response to Robert Way’s comment on Nov 20, 2013 at 1:51 AM, McIntyre wisely suggests that we should carefully examine papers like Shi et al. 2013 and ponder their implications. So I intend to do that, whenever I finish with what Howard calls my “monologue style of conversation”:
          =======================
          What followed were links to WUWT articles and my first comment on each. Maybe without the initial WUWT the spam filter won’t see it? Let’s find out:
          2013/11/06/public-relations-spin-doctors-deliberately-deceived-public-about-global-warming-and-climate-change/#comment-1467978
          2013/11/13/why-and-how-the-ipcc-demonized-co2-with-manufactured-information/#comment-1478101
          2013/11/18/cowtan-and-ways-pausebuster-still-flat-compared-to-models/#comment-1478677
        • RomanM
          Posted Nov 20, 2013 at 7:42 PM | Permalink
          Sean, if the NAS panel didn’t disagree with McIntyre’s criticisms then he, the NAS, and I can all agree that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years has subsequently been supported by an array of evidence.
          Sigh… They just aren’t teaching new PhDs either logic or clear thinking any more. Is it your contention that when someone debating a specific topic raises one peripheral to the conversation, unless the other party immediately drops the current discussion and begins refuting the peripheral issue they accept what has been newly raised as “gospel truth”?
          The matter at hand at the time was the scientific inadequacy of Mann et al’s methodology and proxies – not the broader area of paleo reconstructions. The not-so-impressive “array of evidence” has been dealt with in the intervening time since the NAS deliberations.
          I might also suggest that you unless you learn appropriate blog behavior with regard to introducing issues off-topic to a thread, you could gain yourself a reputation as a troll.
        • Posted Nov 20, 2013 at 8:04 PM | Permalink
          Sigh… They just aren’t teaching new PhDs either logic or clear thinking any more. Is it your contention that when someone debating a specific topic raises one peripheral to the conversation, unless the other party immediately drops the current discussion and begins refuting the peripheral issue they accept what has been newly raised as “gospel truth”?
          The matter at hand at the time was the scientific inadequacy of Mann et al’s methodology and proxies – not the broader area of paleo reconstructions. The not-so-impressive “array of evidence” has been dealt with in the intervening time since the NAS deliberations.
          I was just trying to understand McIntyre’s position better by asking if the NAS panel didn’t disagree with any of McIntyre’s claims. If you’re suggesting that we should move on to examine new research like Shi et al. 2013 and ponder their implications then we’re in furious agreement.
          I might also suggest that you unless you learn appropriate blog behavior with regard to introducing issues off-topic to a thread, you could gain yourself a reputation as a troll.
          I’m sorry to have bothered you. I mistakenly thought that Howard introduced the productive issue of my “monologue style of conversation”. That confused me because of dialogs like yesterday’s in which ATheoK and davidmhoffer introduced a productive issue at 3:16 and 7:44pm that I tried to respond to here:
          2013/11/18/cowtan-and-ways-pausebuster-still-flat-compared-to-models/#comment-1479712
  12. Doug Proctor
    Posted Nov 18, 2013 at 6:09 PM | Permalink | Reply
    I see the difference as more of a step-function at 2005: there are 4 obvious cycles/patterns of rising and falling, the fourth of which was in the last stage of falling when the step-function lurched the data up.
    Our desire to counter CO2 linear forcing functions I think gets us to simplify at times what shouldn’t be simplified.
  13. son of mulder
    Posted Nov 18, 2013 at 6:19 PM | Permalink | Reply
    What is the probability that Arctic temperature change was aligned with the rest of the Hadcrut4 history until 2005 when it started to grow faster? Zilch!
    An extension of this approach to earlier years may well be a reasonable form of adjustment but wouldn’t that mean the the starting points for the models would need to be higher and also 1990′s would be a higher temperature thus meaning a reduced affect on the last 17 year trend.
    The word desperate comes to mind.
  14. RB
    Posted Nov 18, 2013 at 6:38 PM | Permalink | Reply
    It is possible that it might have been on Hansen’s watch list in 2005.
    http://arctic.atmos.uiuc.edu/cryosphere/IMAGES/seaice.anomaly.arctic.png
  15. Posted Nov 18, 2013 at 7:16 PM | Permalink | Reply
    “the difference has arisen entirely since 2005″
    And that’s a key part of the “pause”. Here’s a recent review article by Serreze and Barry on Arctic amplification. They identify important reasons as ice albedo, and exposure (for longer) of warmer sea in place of cold ice. There has been a lot of sea ice melting since 2005.
    • NevenA
      Posted Nov 19, 2013 at 6:56 AM | Permalink | Reply
      Why is no one responding to this? It makes sense, right, that Arctic warming has accelerated because of an acceleration in Arctic sea ice loss?
      • Ian H
        Posted Nov 19, 2013 at 1:31 PM | Permalink | Reply
        There are countervailing factors making it difficult to determine if less ice leads to more warming or less. Certainly it is not a simple one sided effect.
        1. There is no change in ice maximum. Ice regrows each year.
        2. The ice minimum occurs late in the season when the angle of incidence is small. At that angle of incidence there is little difference between the albedo of water and ice.
        3. Darkness is descending on the arctic at that time. No sun = no heating.
        4. Once darkness descends water radiates more heat to space than ice
        5. More open water at the pole is a likely cause of more snow in the northern hemisphere. That snow (which falls on land in places where the sun shines more normally) will cause cooling via the change in albedo.
      • Manfred
        Posted Nov 19, 2013 at 1:55 PM | Permalink | Reply
        “It makes sense, right, that Arctic warming has accelerated because of an acceleration in Arctic sea ice loss?”
        ——————————
        It makes sense, but if this is “the” reason, there would be zero Arctic amplification “model style”, because models did not forsee such ice loss.
        And there is also no Antarctic amplification and no tropical tropospheric ampolification. Then the models got it everywhere wrong.
  16. RB
    Posted Nov 18, 2013 at 7:19 PM | Permalink | Reply
    News releases in 2005:
    On Sept. 21, 2005, sea ice extent dropped to 2.05 million sq. miles, the lowest extent yet recorded in the satellite record.
  17. NevenA
    Posted Nov 18, 2013 at 7:22 PM | Permalink | Reply
    Could it be that Arctic amplification really took off after the record low Arctic sea ice minimum of 2005? Temperature anomalies in the Arctic are extremely large during autumn. If you scroll down for instance to the temp maps and graphs on the Arctic Sea Ice Graphs page, you see something that happens every year around this time (take for instance this NOAA/ESRL temp anomaly map). Does HadCRUT4 catch that? Or GISTEMP for that matter?
    I don’t know if this could be the reason of the post-2005 discrepancy. A lot of sea ice melted in the years following 2005.
  18. Posted Nov 18, 2013 at 7:44 PM | Permalink | Reply
    Difference (Hybrid – HAD4) Data ignored if non-existant in HAD4.
    By Latitude Band for 2005 to 2012
    By Latitude Band for 1998 to 2012
    The data looks kind of jittery.
  19. Gary Pearse
    Posted Nov 18, 2013 at 8:03 PM | Permalink | Reply
    Don’t you think that the record back to 50s should be treated in the same way. Weren’t cooler temps amplified colder in polar areas? It would be cherry picking indeed to just krig the polar regions to adjust the last decade or so. And going forward, wouldn’t the dipping in temps since 2005 result in polar d-eamplification of the poles, reducing global temps more – the rapid freeze-up this year at both poles may reflect some change in this manner. So if the decline since 2005 continues we should make it a steeper slide than otherwise. As you point out,Steve, the effect is small anyway. Also Hadcrut 4 is already a recent adustment of temps upward to try to do the same thing. In reality we should chop of the several tenths of warming that was added since 1998 to to submerge the long standing 30s record for the 150 years. Then do the kriging.Otherwise Krig would roll over in his grave.
  20. Robert Way
    Posted Nov 18, 2013 at 8:08 PM | Permalink | Reply
    The two years which were the most susceptible in the Cowtan and Way paper were 1998 and 2010. There are synoptic reasons which give a reasonable explanation of why coverage bias was so emphasized in those two years. In our FAQ there is a specific discussion (with downloadable .kmz files) related to these two years.
    • Posted Nov 19, 2013 at 11:34 AM | Permalink | Reply
      1998 and 2010. Two strongest recent El Nino years. El Nino Amplification?
      • Robert Way
        Posted Nov 20, 2013 at 2:18 AM | Permalink | Reply
        The synoptic pattern of 2010 is more related to the extreme winter AO conditions whereas 1998 was due to a very strong El Nino.
  21. Chad Jessup
    Posted Nov 18, 2013 at 10:51 PM | Permalink | Reply
    “Cowtan and Way observe that HadCRU does not estimate temperature in many Arctic gridcells. Because Arctic temperatures have risen more than low-latitude temperatures, they state that recent HadCRU temperatures are biased low.”
    OK, but does the same situation apply to the Antarctic where there might be missing gridcells? And if so, would the data cancel out due to possible lower temperatures in that area of the world?
    Just asking.
    Steve: they did a similar procedure on the Antarctic, where they also report recent positive trends.
    • RB
      Posted Nov 18, 2013 at 11:20 PM | Permalink | Reply
      Steve: I had placed my own trend estimates http://www.climateaudit.info/data/observations/gridcell by latitude zone online at CA a few days before Nick Stokes (mentioning this at Lucia’s but not posting here. My estimates for CW match Nick’s but there are differences in the other dsets. Don’t know why. I had to redo collation scripts for hadCRU since they’ve changed their setup and it’s possible that something went awry.
      • Carrick
        Posted Nov 19, 2013 at 6:02 PM | Permalink | Reply
        This must be a private link. As with Lucia’s blog, where I also noted this, clicking on your link gives
        Not Found
        The requested URL /observations/gridcell was not found on this server.
        Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
        Oops. Link corrected.
  22. Steve McIntyre
    Posted Nov 19, 2013 at 12:35 AM | Permalink | Reply
    I’ve added three graphics to the post comparing CMIP5 models to observations, including CW13. I’ve also shown a similar comparison by Bart Verheggen posted a few days ago, which appears to me to be erroneous though i’ve done these calculations quickly and relatively late in the evening here – that’s when errors can be made.
    • Posted Nov 19, 2013 at 1:27 AM | Permalink | Reply
      Am I missing something. Surely the whole point of the current discussion ought to be that whatever the exact discrepancy between C&W13 and HadCRUT4 may be it does very little to diminish the striking discrepacy between model forecasts and actual temperature measurements — even when the temperature measurements are handled in a more spohisticated way.
      Surely Cowtan and Way ought to graciously admit that this is the case.
    • Chip Knappenberger
      Posted Nov 19, 2013 at 10:22 AM | Permalink | Reply
      Steve,
      In this post, we explicitly compare *trends* in models and observations over different time intervals. C&W helped the models a little bit, but not much:
      -Chip
  23. ianl8888
    Posted Nov 19, 2013 at 3:13 AM | Permalink | Reply
    I agree with SMc on interpolation of data with regard to ore reserves, whether by kriging, trendlines or even old-fashioned triangulation. We all have to do this when dealing with sparsely located data
    BUT:
    prior to any form of interpolation, I always insist on an accurate area map with all actual data points posted and identified
    Cowtan and Way’s paper appears to be paywalled, so where are the base area maps with actual data points posted, please ?
    • Jeff Norman
      Posted Nov 20, 2013 at 11:14 AM | Permalink | Reply
      Ian,
      With ore samples are you not interpolating between ore samples that are tens of meters apart while with temperature data they are interpolating between temperature samples that are thousands of kilometres apart?
  24. Posted Nov 19, 2013 at 5:21 AM | Permalink | Reply
    Hi Steve,
    - The figure in http://ourchangingclimate.wordpress.com/2013/11/15/cowtan-and-way-global-average-temperature-observations-compared-to-cmip5-models/ that you reproduced in your post was prepared by my co-blogger Jos Hagelaars.
    - Links to datasources uses are mentioned in that post.
    - Differences with your graph probably arise from different datasources and, perhaps more importantly, different reference periods and vertical allignment. It looks like you’re still using 1990 as starting point, forcing all graphs to be identical in that specific year (why?). That skews the comparison, as has been pointed out before I believe.
    Bart
    Steve: I didn’t use, let alone “still use” 1990 as a starting point. I did two versions, one converting CMIP5 runs to 1961-1990 anomalies and one converting CMIP5 runs to 1980-1999 anomalies, both with similar appearances. I downloaded CMIP5 RCP8.5 runs from KNMI, using the GLB TAS data. I’ve done a number of these comparisons and the appearance of your graph is different than the ones that I’ve done. [Update] I’ve done a detailed reply and diagnosed the difference to the 5-95% envelope.
    • Steve McIntyre
      Posted Nov 19, 2013 at 10:56 AM | Permalink | Reply
      Bart, I’ve re-examined your picture. Your diagnosis of the difference is incorrect. The vertical location of the CMIP5 model runs and vertical location of the Had4/CW observations match closely as shown in the diagram below. The difference arises from the location of the 5-95% percentile envelope, with my calculation located inside yours as shown below. The difference may arise with the data or the calculation. We both used KNMI data: I used an RCP8.5 dset of 82 runs downloaded relatively recently. I’ve placed my collation of runs online and a script below. I calculated the 5-95% quantiles on a very elementary way and do not believe my calculation to be in error. Over to you.
      cw13 annotation of verheggen
      Figure 1. Emulation of Verheggen Figure. Ensemble mean matches closely; location of Had4 and CW13 match almost exactly. Difference is in the location of the 5-95% envelope: my calculation in orange; Verheggen’s in grey.
      here is a script for my figure:
      loc="http://www.climateaudit.info/data/models/cmip5/chron_glb_tas.tab"
      dest= "d:/temp/ chron_glb_tas.tab"
      download.file(loc,dest,mode="wb")
      load(dest)
      chron=chron_glb_tas
      dim(chron) #5412 82
      
      anom=function(x,start=-999999,end=999999,Round=2) { #assumes time series
                work= ts( rep(1:12,ceiling(tsp(x)[2]-tsp(x)[1]+1)), start= floor(tsp(x)[1]),freq=12) 
                work=window(work,start=tsp(x)[1],end= tsp(x)[2] )
                anom=month=factor(work)
                temp= time(x)>=start & time(x)<= (end+1)
                levels(anom)=unlist(tapply(x[temp],month[temp],mean,na.rm=TRUE))
                anom=  round(x- as.numeric(as.character(anom)),Round) 
              return(anom)
      }
      
      work=chron
      P=ncol(chron)
      for(i in 1:P) work[,i]=anom(work[,i],1980,1999.99)
      chrona=work
      ann= ts( array(NA, dim=c( nrow(work)/12,P) ),start=tsp(work)[1])
      for(i in 1:P) ann[,i]= annavg(work[,i]) 	 
      
      ann=window(ann,1880,2099.99)
      ci= t (apply(ann,1, function(x) quantile(x, c(0,.05,.25,.75,.95,1) ,na.rm=T) ))
      ci=window( ts(ci,start=1880),1970,2020)
      ens= window( ts(apply(ann,1,mean,na.rm=T),start=1880,freq=1),1970,2020)
      	year=c(time(ci))
      
      setwd("d:/climate/images/models/cmip5")
      imgv= readPNG("cowtan_way_hadcrut_rcp85-jos-hagelaars-2013.png")
      
      	plot(0:1,type="n", xlim=c(1965,2022),ylim=c(-.7,1.21))
      	rasterImage(imgv,1963.5,-.835,2022,1.27)
      	abline(v= c(1970,2020))
      	abline(h=c(-.4,1.2))
      
      png("cw13 annotation of verheggen.png",w=680,h=520)
      par(mar=c(0,0,2,1))
      plot(0:1,type="n", xlim=c(1965,2022),ylim=c(-.7,1.21),axes=FALSE)
      	rasterImage(imgv,1963.5,-.835,2022,1.27)
      	lines(ens,col=3)
      	lines(ci[,"5%"],col="orange",lwd=5)
      	lines(ci[,"95%"],col="orange",lwd=5)
      	points(window(had4ann,1990)-mean(window(had4ann, 1980,1999.99)),col=2)
      	points(window(hybrid_ann,1990)-mean(window(hybrid_ann, 1980,1999.99)) ,col="blue")
      	legend("bottomright",fill=c("orange"), 
      	  legend=c("5%-95% Quantile"),inset=c(.12,.12) )
      	#lines(ci[,6],lty=3)
      	#lines(ci[,1],lty=3)
      	title("Annotation of Verheggen CMIP5 RCP8.5 Comparison") 
      	mtext(side=3,line=-1,font=2,"Showing 5-95% Quantile") 
      
      dev.off()
      
      • Steve McIntyre
        Posted Nov 19, 2013 at 11:10 AM | Permalink | Reply
        I noticed that my post uses 5-95% quantiles, which is in keeping with IPCC usage. Verheggen might have used 2.5-97.5% quantiles. I re-did the graphic on that basis and it’s still inside his version. On further reflection, it looks like he might have used 0-100% quantiles and smoothed them a bit.
        • Pat Frank
          Posted Nov 19, 2013 at 11:30 AM | Permalink
          Since at its base all this discussion is supposed to be about science, let’s also not forget that the uncertainty limits everyone’s worried about are model run precision, and have nothing to do with physical accuracy or the reliability of the projections.
        • HaroldW
          Posted Nov 19, 2013 at 1:49 PM | Permalink
          Enlarging graphs and reading off (approximate) 5-95% ranges for RCP8.5 for the year 2010…
          McIntyre 0.49 K (0.29-0.78, baseline=1980-1999)
          Hagelaars 0.76 K (0.14-0.90, baseline=1980-1999) (*from graph posted at CA)
          Stott 0.40 K (0.2-0.6, baseline=1986-2005) (from figure 4(bottom) of Stott et al. ERL2013)
      • Posted Nov 19, 2013 at 11:43 AM | Permalink | Reply
        Your data points and the model average in your graph with the yellow lines closely resemble the ones in my graph. As you point out, the difference between your graph and mine is the 2 sigma range. I just used all RCP8.5 model runs available in KNMI Climate Explorer for the calculation of the standard deviation, a total of 984. I extracted all the data in all rcp85 files except the ‘avg’- and the ‘modmean’ files. You can find them here:
        http://climexp.knmi.nl/CMIP5/Tglobal/index.cgi?email=someone@somewhere
        The 2-sigma range for RCP8.5 as displayed in my graph is roughly comparable with 95% range in the CMIP3 model update graph made by Gavin Schmidt on RealClimate:
        http://www.realclimate.org/images/model122.jpg
        Regards, Jos Hagelaars

        Steve: there aren’t “984″ model runs in the linked page. It looks to me like you used a very similar compilation to the 84 RCP8.5 runs that I downloaded from the same source. IPCC used a 5-95% range in their graphic. When you can directly calculate the quantiles, calculating two-sigma from standard deviations when you don’t know whether the data is normal, seems bass-ackwards to me. Nor can I replicate your lines using this methodology. I think that it is more reasonable to use IPCC as a comparison than something at realclimate, but to each his own, I guess.
        • HaroldW
          Posted Nov 19, 2013 at 1:55 PM | Permalink
          Jos -
          Based on your KNMI link, you seem to be including all scenarios, not just RCP8.5.
        • HaroldW
          Posted Nov 19, 2013 at 1:58 PM | Permalink
          Apologies, please ignore previous. Jos specified that he used the RCP8.5 model runs from the set in his link. There are 94 such.
        • Posted Nov 19, 2013 at 2:11 PM | Permalink
          If you use all those files you get 984 different sets of temperature data for the RCP8.5 files (skipping average en modmean files). These files are stored in the KNMI Climate Explorer under the link “download all files”: http://climexp.knmi.nl/cmip5_indices.cgi?id=someone@somewhere
          So I simply used what the KNMI indicated: all files and all data.

          Steve: The individual files (e.g. http://climexp.knmi.nl/CMIP5/Tglobal/global_tas_Amon_ACCESS1-0_rcp85_r1i1p1.dat do NOT contain “12 sets of temperature data, all different”. They contain monthly data :) 12 should have been a clue for you. You do not have 984 CMIP5 runs, but 984/12=82 runs. I suggest that you collate the data correctly and re-issue your graphic.
        • Posted Nov 19, 2013 at 2:49 PM | Permalink
          OK, that was very stupid of me and quite embarrassing.
          Thanks for pointing this out to me. I should have noticed it myself immediately but I just never checked if every file contained 12 lines of data, just downloaded everything.
          I will rebuild my graph, it will surely change the 2 sigma range.
          Steve: with unfamiliar datasets, it’s easy to grab the wrong end of the stick, so to speak. *** happens: no harm done. I think that it makes more sense to directly use 5% and 95% quantiles, rather than calculating standard deviations. I’ve shown a method in my R code. But either way, you should get something like my graph.
        • Steve McIntyre
          Posted Nov 21, 2013 at 10:37 AM | Permalink
          Bart Verheggen’s blog has replaced their incorrect figure with one that looks like mine, with the comment shown below. Their corrigendum didn’t fully undress their actual error.
          The figure below has
          been replaced, since the original was found to be in error during discussions on CA). The confidence interval of this corrected graph is substantially narrower than the erroneous original one. Note that the current graph shows the 5 to 95 percentile range of model runs (i.e. the 90% confidence interval), whereas the previous ones showed the 95% confidence interval. At the bottom of the post a similar figure with both confidence intervals as well as the two sigma range is shown.
      • RB
        Posted Nov 19, 2013 at 12:31 PM | Permalink | Reply
        Ed Hawkins’ 5-95% percentiles seem closer to Jos Hagelaars than Steve McIntyre’s.
  25. Geoff Sherrington
    Posted Nov 19, 2013 at 5:59 AM | Permalink | Reply
    (Please snip if too long)
    Because the CW13 paper is paywalled I have not read it yet and I rely on extracts and the abstract.
    In past work on ore resource calculation using geostatistics including kriging, we have a performance case from the Ranger One uranium mine, number one ore body, open cut. We did a computer-based forecast of grade and tonnes before mining was commenced in 1980, followed by reconciliation after mining ceased in 1995. In the reconciliation, the tonnes of ore grade rock that were shifted were accurate to within 5% of the forecast. The grade of contained uranium mined was within 7% of the pre-mining estimate.
    By example, kriging can work well as a central part of geostatistics. Kriging involves the creation of values where none exist, such as by creating 3D block grades between sparse, linear-like drill holes. Kriging should not be attempted when missing values are frequent or when data points are beyond the reach of the statistical method.
    ………………..
    Next is an example of a danger involved with creating missing values in a thermometer temperature-time record. Suppose that an official had “homogenised” a time series record, by removing all high outliers, believing them to be mistakes, then failed to record the culling. When treated later as missing values, there would be a tendency to ascribe them values close to those typical of the time and place, cooler than the hot anomalies that they were. The outcome is wrong because of undetected bias.
    In reference to this CW13 paper, if the satellite’s estimate of temperatures had an undetected systematic bias, it is analogous to the example with thermometer. The outcome is wrong. But we cannot know, for the satellite data at the Poles have no adequate past check series.
    ………………..
    One step in geostatistics is to estimate the range r via the semivariogram, with r being the separation (distance) where data from one point is held to lose the ability to predict data at another. For a typical 2D case one constructs a search ellipse whose dimensions enclose the range in different directions for an anisotropic case. The search shape is figuratively moved over the known data points to calculate missing values according to a weighting-distance factor related to neighbours. Points that fall outside the search shape wherever it is moved are not candidates for calculations of missing values. The search shape will commonly vary according to surroundings, eg in this case, locations aligned with a prevailing wind might differ in the cross-wind direction.
    What I do not know because of paywalling is the dimension of each search shape that was used by CW13. I seek to know if there are grid cells with missing data that are outside the reach of the relevant search shape. Intuitively, many of them will be. If they are not present in the Arctic, they are likely to be present in the Antarctic.
    Given my background, I would then assert
    1. that it is invalid to assign missing values to cells outside the statistical range. Computational ease has to be sacrificed.
    2. that the ‘global’ temperature has to be reported without the Polar regions.
    3. that it is scientifically invalid to make theories about climate change when a slab of the data is no more than a guess.
    In mining work, making guesses about metal values in an ore deposit with shares available to the public can land one in jail.
    • Matt Skaggs
      Posted Nov 19, 2013 at 10:03 AM | Permalink | Reply
      “…it is invalid to assign missing values to cells outside the statistical range. Computational ease has to be sacrificed.”
      Thanks Geoff, that was the thing that had me scratching my head, and why I went trolling in Thorne05 to see if I could find some data from near 90N. The comments refer to in-filling and interpolation, but the gridcells in question are colder than the vast majority of the measured gridcells, so it is really extrapolation.
    • ThinkingScientist
      Posted Nov 19, 2013 at 12:52 PM | Permalink | Reply
      Geoff,
      I agree with some of your concerns about variogram range (not specified in Cowtan & Way), spatial anisotropy (not specified in Cowtan & Way) and neighbourhood search criteria (again not specified).
      However, I would disagree that the neighbourhood search radious for samples and the variogram are necessarily be linked. In general the search neighbourhood is related to the data sampling pattern whereas the variogram range is a measure of the limit of spatial dependency and is therefore an estimated parameter of the underlying variable. I do agree that you may require a different search orientation (and this can be coded in some software as a trend map for the local behavour, eg wind direction, or anisotropy) but the suggestion that the variogram and search radius should be linked is not necessarily true.
      For example, using Ordinary Kriging with an assumption of a non-stationary variable, restricting the search and the neighbourbourhood to be similar would make sense, not smearing the points beyond their range of influence. However, in a stationary case I would not agree – in this case a unique neighbourhood might make more sense where all samples are used simultaneously for the estimation, as this will better estimate the global mean of the data.
      • Geoff Sherrington
        Posted Nov 21, 2013 at 3:26 AM | Permalink | Reply
        ThinkingScientist,
        If you are still in range, thank you for your comments.
        The geostatistics that I mentioned were done by others, specialist colleagues more capable than I am. It was my recollection that a prime use of the semivariogram was to estimate the range at which sample points could no longer be used, in a statistical sense, to help to forecast each other. Have I got this wrong? Regarding your comments, what would you nominate as the main purpose of the semivariogram?
    • Craig Loehle
      Posted Nov 20, 2013 at 10:16 AM | Permalink | Reply
      I have done some Kriging and agree in large part with Geoff. I have no problem with doing kriging but only “between” the data points one has. In a real sense the N polar region is “outside” the weather stations that we have, because colder and perhaps with different behavior (all ocean, for example). Kriging extrapolation is much less reliable than when used for interpolation.
      If an outsized amount of warming is happening at the poles, that might be of interest for polar bears and sea level rise, but not for the rest of the world. Let’s say that all the warming due to CO2 was at the poles. Then except for melting ice none of us would be affected at all. Thus the current reconstructions without the poles are more informative for policy purposes.
      • ThinkingScientist
        Posted Nov 20, 2013 at 1:18 PM | Permalink | Reply
        That comes down fundamentally to your assumption about stationarity. If you consider the property to be stationary then it is reasonable to extrapolate, although at that point the kriging estimate then tends to the declustered mean of the data and will be featureless.
        However, I don’t think it reasonable to consider polar amplification as a stationary problem for estimation, so any answer will depend on the (likely arbitrary/unknown) non-stationary assumption. And that also knocks out the first method of Cowtan and Way. The second method (the “hybrid”) would have been much more valid had they used say collocated cokriging with Markov-Bayes, which would also allow them to change the correlation coefficient in the cross-covariance to be different for land and ocean (which they report as being different R with the UAH for GISTEMP). It would also treat the satellite data as much softer, which would be reasonable given the weak correlations reported between GISTEMP & UAH of R=0.75 (land) and particularly R=0.32 (ocean). Still find it strange that they report correlations with UAH for GISTEMP, but then apply their method to HadCRUT4. Why not report the correlations for the data actually used?
  26. JamesG
    Posted Nov 19, 2013 at 6:46 AM | Permalink | Reply
    Tamino and Bart like to start their warming trend lines in the mid-70′s while simultaneously claiming that the mid-70′s were artificially cooled by manmade aerosols.
  27. ThinkingScientist
    Posted Nov 19, 2013 at 9:46 AM | Permalink | Reply
    From my read of Cowtan and Way none of the following are clear to me:
    1. What variogram model parameters are set and what is the fit to the experimental variogram? It seems to suggest an Exponential variogram function is used, but the range is not specified. A comparison of the model and experimental variogram data would also give an indication about the validity of the stationarity assumption and whether spatial anisotropy is observed.
    2. Cross-validation presented as tables of RMS difference are of limited use. More usual measures would be crossplots such as predicted vs actual, difference vs actual etc.
    3. What coordinate domain is this being calculated in? Is it computed on a sphere using grand circle distances or does it treat the 5×5 lat/long grid cells as an orthogonal XY system? If the latter, (a) is an area corection performed? and (b) why write this in Python when there are plenty of free software tools available (eg GS-LIB2) which would allow these computations using a known package? (If computed on the sphere, fair enough).
    4. What is the neighbourhood search criteria, to select local samples for a local search or is a unique neighbourhood used? Were negative weights returned? The neighbourhood search is very important and will strongly influence the results and also needs to take into consideration the stationarity assumption used.
    5. In the approach used, kriging should work as an exact interpolator, if the kriging is conducted on the actual 5×5 grid then it should not change any measured values – a weight of 1 for that sample and zero for others should be returned.
    6. For the hybrid method, what is the correlation between the satellite and surface measurements? Are they linearly related? This would also inform the decision in the next question.
    7. Why were well established geostatistical methods of combining two data types not adopted, eg collocated co-kriging, or kriging with external drift? These two approaches are well known and a comparison between them would also be informative as they make somewhat different assumptions (one being across-dependency approach, the ohter a linear function + kriging of residuals).
    Finally, I think a great opportunity is missed here. It would have been much more interesting to subset later years with more coverage to resemble earlier year data recording patterns and then re-predict the missing later year values using the retained data resembling the reduced coverage data sampling pattern of the early years. This would have given valuable information about the relative uncertainty of interpolating early years with a paucity of data in terms of estimating the mean from the available values, rather than just jumping ahead and going for a global mean. Also, omitting strips of latitude tends to over-favour interpolation whereas polar prediction is essentially an extrapolation (and therefore is also sensitive to the neighbourhood search choice and the stationarity assumption).
    • Posted Nov 19, 2013 at 10:31 AM | Permalink | Reply
      ThinkingScientist: Thanks for the final suggestion, which makes a lot of sense to me in providing some kind of check of how uncertain this kind of inter-/extra-polation is.
      On this:
      (b) why write this in Python when there are plenty of free software tools available (eg GS-LIB2) which would allow these computations using a known package? (If computed on the sphere, fair enough).
      I have some sympathy for young researchers who wish to use a more modern language than Fortran, in this case Python. The Python stack has always been free and open source, as I gather is GSLIB2. I assume your point is that in the Python ecosystem there is no library as complete as GSLIB. It’s a very common tradeoff.
      • ThinkingScientist
        Posted Nov 19, 2013 at 12:40 PM | Permalink | Reply
        Geoff,
        For geostatistics in a regular 2D or 3D XYZ coordinate geometry domain GS-LIB is free, available either ready compiled as executables or as source code with complete makefiles and contains more than enough geostats tools you would need for this study, including jack-knife, cross-validation, kriging, variograms etc. You can buy the book too, its relatively cheap (about £40) and fully describes both the theory and application of all the code including worked examples. Its written by two of the most well known researchers in geostastics: Andre Journal (a protege of Matheron himself at Ecole des Mines, Paris) and Clayton Deutsch. Both are well respected and the code is widely used.
        If the Cowtan and Ray paper is computed on a sphere, GS-LIB would not be appropriate. But is Cowtan and Ray computed on the sphere or gridded 5×5 in lat/long but assumed to be XY? If the latter, well a lot of further questions need to be asked…
        • ThinkingScientist
          Posted Nov 19, 2013 at 12:57 PM | Permalink
          Sorry, I meant Richard…:-)
        • Posted Nov 19, 2013 at 1:36 PM | Permalink
          I take it as a compliment :)
  28. Ronan
    Posted Nov 19, 2013 at 10:46 AM | Permalink | Reply
    Hi Robert Way,
    It’s good to see you’ve dropped by. :) Most of the authors whose work is discussed on ClimateAudit don’t bother engaging with us, which is a shame, because there is usually a lot of interesting statistical and scientific discussion, hidden amongst the comments. So, I hope you stick with us, and don’t get too turned off by the name-calling/aggressive banter which unfortunately occurs. Trust me when I say that a lot of us are genuinely here for the scientific discussion.
    By the way, I suspect the fact that you’ve even commented here has already gained you a lot of respect and kudos from many of us! :)
    Anyway, since you’re here, I have a few questions on Cowtan & Way, 2013. Don’t feel under any pressure to reply if you don’t want/don’t have time. Following these comment threads can get time-consuming… :(
    1. You argue in your conclusions that:
    … the problem of coverage bias may exist whenever the Arctic has experienced rapid warming (or cooling) in the past. However, given the magnitude of recent Arctic warming and polar amplification relative to global trends, it is expected those previous periods of warming (or cooling) in the Arctic are unlikely to bias the record to a greater extent than the bias recorded in recent decades by this study.
    I am curious – did you consider estimating the cooling bias introduced to the global temperature estimates for the 1920s/1930s Arctic warm period? Obviously, there was no satellite data for then, so you couldn’t use your hybrid method. But, could you apply your Kriging method to the early 20th century data? It could offer a useful test of your conclusions, for two reasons:
    a) The magnitude of the 1910s-1940s Arctic warming and the 1940s-1970s Arctic cooling seem to be at least comparable to the recent 1980s-2000s Arctic warming.
    b) The data coverage during the early 20th century was worse than now. So, perhaps the cooling coverage bias was even worse then.
    Whatever the results, I suspect it would be an interesting comparison…
    2. I get the impression that you are explicitly assuming that the various temperature records which make up the datasets are either a) non-biased or b) adequately homogenised. Is that correct?
    3. You seem to mostly have focused on your results for the Arctic region, but you do briefly discuss the effects of Kriging on the Antarctic data. However, I noticed you didn’t mention Steig et al., 2009 or O’Donnell et al., 2011 (of which our host was a co-author) in your paper.
    Are you familiar with the Steig vs. O’Donnell debate over the most appropriate methods for spatially interpolating temperature records for the Antarctic region?
    P.S. Best of luck in your PhD studies! You’ve chosen a pretty controversial topic, but it’s an important one, and fascinating too. :)
    • Robert Way
      Posted Nov 19, 2013 at 5:19 PM | Permalink | Reply
      I think there there can be a lot of interesting discussions which go on here and that the statistical and scientific questions that arise can help move things forward in terms of finding better ways of doing science. There are sometimes insinuations about personal motivations and personal issues brought up as well as blanket statements for anyone involved in climate related work and I think that can be what keeps scientists from engaging here in ways they perhaps should. For myself I am able to handle most anything personal thrown at me but many people find it difficult to not take the negativity home with them.
      That being said I do draw the line at what Steve did above. He said basically that in my hacked personal correspondence I said things about him (and many other people) that he didn’t like so he will continue to spread the contents of this hacked correspondence until I “apologize” to him personally. To me this is the type of behavior you very often see in classrooms where a cellphone is stolen and one person says to the other either you apologize to me or I’m going to keep spreading around the bad things you messaged people. You can each yourselves be the judge of what grade level this type of situation occurs the most at ;) I will be issuing no apology to an implied threat or some form of blackmail. If there was an issue that bothered Steve about what was said he could have brought it up to me privately and we could have settled it that way like men. I am still open to that type of discussion but certainly not when there is a threat on the table.
      [1] We have produced a long-kriged series which does show that coverage bias is an issue that affects both warm and cold periods of time with the particular observation that when the arctic(antarctic) warms(cools) quickly there will always be some form of underestimation of those magnitudes when you have incomplete coverage.
      (a) The warming of the 1910s to 1940s is not comparable to that of the last 15 years in the Arctic even considering coverage bias. In some areas of the Arctic it may have been (though I can’t confirm this) but the unique difference is that this period shows extreme warmth over the eastern Canadian Arctic that is certainly not comparable. Note that for my definition of the Arctic I use north of 60. Based on the other types of evidence – the physically based evidence – it is fairly easier to see that warmth in the Arctic is beyond that of the early century warming both using the limited instrumental records, the sea ice record, ice core melt rates and simply observations of glacier changes.
      (b) We have also tried to use the 20th Century Reanalysis to produce such a comparison but it did not perform well – particularly before the Antarctic stations came online. There are some odd features in the reanalysis product and I presume more high latitude surface pressure information (ISPv3 versus v2?) will help to constrain the solution better in the next iteration of the 20CR product.
      [2] We do not explicitly assume that the records making up the temperature records are non biased or adequately homogenized. We used the dataset which had dealt with the SST bias and did not have a form of interpolation (e.g. GISS, NOAA). In the future it would be worth testing this approach on a dataset that is quality controlled and homogenized statistically like BEST but they will need to include oceans and will need to use the bias adjusted SSTs instead of the standard NOAA products. If there were other datasets available that met those conditions then we would have used those as well. The preference is always to include as much data as possible.
      [3] We are investigating the Antarctic region separately and will be taking a more in depth look at the data there in the future. We spent most of our time on the results for the Arctic because there were additional sources of data and a fairly consistent signal there whereas the Antarctic is more nuanced and does not contribute a significant chunk to the overall bias in terms of warming in the past 15 years. The Arctic was the primary contributor to the changes due to coverage bias so it was worth investigating that region in particular.
      We are aware of the work and debate between O’Donnell et al (2011) and Steig et al (2009). Future Antarctic work will include spatial comparisons with O’Donnell et al (2011), Screen and Simmonds (2012), and Steig et al (2009). I think that both papers were interesting contributions with Steig et al (2009) proposing a new method and O’Donnell et al (2011) trying to improve upon it. The debate over those two papers became rather silly with a lot of back and forth between the authors of their respective studies. I anticipate any work in the future we do will be trying to be as inclusive of outside opinions as possible.
      We do have some ideas for better constraining the results in that region and several others. This paper was a good first step and hopefully it leads to new ideas to better understand high latitude regions. The biggest challenge that remains is some regions is still the quality of the satellite products which either lack coverage (RSS) do not have a TLT product (STAR) or may have remaining biases (all three). The NOAA group have recently shown (Weng et al 2013) that some ocean warming has been underestimated over the past 15 years due to cloud and precipitation biases. They used a cloud detection algorithm AND a single sensor covering the entire period for this analysis AMSU-A.
      From the abstract
      “It is shown that the global mean temperature in the low and middle troposphere has a larger warming rate (about 20–30 % higher) when the cloud-affected radiances are removed from AMSU-A data.”
      Weng et al. 2013. Uncertainty of AMSU-A derived temperature trends in relationship with clouds and precipitation over ocean. Climate Dynamics. Published October 6th.
      [P.S.] Thank you – I appreciate the well wishes but I am afraid I must warn you that my PHD studies are on permafrost relationships with climate and will include spatial modelling mostly at the local-to-regional level so this project is not a contributor to my PHD or MSc project (Torngat glacier sensitivity to climate etc..).
      As a side issue related to this I would like to say at the outset that for our paper we suggested reviewers who we knew would give us a difficult time because we wanted to ensure the paper was as strong as possible – it resulted in a very long and difficult review process which greatly improved the manuscript. We also placed the paper in a technical journal rather than a more flashy alternative for the same reason. The code and the majority of the datasets are available and we have made a FAQ section and a website to help people better understand and be able to replicate the work. This has been a clear and transparent process and the hope is that rather than engaging in quote mining and other personal distractions that people take the time to give reasoned scientific discussions(and critiques) like the ones you presented.
      Steve: Robert Way’s comments about me in the post contain the sort of misrepresentation that is far too characteristic of SKS. I observed that Way had made untrue and offensive remarks about me at the SKS forum. Way said that he was young at the time and did not withdraw the false remarks. I said that he ought to withdraw the untrue remarks, presuming that an honorable person would withdraw untrue remarks voluntarily. This was not a “an implied threat or some form of blackmail”, but a simple request that someone do the right thing. Rather than doing the honorable thing, Way characterized his false and offensive remarks as merely saying things about me that I “didn’t like” and tried to claim that I had threatened him:
      That being said I do draw the line at what Steve did above. He said basically that in my hacked personal correspondence I said things about him (and many other people) that he didn’t like so he will continue to spread the contents of this hacked correspondence until I “apologize” to him personally. To me this is the type of behavior you very often see in classrooms where a cellphone is stolen and one person says to the other either you apologize to me or I’m going to keep spreading around the bad things you messaged people. You can each yourselves be the judge of what grade level this type of situation occurs the most at ;) I will be issuing no apology to an implied threat or some form of blackmail
      Yes, I didn’t like the remarks, but my objection was on the basis that they were untrue. And no, I didn’t threaten him. I asked him to behave honorably, unlike, for example, his SKS associate John Cook, who baldfacedly lied to Geoff Chambers about posting the Lewandowsky survey.
      • Lars P.
        Posted Nov 19, 2013 at 5:56 PM | Permalink | Reply
        Robert, you say: “in my hacked personal correspondence” – which you know it is not true. There was no hacking of the secret forum, it was simply left open on the internet. You could blame inadvertently being made public, but not hacked.
        You are a mature person now, would you say now what you said at the time? Is it not the time to put things right?
        I have a lot of sympathy to what Steve says: “over the years, I’ve gotten tired of people privately conceding the validity of my criticisms of paleoclimate practices, but failing to do so publicly”
        I am also tired of being called “denialist” and worse just for just questioning the alarmism. And yes, some of the name callers may later say, they were young and so on, but this does not change the insult, even if I know they are behaving immaturely and stupid.
        • Robert Way
          Posted Nov 19, 2013 at 6:08 PM | Permalink
          “There was no hacking of the secret forum, it was simply left open on the internet. You could blame inadvertently being made public, but not hacked.”
          You are incorrect. It was indeed a hacking and we even know how it was done and in which time period. I am not sure if the details have been made public or not but without an ounce of doubt it was a hacking. Not only that but it was done by a very experienced hacker based on what has been evaluated by the specialists this was brought to. The only thing left open was a means for accessing a portion of the site which was exploited by the hacker to access the rest and to systematically attempt to download everything there. Any insinuation otherwise (as to the origins of the forum material) is against the body of evidence we have collected and frankly a means of rationalizing the reading of private correspondence that should not be read.
          Steve: I’m very puzzled at SKS’s position here. SKS said: “Although we at Skeptical Science (SkS) do not condone the lamentable means through which Peter Gleick obtained the DenialGate documents from the Heartland Institute, the information is nevertheless out there.” SKS then proceeded to vehemently editorialize against Heartland. You yourself personally participated in the discussion and publicization of Heartland documents e.g. here. [Update: Robert says that he was not aware that the documents had been obtained fraudulently at the time that he made the linked comment.] Perhaps you can explain to me why it’s acceptable for SKS to discuss documents that Gleick forged and/or obtained through wire fraud, while it’s not acceptable to comment about statements on the SKS forum. Your position seems inconsistent and hypocritical, but perhaps you can explain. BTW I didnt follow the SKS incident at the time: is there a post in which you set out the evidence for a hack, as opposed to Cook’s contemporary statement that he set security settings to zero?
        • John M
          Posted Nov 19, 2013 at 6:43 PM | Permalink
          Thanks for bringing this up again.
        • thisisnotgoodtogo
          Posted Nov 19, 2013 at 7:04 PM | Permalink
          I find everything the secret war room folks say to be utterly true on the very face of it.
          Therefore this must be true; hacked.
        • thisisnotgoodtogo
          Posted Nov 19, 2013 at 7:23 PM | Permalink
          Rob H. of SkS says that looking through many files is hacking. There are so many hackers, like Google, too.
          It’s a bad world when you only want to plan a secret war.
        • kim
          Posted Nov 19, 2013 at 8:04 PM | Permalink
          Ah, games, and costumes, such fun. I wonder what costumes Robert Way and Kevin Cotwan fancied themselves in.
          =================
        • Posted Nov 19, 2013 at 8:23 PM | Permalink
          “…against the body of evidence we have collected…”
          Does it even matter? You cannot tell the world who did it. You cannot show the evidence. The site was a pooorly secured, home-grown operation, secretive for the purposes of discussing individuals and ideas that are themselves public. There are threads in there, from insiders, offering suggestions on how the closed nature of forum breeds group-thinking and polarization. When things are out in the open, whatever criticism is offered has to not only find its target, but also pass as valid to those who are onlookers. Removal of this crucial function under secret settings makes one say things that he or she wouldn’t probably otherwise. That’s what you did. It is not your age that is at fault.
        • DGH
          Posted Nov 19, 2013 at 10:34 PM | Permalink
          Robert,
          Due respect but that’s pathetic and lame on several counts.
          1. The web hosts of SKS have a track record of incompetence.
          A) John Cook deletes comments to suit the message of the site. Threaded comments disappear without concern for trailing responses. This leads to confusing reads in the comments.
          B) You have seen the photoshopped images of Dana Nuccitelli and John Cook in Nazi regalia that was discovered on the SKS site, haven’t you? Those images were in a public directory.
          C) Despite your pleadings to the contrary, your friends left the front door to the SKS discussion board wide open. And yes your comments were embarrassing. (FYI my children know that anything they post online is or will be public into.)
          2. SKS’s view of Internet security and politeness was summarized by Dana Nucccitelli here…
          Allow me to paraphrase, “we didn’t steal the information from Heartland and we didn’t falsify any documents…”
          You and yours have chosen the low road. Stop whining about the ruts.
          3. I’m no statistician but your comment “Settle it like men” is sexist and offensive to more than 50% of people interested in global temperature.
          How about you apologize for your childish comments made on an insecure website and (at least whiile you’re here) focus on the interesting aspects of your work being discussed? You’re gettiing a great deal of respect from a very tough crowd.
        • Robert Way
          Posted Nov 20, 2013 at 9:51 PM | Permalink
          “Steve: I’m very puzzled at SKS’s position here. SKS said: “Although we at Skeptical Science (SkS) do not condone the lamentable means through which Peter Gleick obtained the DenialGate documents from the Heartland Institute, the information is nevertheless out there.” SKS then proceeded to vehemently editorialize against Heartland. You yourself personally participated in the discussion and publicization of Heartland documents e.g. here. Perhaps you can explain to me why it’s acceptable for SKS to discuss documents that Gleick forged and/or obtained through wire fraud, while it’s not acceptable to comment about statements on the SKS forum. Your position seems inconsistent and hypocritical, but perhaps you can explain. BTW I didnt follow the SKS incident at the time: is there a post in which you set out the evidence for a hack, as opposed to Cook’s contemporary statement that he set security settings to zero?”
          [1] I have not knowingly promoted the use of stolen documents. At the time that I linked to the document in question I did not know that it was received through misrepresentation by Dr. Gleick. I have expressed that I disagree with the means used to get those documents to others and I stand by that. I think you should retract that statement. I do not speak for SKS as a whole as I am sure you do not speak for everyone who participates here as a whole – I think it is important to emphasize that subtlety.
          [2] The constant remarks about the forum being “secret”. There are many organizations that have had internal exclusive forums or mailists – exlusivity does not make them secret. In fact there was never anything “secret” about the forum in the first place. John spoke about it several times during talks and presentations and even included a screenshot of it during one of his discussions long before the hacking took place. I do not think if you are willing to share it publically with hundreds of people that it is considered secret – simply private.
          [3] I thought that there was a post on the details of the hack but apparently not. Perhaps it would be useful for others to be given the details proving it was a hacking. As to whether John left security settings to zero – perhaps that refers to another issue? I could provide the list of steps that were followed in the hacking but I would prefer to consult with John before doing so as it is not really my place to be the “first” to outline this. Were you to email me I would be more inclined to do so.
      • Geoff Sherrington
        Posted Nov 19, 2013 at 8:21 PM | Permalink | Reply
        Robert,
        My attention to this paper was drawn from this line in your abstract “Both methods provide superior results than excluding the unsampled regions, with the hybrid method showing particular skill around the regions where no observations are available.”
        My cloistered past seldom led me to words like these, apart from pop-sci excursions into perpetual motion and lossless recreational gambling, both of which engage scepticism.
        I cannot accept that some of your infills are more than guesses dressed in statistical drag. It follows that I cannot accept that one can assign measures of skill to such guesses.
        Some compilers seem to agree with me that unsampled areas unlinked to measured areas should be kept out of data sets, e.g. HadCRUT4 and Australasian Joint Ore Reserves Committee, see “The Competent Person must not remain silent on any material aspect for which the presence or absence of comment could affect the public perception or value of the mineral occurrence.”
        What would be the core of your rebuttal of my assertion that many of your data points in unsampled areas are guesses? On what predictive linkage between unsampled and sampled areas do you principally rely?
        As to “Both methods provide superior results ….” I ask “superior to what?”. Logically, they cannot be superior or inferior to any data sub-set that is filled with no values.
        • Robert Way
          Posted Nov 19, 2013 at 8:26 PM | Permalink
          Superior during cross-validation. See the cross validation map in the paper.
        • ThinkingScientist
          Posted Nov 20, 2013 at 7:59 AM | Permalink
          Robert – the cross-validation in the paper appears to only test for infilling bands between available data points, this is not the same as cross-validation where data coverage is one-sided relative to the latitude band ie extrpolation at the poles. Your xval will be optimistic.
          Further points:
          1. What variogram model range are you using?
          2. Is the variogram model spatially anisotropic?
          3. You describe experimental semivariogram calculations in distances on a great circle, do you then krige on a 5×5 lat/long grid? If so are you using great circle calculations on the 5×5 grid kriging?
          4. What neighbourhood search criteria are using? Range? Anisotropic? Number of samples?
        • TimTheToolMan
          Posted Nov 20, 2013 at 8:17 AM | Permalink
          Unfortunately I dont have access to the paper…but from what I understand you drop a region and use your hybrid kriging technique to reconstruct the temperature of the dropped region given the satellite readings at high altitude and nearby ground measurements. Yes?
          So the underlying assumption is that cross validation of the dropped regions works because the temperature in those regions varies in the same ways as further North where we dont have ground readings.
          As I said, I dont have access to the paper but are your cross validation regions ones where there is permanent sunlight and permanent darkness? And sea ice boundaries? And within the normal bounds of the polar vortex?
        • Carrick
          Posted Nov 20, 2013 at 9:42 AM | Permalink
          Thinking Scientist:
          Robert – the cross-validation in the paper appears to only test for infilling bands between available data points, this is not the same as cross-validation where data coverage is one-sided relative to the latitude band ie extrpolation at the poles. Your xval will be optimistic.
          This is my worry too.
          Here you only have land stations and you’re using them to infill over open sea (think Arctic Ocean above Russia and Siberia). And because the land-based polar amplification is a maximum for these regions, any errors associated this “crossing the boundary” assumption will be maximized.
        • Posted Nov 20, 2013 at 3:19 PM | Permalink
          “Here you only have land stations and you’re using them to infill over open sea”
          No, they are using HADCRUT gridded data. Most of that information comes from SST.
          There is a set of data (IABP) from ice buoys etc – unfortunately for a limited period 1979-1998. They used that for testing.
        • Carrick
          Posted Nov 20, 2013 at 9:52 PM | Permalink
          Nick:
          No, they are using HADCRUT gridded data. Most of that information comes from SST.
          HADCRUT uses HADSST. Unless something drastically has changed since the last time I’ve looked at it, there are no filled cells in the Arctic.
          If there were, there would be little point to Cowtan & Way, and the argument about HadCRUT ignoring the Arctic Ocean would be spurious.
          There is a set of data (IABP) from ice buoys etc – unfortunately for a limited period 1979-1998. They used that for testing.
          So little data, it has no statistical power.
          This is a case of “as good as we can” is also “not good enough.”
        • Carrick
          Posted Nov 20, 2013 at 10:05 PM | Permalink
          Here is the coverage for the latest HadSST3 release, for October 2013. I’m showing the uncertainty bounds, since it shows which boxes have data, but also which have little weight.
          I’m glad to see they’ve added shipping data at higher latitudes since HadSST2 (which is the set I was looking at previously, it was longer than I had remembered).
          We can replace “no data over open sea” with “virtually no data” and “many empty cells”.
        • Posted Nov 20, 2013 at 10:23 PM | Permalink
          Carrick,
          I think there are many filled SST cells. In fact, just looking at your October plot, it seems to me that 3/4 of the cells bordering the N Polar white area are SST.
          My count, 1997-2102, for NOAA and HAD 4, is here.
        • Carrick
          Posted Nov 20, 2013 at 11:08 PM | Permalink
          Nick, you’re ignoring quality of information again. If there were many cells and the quality of the data were good, there would be no point to C&W, would there?
          What’s with picking 1997-2012 btw? Seems like cherry picking to me (and I’m pretty sure from observation that you would accuse others of it, if they used only that interval).
        • Posted Nov 21, 2013 at 5:52 AM | Permalink
          Carrick,
          I think CW’s 16 years is countering a cherry-pick (17 yrs, Santer etc). But in fact you can see from Steve’s 2005 observation that it’s going to affect any trend period of more than a decade, more or less in proportion to length.
          On quality, I was just pointing out that any interpolation would be mostly from SST, not land as you said.
        • Carrick
          Posted Nov 21, 2013 at 10:52 AM | Permalink
          Nick, you would normally look at other intervals besides an obviously terrible choice. This interval choice is so obviously bad science, I’m surprised to not see you concede it.
          As I said elsewhere:
          I’ll note that their chosen interval 1997-2012 leads to extreme excursions on either ends of two series that otherwise seem to agree well, with little difference between them. These excursions are acknowledged by C&W as associated with anomalous weather patterns.
          In any case, it is well understood that linear regression weights points near the edge more than points at the center, so having selected end points that are just past two extreme events, it is not surprising that the slope found with C&W is significantly different than the slope found with HADCRUT. This comment applies to both series… you’ve amplified the noise and found a desirable result, so you report on it.
          On quality, I was just pointing out that any interpolation would be mostly from SST, not land as you said
          Perhaps you can explain clearly how you factually know from looking at the wrong data set.
          Your link above points to HadCRUT not HadSST that you are counting cells from. HadCRUT combines HadSST with a land-based stations. So I’m not sure why you’re using this as proof that the data is SST rather than land stations.
      • Chad Jessup
        Posted Nov 19, 2013 at 10:13 PM | Permalink | Reply
        Mr. Way, this is just plain country simple – either admit or deny you uttered the offensive, false remarks. It doesn’t matter one hoot if your comments were hacked or you personally emailed them to the NYT.
        • Geoff Sherrington
          Posted Nov 20, 2013 at 5:21 AM | Permalink
          Robert,
          You reply “Superior during cross-validation.”
          What criteria did you use to satisfy the requirement that the prior, measured data was at a statistically useful distance from each of the grid cells that you infilled?
      • sue
        Posted Nov 19, 2013 at 10:39 PM | Permalink | Reply
        Robert Way,
        “There are sometimes insinuations about personal motivations and personal issues brought up as well as blanket statements for anyone involved in climate related work and I think that can be what keeps scientists from engaging here in ways they perhaps should.”
        But openly stating that Steve has an ‘agenda’ on SKS is ok?
        robert way at 12:41 PM on 27 August, 2010
        I disagree with steve’s take on Moberg versus loehle. Loehle was clearly flawed whereas Moberg brought something new to the field with his novel wavelet technique. I think anyone who would give loehle’s analysis the same credibility as Moberg’s clearly has an agenda as methodologically the difference is clear.
      • Scott Basinger
        Posted Nov 20, 2013 at 1:20 PM | Permalink | Reply
        “That being said I do draw the line at what Steve did above. He said basically that in my hacked personal correspondence I said things about him (and many other people) that he didn’t like so he will continue to spread the contents of this hacked correspondence until I “apologize” to him personally. To me this is the type of behavior you very often see in classrooms where a cellphone is stolen and one person says to the other either you apologize to me or I’m going to keep spreading around the bad things you messaged people. You can each yourselves be the judge of what grade level this type of situation occurs the most at ;) I will be issuing no apology to an implied threat or some form of blackmail”
        Let’s put this a different way. You backtalk someone and they overhear it, but you hadn’t intended that they overhear it, and because they overheard it when you hadn’t intended it somehow they’re to blame?
        I’m sorry, but that’s just ridiculous.
        Either stand by the backtalking or retract it. If you don’t retract it, I can assume that you stand by your overheard commentary, and it reflects poorly on you.
        • thisisnotgoodtogo
          Posted Nov 20, 2013 at 1:52 PM | Permalink
          Would Michael Mann retract anything if someone found some dirty, dirty, private laundry he left hanging about?
        • thisisnotgoodtogo
          Posted Nov 20, 2013 at 2:06 PM | Permalink
          SkSdz marks is SkSdz marks.
          Let’s hope the ethical standards on display do not carry over into the research.
  29. Steven Mosher
    Posted Nov 19, 2013 at 11:14 AM | Permalink | Reply
    “Cowtan and Way praise Hansen for being the first person to report the effect of missing Arctic data on global temperature. However, no material discrepancy had arisen between their index and the HadCRUT4 index as of 2005 so that Hansen was, according to Cowtan and Way’s own data, observing a discrepancy that had not yet arisen, making their following praise to Hansen seem somewhat premature:”
    In that time frame Hansen would have been refering to hadcrut3. Hadcrut4 raised the estimate in northern latitudes..
    Steve: Mosh, you’re off your game here. Did you check to see if HadCRU3 makes a difference? It doesn’t. Zero difference to 2005 with HadCRUT3 as well.

    cowtanway2013_difference between hadcru3 and cw2013
    • Steven Mosher
      Posted Nov 19, 2013 at 1:00 PM | Permalink | Reply
      That’s weird. I have to check my differences between had3 and had4.
      Also, recall the climategate email where the difference between giss and hadcrut is put down to lack of arctic coverage in had3,

      Steve: I was a bit surprised as well. Your point sounded sensible, but it was worth checking.
  30. ilmis
    Posted Nov 19, 2013 at 12:08 PM | Permalink | Reply
    Maybe OT or otherwise snipworthy, feel free to. The secret SkS forum gives some interesting background for this paper and other biases.
    Cowtan on finding an error in a Rahmstorf-Tamino paper that claimed there is no slow-down
    Well, I could write a response to ERL, showing why the solar contribution is overestimated. And thus show that warming has in fact slowed in the last decade. … But it is an extremely disheartening prospect, doing a load of work to point out an error by someone whom I greatly respect, and who has contributed more to the debunking of climate denial than I will ever acheive.
    In the end he seems to consider contacting Tamino (in 2011, ”Better that I do it than a denier”) but never follows up, on that thread at least.
    On finding a cooling bias in HadCRUT:
    0.1C! Wow! (Haven’t even looked at the paper yet)

    OK, I clearly need to work on the paper but hold it till the HADCRUT4 data is available, and assess if they’ve picked up all of the bias. I’m excited! I’ll try and post some intermediate results and an outline of the paper in the next month for comments, and see where things go from there.
    In fact he works on this for years, trying several strategies and analyses, writing draft papers etc. to show that HadCRUT3 and later 4 with their lower warming trends are outliers.
    The level of enthusiasm seems to strongly correlate with how the finding fits the alarmist narrative.
    On time periods and trend significance:
    A while back I tried calculating the period over which you’d expect to see 95% significance in the warming trend for the major indices, and it’s not far off 11 years. Which is inconvenient. But if the underlying forcings have changed (e.g. solar and aerosols), then the calculation becomes meaningless.
    On the other hand, his account on how he debunked Judith Curry in private communication with a sceptic:
    Here’s the text I gave:

    Anyone who draw a conclusion based on a temperature trend on only a decade (indeed less than about 15 years for the BEST data), is incompetent or dishonest.
    The unexpectedness of >= 11-year pauses is an inconvenient result only spoken in the secret forum. When communicating climate to a skeptic, even the thought is incompetent or dishonest.
    In climate communication, global warming is a disaster. In non-public communication, the pause is a disaster.
    It’s looking like when 2012 begins, we are again going to be in the sitation where HADCRUT shows 11 years of negative trend. Worse, if the La Nina holds on, it could reach 12 years in 2013. Worst of all, it may turn out to be a statisitcally signficant deviation from the 1975-2000 trend.
    It’s somewhat unsettling. Years of relentless work to push the warming trend up, one small step at a time, with subtle and esoteric corrections. But refusal to notify a friendly team member about a correction that would give support to the pause.
    Things could look different if these people worked to genuinely correct the scientific record instead of pushing a narrative.
    Is there any correction in the other direction? Has UHI become net cooling yet?
    • ThinkingScientist
      Posted Nov 19, 2013 at 12:34 PM | Permalink | Reply
      “Which is inconvenient.”
      An inconvenient truth, perhaps?
    • Steve McIntyre
      Posted Nov 19, 2013 at 2:08 PM | Permalink | Reply
      Here’s another one of Kevin Cowtan’s suggestions:
      If you want it to be a good game, the best way to do it is to pick a game out of the top 500 rankings on boardgamegeek and retheme it (although Cleudo is probably the best of the classics).
      Off the top of my head, ‘Pandemic’ might be a good candidate. It’s a cooperative game in which all the players are working together to save the world from 4 pandemic diseases. Change the diseases to climate myths. The Epidemic cards can have pictures fro the deniers on. The operatives (players) could either be well known mythbusters, like John, or climate scientists, or more generic roles (climate scientist, environmental journalist, climate blogger etc).
      • NevenA
        Posted Nov 19, 2013 at 3:28 PM | Permalink | Reply
        It would seem this paper is annoying you.
        • Scott Basinger
          Posted Nov 19, 2013 at 4:43 PM | Permalink
          The possibility exists given the press releases that this is yet another example of team advocacy masquerading as science. We have to keep a close eye on the pea.
        • kim
          Posted Nov 19, 2013 at 5:04 PM | Permalink
          It annoys me that this filthy disease is not finally solved.
          ===============
        • Steve McIntyre
          Posted Nov 19, 2013 at 6:11 PM | Permalink
          No, the paper doesn’t annoy me. However, the comments in the SKS forum did annoy me. I hadn’t heard of Cowtan prior to this paper or associated him with the SKS forum until a commenter did so. I intended to post on them at the time; it deserved criticism; but I was a bit worn out with such nonsense and didn’t at the time. The appearance of the paper by two participants reminded me of the SKS forum, which deserves more commentary than it’s received thus far.
        • NevenA
          Posted Nov 20, 2013 at 2:58 AM | Permalink
          What a coincidence then that you have decided just now that the SKS forum hack deserves more commentary.
          Here’s to hoping you never get hacked…
        • Brandon Shollenberger
          Posted Nov 20, 2013 at 4:20 AM | Permalink
          NevenA, Steve McIntyre hasn’t decided that “just now.” I’ve seen him make a number of comments indicating it deserves more commentary in the last few months.
          Your attempts at mind-reading seem to be clouded by a personal bias.
        • b4llzofsteel
          Posted Nov 20, 2013 at 5:28 AM | Permalink
          It would seem Steve McIntyre is annoying you.
        • kim
          Posted Nov 20, 2013 at 4:09 PM | Permalink
          Neven expects his environment to melt the ice.
          ====
      • daved46
        Posted Nov 20, 2013 at 12:15 AM | Permalink | Reply
        We rather enjoy playing Pandemic in our family, but I’d bet a version created by skeptics would be much more fun than ones put out by the team. I can imagine “cities” like “Tiljander” or “Urals” and powers by the players might include teleconnections and “adjusting” the number of blocks on a given city.
      • John Ritson
        Posted Nov 20, 2013 at 4:26 AM | Permalink | Reply
        Climate Cleudo – It was Professor Peat in the varve with a hockeystick.
      • Scott Basinger
        Posted Nov 20, 2013 at 1:14 PM | Permalink | Reply
        “Here’s to hoping you never get hacked…”
        When you have nothing to hide, it doesn’t really matter.
        • NevenA
          Posted Nov 20, 2013 at 3:53 PM | Permalink
          “When you have nothing to hide, it doesn’t really matter.”
          Here’s to hoping Steve McIntyre has nothing to hide.

          Steve: What would you like to know about? I don’t correspond with very many people and my email correspondence is mostly fairly terse.
    • ilmis
      Posted Nov 20, 2013 at 12:30 PM | Permalink | Reply
      Correction, Dr. Cowtan’s proof of Judith Curry ”lying with statistics” was made in public, here.
  31. igsy
    Posted Nov 19, 2013 at 12:56 PM | Permalink | Reply
    I am pretty certain what follows is a dumb question, but I’ll ask it anyway. There’s no mention in the paper of the surface-to-lower troposphere amplification ratio, generally assumed to be of the order of 1.2. So, if this isn’t taken into account in the mapping step, some part of the “extra” warming in the hybrid series would be gained at the expense of consistency with the lapse rate enhancement assumption in the GCMs. They did allow for this, didn’t they? Or, if not, why not?
    • phi
      Posted Nov 19, 2013 at 2:15 PM | Permalink | Reply
      There is no tropospheric amplifcation expected in high latitudes, but rather a global warming mitigation in altitude. That said, the question is interesting because globaly (and not just for the tropics) it is an amplification that we should see. However, and this is a big problem, this amplification is already absent over HadCRUT4, and worse, for the land, the divergence is about 0.1 ° C per decade in the wrong direction. The results of CW13 make the puzzle even more obscure. Well, actually, the solution is crystal clear, but nobody wants to see it.
      • igsy
        Posted Nov 19, 2013 at 3:32 PM | Permalink | Reply
        Yup, it was a dumb question. I appreciate your informative reply.
  32. MarkB
    Posted Nov 19, 2013 at 2:08 PM | Permalink | Reply
    Steve, I’m unable to reproduce the linear trendline since 2005 as you show it in figure 2. I get a step discontinuity in the before/after 2005 trendlines like Bob Tisdale is showing here: http://bobtisdale.files.wordpress.com/2013/11/figure-44.png . The post 2005 trendline you’re showing is also much steeper in slope. Dare I say “hockey stick”? To get trendlines which intersect with a common inflection point, the before/after point is circa 2000.

    Steve: I constrained the regression line to begin at the endpoint of the previous linesegment.
    • MarkB
      Posted Nov 19, 2013 at 4:39 PM | Permalink | Reply
      Steve: I constrained the regression line to begin at the endpoint of the previous linesegment.
      I’m troubled because a step discontinuity (beyond sampling noise tolerances) in a temperature trend would imply a huge energy impulse into the system at the discontinuity point. The mathematics don’t seem physically plausible. Hence postulating an inflection in trend around 2000 makes more physical sense, and any hypothesis attempting to explain the slope change would want to be consistent with that time frame.
      Discontinuity aside, the 2005-2012 trend slope in your graphic looks to be considerably overstated compared to Tisdale’s figure which agrees with my own calculations. I’m guessing that in generating the figure you subtracted the value of the step from start point of the regression line but inadvertently missed doing the same with the endpoint.
      • MarkB
        Posted Nov 20, 2013 at 11:02 AM | Permalink | Reply
        Figures illustrating the points made in previous post:
        [IMG]http://i41.tinypic.com/2f07lac.jpg[/IMG]
        • MarkB
          Posted Nov 20, 2013 at 11:02 AM | Permalink
  33. Gary Pearse
    Posted Nov 19, 2013 at 2:13 PM | Permalink | Reply
    Steve: When looking at images of polar amplification (ones that don’t just brick-red the arctic in or make a jack o’ lantern earth with broad orange covering several temperature ranges)it would appear that the main amplification is in a rough ring waggling between 60-80 N and it then is less over the poles. Most of this ring has thermometers in it and certainly is covered by UAH. If this is true, then the extrapolation would add too much warmth. Also in Antarctica (which you know well)the “amplification is largely in West Antarctica. Amplification is not evident in the polar area. Extrapolating here would, too, warm too much. Have I got this wrong?
  34. Posted Nov 19, 2013 at 6:52 PM | Permalink | Reply
    Hi Steve,
    If you’re interested then an update to IPCC Fig. 11.25, which includes Cowtan & Way and provisional HadCRUT4 data for 2013, is here:
    Ed.
  35. AndyG55
    Posted Nov 19, 2013 at 7:39 PM | Permalink | Reply
    “Because Arctic temperatures have risen more than low-latitude temperatures, they state that recent HadCRU temperatures are biased low.”
    Hey? either they have temperature readings, or they don’t.
    If there are temperature readings, use them..
    If there are not temperature readings, they can’t make this statement.
  36. Paul Baverstock
    Posted Nov 19, 2013 at 7:49 PM | Permalink | Reply
    I am not fully conversant with all of the scientific arguments here, but I have been a reader of Climate Audit since it’s inception. I would like to say that over the years Steve has been more than willing to accept when he has made errors and to correct them forthwith. I believe he is a man of great integrity and that his contribution to the discussions on climate and his research on many matters has been hugely underestimated by the scientific community.
    I believe that in the years to come he may well receive accolades for his work that will be greatly deserved.
  37. Posted Nov 19, 2013 at 11:28 PM | Permalink | Reply
    Are there tables where one can interrogate datums used for infilling?
    Here I am, looking at a global temperature map, that as Steve presents and particularly illustrates in Figure 2. C&W’s effects of infilling Arctic temperatures to the global temperature.
    Normally, in my mind it takes large temperature changes over a large area to affect a global average. Yet, if I understand Figure 2 correctly, starting oddly enough in 2005 temperatures in the Arctic increase global averages by .01C per year.
    Just for the Arctic? Is this some sort of a Mercator temperature version of the Arctic? Just how high are these C&W hybridized temperatures?
    Then again, .01C temperature changes per year effect on global temps? what are the confidence levels for each infill? I assume that confidence levels are part of the data tables? I would like to know how much confidence one can place in this sudden temperature spike.
    If it seems to good to be true, especially to those desperate for shifting the pause explanations, then quite likely…
  38. thisisnotgoodtogo
    Posted Nov 20, 2013 at 12:55 AM | Permalink | Reply
    Recalling Guy Callendar’s study and the mention of too-high arctic temps which were therefore discounted.
    Did that appreciation of difference continue on in other studies and datasets?
    How different were they?
  39. Brandon Shollenberger
    Posted Nov 20, 2013 at 4:07 AM | Permalink | Reply
    Robert Way makes an intriguing comment upthread. I’m not going to dwell on the issue of hypocrisy McIntyre raised in his inline response. I’m going to try not to dwell on the fact I was accused of hacking Skeptical Science for simply viewing a publicly posted directory, a directory even Google viewed. Instead, I’m going to dwell on the absurdity of the situation as a whole. To emphasize, I’ll excerpt (a?) the absurd portion:
    Any insinuation otherwise (as to the origins of the forum material) is against the body of evidence we have collected and frankly a means of rationalizing the reading of private correspondence that should not be read.
    This is absurd on its face. One needs no rationalization for reading material that is in the public domain. The idea that people would need to do anything to rationalize reading material in the public domain is absurd.
    More importantly, Robert claim necessarily demeans anyone who has any doubt that Skeptical Science was hacked. Some people feel it is appropriate to be skeptical of serious claims. These people feel it is reasonable to require evidence before believing serious claims. Robert denies this. Robert claims the only reason people don’t agree Skeptical Science was hacked is they need to rationalize their behavior. Robert is effectively arguing refusing to blind accept his group’s statements is a sign of close-mindedness.
    Back when the “hack” first happened, Robert and Tom Curtis both participated in a discussion at Bishop Hill. That was a year and a half ago. It’s interesting to see the arguments were the exact same then. For example, Robert said:
    Not only do we know it is a Hack, the hacker was slopping and left clues. I’m sure some people will imply these clues were made up to make it look like a hack too… it’s all one big conspiracy right?
    Robert criticized people (in advance) for being conspiratorial for not accepting evidence which nearly two years later nobody has provided. Tom Curtis was worse, saying things like:
    The material is hacked without question. I am not in a position to say more than I have because it is not my website. The fact remains, however, that I am in a position to know, and I know that your various canards are exactly that and have no bearing on what actually happened.
    Plainly you are not going to believe me because that would be inconvenient. I must say that it must be terribly convenient to have beliefs so easily determined by your wants.
    Never-the-less, I am in a position to know that it was a hack. As can be seen from the various quotes of my “secret” comments above, I am a person of impeccable honesty.
    When a user, TerryS, responded he didn’t believe it was a hack because no evidence had been provided that it was, Tom Curtis responded:
    terryS, bull ****. If that was your basis for a plea of not guilty on a charge of receiving stolen goods you’ld end up rotting in jail.
    As we can see, the argument hasn’t changed at all. Nearly two years after the “hack” happened, Robert is still relying on the same absurd argument. He claims the reason people don’t accept a claim no evidence has ever been provided for is they’re close-minded. Anyone with integrity or honor would just blindly accept what he says is true (as Tom Curtis says):
    If your intention is to not act honourably, nothing I can say will be sufficient to stop you. Unfortunately the evidence on this thread is that nothing is going to stop you or anyone else doing what you want to – and you will grasp at any feeble excuse to feel justified in so doing.
  40. Brandon Shollenberger
    Posted Nov 20, 2013 at 4:15 AM | Permalink | Reply
    There’s an interesting note in Robert Way’s comments from way back then. He said:
    Short answer is yes. There’s a variety of information which is being collected. We are to the point where we have a short list of suspects based on the evidence that is collected.
    I’m intrigued by how Skeptical Science could “have a short list of suspects.” I’d love to know who’s on it. Unfortunately, the Skeptical Science crowd has repeatedly avoided giving out information. One may have slipped up though. John Hartz said:
    The SkS author team is unlikely to catch the perp. We’re relying on Interpol to do so.
    Counting on Interpol to catch a criminal means the criminal must be outside the United States. That means blind faith in the Skeptical Science team tells us they have a short list of non-US residents as suspects in the “hack.”
    I guess it wasn’t me this time.
    • AJ
      Posted Nov 20, 2013 at 6:22 AM | Permalink | Reply
      I was probably Google’s European crawler :)
  41. Posted Nov 20, 2013 at 4:21 AM | Permalink | Reply
    Hi Steve,
    A small point perhaps, but your comparison using Fig. 11.12 SOD is not quite fair as that particular graph is for decadal means which is why the observation lines don’t extend until 2012. Annual means would produce a slightly wider range of model estimates.
    See the final AR5 version of the figure here:
    http://www.met.reading.ac.uk/~ed/bloguploads/AR5_11_9.png
    Ed.
    • Geoff Sherrington
      Posted Nov 21, 2013 at 3:33 AM | Permalink | Reply
      Ed,
      To save the time of cross-checking, is your vertical positioning of the IPCC data relative to the actual data done in knowledge of the discussion at
      http://climateaudit.org/2013/10/08/fixing-the-facts-2/
      and earlier related threads at CA?
      Which of the various positions did you display here?
      • Posted Nov 21, 2013 at 5:29 AM | Permalink | Reply
        Hi Geoff,
        In my figures the reference period (i.e. zero anomaly) is 1986-2005 period, which was chosen by the IPCC for the future projections. So, the mean of each simulation and the observations is zero over this period.
        Ed.
  42. Green Sand
    Posted Nov 20, 2013 at 9:57 AM | Permalink | Reply
    As the majority of the conversation is way out of my comprehension, this may be a daft question but could the circa 2005 “trend shift” be as a result of station availability?
    Two years ago whilst making their “Global-average annual temperature forecast” the Met Office took time out to make the following statement:-
    • Over the last 10 years there has been a small tendency for the Met Office – CRU data to be in the lower half of the predicted range of temperatures. It would appear that the limited number of observations in the Arctic may have led to this tendency.
    • The limited surface observations that we have (together with Satellite observations) show that the Arctic is warming faster than other parts of the globe. Fig 3 shows that the data coverage has decreased between the 1990s and 2000s Some observational estimates take more account of the Arctic warming than Met Office – CRU, hence the need to compare our global forecast temperatures to the range of global records.
    This was followed by
    Figure 3: The difference in coverage of land surface temperature data between 1990-1999 and 2005-2010. Blue squares are common coverage. Orange squares are areas where we had data in the 90s but don’t have now and the few pale green areas are those where we have data now, but didn’t in the 90s. The largest difference is over Canada.
    I can remember at the time trying to find out out why the MO no longer had Canadian data that it had in 1990′s.
  43. Don Monfort
    Posted Nov 20, 2013 at 1:49 PM | Permalink | Reply
    I think that SkS kidz Way and Cowtan are deserving of their own blog in the Guardian. They could call it “The Pause Ain’t Really Killing the Cause: and we are 97% sure of that”.
  44. Ronan
    Posted Nov 20, 2013 at 2:14 PM | Permalink | Reply
    Robert,
    Thanks for your considered reply.
    With regards to the name-calling and the “he said, she said” arguments, I’m reluctant to get involved, as I am a fan of the following quotation:
    Great minds discuss ideas; average minds discuss events; small minds discuss people. – Eleanor Roosevelt
    Having said that though, when you say that there “…are sometimes insinuations about personal motivations and personal issues brought up as well as blanket statements for anyone involved in climate related work…”, do you agree that this could equally apply to critics of Steve and the commenters on ClimateAudit?
    Unfortunately, there seems to be a lot of bad blood on all sides of the climate debate. But, I fear, if we spend all of our time assigning the bad blood to different sides, all we will get is a “bloody mess”! :(
    Instead, I think we should remember that just because somebody doesn’t agree with us, that does not in itself mean they’re stupid, wrong, in denial, uninformed, part of some “denial machine” or “vested interest”, or any such nonsense. All it tells us is that they disagree with us! ;) If we want to know why, then maybe we should just respectfully ask them. Who knows – maybe in the ensuing discussion, both parties might learn something…?
    “That was excellently observed”, say I, when I read a passage by an author, where his opinion agrees with mine. When we differ, there I pronounce him to be mistaken. - Jonathan Swift
    With that out of the way, can we get back to the science? Please?
    First, I’m glad to see you were aware of the Steig vs O’Donnell debate, were keeping it in mind, and that you are planning on looking at Antarctica separately.
    However, I do think it is relevant to your analysis in Cowtan & Way, 2013, since you were also carrying out spatial interpolation of temperature records, e.g., kriging.
    This is especially so if you are claiming that the Arctic is showing “extreme warmth over the eastern Canadian Arctic”. One of the main points I got out of O’Donnell et al. was their argument that Steig et al. had incorrectly interpolated the pronounced warming apparent in the West Antarctica peninsula over too much of the continent.
    Is it possible that your interpolations have similarly spread the eastern Canadian Arctic warming over too large an area?
    Second, I’m confused at your claim that the “.. warming of the 1910s to 1940s is not comparable to that of the last 15 years in the Arctic even considering coverage bias.”.
    (By the way, I normally would define the Arctic as north of 66.33N, but I don’t mind your definition)
    Re-reading your paper I see that you consider three possible sources of bias in the global temperature records:
    1. Instrumental biases in the land records, e.g., urbanization bias
    2. Instrumental biases in the SST estimates, e.g., changes in buckets, transition to engine intake
    3. Coverage biases due to incomplete sampling
    Let us assume that those are the only biases we need to consider (I agree they’re among the most important).
    Then we have HadCRUT4 trends = true climatic trends + non-climatic biases. Is that reasonable?
    You seem to be implying that to determine the true climatic trends from the HadCRUT4, the only non-climatic bias we need to consider is the Arctic coverage bias during the 2000s. Am I wrong?
    In my opinion, the land records for quite a few Arctic stations have been affected by warming biases for recent years, e.g., urban heat islands and particularly “airport heat islands” (especially if we use your >60N definition).
    These biases are probably greater for the 1990s-2000s Arctic warm period than the 1930s-1940s warm period.
    What are your thoughts on the matter? I noticed that you said:
    “Sources of measurement bias in the land temperature record have been widely studied and include time of observation, instrumentation type and location (Williams et al. 2012), and the ‘Urban Heat Island’ (UHI) effect (Hausfather et al. 2013).”
    When you said ”…have been widely studied…”, did you mean “and resolved”, or were you just pointing out that it’s a subject of importance?
    Personally, I find the SST estimates more problematic. There have been too many changes and inconsistencies in the grid-box inputs for each year to rely on them as our primary source for the early 20th century.
    From your reply, you seem to be concerned about the biases in the SST estimates. So, I suspect we’re in agreement that they are somewhat unreliable, no?
    As for coverage biases, I know that there was an unfortunate post-1990 drop in station coverage in the Arctic region. But, surely you agree that the coverage during the 1930s-1940s warm period was worse than now.
    So, if you’re saying that global temperatures have been biased cold for the 2000s from inadequate Arctic coverage, shouldn’t the 1930s-1940s Arctic warm period have been biased cold even more?
    I get the impression your main interest seems to be on countering “the pause” in the 21st century.
    But, if the lack-of-Arctic coverage during Arctic warming periods is as important as you claim, it should also have led to a significant global cooling bias for the estimates of the mid-20th century warm period.
    When you add to that the uncertainties over the pre-WW2 SST estimates, and the potential warming biases in the Arctic for recent decades due to urban heat islands and airport heat islands, it could be quite substantial. Do you not think so, yourself?
    P.S. Steve/Moderators, apologies for the long comment. But, I’m interested in hearing Robert’s thoughts on the matter…
  45. Walter Manny
    Posted Nov 20, 2013 at 2:21 PM | Permalink | Reply
    For Robert, there is another way: own it — it’s what grown-ups do. If he still considers himself a young person, well, he can hold on to that and continue to obfuscate the issue, to make it other than it is. He said what he said, he meant what he said, and he is either sorry he said it or he’s not. The reason the two haven’t yet “settled it like men” is because, so far, there’s only one man involved. If he wants to be taken seriously as a scientist, he needs to disassociate himself from the SkS agendists as fast as his feet will take him, which, admirably, he has begun to do by appearing on all these independent sites to discuss his work. Now to reach the finish line.
    • Scott Basinger
      Posted Nov 20, 2013 at 3:29 PM | Permalink | Reply
      Good advice.
    • DCA
      Posted Nov 20, 2013 at 3:42 PM | Permalink | Reply
      ” If he still considers himself a young person”
      In looking at the references on CW13, it appears Mr Way is still young(relatively speaking) because it’s his first paper. I didn’t find him listed as falcuty member at U of Ottawa either (listed as his contact info). My guess is that he’s a grad student.
      • Ronan
        Posted Nov 20, 2013 at 5:01 PM | Permalink | Reply
        I believe this is Robert’s website.
        I think he has a fairly respectable CV for a grad student, and I think he has a promising career ahead of him.
        Personally, I think it is good that he is engaging with us here at Climate Audit. I acknowledge that there have been harsh words spoken on both sides and many of us probably disagree with him on many aspects. But, I do hope we can shift the dialogue back towards the science.
        As far as I remember, similar name-calling and disagreements occurred when Prof. Judith Curry first started commenting here. Although, maybe Steve or some of the others might disagree?
        • thisisnotgoodtogo
          Posted Nov 20, 2013 at 5:19 PM | Permalink
          “Personally, I think it is good that he is engaging with us here at Climate Audit.”
          Agreed. He has some courage, the thing that so many of The Team lack.
  46. hswiseman
    Posted Nov 21, 2013 at 2:15 AM | Permalink | Reply
    Let’s accept the proposition that the temperature record lacks spatial resolution in the arctic regions and that C&W represents a reasonable method of infilling mixed data using acceptable statistical methods. In the time-frame studied, tens, if not hundreds of billions have been expended on the global warming enterprise. Meanwhile this enormous blind-spot in the temperature record is un-addressed, when 20-30 automated temperature stations could have easily been installed and maintained throughout the region and delivered REAL OBSERVATIONS for peanuts. Instead we end up debating yet another Frankenstein data grafting exercise, albeit nowhere near the proxy/temperature monstrosity graft delivered by Al Gore. Genuine scientific inquiry is conducted through hypothesis and observation. The lack of insistence for real data among the climate community reveals all of this paper-pushing as a dubious exercise indeed.
  47. phi
    Posted Nov 21, 2013 at 7:37 AM | Permalink | Reply
    CW13 mainly concerns the Arctic amplification. In this regard, I think a graphic I already linked on CA has not received the attention it deserves.
    This is of course only a small portion of the Arctic but I don’t think we can conclude otherwise than by adding regional studies. Anyway, some general information can be drawn given the outstanding quality of the correlations.

13 Trackbacks

  1. By | The Global Warming Policy Foundation (GWPF) on Nov 18, 2013 at 6:56 PM
    […] Full story […]
  2. By CSALT Ju-Jutsu | context/Earth on Nov 19, 2013 at 12:31 AM
    […] hybrid correction to the HadCRUT global temperature series [1]  has provoked expected interest by auditor Steve McIntyre.  This is always welcome, because as with the majority of of these nosy irritants, the more that […]
  3. By The Climate Change Debate Thread - Page 3360 on Nov 19, 2013 at 5:49 AM
    […] Posted by Trainwreck Oops. The Cowtan and Way "pause busting" fudge paper isn't the Messiah after all. Even using that data manipulation – which is engineered to have the largest trend over the last […]
  4. […] Steve McIntyre’s post (here) illustrated the apparent 2005 breakpoint in his Figure 2.  I’ll discuss in this post why […]
  5. […] Steve McIntyre’s post (here) illustrated the apparent 2005 breakpoint in his Figure 2. I’ll discuss in this post why that’s […]
  6. […] Cotwan and Way 2013 (climateaudit.org) […]
  7. By kostnadseffektiv NFL Cykling tröjor 2013 on Nov 19, 2013 at 8:27 PM
    … Senaste Blogroll Tillsatser
    [ ... ] jag är inte längre visst platsen du få din Information , men stor . topic [ ... ]
  8. […] 17 Years, and taking into account the good work already done in analyzing Cowtan and Way by Steve McIntyre, Judith Curry, Bob Tisdale, David Whitehouse, and Christopher Monckton, I will focus on one of […]
  9. […] Cotwan and Way 2013 (climateaudit.org) […]
  10. […] Steve McIntyre (om rapporten som Jacob hänvisar till) […]
  11. […] discussione va avanti su molte pagine web, WUWT, Climateaudit, Climate etc., Real climate, SkepticalScience e altre ancora, con quelle orientate ad una visione […]
  12. […] [Update: The figure below has been replaced, since the original was found to be in error during discussions on CA). The confidence interval of this corrected graph is substantially narrower than the erroneous […]

Post a Comment

Required fields are marked *
*
*
Follow

Get every new post delivered to your Inbox.
Join 1,724 other followers
%d bloggers like this:
    0%
    10%
    20%
    30%
    40%
    50%
    60%
    70%
    80%
    90%
    100%