Remind me what counts as incomplete. When I looked at that Antarctic data I found cases where Tmax was present, but no Tmin as well as missing both data and the occasional way out of line figures (e.g missing the negative sign off). To my mind there is no point in using faulty or incomplete data but any data cleaning should leave an audit trail of the adjustments: beforedata + adjustments = finaldata. Does anybody do this? Those meticulous Germans and Austrians perhaps?
A good question. You have discovered the delights of raw station data! For this analysis complete means 100% daily capture for average temperature (Tmean), quite why Tmax and Tmin have to be different when it's the same sensor at the same station is anyone's guess. Yes indeed, there are nonsensical values sitting down in the records and in this age of techno-wizz this is unforgivable. Quite how the big names handle all this to produce those slick gridded products is another matter and I suspect they're not as fastidious as they should be.
I am struggling to understand exactly what the 'temperature anomalies'. OK it makes sense to to compare the change in temp relative to a reference period at each station. That way we can compare a
of station at sea level with one at altitude.
However, what happens when new stations are added starting at some later date than the reference period?
Also, as you have been showing here, what happens with missing data. My limited understanding is that when data are missing, it is estimated from the 'anomaly values' at adjacent stations.
It would be great to have some sort of precise of how these temp eg HADCRUT, NASA GISS etc series are derived by the various organisations, how their adjustments are made, and whether these methods are statistically sound. Just a thought?
Bang on! The new station issue bothers me because I suspect the big names are being rather naughty. Homogenisation is the word you need - they use records from whatever stations they do have to fill in the many holes in rural areas. Since these come from the larger towns and cities it means the global record is being contaminated by the urban heat island effect. Tony Heller on YT covers the crime of homogenisation nicely in a couple of vids. I have yet to find an honest account of how these substantial problems are tackled in detail by the big organisations - they throw out glib phrases like 'statistical modelling' that could mean anything
I have always been a little suspsicious of Mr Heller - fraud and tampering are strong words.
But the video does make sense - we do indeed seem to have a sparsity of long term historical data, so when error limits are applied, we really cannot say much about past global temps, whether tmax tmin or Tav anomalies.
In this video https://newtube.app/TonyHeller/rUo4RmU, Heller show the effects of adjustments. I note that these are adjustments to Tmax. This begs the questions as to what adjustments are made to Tmin and thus the much vaunted Tav values.
Seems a bit like a game of 'think of a number, any number' (just as long as it fits my model prediciton!)
Quite. My main gripe is that the big guns don't come clean on how poor the historic data really are. I can understand them trying to fill the holes and get something workable but to then place great emphasis on such a contrivance is unforgivable!
As it reminded it me of Tweedledum and Tweedledee I decided to look up "Tweedie Distribution". In stead of two fat foolish little men quarrelling over a rattle I found:
"In probability and statistics, the Tweedie distributions are a family of probability distributions which include the purely continuous normal, gamma and inverse Gaussian distributions, the purely discrete scaled Poisson distribution, and the class of compound Poisson–gamma distributions which have positive mass at zero, but are otherwise continuous."
Let's get this straight: Tweedle-de-dum's family are probably continuously distributing their very pure but normal grandmother. (But why does she need distributing?)
This has to be done with discretion because the upside-down goose (Gausse) and the Poisson fish are also involved with distributing Gamma in his compound (the poor old lady must be all in bits and in such a muddle!)
This is otherwise continuous and has zero positive mass (meaning the Fish floats in air?)
... OH! it's Michael Fish, who very famously cancelled a hurricane. That's beginning to make sense now.
Sorry, had to have some fun! Good article, but I feel sorry for poor Gamma.
Do you know who georgie&donny are John? I was talking to Joel Smalley on his substack and recommending your substack - which he already knew about it seems when up pops 'georgie&donny' saying 'John Dee talks nonsense' or something similar. I had a quick flick across on the link they posted and it all seemed very 'Alice in Wonderland' over there ie an inverted reality such as 'I believe transactivism has been deliberately stirred by the patriarchy to create homophobia and tension' - so I didn't bother to read whatever they had to say about you. No doubt they are fully paid up believers in the climate cult.
No idea who they are but, yes, there have been several attempts at character assassination including some solid gutter work by the 77th brigade. I may indeed talk nonsense from time to time but the data doesn't talk as much nonsense (at least it is not supposed to) and I invite alarmists to download a dollop and see if they can replicate my analyses. When invited to do such they usually resort to insults, which I take to be confirmation of the veracity of my work.
I don't think you ever talk nonsense - I think you talk perfect sense which is why I was recommending you. You present the data in ways that us non statisticians can understand. I just wondered who the trolls were - maybe UN recruits to track down us conspiracy theorists.
Remind me what counts as incomplete. When I looked at that Antarctic data I found cases where Tmax was present, but no Tmin as well as missing both data and the occasional way out of line figures (e.g missing the negative sign off). To my mind there is no point in using faulty or incomplete data but any data cleaning should leave an audit trail of the adjustments: beforedata + adjustments = finaldata. Does anybody do this? Those meticulous Germans and Austrians perhaps?
A good question. You have discovered the delights of raw station data! For this analysis complete means 100% daily capture for average temperature (Tmean), quite why Tmax and Tmin have to be different when it's the same sensor at the same station is anyone's guess. Yes indeed, there are nonsensical values sitting down in the records and in this age of techno-wizz this is unforgivable. Quite how the big names handle all this to produce those slick gridded products is another matter and I suspect they're not as fastidious as they should be.
Great stuff!
There was also a very interesting talk on Tom Nelson regarding the accuracy of global temperature measurements.
https://youtu.be/0-Ke9F0m_gw?list=PL89cj_OtPeenLkWMmdwcT8Dt0DGMb8RGR.
I am struggling to understand exactly what the 'temperature anomalies'. OK it makes sense to to compare the change in temp relative to a reference period at each station. That way we can compare a
of station at sea level with one at altitude.
However, what happens when new stations are added starting at some later date than the reference period?
Also, as you have been showing here, what happens with missing data. My limited understanding is that when data are missing, it is estimated from the 'anomaly values' at adjacent stations.
It would be great to have some sort of precise of how these temp eg HADCRUT, NASA GISS etc series are derived by the various organisations, how their adjustments are made, and whether these methods are statistically sound. Just a thought?
Bang on! The new station issue bothers me because I suspect the big names are being rather naughty. Homogenisation is the word you need - they use records from whatever stations they do have to fill in the many holes in rural areas. Since these come from the larger towns and cities it means the global record is being contaminated by the urban heat island effect. Tony Heller on YT covers the crime of homogenisation nicely in a couple of vids. I have yet to find an honest account of how these substantial problems are tackled in detail by the big organisations - they throw out glib phrases like 'statistical modelling' that could mean anything
As it so happens Tony Heller put out a new vid on homogenisation a day ago...
https://youtu.be/QQK2_kNSCnc?si=LeYDAc3QTqT18Ds9
Thanks John.
I have always been a little suspsicious of Mr Heller - fraud and tampering are strong words.
But the video does make sense - we do indeed seem to have a sparsity of long term historical data, so when error limits are applied, we really cannot say much about past global temps, whether tmax tmin or Tav anomalies.
In this video https://newtube.app/TonyHeller/rUo4RmU, Heller show the effects of adjustments. I note that these are adjustments to Tmax. This begs the questions as to what adjustments are made to Tmin and thus the much vaunted Tav values.
Seems a bit like a game of 'think of a number, any number' (just as long as it fits my model prediciton!)
Quite. My main gripe is that the big guns don't come clean on how poor the historic data really are. I can understand them trying to fill the holes and get something workable but to then place great emphasis on such a contrivance is unforgivable!
As it reminded it me of Tweedledum and Tweedledee I decided to look up "Tweedie Distribution". In stead of two fat foolish little men quarrelling over a rattle I found:
"In probability and statistics, the Tweedie distributions are a family of probability distributions which include the purely continuous normal, gamma and inverse Gaussian distributions, the purely discrete scaled Poisson distribution, and the class of compound Poisson–gamma distributions which have positive mass at zero, but are otherwise continuous."
Let's get this straight: Tweedle-de-dum's family are probably continuously distributing their very pure but normal grandmother. (But why does she need distributing?)
This has to be done with discretion because the upside-down goose (Gausse) and the Poisson fish are also involved with distributing Gamma in his compound (the poor old lady must be all in bits and in such a muddle!)
This is otherwise continuous and has zero positive mass (meaning the Fish floats in air?)
... OH! it's Michael Fish, who very famously cancelled a hurricane. That's beginning to make sense now.
Sorry, had to have some fun! Good article, but I feel sorry for poor Gamma.
"In stead of two fat foolish little men quarrelling over a rattle I found..."
Ah, got it! The rattle was in the Distributor of the truck that was supposed to take Gamma to the Goose's compound!
Do you know who georgie&donny are John? I was talking to Joel Smalley on his substack and recommending your substack - which he already knew about it seems when up pops 'georgie&donny' saying 'John Dee talks nonsense' or something similar. I had a quick flick across on the link they posted and it all seemed very 'Alice in Wonderland' over there ie an inverted reality such as 'I believe transactivism has been deliberately stirred by the patriarchy to create homophobia and tension' - so I didn't bother to read whatever they had to say about you. No doubt they are fully paid up believers in the climate cult.
No idea who they are but, yes, there have been several attempts at character assassination including some solid gutter work by the 77th brigade. I may indeed talk nonsense from time to time but the data doesn't talk as much nonsense (at least it is not supposed to) and I invite alarmists to download a dollop and see if they can replicate my analyses. When invited to do such they usually resort to insults, which I take to be confirmation of the veracity of my work.
I don't think you ever talk nonsense - I think you talk perfect sense which is why I was recommending you. You present the data in ways that us non statisticians can understand. I just wondered who the trolls were - maybe UN recruits to track down us conspiracy theorists.
Thank you. Most definitely trolls but contacts have revealed some of these are operatives. Their job is to minimise my impact.
In that case I shall make sure I recommend you everywhere I go :)
Rascal!