BoM Blast for Dubious Record Hot Day

IN September 2017, the Australian Bureau of Meteorology (BoM) claimed a series of new record hot days across south eastern Australia, including on 23 September at Mildura. At that time, the mainstream media reported this as a new record for the state of Victoria, specifically claiming it was the hottest September day ever recorded – all the way-back to September 1889. This claim, however, cannot be verified because the BoM uses a non-standard method for recording temperatures at Mildura, and furthermore the parallel data provided to me in December 2017 as proof of equivalence is flawed and deficient.

Background

On 23 September 2017, a new record hot day for Victoria was claimed at the Mildura airport using an electronic probe in an automatic weather station (AWS) housed in a Stevenson screen.

The BoM claims that measurements from such devices are ‘comparable’ to measurements from traditional mercury thermometers, which were used to measure official air temperatures at Mildura from 13 June 1889 until 1 November 1996.

There is no documentation, however, supporting this contention for Mildura or any of the other nearly 500 AWS spread across the Australian continent. Furthermore, the BoM does not have World Meteorological Organisation, or any other form of accreditation (i.e. ISO 17025) for any of its AWS. In addition, the BoM uses a non-standard method of recording temperatures from such devices. Specifically, while one­-minute averaging of one­-second readings is standard across the world (e.g. in India, UK, Germany, Holland, Switzerland, Saudi Arabia and Kuwait) the BoM records the instantaneous highest one-second readings from a probe as the maximum temperature for that location for that day.

Given electronic probes generally respond more quickly to fluctuations in air temperature than mercury thermometers, it follows that this method – instantaneous recordings from an electronic probe – would result in new temperature maxima under the same weather conditions. The BoM, however, claims that temperature measurements from electronic probes are nevertheless ‘comparable’ with measurements from mercury thermometers because the BoM’s ‘purpose-designed’ probes ‘closely mirror’ the behaviour of liquid-in-glass thermometers, including the time constant.

While this is theoretically possible, to know if it is being achieved in practice it is necessary to analyse parallel measurements i.e. data from an electronic probe and mercury thermometer operating side-by-side for a period of time.

While the BoM has never released reports with parallel data supporting the claim of equivalence, in late October and early December 2017 a first and second lot of A8 forms were released to me – this followed my request to Minister Josh Freydenberg on 26 September for parallel measurements, and more specifically on 22 October for these A8 forms… immediately after I was informed by a whistle-blower that these forms contained the relevant information.

Parallel data from Mildura – preliminary findings

After I manually transcribed and analysed relevant data from a subset of the first batch of over 4,000 scanned A8 forms received on 28 October, I wrote to Minister Freydenberg on 12 November explaining that the values recorded manually on the A8 forms from the mercury thermometers for the period November 1996 to December 2000 at Mildura are significantly different from the official values recorded from the electronic probes.

Just considering the values for September, the mean difference is statistically significant at the 0.05 level of probability, and is +0.34 °C, +0.27 °C and +0.28 °C for the years 1997, 1998 and 1999, respectively. Somewhat surprisingly, the automatic weather stations at Mildura for these three Septembers recorded statistically significantly cooler temperatures than the mercury thermometers.

Analysis of the second lot of forms, received on 4 December, has proven more problematic because of the quality of the available data, and absence of critically important data.

The electronic probe that measured the record hot day on 23 September 2017 was installed on 27 June 2012 and I was initially told that there was parallel data only available through until January 2015. So, there is no reading from a mercury thermometer for Mildura for 23 September 2017.

After scrutiny of the A8 forms actually provided, however, it appeared that the extent of parallel readings for the probe installed on 27 June 2012 would be limited to just the eight months July 2012 to February 2013… except that the BoM had omitted to scan September 2012 – the one month that could provide a direct measure of the equivalence of the relevant probe for that time of year at that location. After informing the BoM of this omission, I have been told it is being looked-into… that the relevant officer will follow-up on the missing month of data.

Meanwhile, if we consider the residual available parallel data… the very hottest days according to readings from the electronic probe (30 November 2012, 18 January 2013, 5 January 2013, 8 January 2013, 6 January 2013, 1 December 2013, highest to lowest) have no equivalent reading from a mercury thermometer. In short, it appears that on the hottest days in Mildura – during the period that manual readings were being taken after installation of the most recent probe – no one was turning-up to take the manual reading from the mercury thermometer. As a consequence, the data for this period from the mercury thermometer is not normally distributed, as shown in Figure 1. This makes statistical analysis using standard techniques impossible as assumptions implicit, for example in a standard paired T-test, are violated.

There are many more maximum temperatures measurements available for the electronic probe (n=948) than for the mercury thermometer (n=115), and the distribution is quite different, with a somewhat more normal distribution for the probe data, as shown in Figure 2.

Fig 2. Distribution of measurements from the electronic probe recording in an AWS.

Considering days when there is parallel data available in the temperature band of interest (the claimed-record hot day in September 2017 measured 37.7 degrees Celsius) the new probe has been found to measure up to 0.4 degrees hotter (e.g. 26 February 2013 the recording for the probe is 37.3, while the mercury thermometer recorded 36.9 on the A8 form). In fact, Table 1 shows that for the last month of available parallel measurements the electronic probe (Tmax-Probe) often recorded considerably warmer than the mercury thermometer (Tmax-LIG).

Tbl 1. This screenshot of the Excel file where I record the transcribed values shows that for the very last month of parallel recordings at Mildura, the electronic probe was often recording hotter than the mercury thermometer by up to 0.4 degree Celsius. (The abbreviation LIG stands for liquid-in-glass i.e. mercury and alcohol thermometers. Also, note that the measurements/data points shown here are as recorded on the A8 forms that are one full day different from the values in the CDO and ADAM databases… this is because the actual recording is of the temperature the afternoon before.)

In conclusion

While it is official BoM policy to ensure that there is approximately five-years of overlapping parallel data when there is a site move or equipment change at an official weather station, this policy appears to be rarely implemented. Indeed, while it would seem reasonable to assume that there would be dozens of reports detailing the results from such parallel studies – none have been made publicly available.

In the case of Mildura, the quality and length of the available parallel data makes it difficult to draw any real conclusions about the equivalence of measurements from the electronic probe installed in July 2012, with measurements from earlier probes and/or the mercury thermometer first installed back in 1889.

The issue of verifying the claimed record hot day on 23 September 2017 is compounded by the BoM’s method of measuring temperatures – in particular the absence of averaging over at least one minute which is standard for electronic probes.