Without AI, The Forecast Is Grim

10 May 2024
Without AI, The Forecast Is Grim - Featured image

The Bureau of Meteorology’s refusal to explore advanced forecasting technologies is putting Australians at risk while stoking climate change paranoia, argues IPA Senior Fellow Jennifer Marohasy.

Australia’s Bureau of Meteorology (BoM) is ignoring more reliable new weather forecasting techniques in favour of old models. The consequences have been catastrophic for many Australians, perhaps none more so than the residents of Cairns and Lismore who have been left unsafe and with a terrible fear of climate change. Consider, for example, the climate activist who caused traffic chaos when she bike-locked herself to her steering wheel to block the Sydney Harbour Tunnel during peak hour. All charges were dismissed in court after the defence successfully argued Mali Cooper, then 22, was traumatised by the Lismore floods in 2022.

When the Premier of New South Wales handed down the three-volume 2022 Flood Inquiry report, he also made public 1,498 submissions. Many make for harrowing reading, especially the first-hand accounts from Lismore residents. There are very personal stories explaining how there had been limited flash-flooding up to 27 February 2022. Many thought the worst had passed. There was no warning from the Bureau or the local emergency services of the imminent catastrophe. As the Lismore City Council’s submission explains, the SES issued a flood evacuation for North Lismore at 9.30pm on Sunday evening, with advice to be evacuated by 10pm. To be clear, residents were given an advance warning of just 30 minutes!

The waters rose, creeping in backdoors, covering floorboards, then windowsills. Families eventually climbed into roof cavities, before being forced to break through onto rooftops. From rooftops, in the early hours of the morning of Monday, 28 February, they called out for help, into the darkness.

Eventually, of course, the authorities blamed climate change. That is easier than taking responsibility.

The submissions to the 2022 Flood Inquiry described the rain as being heaviest on Sunday evening and flooding as being worst on Monday. These dates—Sunday 27 and Monday 28 February 2022—are etched into the collective memory of the survivors. They describe the rain as like a river which fell from the sky, drenching, sheeting down vertically, with unrelenting intensity.

The rain-bearing weather system was categorised by meteorologists as stationary, that it had stalled in place.

But nowhere in the 2022 Flood Inquiry report are we told how much rain the Bureau forecasts for those days, nor are we told how much rain fell. These are important statistics if the flooding of Lismore is to be accurately characterised and understood as a meteorological phenomenon. Furthermore, accurate historical data is critical if we are to be able to accurately forecast future flooding using emerging new technologies.

FORECASTING USING AI

Over the last few months much has been written about the power of artificial intelligence (AI). An article in The Guardian on 21 January 2024 titled ‘Can the power of artificial intelligence be harnessed to help predict Australia’s weather?’ opened with a description of one of the terrible storms at the Gold Coast late last year. Artificial intelligence is everywhere, because it is good at finding patterns in the information that it is fed and using this information to decide what to do next—even what news item to serve up to you online, such as by knowing that you are more interested in politics than AI.

I actually have been surprised at how slow the adoption of AI has been, and continues to be, for weather forecasting. The official line, from BoM, has been that weather patterns are changing and elevated levels of carbon dioxide mean the climate is on a new trajectory. Therefore, AI won’t work. The Bureau has so far stuck to technology that is decades old, particularly ‘general circulation models’ that are a type of simulation like video gaming technology.

I first became interested in rainfall forecasting using AI in January 2011, following the flooding of Brisbane. My colleague John Abbot used Artificial Neural Networks (ANNs), a form of AI, for share market trading, and bought a red Corvette with the winnings one day. That same sports car drowned in the 2011 Brisbane floods.

Our Bureau scoffed at the idea that AI could be used for weather forecasting.

Over the next five years to 2017, Abbot and I published a dozen research papers on our new technique using AI for monthly rainfall forecasting: the total amount of rain likely to fall each month is very useful, particularly if you are a farmer or dam operator.

We published in the best international peer-reviewed journals, and as book chapters following AI conferences. Our very first paper about forecasting monthly rainfall (for 17 locations in Queensland, 12 months in advance) was published in Advances in Atmospheric Sciences, which is an Elsevier journal sponsored by the Chinese Academy of Sciences.

The Chinese were very interested 12 years ago when we were pioneering the technique and were prepared to publish us, while our own Bureau scoffed at the idea that AI could be used for weather forecasting.

TYPES OF FLOOD FORECASTS

Whether considering the next three hours, the next three months, or the next three decades, BoM uses the same supercomputer and the same general circulation model to forecast rainfall for locations across Australia’s landmass. The Bureau does not use AI that relies on input data and finding patterns among numbers. I generated large arrays of numbers for our AI program that included information on changing patterns of sea surface temperature and air pressure across the South Pacific Ocean going back 100 years. That data is available. It just needs to be compiled in ways that are intelligible to the AI software. I’m good at that.

Rather than compiling such data, the Bureau has what is known as a dynamic model that attempts to simulate every relevant aspect of the known atmospheric and oceanic physics. So, the Bureau uses a physical model while Abbot and I were generating statistical models with commercially available AI software.

The initial starting conditions are very important when using a simulation model and can be changed. In contrast, with AI, there are no starting conditions, just arrays that are very large spreadsheets of data. The starting condition is the most recent data, the most up-to-date numbers. Running the AI program was easy, but keeping the data up-to-date was very time consuming and exhausting.

At the time of writing, February 2024, the Bureau specifically uses the simulation model developed by the UK Met Office known as ACCESS-S2, that is also one of the Intergovernmental Panel on Climate Change’s CMIP6 general circulation models. These general circulation models are underpinned by the assumption that carbon dioxide drives climate change, and they are focused on global-scale phenomena.

Of course, the Bureau also uses these simulations to forecast temperature, which it claims it can forecast accurately to within two degrees Celsius on any day. This may be of intense political interest, but it is of limited value to much of the Australian community. Being able to accurately forecast rainfall would arguably be much more useful to larger sectors of the community.

Dr Jennifer Marohasy with a local official at the Indonesian Bureau of Meteorology in 2018.
Photo: David Lowe/Echo

General circulation models, while accurately simulating general global patterns of temperature and rainfall, struggle with high intensity rainfall events over small areas, including rainfall associated with cyclones and low-pressure systems. Yet these are the events that cause flooding, these are among the most threatening processes in real time.

The Bureau was able to accurately forecast the trajectory of Cyclone Jasper as it presented as a large and slow-moving weather system off Cairns in December 2023. But once the structure of the cyclone began to break down, and this occurred from December 9, ACCESS-S2 was unable to accurately forecast either the direction of the low-pressure system or the intensity of rainfall.

CAIRNS FLOODING, DECEMBER 2023

What happened at Lismore in February 2022, happened almost two years later in the aftermath of Cyclone Jasper, right down to the main rainfall gauge failing. So again, there is no accurate historical rainfall record; this time it is the rainfall data for Cairns that is inaccurate.

Just before Christmas 2023, friends and family who live at Holloways Beach, north of Cairns, assured me the rain was about to stop and the cyclone was over—they knew because they were listening to Bureau updates. They assured me they had pulled through with little more than the inconvenience of no electricity or running water. Then the rain started all over again. Contrary to Bureau forecasts, instead of Cyclone Jasper tracking west, the low-pressure system remained more-or-less stalled in the one place.

The beachside suburb of Holloways at the bottom of the Barron River catchment suddenly experienced terrible flooding, and again there was no real warning. This time no one climbed into a roof cavity as that would have been deadly. Neighbours evacuated each other by boat, and then part of the esplanade washed into the sea and so the water damning up behind it was released and suddenly the flooding was over.

Again, in the immediate aftermath there was a focus on how the warning systems failed, again politicians promised an inquiry. Of course, much was said in the popular press, and repeated among the local community, that such flood events are impossible to forecast and a consequence of human-caused global warming. Many repeated it was ‘unprecedented’.

Google is now using AI for weather forecasting.

Yet there have been higher rainfall totals in the past: higher daily, weekly, and annual totals. In the aftermath, nothing was said by Bureau officials about how much rain fell at the official rainfall gauge at Cairns airport because we do not know. The official rainfall gauge failed at the Cairns airport (station number 31011), just as the official rainfall gauge failed at Lismore airport (station number 058214) two years earlier. The official record shows no rain fell in either Cairns or Lismore on the days of most intense rainfall. This is a travesty that needs to be corrected.

The longest rainfall record for the Cairns catchment is at Kuranda, a little town on the top of the mountain range to the west of Cairns. That gauge did keep working through the recent downpours. Unlike more modern contraptions with ‘buckets’ that are emptied automatically, the gauge at Kuranda is an old-fashioned device that is read and emptied manually.

As the story goes, 1882 was another ‘unprecedented’ year of heavy rain. It cut the supply routes from the mining towns beyond the mountains to the coastal settlements, including Cairns. Legendary bushman Christie Palmerston was tasked to find a reliable supply route for a railway to link the rich mining area to the sea. So, the Kuranda railway was built and opened in 1891, with an official BoM rainfall record from 1896. The wettest year on record with nearly 5,000mm (4,921mm) of rain is 1911, followed by 1979 (4,657mm). This last year, 2023, is the third wettest year on record with 4,417mm. Most of the rain fell in December, when the official Bureau seasonal forecast was for drought.

AI FOR MORE SKILFUL FORECASTS

The Executive Director of the IPA, Scott Hargreaves, commented recently in Spectator Australia that:

While opinions differ on whether we are being too hard or too soft in criticising the Bureau we see around us too much application of what behavioural economists call hindsight bias, criticising an expert merely because their predictions turned out to be wrong.
But as that great Australian psychologist and analyst of decision-making Professor Leon Mann taught me 25 years ago, the more important question is whether the expert based their prediction on the best available information and decision-making processes at the time it was made. It is on this basis the Bureau’s long-standing and obdurate refusal to investigate the potential applications of AI for weather forecasting is a black mark.

Since August 2011, John Abbot and I have repeatedly offered to collaborate with the Bureau by suggesting the Bureau develop a capacity in AI alongside their general circulation modelling. I knew it would be difficult for us to commercialise our AI technique, and that the best chance for its success would be collaboration. This did happen eventually with the Indonesian Bureau of Meteorology in 2017, but not with our own BoM.

Meanwhile, Google is now using AI for weather forecasting with their GraphCast, run on a desktop, outperforming all the GCMs run on supercomputers.

Google’s GraphCast works from the same principles John Abbot and I used: recurrent cycles that can be found in weather and climate data—so long as the data hasn’t been remodelled to fit the human-caused global warming theory, and as long as the inputted rainfall data is reliable.

I have no doubt that the rainfall forecasts for Lismore and Holloways Beach would have been far superior if the Bureau had begun to invest in AI technology 10 years ago and begun to develop some capacity in this very different technique.

Aerosols can supercharge the atmosphere, making rainfall more intense.

The extraordinary rainfall associated with the flooding of Lismore and surrounds in early 2022 may have been exacerbated by the increase in atmospheric aerosols and water vapour content from the eruption of the Hunga Tonga–Hunga Ha‘apai volcano from December 2021 to January 2022. It could be that this eruption also caused a depletion of ozone in the stratosphere that continues to disrupt weather systems.

General circulation models have difficulty simulating the local impact of volcanic ash on rainfall intensity and global temperatures, and this is a problem because aerosols can supercharge the atmosphere, making rainfall more intense. Such variables could easily be included in an AI forecasting system.

At the same time there is a need to undertake significant quality assurance of the historical temperature and rainfall records before they can be used for forecasting using AI. All of this is possible, if only there was the political will.

Amy McGovern leads the US-based NSF (National Science Foundation) AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography (AI2ES). She recently commented in a feature story for the US National Academies of Sciences, Engineering, and Medicine that while AI has much potential for weather forecasting, as regards the government institutions, “I think a lot of people are just like, yeah, we see that, but we’re going to keep doing our thing”.

ACCURATE HISTORICAL DATA MATTERS

AI works because every location has a unique geography that will affect the magnitude and frequency of extreme rainfall events. Locations along the east coast of Australia, including Cairns and Lismore, are very affected by changing sea surface temperatures and pressures across the South Pacific that have been measured since the late 1800s.

The official temperature database for Australia, known as The Australian Climate Observations Reference Network – Surface Air Temperature (ACORN-SAT) is used to generate an average Australian temperature, and this database only begins in 1910.

This record only begins in 1910 because many weather stations did not record temperature in what is known as a Stevenson screen (basically a white louvred box) until about 1910. Before 1910 the mercury thermometers used to measure maximum temperatures were not necessarily kept in a standard housing, and this could result in higher temperatures for the same weather.

A Stevenson screen did not become the official housing for the thermometers at the Bureau’s official weather station in Sydney until 1910. In Melbourne a Stevenson screen was not installed until 1908, and in Brisbane a Stevenson screen was installed earlier in 1896.

Patrons of Hotel Euramo in northern Queensland were not kept away from their local watering hole by severe flooding in December 2023. Photo: Hotel Euramo/Facebook

Darwin has the longest record, with an official temperature record from a mercury thermometer in a Stevenson screen starting in March 1894. This is all documented in the online archive for Darwin at the Bureau’s website, and I have found photographs of different shelters and other instruments in the Darwin public library including a photograph taken in January 1890 of a Stevenson screen in the post office’s yard.

Charles Todd is the person to thank for Darwin’s exceptionally long, continuous, and reliable early temperature record. He was an avid meteorologist, astronomer, and electrical engineer who oversaw the construction of the Overland Telegraph line connecting Darwin with Adelaide that was completed in 1870. That was the same year Todd became Australia’s first Postmaster-General.

After the completion of the Overland Telegraph, telegraphic officers in South Australia and the Northern Territory were required to report temperatures, barometric pressure, and rainfall on a daily basis to his West Terrace Observatory in Adelaide.

Perhaps as unexpected as the exceptionally long continuous and reliable temperature record for Darwin, Darwin also has the earliest reliable atmospheric pressure measurements. So, the Southern Oscillation Index (SOI) is still measured as the pressure gradient difference—not between Brisbane and Tahiti or Sydney and Tahiti—but between Darwin and Tahiti. These SOI values (expressed as an index) are still derived from the 1887–1989 base period, with the first 10 years of measurements part of the network established by Charles Todd, and still, to this day, updated daily by the BoM.

Changing daily patterns in the SOI were incorporated into the statistical models that John Abbot and I used to forecast monthly rainfall for locations on Australia’s east coast.

There will obviously be problems if rainfall has not been accurately recorded for the location of interest—if the historical record has been corrupted—because the AI will be considering the rainfall total, relative to pressure and temperature gradients and pressures across the Pacific, including at Tahiti.

AI is only as good as the data inputted; AI forecasts are only as good as the data provided for model building, and then for training the model that will be used to make the forecast. Training essentially involves running segments of data to give the model some idea of what to expect. In this regard, AI is like human intelligence: it can get better at anticipating what will happen next, if it is given some practice and good data (reliable information).

AI is like human intelligence: it can get better at anticipating what will happen next.

After the Lismore flooding, there was a Special Climate Statement 76 – Extreme rainfall and flooding in south-eastern Queensland and eastern New South Wales published by the BoM on 25 May 2022, replete with regional distribution maps of rainfall across broad geographic bands and with intervals of varying quantity, with the highest value of 200mm to an unknown total amount. In fact, the rainfall totals were much higher than this.

The ‘special climate statement’ claims there has been an increasing trend in extreme hourly rainfall, but no actual data is provided: the reference is to State of the Climate 2020. I have been through that report. It provides no actual measurements. The ‘special report’ was intended as a formal record of the extreme rainfall, yet there was no actual assessment of the data from that period and no mention of the 24-hour rainfall total for Lismore for 28 February.

In fact, the Bureau’s main rainfall gauge at Lismore Airport failed on 28 February. The Bureau has never acknowledged this, not in the special statement nor in the Flood Inquiry report. This will be a problem for training models in the future.

The Bureau has a network of gauges for flood warning, including at Dawson Street, Lismore. This data is not publicly available beyond the week in which it is issued as a daily total. A screenshot shows 467mm was recorded at the Dawson Street gauge by the Bureau for the 24 hours to 9am on 28 February.

Yet the Bureau in its monthly summary for February 2022 claimed a new 24-hour rainfall record for Lismore on Thursday, 24 February, of only 146.8mm. It is the case that 146.8mm fell at Lismore airport during the 24 hours to 9am on that Thursday, but this was not the main event, and it is not even a new record. The previous 24-hour record for Lismore was 334.3mm that fell at Centre Street in Lismore on 21 February 1954.

For further reading, please download Dr Jennifer Marohasy and Chris Gillham’s IPA research report Are Extreme Rainfall Events Increasing In Frequency?, which formed the basis of a submission to the NSW Flood Inquiry in May 2022. https://ipa.org.au/research/ipa-report-are-extreme-rainfall-events-increasing- in-frequency


This article from the Autumn 2024 edition of the IPA Review is written by IPA Senior Fellow Jennifer Marohasy.

Support the IPA

If you liked what you read, consider supporting the IPA. We are entirely funded by individual supporters like you. You can become an IPA member and/or make a tax-deductible donation.