One of the modern computer’s first killer apps was predicting the weather.
John von Neumann, who built the initial ENIAC computer, became fascinated with predicting weather in the 1930s. He called it “the most complex, interactive, and highly nonlinear problem that had ever been conceived of.” In 1948, he assembled a team of meteorologists to create a mathematical model that would describe what weather would occur based on conditions in the atmosphere. These first weather predictions took more than 24 hours to compute, but proved the idea was possible—and that we needed faster computers.
Today, governments and corporations constantly run data through more than 150 accepted weather models to forecast conditions around the globe. But the field is still changing. The ability for machines to learn from incorrect forecasts and adjust weather models themselves has been studied since the 1970s, but has only become practical in recent years due to an explosion in access to computing power.
The Weather Company, owned by IBM, currently uses machine learning to make 2.2 billion forecasts every 15 minutes. The system can access 162 weather models, taking data from satellite imagery, weather stations and weather balloons, and then adjusts the forecasts based on how accurate models are in certain locations at that time of the year, and a litany of other variables. When comparing the outputs of the machine learning models against the traditional models, machine learning fares 10-20% better when predicting the track of weather and up to 30% better predicting the intensity of hurricanes.
“If you’re expecting a human to derive a point forecast all over the globe, that’s a lot of locations and a lot of time. It’s an information overload,” says James Belanger, a senior meteorological scientist at The Weather Company. “We’ve used machine learning to allow humans to focus on areas of their particular expertise. For instance, our team is probably spending more time on Irma [than other forecasting] given the uncertainty in the forecast.”
Events that happen in a location only a few times a year, like hurricanes, would be considered edge cases for machine learning algorithms—data that doesn’t fit the typical pattern. Machine learning doesn’t operate well on these edge-cases, instead relying on massive amounts of data to make simpler decisions. But since forecasters have already been building separate models of how hurricanes act for decades, alongside the global weather models, the Weather Company’s machine learning can operate those models, avoiding this problem that plagues other industries, like autonomous vehicles.
From that baseline, Belanger says the company runs a whole bunch of possible scenarios: the hurricane will run west, it’ll peter out, it will gain strength in the ocean for longer. Then, an algorithm is used to find the most likely outcome based on how we’ve observed hurricanes to operate in the past, given similar conditions. This algorithm, known as a Monte Carlo tree search, is the same kind of approach used by DeepMind’s AlphaGo to beat top professionals at Go, a board game with more potential moves than atoms in the universe.
You know the cone that you often see with hurricane predictions, indicating that the storm can take a wide path? The Weather Company’s approach is able to narrow that cone based on its AI’s confidence, giving predictions over 10-15 days, compared to the 5-day forecast typically given by the US government.
The company can also predict how it will affect power outages to help utility companies prepare for the storms, by looking at how the storm’s estimated wind and precipitation has historically correlated to power outages.
“It helps the utility get ahead of the situation by forecasting the impact in advance, and helping them to plan what level of resource they need to have available so they can mobilize quickly and get people back up on their feet quicker,” Maia Sisk, general manager of new B2B markets at The Weather Company, told Datanami earlier this year.
The Weather Company typically sells these forecasts to businesses, as Belanger says the company doesn’t want to contradict national weather services.
While machine learning has taken over a large portion of running the models, meteorologists still play an important role. Instead of figuring out what the weather will be, they’re able to work on more applied problems such as nailing down how hurricane conditions will affect infrastructure or cause streams to overflow and flood.
“You can envision a day where maybe forecasters are working less on the track of the storm and more on the impacts, and communicating those impacts to the public more than they are right now,” Belanger says. “Humans are still critically involved.”