Earth’s ever-changing weather patterns are deceptively simple to explain. The sun warms our planet unevenly (think of the hot and humid equatorial regions versus the frigid north and south poles), and that causes weather. Our atmosphere redistributes this heat, creating high and low areas of pressure throughout the world. This results in the development of clouds, winds, precipitation, and widely varying temperatures.
Because of the size and complexity of these weather patterns, they can be very difficult to predict more than a few days in advance. So researchers continually refine their instruments and processes to better understand the climate and make more accurate weather forecasts.
Today, with an abundance of satellites and remote-sensing devices monitoring weather systems all over the world, meteorologists have more data available to them than ever before. But more data doesn’t necessarily translate into improved predictions, which is why Esri partner Weather Decision Technologies (WDT) uses advanced GIS to better organize and analyze this big data.
Advances in Weather Surveillance
Weather has always had an enormous impact on our planet and the things happening around it. Monsoons reportedly hindered the Mongols from invading Japan in the thirteenth century, and winds certainly played a role when, in 1588, the English defeated the Spanish Armada, since the powerful flotilla could only sail with the wind at its back. Clouds spared the Japanese city of Kokura from being the target of the second atomic bomb in 1945, and over the last few years, drought in the western United States has affected food production and prices all over the country. That is why monitoring the weather has always been imperative.
During the Renaissance, inventors created various instruments to measure local weather phenomena, including temperature, humidity, and atmospheric pressure. When Samuel Morse developed the telegraph in the 1800s, local weather observations and measurements were connected in a rudimentary weather surveillance system. The secretary of the Smithsonian Institution at the time, Joseph Henry, envisioned that the telegraph would “furnish a ready means of warning the more northern and eastern observers to be on the watch for the first appearance of an advancing storm.” By the end of 1849, 150 volunteers throughout the United States were helping to make that a reality by regularly reporting weather observations to the Smithsonian. That’s when the science of meteorology emerged.
When computer technology was introduced to weather forecasting in 1950, a group of meteorologists at the Institute for Advanced Study in New Jersey produced the first numerical weather prediction. This ushered in the era of modern meteorology and greatly expanded research opportunities.
To better understand developing weather patterns, however, researchers needed satellite imagery. This came in 1964 with the introduction of the Nimbus program. These satellites were specifically designed for meteorological research and paved the way for most of the earth observation satellites launched by the United States over the last 30 years. The satellites provide tremendous amounts of data.
Analyzing Clouds in the Cloud
All the data from satellites and remote-sensing devices found both on land and in the oceans is greatly helping to improve weather forecasting services.
“The amount of data that we collect is enormous, about one terabyte per day,” said Matt Gaffner, GIS solutions expert at WDT. “Over the years, we have assembled an archive of almost half a petabyte of weather data.”
WDT provides weather forecasting and mapping services to a number of industries, including energy corporations, to help them predict electrical outages and keep offshore oil rigs safe; agriculture agencies for crop insurance; freight transportation companies to aid with route design; and concert and sporting event organizers for planning and safety. The company currently uses ArcGIS Server to develop all its map services for customers, though it plans to upgrade to ArcGIS Enterprise later this year.
“One of the great things about the ArcGIS Server platform is that it’s extremely easy to publish live, dynamic, rapidly updated data and then host it as a service for use with other applications,” said Gaffner. “This allows our users to quickly add past, present, and future weather data to their maps and apps.”
It also enables WDT’s partners that build apps for specific vertical markets, such as utilities, to add weather data to their apps using WDT’s map services.
Because of the quantity of data WDT collects and analyzes, cloud computing is key to its operations. The company uses Amazon Web Services (AWS) to deploy the analytical and mapping services provided by ArcGIS Server.
“There are so many advantages available to us by using Amazon Web Services,” Gaffner pointed out. “For one thing, it allows us to implement the ‘fail faster’ mantra. Because ArcGIS Server has an Amazon Machine Image (AMI) capability, we can easily stand up a version of the server in the cloud and try something new—like using different machine hardware specifications or configuring data services differently—to see if it works or not. If it works, great. If it doesn’t work, then we stand up another instance and try something else.”
With cloud services, the company can stand up multiple machine instances of ArcGIS Server to determine the operational stack that runs the best.
“AWS provides us with reliability because if something goes wrong, we can easily replicate ArcGIS Server on another machine in the cloud,” said Gaffner. “It also provides us with load balancing—that is, we can redistribute the many requests we receive for weather data and map services between our servers. This extra demand normally happens when the weather changes and storms begin to develop. Our datasets increase in size then because there is more radar data accumulating and our customers need access to that data.”
WDT is also currently doing a lot with time-enabled map services. For example, users can loop through the last 60 minutes of radar data to see where a storm has been and where it is headed. WDT is also launching a time-enabled global forecasts map service that will provide both daily and hourly forecasts for all the normal weather variables—such as temperature, precipitation, wind speed, and direction—out to 10 days.
Because the company is serving out bigger datasets to accommodate more requests, using cloud services makes WDT’s servers scalable at critical times.
Environmental Conditions Affect Business—and Decisions
WDT’s ever-growing collection of big geospatial datasets allows users to conduct unique analyses across time and space. For example, one of its customers—an oil and gas company—wanted to determine how much the weather affected the productivity of its crews in Oklahoma versus Colorado.
“They wanted to figure out the productivity of each, but they wanted to statistically normalize that comparison by taking into consideration the local weather conditions,” explained Gaffner.
During the summer, it could be 90 degrees in both Oklahoma and eastern Colorado, but with more humidity in Oklahoma, the heat index would likely be higher than in Colorado.
“By using historical weather data,” said Gaffner, “we were able to perform this analysis and found that weather can impact crews in two ways: decrease worker efficiency under heat stress and push the heat index above a critical threshold where workers are required to take mandatory breaks.”
WDT is also considering the impact that the Internet of Things (IoT) will have on geospatial analysis.
“Take, for example, the connected car,” said Gaffner. “We can provide real-time weather or a weather forecast to your car that might tell you, ‘Hey, you should probably stop driving to avoid that storm,’ or, ‘There’s a line of thunderstorms moving through your area; you might as well stay at work for another 20 minutes and wait until it passes through and then drive home.'”
If weather and environmental data are thus combined, they can help mitigate risk and enable people to make smarter decisions. And, as Gaffner hopes, the confluence of big data with smart analysis can save lives and property in the long run.
Get more information about WDT, and learn more about Esri’s big data and IoT solutions.
