While augmented reality (AR) has been around in some fashion for decades, it may be the next big thing in GIS. AR takes a live view of a real-world location and adds contextual information. A familiar example would be the heads-up display in some military aircraft. AR holds the promise of even greater integration of GIS attribute data using mobile devices.
Brady Hustad, director of partner solutions for Esri partner Astadia, Inc., a global business consulting, marketing, and technology services company headquartered in Jacksonville, Florida, spoke with Keith Mann of Esri about the integration of AR with GIS in his firm’s AR framework.
Why did you decide to integrate GIS into Argis?
Understanding why GIS is integrated with Argis begins with understanding how AR, and more specifically, how most AR, works today. It breaks into two groups that I have coined as inside AR and outside AR.
Inside AR is the standard AR you see in most applications today. Any augmented reality has to anchor to the real world in some form or it is no longer augmented reality but something else. In order to augment reality, you have to know where you are in that reality. Inside AR uses markers, or object recognition, to anchor itself to a specific object, not a location. This does have some strong stories to tell, like in maintenance applications and extension to the Internet of Things, but is limited to knowing what object you are attached to and nothing else.
Argis works in what I coined outside AR or augmented reality that is anchored in the real world through standard geospatial sensor technologies. Using GPS, and then the normal AR suite of sensors, we anchor to the real world (instead of a specific object) to allow users to visualize their data wherever they are, impacting whatever they are doing there. The only way to know where data is in the real world is through location, and GIS owns data that has location.
Why did you choose ArcGIS for the geospatial component of Argis?
Ha! That’s simple—Esri owns the standards in the geospatial spectrum. Who is going to care about outside AR, or Argis? People and organizations with assets that are either hidden, underground, complex, or otherwise challenging to a crew on the ground. These people are in utilities, energy, and the public sector, and that’s who makes up a large portion of Esri users. Now, these Esri users own most of the data that would need to be seen in Argis, so if we didn’t conform to this standard, we would constantly be battling data standards as we tried to roll out the product.
But it also goes even deeper. Esri also is the technical choice. A key factor is ArcGIS Online. When you need data on a mobile platform in the field, you need the data accessible no matter where you are. ArcGIS Online opens this door by creating a secure platform created to specifically manage and distribute spatial data. This platform [the ArcGIS platform] allows us to extend its capabilities to make a truly unique visualization of that data.
When you search the web for “augmented reality (AR),” you see a lot of cool demos but not too much in the way of practical applications. Is this one of the reasons Astadia decided to create Argis? What are other reasons that Astadia created Argis?
Developing a practical application for augmented reality was definitely a driving force behind creating Argis. Augmented reality has amazing potential to be truly useful, but the market mostly currently offers applications and solutions that range from toys to marketing gimmicks.
While it’s true that we must start somewhere, the current state of AR technology offers real opportunities for paradigm-shifting applications. At Astadia, we saw an opportunity to make a new impact and jumped at it. Beyond this, Astadia is committed to bringing our clients compelling solutions that can directly impact their ROI. As part of my role in R&D, I look for solutions that not only impact the tech ecosphere but also fit our skills and abilities. AR made perfect sense to Astadia because of our mobile, GIS, 3D, and user experience backgrounds. It was a natural progression.
What were some of the issues/concerns/obstacles you were thinking about when you designed Argis?
One of the key challenges is always contextualization. When you give someone a table of numbers, it can take hours—if not days—for them to get the answers they’re seeking. Convert this information to graphs and charts and the answers may come faster. But with a map, insight can occur in seconds. You still have to figure out where you are or where the thing you care about is located. With AR, you open up your iPad and see the pipe that runs four feet below the ground in your backyard. You instantly know exactly what you need to know.
Another issue Argis aims to address is making the right data available to users when and where it’s needed. With crews working in the field, there may be one guy at an organization who’s been there 35 years and has always known that the valve they can’t see is hidden in an old shed. But when that guy retires in six months, his replacement simply won’t have that institutional knowledge. So how do you reliably transfer that knowledge to a new generation?
Today we can give teams in the field the data they need, where and when it matters. And as an added benefit, new, young crew members can share in real time everything they are seeing in the field with an expert back at the office who can help guide them. With a good system and a wearable device, the supervisor would even be able to communicate back and draw actions on the viewport of the field crew.
How do you think AR and Argis will change your business/operations? Where do you expect to see the most dramatic changes in your business?
I think people now are only getting a glimpse of the impact augmented reality will have on their world. Concepts being shown today include removing blind spots from cars (by perfectly projecting the outside on the inside) to Argis’ ability to offer what some call x-ray vision. This is going to be a new standard in visualizing data. Since Astadia aims to deliver actionable intelligence for our customers who need it, AR and Argis will go a long way to helping our customers use their data in new, more efficient ways.
The most dramatic impact I expect for the near term is the visualization of the unseen or hidden. It is an absolutely simple, but profound, use case—to be able to see what you need to see, even if it is hidden, destroyed, or missing. The ROI is clear: How much do you lose by digging up the wrong asset or wrecking something you didn’t know was there? How much time do you lose when you take the wrong tools into the vault?
Please describe the solution in as much detail as you can.
Argis, put simply, translates your spatial data into the real world, for visualization through a mobile device or wearable. In more detail, Argis takes data that has a location, and creates a 3D model of the data within a distance from the user, with a default setting to 400 meters. Argis extrapolates elevation if the data doesn’t have the elevation data. Argis then displays this data, either overlaying it on a camera display from a mobile device or on the viewport of wearable AR glasses. Argis then integrates with all the sensor systems of the device to allow the user to interact and move among their data.
Then the fun begins. Argis is built as a framework so the workflow can be directly customized to an organization’s specific needs or to a specific use case. Astadia will be customizing a few use cases but leaving the door open to more possibilities depending on the customer’s needs.
What are the coolest features of Argis?
I will break this into two sections, the first about the visible features, the second about the technical features.
The coolest features that users will see in Argis have to do with immediate integration with their data. With just a few clicks, users will have their data—any data—visible within the solution. This data will have the symbology that they set up using standard tools from Esri. Then they’ll have all the standard tools to turn data on and off, connect to the attributes, stuff like that.
What gets awesome from here is the ability to turn off GPS and fine-tune the data. One of the challenges with using GPS as a sensor to anchor you to the world is that most GPS [receivers] are not that accurate. They are accurate enough for most situations (as friends of mine say, “get you within a bucket width” of, say, a meter or so), but what if you need to be more accurate?
Well, Argis gives you that ability. If you know where a few visible assets are, you can turn off the GPS and calibrate your data to match exactly to those visible assets, giving you inches of variance. Also, if you want to be able to move and have that level of accuracy or if all your assets are hidden, the system has the ability to link with Bluetooth sensors, including GPS, so you can override with a highly accurate GPS to get the system as accurate as your data can be.
Internally, the real magic occurs. You would be surprised to know that the actual technology to display data in a 3D space isn’t that complicated. What is complicated is making sure that data is accurate, smooth, moving correctly, and staying at the correct location. Sensors in commercial devices are not perfect—they drift, they shift, and with them your data will drift and shift.
Astadia spent the time and effort in Argis to smooth the user experience, maintain accurate data, and remove as much sensor error as possible. We have a system that is now ready to truly show you your data in a new story.
Are there special requirements for using Argis?
I would love to say there are no special requirements for using Argis, but there definitely are. Whatever device you use, it must have sensors that allow you to know where you are and what you are looking at. Necessary sensors include gyroscopes, magnetometers, GPSs, good accelerometers, cameras, or view ports. Not all are needed, but these are what we can use. Luckily, most of these sensors are just part of your smartphone or table.
Additionally, Argis uses your data and can only be as accurate as the data you have. It requires solid, accurate data.
What benefits will your customers realize once they start using Argis?
Time, accuracy, and efficiency.
TIME: Anyone who has worked the field has spent hours searching for what they are working on. This can simply be eliminated. If you can get within one to two meters of what you are looking for, you can usually find it pretty quickly, whether it is covered in snow, dirt, or structures.
ACCURACY: If the asset you are to work on is highlighted in red, and the asset beneath your feet is not red, perhaps you are not at the right location. I know many field crews have found ways to stop from working on the wrong asset, but are any of those ways faster than simply looking at it?
EFFICIENCY: You can send the newer, less experienced crew and can have more cross-trained teams. You can get it done right, faster, the first time.
Describe how what you are doing is unique compared to other AR applications.
Arguably, I have seen a few attempts in the same direction as Argis. I imagine that as time passes and technology improves, more solutions in this vein will appear, as the need is obviously there. Currently, what is unique about our work with Argis is that we are fully autonomous from requiring any visible marker to which to anchor. Argis anchors to a location through GPS, not through object recognition or markers. Additionally, we are a fully outdoor-capable solution. Argis is made to solve the problems of field crews—the teams doing the “last mile” of work.
Are you planning on expanding your use of AR and GIS in the near future? If yes, can you elaborate?
Yes, we are definitely not done yet. We have this ready to be introduced into the industry, but we are continuing to improve and extend the functionality. For example, elevation is a real challenge. It is hard to get accurate elevation data and to accurately model where your data should be in that elevation. I can’t go into detail, but we are building a dynamic methodology to create accurate elevation and therefore more accurate data.
Another area we are pushing into is interior dynamic mapping and locating. A large field of interest is the ability to locate people and assets dynamically within a building in real time. We have much of this in place and are working on how to seamlessly switch from a GPS-based system to a SLAM [simultaneous localization and mapping] or other technology-based system and what devices would be able to do this.
Finally, we have the ability to handle Bluetooth sensors and web service sensors in the Internet of Things. Unfortunately, each device and sensor array differs slightly, so we have to code specifically for each one in a small way, at least. So we are working to establish solutions for more and more devices. We started with iDevices [an informal term referring to mobile or portable devices manufactured by Apple] as these are ubiquitous in the industry, cost effective, and already have a passable sensor group. We also have the Meta glasses working. We will be working through Android, Windows, and other wearables soon.
Is there anything else you’d like to share?
Currently the researchers are indicating that AR is around a $250 million industry and expected to grow to $2.5 billion. This growth is not expected to slow down over the next few years. Every major technology player has jumped into the industry—even Apple with a late play in buying Mateio [German software company specializing in augmented reality technology]. This method of visualizing your data and empowering your team will be around for the long haul and will revolutionize how you expect to do your work. I would urge you to look into this technology; this is a game changer and could give you that distinct advantage you need in your business.
