Access to data is as important as ever. In the GIS field, data has traditionally been accessed through file system-based methods, such as coverages, shapefiles, and file geodatabases, and through direct connections to relational enterprise geodatabases on engines such as Microsoft SQL Server, PostgreSQL, and IBM Db2. These methods of connecting to data served the needs of a desktop GIS world for decades, but then in the mid-1990s, the information superhighway came along, and everything changed. By the early 2000s, a new era of geospatial data dissemination was dawning: the era of Web GIS.
What Is an API?
The topic of application programming interfaces (APIs) is very broad. APIs enable applications to easily and securely exchange data and functionality. Today, the term API typically refers to web APIs. They are a modern approach—using the internet—to provide access to machine-readable data in formats such as JSON or XML, usually over HTTP. Simply put, APIs make it possible for digital services and applications to easily talk to each other.
This article introduces the topic of APIs, why they are important, how they are used in GIS, and why they should be tested. Building and executing tests is a broad topic that is beyond the scope of this article. It could easily be the topic of an entire book. However, this article introduces high-level examples of tests that will (hopefully) engage and interest the reader to pursue more information on the topic.
Why Are APIs Important?
Arguably, APIs have been around since the earliest days of computing. Modern APIs began their rise to stardom in the early 2000s with the commercialization of the internet by companies such as Salesforce, eBay, and Amazon. These pioneers used APIs to make their products and services available to their customers on a single site while also providing partners and third-party resellers with the ability to interact with and extend their platforms. Fast-forward to today and APIs are the backbone of virtually all the digital services people use daily. APIs power desktop applications with data, fetch data from databases for websites, provide the building blocks of mobile applications, and allow for the Internet of Things (IoT) to exist.
Where Do APIs Fit In with GIS?
This brief introduction to APIs highlights the value of APIs in disseminating data. Currently, the world is dependent on the fast and reliable delivery of data. GIS is no different. By effectively eliminating barriers to data dissemination, APIs allow the outgoing delivery and sharing of all types of data both inside and outside an organization while allowing data from outside organizations to be consumed and utilized.
In the late 1990s to early 2000s, GIS data sharing within an organization typically occurred in the form of a digital or printed map. Outside an organization, datasets were either emailed (if they were small enough), transferred via FTP, or transferred using physical media such as CDs or ZIP disks. This was, at times, a painful process.
There were predecessors to Web GIS software, such as ArcIMS. However, in 2004, ArcGIS Server was released with ArcGIS 9.0 and it revolutionized the way GIS data was used both inside and outside of organizations. Web GIS powered by ArcGIS Server allowed maps to be published as services that could be distributed via an API and consumed in web mapping applications that were viewable in a standard web browser. Since then, products such as ArcGIS Enterprise rely on internal APIs for all their constituent parts such as Enterprise portal, ArcGIS Server, and ArcGIS Data Store. APIs provide the foundation for modern Web GIS. Nowhere is the use of APIs more prevalent in ArcGIS Enterprise than in the ArcGIS REST APIs.
Other Uses of APIs in GIS
APIs are not limited to those that come out of the box in commercial off-the-shelf software such as ArcGIS REST APIs or the ArcGIS API for Python. APIs can be built from scratch to serve up virtually any type of data from any source. Although the ArcGIS REST APIs are incredibly powerful and can often serve the needs of most organizations, sometimes additional capabilities are required. In these cases, a custom API can be written in languages such as Java, .NET, or Python and deployed either on premises or in the cloud using methods such as Koop, Flask, AWS Lambda, Azure Functions, or application containerization with Kubernetes. Building custom APIs is a broad topic, and the reader is encouraged to research it further.
Testing APIs
What is API testing? Simply put, testing an API checks its functionality, reliability, performance, and security to ensure that it works as intended. For example, if an ArcGIS Enterprise feature service is configured in a search widget within a web mapping application, users can enter text in a search bar to get results that are displayed on the map.
How can this endpoint be tested to ensure that, for a given search term, the user will get back the intended and correct results? Searches can be manually executed by a human using the web application search bar, but that’s not very efficient. A more efficient, repeatable, and faster method would be to test just the API’s REST endpoint by submitting known requests and ensuring that the responses are as expected. This is an example of a smoke test.
Smoke Testing
The term smoke testing has an interesting history. Some say it originated in electronics and computer hardware development, where a board passed a smoke test if it was plugged in, and it didn’t start smoking and catch fire due to failure. However, the plumbing industry has long conducted smoke tests on systems. Smoke is put into pipe systems and used to find leaks in the system.
Regardless of where the term came from, it is easy to see how it transferred to software testing, specifically API testing. Smoke testing an API checks that its basic functionalities work and that it performs as intended. It allows major errors and flaws to be quickly discovered. Smoke tests are some of the simplest of tests. They are often performed in the postdeployment of an API, sometimes from within continuous integration/continuous deployment (CI/CD) pipelines. Once API functionality has been tested and validated, it’s time to put stress, or load, onto the system.
Load Testing
In load testing, demand—in the form of expected usage—is put on a software system. The response of the system is measured to gauge performance and ensure stability before deployment. Unlike smoke testing, which tests whether something works or not, load testing:
- Helps determine if current infrastructure is sufficient for the expected load.
- Tests if the system is sustainable under peak load.
- Allows for concurrent user simulation.
- Allows for scalability testing.
Why Testing Is Important
Generally, testing any software system can determine whether it works. However, works is a vague term and doesn’t fully describe levels of functionality, and performance. Smoke testing checks for basic functionality while load testing can assist in assessing performance.
Software development continues to evolve and progress at a rapid pace. Developers—like everyone else—are pressured to do more with less. As a result, no one has time for improperly functioning or broken software systems. Testing software, including APIs, identifies flaws and bottlenecks that can be addressed before deployment to other systems and end users.
How Testing Can Be Used in GIS
Smoke tests check for basic functionality. Often this is done by testing that a particular API request returns the intended and correct response and ensuring that the API is functional and provides the correct data to applications and end users. This can be accomplished through a variety of methods, ranging from simple manual testing in a web browser to scripted and automated testing.
Chances are that many readers have done manual smoke testing in a browser without even realizing it. If you have ever gone to a /query REST endpoint of an ArcGIS Server map or feature service and submitted a query to see if the expected response is returned, you have conducted a simple manual smoke test. For quick ad hoc tests, that is fine, but there are tools that can help make this task easier and more efficient.
There is no shortage of API smoke testing tools available. Some tools are open source, but most tools have free or multiuser and enterprise pricing plans. From a smoke testing perspective, these tools can help organize, parameterize, and automate tests. Many packages are designed around collaboration and allow tests to be shared among users. Some popular smoke testing tools include Postman, Katalon Studio, and SoapUI.
A Smoke Testing Example
For example, when testing a web application that searches a point feature layer of US cities and zooms the map into the area in which the city is located, the where query might look like Listing 1.
Note that the WHERE clause has been URL encoded from where=NAME=’seattle’ to where=NAME%3D%27seattle%27, because browsers don’t like special characters in URLs. When the query in Listing 1 is executed, the JSON response returned should resemble Listing 2.
In a smoke testing framework such as Postman, assertions can be executed against response content. For example, an assertion can test to ensure that there is a feature object within the response and that within features there are x and y geometry values. When automated, smoke tests with assertions are powerful tools that can be used to routinely validate the functionality of a REST API.
Load Testing
While smoke testing verifies API functionality, load testing puts the system under stress to either mimic production user load or exceed user load to test performance, scalability, and robustness. Load testing can also be used to push a system to its limits to see if, when, and why it breaks. This makes load testing very different from smoke testing. Creating effective, realistic, and useful load tests is somewhat of an art form, and it takes experience to master.
Load Testing Tools
There is a plethora of load testing tools available. One of the most popular is Apache JMeter, which was initially released in 1998. It is open-source, platform independent, and highly documented, so it is an ideal tool for both beginners and experts.
In its simplest form, a load test might simulate requests to an API at certain rates over time to stress the system and see how response time is affected. More complex load tests might include simulating users logging in to a site and performing many user actions such as filling out forms. User counts can be ramped up and down to simulate peak usage times.
A Load Testing Example
Use load testing to measure the performance of querying a hosted feature layer on a company’s ArcGIS Server instance under user load. Network services, such as routing and geocoding, can also be load tested. Smoke testing or very light load testing with a minimal number of users simulated can provide an idea of optimal response times. Load testing with a higher number of simulated users can help assess performance and determine if current infrastructure can sustain acceptable response times under the stress of additional users.
With a load testing framework, such as JMeter, requests can be configured as tasks to be run by simulated users. For example, it can be configured to simulate a query to a hosted feature layer REST endpoint to measure performance when 50 users are sending one request per second over a 30-minute period. Assertions can be utilized in load tests to ensure that responses meet expectations in both content and response time. Detailed reports can be generated to help analyze results and determine where potential weaknesses exist within a system.
Integration with Development Pipelines
Smoke tests can be integrated into many CI/CD pipeline systems such as those available with Amazon Web Services, Microsoft Azure, or even Jenkins, although for many smaller organizations this may be out of scope. In a CI/CD pipeline, smoke tests can be run after code deployment, ensuring that no unintended consequences arise. The code referenced here does not necessarily have to be API code. It can be code that calls or is dependent on an API. If the smoke test fails, the pipeline can even be configured to roll back the deployment to a previous working version, thus minimizing user impact and disruption.
Best Practices
API test suites can range from simple to incredibly complex. Regardless of the type of tests being performed or their complexity, there are a few simple best practices that can always be followed.
Perform negative tests along with positive tests. While it’s important to know that the API returns correct, valid responses as expected, it’s also important to know that it should fail when expected. Test to ensure that invalid requests yield the proper expected responses.
Smoke test early in the development process. The earlier testing begins, the sooner bugs can be caught and dealt with.
Always try to simulate production conditions. Although this might seem like a given, simulating production conditions and/or loads can be very challenging. It is easy to get caught up in configuring clever tests and lose sight of the bigger picture. Don’t fall into this trap.
Save and track responses. The requests made are important, but so are the responses. As an API matures and changes over time, so will requests and responses. Requests are easier to track, as they are often embedded in the tests. Responses are not, and they are ephemeral and easy to forget. Save responses with their related tests.
Enforce service-level agreements (SLAs) during testing. If an organization has defined SLAs around API response times, those should be enforced during testing to ensure response times are within acceptable ranges.
Think like a consumer. Don’t waste time creating tests that aren’t realistic. Try to mimic how user actions will impact the requests made to and usage of the API.
Conclusion
Access to data today is paramount in any system, regardless of size. In a Web GIS, that access happens via APIs. Knowing that APIs are properly deployed, configured, and tuned ensures that users get the data they want, when they want it. Smoke tests verify that an API is returning expected results. Load tests assess performance and scalability. Having a suite of tests available to run on an ad hoc basis—or perhaps on a set schedule—to monitor functionality and performance will keep the APIs that power applications across an organization healthy, performant, and reliable.
For more information, contact Chad Cooper.
About the author
