The scientific method is getting quite a bit of attention these days. How scientists work and produce results is under widespread scrutiny.
Of course, scientists have conducted innumerable studies that have followed the appropriate process of testing ideas and theories via observations and experiments and then verifying the results. But some individuals, from various scientific disciplines, have made up data, executed false experiments, and followed biased reasoning—all resulting in irreproducible results.
A Dutch psychology professor, for example, published several experiments that attracted a lot of attention. One allegedly proved that trash-filled environments can lead people to have racist tendencies. In another experiment, he claimed that eating meat made people less social and more selfish. Both experiments, however, were based on fabricated data and dubious research methods.
It is claimed that behavior like this happens because of the heavy pressure put on scientists to publish their work—that this is what stimulates carelessness and even fraud.
Two keywords that go with the scientific method are reproducibility and replicability. Reproducibility refers to using the same data with the aim of getting the same results. It requires original data and an overview of all the processing steps and results. Replicability refers to using new data but following the same workflow, as closely as possible, to get similar results.
I thought it would be an interesting exercise to see how using the scientific method would affect cartography—principally because if five professional cartographers are given the same dataset to visualize, they will produce five different map designs that could all be classified as good. So how do reproducibility and replicability work here?
To answer that question, it is best to start by looking at the definition of the discipline. The International Cartographic Association (ICA) defines cartography as the art, science, and technology of making and using maps.
The science part of cartography is related to gaining a systematic understanding of the data that needs to be mapped. It also involves taking into consideration the primary objective(s) of the map and who its potential users might be. But let’s concentrate on the data analysis first.
In short, a cartographer needs to establish the character of the data: is it qualitative or quantitative? The mapmaker also needs to define the perceptual properties of the symbology that will be used. For instance, with quantitative data, the symbol should evoke the perception of amounts. This indicates that the symbols should be variable in size and able to be applied in relevant map types. Because this part of the workflow is based on fixed rules and guidelines that can be found in any cartographic textbook, it is very likely that it can be executed in a reproducible way.
Why, then, do we still get five different map designs? Well, because of the art part of cartography—the cosmetics of the map.
The reproducible theory stops at a point symbol that changes size. Nobody tells us if the symbol should be rectangular or circular in shape. Sure, there is literature explaining which shapes work best for which situations. But this does not yield clear, definitive results.
So out of the five cartographers, one might select a circle, another a square, and yet another might opt to use a pictorial symbol. Even if somehow they all choose circles for the symbology, each cartographer still has enough artistic freedom to make them all different. One might make them bright red, green, and orange circles with shading. Another might use pastel colors without gradients. The options are unlimited, yet they could all result in forming a good, communicative map.
Does this disqualify cartography as non-reproducible? Personally, I do not think so. The science part of the cartographic workflow—the cartographic data analysis—is reproducible. The artistic part of the workflow is not 100 percent reproducible, but one’s choices can often be explained.
Of course, given the context in which the map will be used and the message it’s supposed to convey, cartographers can still deviate from the expected artistic logic. Assuming most people associate green with good and safe, what if a cartographer chooses green colors to indicate polluted areas or places with high unemployment rates? In theory, the choice is not wrong. But from a moral standpoint? In other words, if we see a map like this, can we still reason why the cartographer made these artistic choices?
If we look at the use part of the definition of cartography, then cartographers should be very much involved in usability research to judge whether or not their map designs make sense. Here is where reproducibility and replicability play a significant role.
But the protocols of usability research are still in full development. Usability experiments have proven to be difficult to reproduce, not only because the authors of research papers don’t describe the conditions well enough, but also because the original observational data is rarely still available. Questions abound on how to reproduce similar test environments, which maps should be used, and who should take the tests.
These days, researchers are obliged to keep their data from all steps of the research process. For cartography, that means cartographers should describe how they applied the scientific method when making the map and document why they made the design choices they did (e.g., why the orange circles?).
This would help validate the scientific elements of cartography. On the other hand, it introduces another burden—that of the time-consuming process of putting together detailed procedural workflow descriptions, similar to writing extensive metadata descriptions. That is a topic we will have to keep an eye on.
