Consumer movement data helps businesses understand their customers and more effectively deploy marketing strategies. Understanding how customers are attracted to locations such as hotels, restaurants, theaters, museums, shopping malls, and tourist attractions can improve segmentation and personalization strategies.
This data can include not only the types of visitors but also the frequency of visits, and overall movement patterns. Also, it is critical that this data is analyzed and reported on in a timely fashion to support decision-making.
JAKALA, a marketing technology company headquartered in Milan with offices worldwide, specializes in marketing and sales with a focus on artificial intelligence. By utilizing the power of location intelligence and the analysis of consumer movement data, JAKALA helps more than 900 clients in more than 30 countries unlock their potential by uncovering new opportunities, empowering decision-making, and enabling sustainable growth.
Gaining a comprehensive understanding of consumer movement demands a significant volume of data, and processing this data can be time-consuming. JAKALA needed a way to efficiently analyze large volumes of data so that the company could provide valuable and timely insights for its clients. JAKALA faced several challenges around scaling up its analytic workflows to deliver quality data and analytics on the timeline needed by its clients.
Data volume and analytic complexity present a challenge when performing big data analysis. JAKALA works with large-volume people movement data that the company sources from a vendor that amalgamates opt-in location tracking. The sourced data relies on more than 10 million devices, generating around 1 billion data points per day. For monthly analytics, this typically means 30–40 billion data points that need to be incorporated in the analyses.
The size of the data made it so that many geospatial analysis tools could not handle the entire dataset at once. The JAKALA team had to subset the data into weekly segments to complete the workflow. While segmented data is useful in some instances, longer time periods of data need to be used for forecasting as well as achieving richer pattern analyses.
For big data projects, it is important to have systems that scale easily. JAKALA’s large volume of data in the workflow could not all be analyzed in one place or with one tool. This led to the shuffling of data between different analytics tools and systems to combine the data and generate final products. Once one month’s data was received, it would take almost another month to complete data cleaning and analysis, resulting in a significant lag between data collection and analysis delivery.
For this workflow, manual data compilation and analysis were time-consuming and inefficient, hindering JAKALA’s ability to deliver timely insights to its clients. Shortening the time to delivering insights is critical to decision-making and requires processing large datasets quickly.
JAKALA needed a single environment that would facilitate working with large volumes of data and spatial and tabular inputs, as well as allow easy interoperability across all the company’s data warehouses and analytics environments. To more efficiently provide customer insight for the company’s clients, JAKALA used ArcGIS GeoAnalytics Engine in the Azure Databricks environment.
JAKALA is a Databricks partner and utilizes a comprehensive integration of Databricks’ platform to leverage the scalability and performance provided by that platform. Databricks provides an Apache Spark-based cloud platform with a unified set of tools for building, managing, and working with data at scale. GeoAnalytics Engine can work with Databricks using Azure, Amazon Web Services (AWS), or the Google Cloud platform as part of an integrated spatial analytics environment connected with the Databricks lakehouse architecture.
“Since our infrastructure is all on Azure, we can take our data stored there and easily combine it on Databricks with Esri’s GeoAnalytics Engine. It’s easier and more efficient to combine these environments instead of having to move the data somewhere else for analytics,” said Stefano Angarano, manager of web development and mobile data analysis at JAKALA.
ArcGIS GeoAnalytics Engine provided the perfect solution with a Spark-native library of more than 160 spatial functions and tools that work seamlessly within a Databricks workflow. As a bonus, JAKALA is now able to save time and energy on maintenance because working with GeoAnalytics Engine in Databricks allows staff to focus on the analytics instead of managing the company’s cloud or server infrastructure.
The integration of GeoAnalytics Engine with Azure Databricks substantially improved analytics processing time for JAKALA. In addition, GeoAnalytics Engine has streamlined the overall analysis process and improved automation, scalability, and maintenance efficiency.
“With GeoAnalytics Engine, we are able to receive all the data and perform the necessary analysis within a couple of days, whereas without it, it would take us weeks. This makes it much easier to provide our clients the data that they need, at the speed needed to make timely decisions,” said Angarano.
The shortened analysis time speeds data delivery and enables JAKALA to complete analysis on much larger datasets. Staff members can now examine annual patterns instead of weekly or monthly patterns to facilitate longer-range analysis and forecasting. This helps staff members conduct more comprehensive and accurate analyses and provide valuable insights and forecasts to the company’s clients based on a more detailed understanding of annual patterns and trends.
JAKALA also found that GeoAnalytics Engine helped streamline the overall process. Using GeoAnalytics Engine in the Databricks environment improved staff’s ability to automate and scale analysis. They were able to analyze large volumes of data in a more efficient manner and could adjust the scale of resources based on the complexity of the analysis. Maintenance of the system was also simplified, as GeoAnalytics Engine is cloud based and requires minimal manual updates.
“Performing monthly data analysis is a recurring task for us, and each month we have a consistent set of operations to perform. Having an automated solution is crucial for us to streamline the analysis process and eliminate the need for manual analysis, ensuring efficiency and accuracy in our operations,” said Angarano.
Using GeoAnalytics Engine to drive the analysis workflow has allowed JAKALA to offer new services to its customers, such as additional movement analysis and the ability to bring in larger amounts and longer time spans of data for improved context and forecasting. In the future, JAKALA will add new data sources, such as vehicle-generated data, to complement the data from personal devices it uses. Incorporating these new data sources will provide richer analyses about human movement and patterns.
“We are excited about the network analysis capabilities offered by GeoAnalytics Engine. The ability to leverage network-based service areas and closest facilities opens up new possibilities for us to incorporate additional metrics and enhance our analysis,” said Angarano.
