“See” the Source of the Problem, “Understand” its Cause and Nature, “Intervene”: Enerjisa Case Study

Enerjisa is one of the largest organizations in Turkey operating in Electricity Production, Distribution and Sales. It serves more than 21.5 million users in 14 provinces across 3 distribution regions: Başkent, Ayedaş and Toroslar.

In order to increase efficiency in the electric distribution industry, Enerjisa has integrated the Smart Networks concept and associated new business models into its business processes. Enerjisa Distribution Companies are responsible for ensuring the continuity of energy supply for customers and carrying out infrastructure activities such as fault, maintenance, repair and investments in the distribution regions where it provides service. The organization also carries out on-site operations such as meter reading, connecting/disconnecting electricity, new connections, and detection of illegal use.

Establishing a strong and sustainable IT infrastructure and maintaining the services without interruption is of utmost importance in organizations such as Enerjisa that provide strategic and critical services. Once the investment is made, the critical expectation is that this investment will meet the desired performance standards and provide the organization with the right results. When the expected results are obtained through this investment, the aim is to increase the productivity of the workforce and equipment as well as ensuring early intervention is possible. As a result, internal and external customer satisfaction as well as brand reliability are improved.

As a result of the continuous growth of the network, the increase in network data utilization and the number of integrated systems/applications and users, there is a subsequent increase in integrations and resource utilization, causing bottlenecks and performance issues in the systems. As a result of the work performed by the network teams in coordination with the system and infrastructure, it became apparent that there was a need to examine systems from a larger perspective, using parameters specific to GIS applications. It revealed the need to find an urgent and permanent solution before the working experience and processes of the users are affected.

There were issues such as interruptions due to timeouts, operations freezing on the screens, long processing times and not being able to receive a response to requests during the work carried out in desktop and web applications. On the other hand, because the resource pool did not increase in parallel with the huge number of requests for published map services and web applications, interruptions and service time-outs were encountered. For this reason, the need for resource planning in GIS infrastructure became more and more evident.

These chronic issues experienced by experts worsened to the point where it had a negative effect on their motivation. Some temporary measures for solving these issues had been put forward previously, but they proved inadequate, so the teams developed a negative attitude in terms of finding a proper solution. At this point, the GIS Team started an extensive study and examined the GIS infrastructure, including desktop, web and mobile, in coordination with the system, network, architecture teams and users within a certain time frame. In this study, the infrastructure was reviewed in detail by creating parameters and reports specific to Enerjisa GIS in tandem with systemic parameters.

EnerjiSA CBS infrastructure hosts more than 70 systems, more than 20 ArcGIS Enterprise environments, over 100 services, applications and integrations.

Within the scope of the work that was undertaken, ArcGIS Monitor was installed and then parameters such as systems and resource utilization, Windows services and applications, databases and resource utilization of the systems it works with, the number of connections and queries, the number of users and user-based system usage, applications and transactions on the Desktop side, and response times for services, requests for services, consumption information related to services, the number of transactions per second, load and processing capacity of services and systems when idle and busy on the Enterprise side were monitored with data collected at 5-minute intervals, and daily and weekly reports were issued within the EnerjiSA GIS infrastructure. The installation of the ArcGIS Monitor application was meant to monitor all components of the existing GIS infrastructure and all spatial data flow, and alarms were generated based on the specified criteria, and notifications were delivered to users (e-mail service) by the EnerjiSA GIS team. Monitoring counters were defined for organization-specific monitoring metrics.

After configuring ArcGIS Monitor to track the above parameters of existing systems and applications, analyses and evaluations of the root cause of the problem were carried out, in addition to problem detection, with the help of the Root Cause Analysis (RCA) feature, which directly contributes to the decision-making processes in capacity planning studies.

The reports generated on the ArcGIS monitor are run periodically, in order to monitor, for example the locations, types and repetition of bottlenecks. Once these reports were completed, the planning phase began. After capacity planning and resource optimization were performed on GIS infrastructure, controlled resource increases were started.

Actions were taken regarding users changing the utilization status. After a 2-2.5 month period, it was possible to create solutions for the issues directly affecting the usage experience of users. A mechanism was created that made it possible to make decisions and interventions for problems with pinpoint accuracy. Solution teams related to the source of the problem were brought into play (Database Teams, System Teams or GIS teams directly for user-related issues).

As a result of these interventions, the chronic slowness issue began to be resolved quickly and in a noticeable fashion. This gave teams the time and initiative to plan what actions needed to be taken and change the system. As a result, user experience was greatly enhanced. Previously, when on-site teams had issues with the system, they notified the system support teams. Now, there is a process in place where the teams see the problem and can proactively intervene before these notifications are delivered.

After interventions, the processing times of users on the desktop and web side decreased substantially. Once the slowness issue was resolved, processing times on the desktop and web side were faster. For example, before the system enhancements, it was possible to work with only 10,000 records, but now it is easy to process 50,000 records. After the system interventions, there was a lot of positive feedback received from end users.

Thanks to the capabilities of the ArcGIS Monitor application and the efforts of the EnerjiSA GIS team, data-driven decision-making processes were enhanced in the areas of cost management and investment management.The unit became capable of directly affecting resource planning in GIS infrastructure and managing its resources.

Here are a few examples.

• If a web service handles 100,000 requests per month and another service handles 10,000 requests, the GIS unit becomes the decision maker to decide whether to shift the resources to the service that receives 10,000 requests. In the present situation SOC Optimization is carried out and this resource allocation is automatic between services. As a result, the resource pool is used more efficiently, which directly shortens users' processing times.
• The number of time-outs per week was 2863 in the ArcGIS Enterprise, but after the intervention, this number initially dropped to less than 10 and finally reached 0.
• In another case, while the number of shutdowns and non-responsive incidents experienced in applications were over 15 per month, this value was reduced to 0 after the intervention, and instantaneous shutdown issues were eliminated in the application.
• While there was more than 20 bottlenecks experienced per week in the monitored systems, this number was reduced to 0 after resource increases and capacity planning work, and existing systems began to be used more effectively.

In this manner, both the data set processed by the user and the scope of the process were expanded, and a monitoring system was installed that allowed intervention by detecting instantaneous bottlenecks and issues affecting the current state of the user before they notice them.

• Launched in June 2021, the first actions of the project became clear in July 2021. The positive comments coming from end users after the first interventions provided time for other actions and steps to be taken. 
• Before ArcGIS Monitor was installed, GIS teams were unable to convince system teams about the cause of the problematic bottlenecks. Because the system teams were not able to clearly see the root of the issue with the infrastructure they possessed, the issue persisted even if after they intervened. As the accuracy and reliability of the monitoring outputs were proven, the GIS unit became more influential in decision-making. The reason for this is that they were able to go directly to the root cause of the issue thanks to the output from ArcGIS Monitor.
• Because it was possible to detect the root cause of the issues, a structure was set up that allowed teams to detect the issue even before the user experienced a problem and then quickly resolved them.
• ArcGIS Monitor outputs were reviewed every week by the CIO, the Information Technologies Director, the Information Systems Applications Group Manager and started to be used as a support mechanism in decision-making processes.
• Thanks to this system, a team structure was put in place that was capable of detecting issues in advance, making fast decisions and thus properly operating quick response mechanisms.