XenonStack Recommends

TestOps

Troubleshoot Performance Issues in Four Simple Steps

Navdeep Singh Gill | 10 Apr 2020

Introduction to Troubleshooting Performance Issues

Even the most advanced and feature-packed application might not survive significant performance problems. Users typically experience reduced response time. This is mostly caused by resource congestion. While the explanation is simple, it is often very tricky to find out what is getting congested. An application might have millions of lines of code, trying to find the exact sequence of the system causing the resource congestion is hard.
Find the Best Solution for Digitally Transforming your Enterprise. Talk to an End-End Testing Expert
In this tutorial, let's learn how to find out the exact line of code which is causing the resource congestion. In this tutorial, we will be showing you the troubleshooting process of a Demo Application.

1. Identify Processes that are Consuming High CPU

The first step of Troubleshooting Performance Issues is to identify the processes which are causing the CPU to spike. The simplest way to do this is by using the 'top' command. Run command 'top' from the console: $ top This command will display all the current processes that are running in the system, and the methods are sorted by high CPU consuming processes displayed at the top to bottom. After running the command, the console will show the output similar to the products given below. The output shows, the process 31294 is consuming about 199.1% of CPU. It is not usual. So it is pretty clear that the process 31294 is creating the problem. Now the next step is to identify what threads within the process are causing the CPU to spike up.

2. Identify Threads Within the Process that are Consuming High CPU

The step two of Troubleshooting Performance Issues is to check the CPU consumption of the threads within the process, run the command' top -Hpp {pid}' $ head -Hpp 312943 This command will show all the threads within the thread and their CPU usage. After running the power, it will show you the output similar to the products given below. From the output, it is clear that which threads are causing the spike in CPU 1. 69.3% of CPU is being utilized by Thread Id 31306. 2. 65.6% of CPU is being used by Thread Id 31307. 3. 64.0% of CPU is being used by Thread Id 31308. 4. All the other threads are using the negligible amount of CPU. Now, as we have precisely identified the threads which are causing the problem, the next step is to capture thread dump so that we can identify the exact lines of code that are responsible for the CPU spike.

3. Capture Thread Dumps

The Step three of Troubleshooting Performance Issues is - A thread Junkyard is a snapshot of the state of all threads that are part of the process. A thread dump reveals all the information about the thread activity and can help in diagnosing problems and optimize the application. There are so many options to capture thread dumps. We will be captured using the tool 'stack' which is packaged in JDK. The command for capturing thread dump is "jstack -l {pid} > {file-path}" jstack-l 31294 > opt/tmp/threadDump.txt A thread dump has three parts: 1) Timestamp at which thread was triggered. 2) Information about Java Virtual Machine 3) Thread Details (discussed below) Again in thread details, It has information of thread :
  • Thread Name

The first part of the thread information in 'Thread Name'. In the above example, The name of the thread is "Invoice Thread-A996". This name is assigned by the developer.

  • Priority

The next part is Priority of the thread. It may has values from 1 to 10. Always the thread with higher priority value gets the chance to be executed in case of a clash.

  • Thread Id

The next part is the Thread Id of the thread. This Id is assigned by the JVM to the thread. This Id is always unique, whereas different threads can share the same Thread Name.

  • Native Id

The next part is the native Id. This Id is assigned by the Operating System to the thread. This Id is used while performing tasks which involve Operating System (Like operation 'top').

  • Address Space

The next part is the address space. This gives the address where the thread is currently running in the memory.

  • Thread State

 The next part is the Thread State. Usually, a thread has six different states.

  • Stack Trace

The last part is called the Stack trace. The stack trace shows the code's execution path. It shows the information about where the thread started and its movement across the system.

The dump captured will be captured into the file threadDump.txt. Now the next step is to identify the lines of code, causing CPU spike.

4. Identifying Lines of Code Causing CPU Spike

In the Fourth step of Troubleshooting Performance Issues after capturing the thread dump, now we upload the thread dump to fastThread tool. FastThread tool generates visual data report. The report has multiple sections. On the top right part, there is a search box. Enter the Thread ids of threads which are consuming high CPU. FastThread tool will display all three stack trace, as shown in the image. Now, as know what is causing the problem, you can make necessary changes required to make the application run smoothly again. Example of a Faulty Application Source Code 1: package com.buggyapp.cpuspike; 2: 3: /** 4: * 5: * @author Test User 6: */ 7: public class Object1 { 8: 9: public static void execute() { 10: 11: while (true) { 12: 13: doSomething(); 14: } 15: } 16: 17: public static void doSomething() { 18: 19: } 20: } As you can see in the above program, 'doSomething' method is invoked an infinite number of times. This is because of non terminating while loop. If a thread goes inside the circle, it executes it for unlimited times. This causes CPU to spike up. This is what can happen in an incorrect source code. If the non-terminating loop is fixed, the problem with CPU spike will also be adjusted.

Conclusion - Troubleshooting Performance Issues in Four Steps

With these four steps, all your performance issues will be solved quickly and effectively. At XenonStack our team makes sure the performance of everything we are developing is as per the industry standards. You can read more here A Complete Guide to Security Testing in Devops