Jira Service Management 5.12.x 長期サポート リリースのパフォーマンス レポート

We are continuing to evolve and improve the quality of our performance reports. For this LTS performance report, we extensively tested the performance of Jira Service Management Data Center using significantly larger datasets and virtual loads generated by a cohort of heavy virtual users. We also broadened the scope to include common actions and analyzed data that reflects real-world usage profiles of Jira Service Management, Service-management heavy and Assets-heavy usage profiles. We hope these enhancements provide richer insights and give you a better understanding of how Jira Service Management 5.12 is performing.

このページの内容

長期サポート リリースについて

Jira Service Management を定期的にアップグレードすることをお勧めします。ただし、組織のプロセスによって年に 1 回程度しかアップグレードできない場合は、長期サポート リリースへのアップグレードが良い選択肢となります。これにより、該当するバージョンのサポート終了まで、重大なセキュリティ、安定性、データ整合性、パフォーマンスの問題に継続的に対応できるようになります。

要約

The report below compares the performance of versions 5.12 and 5.4. You may notice that version 5.12 shows slower metrics. We’ve added many new features to Jira Service Management since version 5.4. As a result, the perceived performance remains comparable, making these slight regressions an acceptable trade-off.

As seen from our past roadmap, we’ve shipped more features that support the Service-heavy usage profile. This is reflected in performance improvements skewed towards the Assets-heavy usage profile. The perceived performance for both profiles is similar, and you can be assured that Assets can be adopted at scale without any detrimental impact on the overall system health.

テスト手法

The following sections detail the testing methodology, usage profiles, and testing environment we used in our performance tests.

テスト方法

Before we started testing, we needed to determine what size and shape of dataset represents a typical large Jira Service Management instance. To achieve that, we used data that closely matches the instance and traffic profiles of Jira Service Management in a large organization.

The following table represents the baseline dataset we used in our tests, which is equivalent to an XLarge size dataset.

Jira Service Management profiles

In this report, we conducted tests on two different profiles: Service-management heavy and Asset-management heavy. These profiles represent two common profiles we’ve observed and aim to provide the reader with a clearer understanding of performance when adopting Assets. 

Refer to the table below for more information on these profiles:

実行した操作

We conducted tests with a mix of actions that would represent a sample of the most common user actions for Jira Service Management, like creating requests through the customer portal, viewing agent queues, and commenting on tickets. An action in this context is a complete user operation, like opening a page in the browser window. The number of actions performed varies for each of the two mentioned profiles.

The following table details the actions we tested for our testing profiles and indicates how many times each action is repeated during a single test run.

テスト環境

The performance tests were all run on a set of AWS EC2 instances. For each test, the entire environment was reset and rebuilt, and then each test started with some idle cycles to warm up instance caches. Below, you can check the details of the environments used for Jira Service Management Data Center, as well as the specifications of the EC2 instances.

Our tests are a mixture of browser testing and REST API call tests. Each test was scripted to perform actions from the list of scenarios above.

各テストは 40 分間実行され、その後、統計情報が収集されます。

The test environment used the hardware below, which aligns with the recommended X-Large Jira instance. Learn more about Data Center infrastructure recommendations

• 5 Jira ノード
• 別のノード上にあるデータベース
• Load generator on a separate environment
• 別のノード上にある共有ホーム ディレクトリ
• ロード バランサ (AWS ELB HTTP ロード バランサ)

Jira Service Management Data Center

データベース

ロード ジェネレーター

Performance summary for Service-management heavy profile

Constant load test

For this test, a constant load was applied for 40 minutes and the median response time observed.

The graph below shows the differences in response times of individual actions between 5.12 and 5.4.10. The data used to build the graph is below.

The regressions we see in the above graph can be attributed to:

• the functional improvements introduced in the product from 5.4 to 5.12, which provide significant feature values.
• the changes related to A11y improvements, as we have been actively addressing critical A11y issues following our recent VPAT assessment update. Particularly the viewMyRequestsOnPortal action, that we are aiming to resolve in the upcoming 5.12 bugfix release.

Increasing load test

For this test the number of concurrent users started at 0 and increased to a maximum of ~160 users over 30 minutes.  The users themselves we’re highly active (above standard) performing high load actions with no regressions in overall performance and hardware metrics and no errors were detected during execution. These were the results observed:

• No regressions in the perceived performance of the product (as measured by the response time) between 5.4 and 5.12. With an increasing virtual load by an increasing number of virtual users, the response time remained stable. Note that the spike in response time at the beginning and end are due to start/end of tests.

• Similar throughput achieved between 5.4 and 5.12.

• Heap memory usage was steady and roughly the same for both 5.4 and 5.12.

• CPU usage was more efficient on 5.12 compared to 5.4.

Multi-node comparisons

Response time for 3,5, and 7 node 5.12 instances

Throughput time for 3,5, and 7 node 5.12 instances

For this test the number of concurrent users were gradually increased from 0 to 520 virtual users over 30 minutes on 3, 5, and 7 node 5.12 environments. The results indicate significant performance improvements between instances with 3 and 5 nodes. An instance with 5 nodes responds faster, uses less CPU while keeping the same throughput (Hits/s). The occurrence of timeouts also notably decreases on the 5-node instance. However, the performance difference between 5 and 7 nodes is marginal. Therefore, for the scenario and instance sizes simulated, opting for a 5-node configuration is deemed more cost-effective. 

Overall average heap memory usage for 3, 5, and 7 node is similar. 

Performance summary for Assets heavy profile

Constant load test

For this test, a constant load was applied for 40 minutes and the median response time observed.

The graph below shows the differences in response times of individual actions between 5.12 and 5.4.10. The data used to build the graph is below.

Note that, the performance of the Assets heavy profile is similar to the Service heavy profile. The data strongly indicates that the adoption of Assets doesn’t negatively impact the overall health of the system. Assets usage can involve heavy actions that add to the load. Learn more about the significant performance improvements we’ve shipped in Assets

Increasing load test

For this test the number of concurrent users started at 0 and increased to a maximum of ~320 users over 30 minutes. The load was then held for 10 minutes. The users themselves we’re highly active (above standard) performing high load actions with no regressions in overall performance and hardware metrics and no errors were detected during execution. These were the results observed:

• No regressions in the perceived performance of the product (as measured by the response time) between 5.4 and 5.12. With an increasing virtual load by an increasing number of virtual users the response time remained stable. Note that, spike in response time at the beginning and end are due to start/end of tests. 
• Similar throughput achieved between 5.4 and 5.12. 
• CPU load for 5.12 and 5.4 started off higher and lowered when load stabilized. Average CPU usage for 5.12 is higher than 5.4. 
• Heap memory usage was steady and roughly the same for both 5.4 and 5.12. 

Multi-node comparisons

For this test, an increasing load of up to ~500 users was applied to 3, 5, and 7 node 5.12 environments.

The multi node comparisons for an Assets heavy profile are for the most part similar to the result of the Service heavy. However, for the Assets heavy profile the 7-node profile was much more performant than 5-nodes. Most likely due to the CPU and memory usage. We can observe that having more nodes improved the hardware metrics' usage performance in this profile.

Response time for 3,5, and 7 node 5.12 instances

Throughput time for 3,5, and 7 node 5.12 instances

既知の問題

As noted earlier, are aware of performance issues around the ‘View my requests on the customer portal’ scenario when rendering one of the combobox filters. We aim to fix the regression in Jira Service Management 5.12.1 bugfix release or 5.13.0.