JVM is not reachable with jstat and jstack

アトラシアン ナレッジベース

このページの内容

お困りですか?

アトラシアン コミュニティをご利用ください。

コミュニティに質問

要約

Generating Jira thread dumps using "jstack" returns error "Unable to open socket file"

環境

Jira Software/Service Management running on Linux hosts with Java versions 8 or 11

診断

Trying to generate Jira thread dumps using jstack, the command returns the following error: 

$ sudo -u jira /usr/lib/jvm/jdk-11.0.2+7/bin/jstack 1455
1455: Unable to open socket file /proc/1455/root/tmp/.java_pid1455: target process 1455 doesn't respond within 10500ms or HotSpot VM not loaded

The path "/proc/<PID>/root" is a symbolic link to the root path ( / )

ls -l /proc/<PID>/root
lrwxrwxrwx 1 jira jira 0 dez  3 18:29 /proc/<PID>/root -> /

So the path "/proc/<PID>/root/tmp/" is pointing to "/tmp" to store temporary files for the processes.


Checking the "/tmp", the Java socket file was not present:

ls -l /tmp/.java_pid<PID>
ls: cannot access '/tmp/.java_pid<PID>': No such file or directory 

原因

There are some external factors that can cause the socket file to be deleted from "/tmp":

  1. External routine tools to clean files older than X days;
  2. Linux "systemd-tmpfiles-clean" removing Java Hooks
    (More information can be found in Red Hat Bugzilla – Bug 1704608 but this is not limited to RedHat)
  3. Manual removal of the files from "/tmp"

ソリューション

1 - If any external tool is being used to delete old files from "/tmp", configure the tool to ignore the patterns below: 

/tmp/hsperfdata_*
/tmp/.java_pid*


2 - If it is found to be the Linux Systemd Cleanup task deleting the files, do the following: 

Edit the file "/usr/lib/tmpfiles.d/tmp.conf" and add the 2 lines below: 
(The "x" in both lines are to indicate that the patterns must be excluded from the cleanup task)

# Exclude java-hooks
x /tmp/hsperfdata_*
x /tmp/.java_pid*

回避策

If the socket file was deleted and for a troubleshooting situation it is necessary to generate thread dumps before restarting the application, the command below can be used to send thread dumps to "Catalina.out" log.

See more details in the document Generating a thread dump under the section "Linux/Unix/OS X environment" > "Linux/Unix command line"

kill -3 <PID>

追加情報

Additional information can be found in the following links:

HotSpot Dynamic Attach Mechanism
Why jstack not working when the /tmp/.java_pid<num> socket file has been deleted
Target process xxxx doesn't respond within 10500ms or HotSpot VM not loaded
Red Hat Bugzilla – Bug 1704608 - systemd-tmpfiles-clean deletes /tmp/hsperfdata_*


最終更新日: 2020 年 12 月 7 日

この内容はお役に立ちましたか?

はい
いいえ
この記事についてのフィードバックを送信する
Powered by Confluence and Scroll Viewport.