Jira notifications piling up in the mail queue due to IPv6 issues on JVM

お困りですか?

アトラシアン コミュニティをご利用ください。

コミュニティに質問

症状

Jira is not able to connect to a specific mail server or mail queue keeps getting overloaded as each message takes a long time to leave the queue. As a result all notification emails from Jira (batched notifications, non-batched notifications, customer notifications) are piling up in the queue and are sent with a very long delay.

診断

Looking at atlassian-jira-outgoing-mail debug logs, we can see the message is always taking longer between these two messages below:

2014-05-06 15:36:06,848 DEBUG [jira.local@domain.com] Sending mailitem To='user@domain.com' Subject='(XXX-1) Summary' From='null' FromName='JIRA updates' Cc='null' Bcc='null' ReplyTo='null' InReplyTo='null' MimeType='text/html' Encoding='UTF-8' Multipart='null' MessageId='null' ServiceRunner    Mail Queue Service DEBUG SMTP: connected to host "mail.domain.net", port: 25
2014-05-06 15:36:18,860 DEBUG [jira.local@domain.com] Sending mailitem To='user@domain.com' Subject='(XXX-1) Summary' From='null' FromName='JIRA updates' Cc='null' Bcc='null' ReplyTo='null' InReplyTo='null' MimeType='text/html' Encoding='UTF-8' Multipart='null' MessageId='null' ServiceRunner    Mail Queue Service EHLO jira.domain.net

The following error can also appear on the outgoing mail log files:

2015-02-04 21:00:26,605 WARN [Mail Server - JIRA] QuartzScheduler_Worker-0 ServiceRunner    TEST TEST[10100]: 
javax.mail.MessagingException: Connect failed while connecting to host 'mailserver.example.com' as user 'user@example.com' via protocol 'smtp': javax.mail.MessagingException: Connect failed;
  nested exception is:
	java.net.UnknownHostException: mailserver.example.com
javax.mail.MessagingException: Connect failed;


Taking thread dumps while the problem is happening will reveal the mail threads are taking a long time in the method java.net.Inet6AddressImpl.lookupAllHostAddr:

"Sending mailitem To='email3@test.com' Subject='[ABC-123] - Some Jira ticket' From='null' FromName='Some Name' Cc='null' Bcc='null' ReplyTo='email3@test.com' InReplyTo='null' MimeType='text/plain' Encoding='UTF-8' Multipart='javax.mail.internet.MimeMultipart@2cb1caa8' MessageId='null' ExcludeSubjectPrefix=true'" #241 daemon prio=5 os_prio=0 tid=0x00007ff58eba6000 nid=0x4361 runnable [0x00007ff54665a000]
java.lang.Thread.State: RUNNABLE
	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at java.net.InetAddress$1.lookupAllHostAddr(Unknown Source)
	at java.net.InetAddress.getAddressesFromNameService(Unknown Source)
	at java.net.InetAddress.getLocalHost(Unknown Source)
	- locked <0x00000007802e1cc8> (a java.lang.Object)
	at com.sun.mail.smtp.SMTPTransport.getLocalHost(SMTPTransport.java:235)
	- locked <0x00000007d61a5e58> (a com.sun.mail.smtp.SMTPTransport)
	at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:646)
	- locked <0x00000007d61a5e58> (a com.sun.mail.smtp.SMTPTransport)
	at javax.mail.Service.connect(Service.java:295)
	- locked <0x00000007d61a5e58> (a com.sun.mail.smtp.SMTPTransport)
(...) 

原因

The IPv6 lookup method can enter an infinite loop when a certain race condition occurs. This occurs infrequently, however can cause deadlocks where some threads end up blocked.

Please check the Java bug: https://bugs.openjdk.java.net/browse/JDK-8158038 for more information.

ソリューション

  • Set the following parameter on JIRA startup:

    -Djava.net.preferIPv4Stack=true
    

    With the parameter stated above set to true, Java will not execute IPv6 lookup methods and the problem should be avoided.


  • Please ensure you have your JIRA server hosts file updated with JIRA server hostname and IP addresses, both IPv4 and IPv6.

Last modified on Mar 21, 2024

この内容はお役に立ちましたか?

はい
いいえ
この記事についてのフィードバックを送信する
Powered by Confluence and Scroll Viewport.