症状

Bamboo is stuck during the startup and wouldn't finish coming up. The server log doesn't give a hint on why it is not starting up.

環境

Any Operating system with network filesystem (NFS) mount points attached to the system.

診断

• check if Security-Enhanced Linux is turned ON. 

  /usr/sbin/getenforce
  /usr/sbin/sestatus

  If enabled, you may have to disable it as detailed in this KB article - Bamboo hangs during the installation on Linux and uses all CPU - Disable SELinux. 

• When the process gets stuck, generate a set of thread dumps (6 thread dumps taken 30 seconds apart) by following this article

• Check if <bamboo-home> is mounted on NFS volume. If so, check if all the mount points in NFS are accessible and if you can run the below commands successfully:

  $ mount -l
  $ df -h

原因

• A hanging NFS mount can cause the problem even if it isn't where Bamboo home is located. This is because Bamboo uses Lucene and this iterates over all the mount points, even when it is not a path utilized directly by the application.
• Thus, by checking the output of mount -l we can check if there are any NFS mounts then verify if they're working.
  

• Thread Dumps show setup stuck here across all dumps:

  "setupDefaultGeneralBackgroundThread" #48 daemon prio=5 os_prio=0 tid=0x00007f1f44004000 nid=0x1ab5 runnable [0x00007f1fac209000]
     java.lang.Thread.State: RUNNABLE
  	at sun.nio.fs.UnixNativeDispatcher.stat0(Native Method)
  	at sun.nio.fs.UnixNativeDispatcher.stat(UnixNativeDispatcher.java:286)
  	at sun.nio.fs.UnixFileAttributes.get(UnixFileAttributes.java:70)
  	at sun.nio.fs.UnixFileStore.devFor(UnixFileStore.java:55)
  	at sun.nio.fs.UnixFileStore.<init>(UnixFileStore.java:70)
  	at sun.nio.fs.LinuxFileStore.<init>(LinuxFileStore.java:48)
  	at sun.nio.fs.LinuxFileSystem.getFileStore(LinuxFileSystem.java:112)
  	at sun.nio.fs.UnixFileSystem$FileStoreIterator.readNext(UnixFileSystem.java:213)
  	at sun.nio.fs.UnixFileSystem$FileStoreIterator.hasNext(UnixFileSystem.java:224)
  	- locked <0x00000006dc61e538> (a sun.nio.fs.UnixFileSystem$FileStoreIterator)
  	at org.apache.lucene.util.IOUtils.getFileStore(IOUtils.java:584)
  	at org.apache.lucene.util.IOUtils.spinsLinux(IOUtils.java:528)
  	at org.apache.lucene.util.IOUtils.spins(IOUtils.java:517)
  	at org.apache.lucene.util.IOUtils.spins(IOUtils.java:492)
  	at org.apache.lucene.index.ConcurrentMergeScheduler.initDynamicDefaults(ConcurrentMergeScheduler.java:412)
  	- locked <0x00000006dc61e5f8> (a org.apache.lucene.index.ConcurrentMergeScheduler)
  	at org.apache.lucene.index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:501)
  	- locked <0x00000006dc61e5f8> (a org.apache.lucene.index.ConcurrentMergeScheduler)
  	at org.apache.lucene.index.IndexWriter.waitForMerges(IndexWriter.java:2557)
  	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1037)
  	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1079)
  	at com.atlassian.bamboo.index.bonnie.BonnieLuceneConnection.upgradeIndexIfNecessary(BonnieLuceneConnection.java:545)
  	at com.atlassian.bamboo.index.bonnie.BonnieLuceneConnection.ensureCorrectIndexFormat(BonnieLuceneConnection.java:520)
  	at com.atlassian.bamboo.index.bonnie.BonnieLuceneConnection.<init>(BonnieLuceneConnection.java:101)
  	at com.atlassian.bamboo.index.bonnie.BonnieLuceneConnection.<init>(BonnieLuceneConnection.java:115)
  	at com.atlassian.bamboo.index.bonnie.BonnieLuceneConnection.<init>(BonnieLuceneConnection.java:127)
  	at com.atlassian.bamboo.index.LuceneConnectionImpl.<init>(LuceneConnectionImpl.java:17)
  	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:172)

ソリューション

This confirms that Lucene thread is blocked as it is waiting for all the NFS mounts to be accessible. If one of the NFS mounts is hanging/unresponsive on this system,  please unmount them or make them properly available and then attempt the Bamboo installation process again.