Bitbucket Server Inaccessible After Upgrade - HTTP 400 error
症状
Bitbucket Server cannot be accessed after upgrading to 5.13.0 or later. The following type of error appears when navigating to any Bitbucket page, alongside an HTTP 400 status:
java.lang.IllegalArgumentException: The character [_] is never valid in a domain name.
org.apache.tomcat.util.http.parser.HttpParser$DomainParseState.next(HttpParser.java:926)
org.apache.tomcat.util.http.parser.HttpParser.readHostDomainName(HttpParser.java:822)
org.apache.tomcat.util.http.parser.Host.parse(Host.java:71)
org.apache.tomcat.util.http.parser.Host.parse(Host.java:45)
org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:288)
org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:809)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:384)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:748)On versions prior to 6.0, the above error will not display - and you will receive an HTTP 400 error without any other information, such as the following:
To confirm that you are being impacted by this issue while on a version lower than 6.0, we recommend enabling Apache Tomcat access logging and reviewing the error in the Apache access log.
To enable Apache Tomcat access logging, we recommend setting the Tomcat access log properties in your bitbucket.properties file to something like the following values:
server.tomcat.accesslog.enabled=true
server.tomcat.accesslog.directory=${bitbucket.home/log}
server.tomcat.accesslog.prefix=tomcat-access
server.tomcat.accesslog.pattern=%t [%I] %{X-AREQUESTID}o %h (%{X-Forwarded-For}i) "%r" %s %b
原因
When upgrading to Bitbucket Server 5.13.0 or later, the version of Apache Tomcat is upgraded to 8.5.31 or above. Within this Apache Tomcat update, the following change was introduced:
- "Enable strict validation of the provided host name and port for all connectors. Requests with invalid host names and/or ports will be rejected with a 400 response. (markt)" - Apache Tomcat 8.5.31 Release Notes
If you are receiving the above error, this means that the "Host" http header is being formatted in a way where it is now being denied access by Apache Tomcat, and the request is now receiving a 400 HTTP response. A known cause of the "Host" http header being modified is when a reverse proxy modifies this http header before passing the http request to the proxied server (Bitbucket Server's Apache Tomcat instance).
ソリューション
Locate where the "Host" http header is being modified before it reaches Apache Tomcat, and modify it to remove any special characters/incorrect formatting.
Here is some example portions of two nginx.conf files where the 'Host' http header was being modified to include an '_' (an invalid character), and then the change made to the "proxy_set_header" portion to remove the invalid character and allow access to Bitbucket:
Before
...
server
{
listen 80;
server_name bitbucket.mybitbucket.com;
location /
{
proxy_set_header Host "Bitbucket_Server_Proxy";
proxy_pass http://bitbucket.mybitbucket.com.upstream;
proxy_redirect off;
}
}
upstream bitbucket.mybitbucket.com.upstream {
server localhost:7990;
}
...After
...
server
{
listen 80;
server_name bitbucket.mybitbucket.com;
location /
{
proxy_set_header Host "BitbucketServerProxy";
proxy_pass http://bitbucket.mybitbucket.com.upstream;
proxy_redirect off;
}
}
upstream bitbucket.mybitbucket.com.upstream {
server localhost:7990;
}
...