Closed
Description
We ran into an issue today where we needed to take a heap dump (jmap). But it wasn't working.
Image was java:openjdk-8u92-jdk-alpine
.
$ docker exec server1_v3_1 ps aux
PID USER TIME COMMAND
1 root 1:54 java -server -verbose:gc ...
121 root 0:00 ps aux
$ docker exec server1_v3_1 jmap -dump:live,format=b,file=/data/dump.map 1
1: Unable to get pid of LinuxThreads manager thread
Here's the source of the error: http://code.metager.de/source/xref/openjdk/jdk8/jdk/src/solaris/native/sun/tools/attach/LinuxVirtualMachine.c#255
We suspect it might be because alpine is using musl instead of glibc.
jmap works in java:openjdk-8u91-jdk
.
Activity
niloc132 commentedon May 25, 2016
Other tools that don't work:
jstack
,jinfo
,jhat
.paranoid945 commentedon Aug 30, 2016
Not working in openjdk 7 neither.
does any openjdk engineer watching this issue?
yosifkit commentedon Sep 1, 2016
This seems related: gliderlabs/docker-alpine#11.
michaelzg commentedon Jan 15, 2017
Tried what @yosifkit linked.
Installed
glibc
as suggested by sgerrand in https://github.com/sgerrand/alpine-pkg-glibc/issues/1 and still no luck:Docker image:
8u111-jdk-alpine
tianon commentedon Jan 27, 2017
Indeed -- I don't think there's much we can do about this. 😞
dkolog commentedon Jan 31, 2017
I noticed that too. This happens because your java process PID is 1.
I use the same setup (alpine + glibc + jmap) and it works well if the PID is not 1 (when bash scripts forks the java process)
yosifkit commentedon Jan 31, 2017
If you keep bash as PID 1, you'd no longer get signals from
docker stop
anddocker kill
(without having to write traps). With docker 1.13 you can use--init
to have docker put in tini as PID 1; it'll forward signals and reap zombies.dkolog commentedon Feb 1, 2017
@yosifkit you are absolutely right.
@devinrsmith I was able to get heap/thread dump from PID 1 using Oracle's JDK.
elyscape commentedon Feb 20, 2017
This may be a result of the debug symbols not being installed in the image. See comment 9 on this issue or comment 15 on this issue.
sgerrand commentedon Jun 20, 2017
For those arriving late to the party, like me, this issue appears to be unrelated to the underlying libc implementation.
sitnik commentedon Jul 5, 2017
I have to use docker 1.12 and don't know if '--init' will be used in Kubernetes, so I solved this issue by wrapping java process in shell script and adding trap for TERM signal. At least jmap now works for me.
tianon commentedon Jan 3, 2018
Closing, since this appears to be solved by either using Docker's
--init
flag, or adding something liketini
/dumb-init
to images which suffer from this problem. 👍28 remaining items