Java libnio.so NET_Bind not defined in file libnet.so

I run this same application on both x86_64 and s390x servers, but I only see this error on the s390x side for some reason.

Exception in thread “main” java.lang.NoClassDefFoundError: sun.util.calendar.ZoneInfoFile (initialization failure)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:176)
at sun.util.calendar.ZoneInfo.getTimeZone(ZoneInfo.java:675)
at java.util.TimeZone.getTimeZone(TimeZone.java:592)
at java.util.TimeZone.setDefaultZone(TimeZone.java:725)
at java.util.TimeZone.getDefaultRef(TimeZone.java:656)
at java.util.TimeZone.getDefault(TimeZone.java:643)
at org.joda.time.DateTimeZone.getDefault(DateTimeZone.java:149)
at org.jruby.RubyTime.getLocalTimeZone(RubyTime.java:151)
at org.jruby.RubyTime$5.allocate(RubyTime.java:230)
at org.jruby.RubyClass.allocate(RubyClass.java:224)
at org.jruby.RubyTime.newInstance(RubyTime.java:924)
at org.jruby.RubyTime$INVOKER$s$0$0$newInstance.call(RubyTime$INVOKER$s$0$0$newInstance.gen)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
at logstash.runner.file(logstash/runner.rb:2)
at logstash.runner.load(logstash/runner.rb)
at logstash.runner.main(logstash/runner.rb)
Caused by: java.lang.UnsatisfiedLinkError: nio (/usr/lib64/jvm/java-1.7.0-ibm-1.7.0/jre/lib/s390x/libnio.so: symbol NET_Bind, version SUNWprivate_1.1 not defined in file libnet.so with link time reference)

server:/opt/logstash # strings /usr/lib64/jvm/java-1.7.0-ibm-1.7.0/jre/lib/s390x/libnet.so | grep NET
NET_Wait
NET_SockaddrEqualsInetAddress
NET_SockaddrToInetAddress
NET_SetTrafficClass
NET_GetPortFromSockaddr
NET_Bind
NET_MapSocketOption
NET_SetSockOpt
NET_GetSockOpt
NET_AllocSockaddr
NET_InetAddressToSockaddr
IOCTL SIOCGIFNETMASK failed

server:/opt/logstash # strings /usr/lib64/jvm/java-1.7.0-ibm-1.7.0/jre/lib/s390x/libnet.so | grep SUN
SUNWprivate_1.1

Is there any reason why this library would act differently?

Gotta hand it to you; you’ve tried everything I would have thought to do
except I would have used ‘nm’ instead of ‘strings’ to list symbols. Does
that matter? Probably not, but may be worth checking.

Assuming you’re using the default IBM JRE on the other SLES boxes maybe
this is just a difference in Java on s390x, though that’s not supposed to
happen, especially considering what you’ve shown us.

Still looking…

Good luck.

Could you confirm the Java vendor on the x86_64 systems is also IBM (not
Sun)? Do you by chance have some sample code that you know causes this
path to be taken that could be shared?

Good luck.

server:~ # java -fullversion
java full version JRE 1.7.0 IBM Linux build pxz6470sr4fp1-20130325_01(SR4 FP1)

server:~ # java -version
java version “1.7.0”
Java™ SE Runtime Environment (build pxz6470sr4fp1-20130325_01(SR4 FP1))
IBM J9 VM (build 2.6, JRE 1.7.0 Linux s390x-64 Compressed References 20130306_14 0761 (JIT enabled, AOT enabled)
J9VM - R26_Java726_SR4_FP1_20130306_1011_B140761
JIT - r11.b03_20130131_32403ifx1
GC - R26_Java726_SR4_FP1_20130306_1011_B140761_CMPRSS
J9CL - 20130306_140761)
JCL - 20130315_01 based on Oracle 7u13-b08

The code is a jar file.
http://logstash.net/ (Download) logstash-1.1.12-flatjar.jar
It requires that ruby and libyaml be installed. These don’t seem to be the problem.

server:~ # nm /usr/lib64/jvm/java-1.7.0-ibm-1.7.0/jre/lib/s390x/libnet.so | grep NET
000000000000eef4 t NET_Accept
000000000000bd4c T NET_AllocSockaddr
000000000000add4 T NET_Bind
000000000000e2fc t NET_Connect
000000000000da38 t NET_Dup2
000000000000ab68 t NET_GetFileDescriptorID
000000000000abe8 T NET_GetPortFromSockaddr
000000000000afe0 T NET_GetSockOpt
000000000000d818 t NET_GetThreadTag
000000000000ac40 t NET_IPv4MappedToIPv4
000000000000c334 T NET_InetAddressToSockaddr
000000000000d7bc t NET_Interrupt
000000000000d798 t NET_IsBlocked
000000000000ac70 t NET_IsEqual
000000000000abf4 t NET_IsIPv4Mapped
000000000000ae2c T NET_MapSocketOption
000000000000e0c0 t NET_Poll
000000000000e84c t NET_Read
000000000000ecd0 t NET_ReadV
000000000000ea70 t NET_RecvFrom
000000000000de88 t NET_Send
000000000000f170 t NET_SendTo
000000000000af28 T NET_SetSockOpt
000000000000abc8 T NET_SetTrafficClass
000000000000a2f8 T NET_SockaddrEqualsInetAddress
000000000000a534 T NET_SockaddrToInetAddress
000000000000dd08 t NET_Socket
000000000000da28 t NET_SocketClose
000000000000c180 t NET_ThrowByNameWithLastError
000000000000c14c t NET_ThrowCurrent
000000000000c08c t NET_ThrowNew
000000000000e520 t NET_Timeout
000000000000acf4 T NET_Wait
000000000000f3c4 t NET_WriteV
000000000000acc8 t NET_addrtransAvailable

server:~ # nm /usr/lib64/jvm/java-1.7.0-ibm-1.7.0/jre/lib/s390x/libnet.so | grep SUNW
U JCL_Accept@@SUNWprivate_1.1
U JCL_Bind@@SUNWprivate_1.1
U JCL_Close@@SUNWprivate_1.1
U JCL_Connect@@SUNWprivate_1.1
U JCL_Dup2@@SUNWprivate_1.1
U JCL_Listen@@SUNWprivate_1.1
U JCL_Poll@@SUNWprivate_1.1
U JCL_ReadV@@SUNWprivate_1.1
U JCL_Recv@@SUNWprivate_1.1
U JCL_RecvFrom@@SUNWprivate_1.1
U JCL_Send@@SUNWprivate_1.1
U JCL_SendTo@@SUNWprivate_1.1
U JCL_Shutdown@@SUNWprivate_1.1
U JCL_Socket@@SUNWprivate_1.1
U JCL_Socketpair@@SUNWprivate_1.1
U JCL_WriteV@@SUNWprivate_1.1
U JNU_GetStringPlatformChars@@SUNWprivate_1.1
U JNU_NewObjectByName@@SUNWprivate_1.1
U JNU_NewStringPlatform@@SUNWprivate_1.1
U JNU_ReleaseStringPlatformChars@@SUNWprivate_1.1
U JNU_ThrowByName@@SUNWprivate_1.1
U JNU_ThrowByNameWithLastError@@SUNWprivate_1.1
U JNU_ThrowIOException@@SUNWprivate_1.1
U JNU_ThrowIOExceptionWithLastError@@SUNWprivate_1.1
U JNU_ThrowNullPointerException@@SUNWprivate_1.1
U JNU_ThrowOutOfMemoryError@@SUNWprivate_1.1
0000000000000000 A SUNWprivate_1.1
U dbgCalloc@@SUNWprivate_1.1
U dbgFree@@SUNWprivate_1.1
U dbgMalloc@@SUNWprivate_1.1
U dbgRawFree@@SUNWprivate_1.1
U dbgRawMalloc@@SUNWprivate_1.1
U dbgRealloc@@SUNWprivate_1.1

I upgraded to SDK SP3, and that helped some, now I’m getting a different error.

Problem: nothing provides libyaml-0-2 = 0.1.3 needed by libyaml-devel-0.1.3-0.6.16.s390x
It turns out, libyaml is on OBS for x86_64, but nothing for s390x

Sorry for the delay. Some alpha geek went crazy with strace and has this
analysis:

I ran an strace on this and found that there is a libnet.so in the /lib64 directory that was being referenced instead of the /usr/lib64/jvm/ java-1.7.0-ibm-1.7.0/jre/lib/s390x/libnet.so ...

The /lib64/libnet.so does not have the NET_Bind defined, thus the error.

It can be fixed by putting /usr/lib64/jvm/java-1.7.0-ibm-1.7.0/jre/lib/
s390x in the /etc/ld.so.conf file before the /lib and /lib64 entries,
then run ldconfig.

Care to try and let us know? Note that modifying things in
/etc/ld.so.conf* can affect system behavior; in this case, I assume that’s
desired. There are also other ways of implementing this change I believe,
like setting LD_LIBRARY_PATH before calling Java (or at least that SHOULD
work), or using LD_PRELOAD.

Also note that this appears to address your original issue, so I’m not
sure how things will be now that you have an issue with libyaml. I’m
hoping somehow this will help that too.

Good luck.

Yes this fixed it. In my case, I just relinked the libraries. I would have thought them to be identical.