I have some (commercial, third-party) software that was previously executing correctly on SLES11 SP2, but now it exits with the following error:
code: relocation error: /lib64/libnsl.so.1: symbol strlen, version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference
According to my administrator, nothing on the system has changed. I can’t think of anything in my path/environment that would have changed. Has anyone seen this problem before? How do I resolve it?
Thanks,
Greg
On 18/11/2013 16:04, fischega wrote:
[color=blue]
I have some (commercial, third-party) software that was previously
executing correctly on SLES11 SP2, but now it exits with the following
error:code: relocation error: /lib64/libnsl.so.1: symbol strlen, version
GLIBC_2.2.5 not defined in file libc.so.6 with link time referenceAccording to my administrator, nothing on the system has changed. I
can’t think of anything in my path/environment that would have changed.
Has anyone seen this problem before? How do I resolve it?[/color]
I’m sure it would help if you could let us know what this software is …
Simon
SUSE Knowledge Partner
Hi Greg,
code: relocation error: /lib64/libnsl.so.1: symbol strlen, version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference
first thing to check is if the glibc RPM is still consistent:
:~> rpm -qf /lib64/libnsl.so.1
glibc-2.11.1-0.20.1
:~> rpm -V glibc-2.11.1-0.20.1
..?..... /usr/sbin/glibc_post_upgrade
:~>
(this is from a SLES11SP1 system, so the package version will be different on your system)
Next thing would be to check if you have some interfering version of the libraries that are referenced by i.e. LD_LIBRARY_PATH, as set by the startup routines of your commercial, third-party software. You’ll probably have to track down where/how the software is started and instead of loading the actual binary, invoke “ldd” on the binary and /lib64/libnsl.so.1 - the latter should only access libraries as distributed by the system: If libs from other pathes appear, then most likely these are the culprit.
Library path lookup can either be via environment variables (LD_LIBRARY_PATH) or via /etc/ld.so.conf(.d/*) entries. Depending on the software you’ve installed, one or more of these could have been modified to include application-specific versions of libraries.
Regards,
Jens
My administrator checked the RPMs and didn’t find anything amiss.
So far as I can tell, LD_LIBRARY_PATH is not being modified by any of the scripts that launch the sofware, and there is very little in my LD_LIBRARY_PATH. We have another “fresh” SLES11 SP2 system, and the software works properly on this system. The contents of my LD_LIBRARY_PATH variables are the same on both systems. The same is true for /etc/ld.so.conf(.d/*).
Very strange…
Hi fischega,
[QUOTE=fischega;17555]My administrator checked the RPMs and didn’t find anything amiss.
So far as I can tell, LD_LIBRARY_PATH is not being modified by any of the scripts that launch the sofware, and there is very little in my LD_LIBRARY_PATH. We have another “fresh” SLES11 SP2 system, and the software works properly on this system. The contents of my LD_LIBRARY_PATH variables are the same on both systems. The same is true for /etc/ld.so.conf(.d/*).
Very strange…[/QUOTE]
does running ldd on the lib in question reveal anything suspicious?
Regards,
Jens
Hi Jens,
No, I don’t see anything unusual. The working and non-working systems both report the same results.
1001 fischega@bl2005[~]> ldd /lib64/libnsl.so.1
linux-vdso.so.1 => (0x00007fff8a6fb000)
libc.so.6 => /lib64/libc.so.6 (0x00002ba631dec000)
/lib64/ld-linux-x86-64.so.2 (0x00002ba63197a000)
1001 fischega@bl1309[~]> ldd /lib64/libnsl.so.1
linux-vdso.so.1 => (0x00007ffff4b60000)
libc.so.6 => /lib64/libc.so.6 (0x00002b3ddd798000)
/lib64/ld-linux-x86-64.so.2 (0x00002b3ddd326000)
All three libraries have the same CRC checksum on both systems.
Greg
Hi Greg,
No, I don’t see anything unusual. […] All three libraries have the same CRC checksum on both systems.
yes, that looks perfectly normal (and the CRC result match the test result of your IT folks, I assume they ran rpm -V on the RPM in question).
Hadn’t you already checked, my bet would have been on a modified LD_LIBRARY_PATH during program startup - but as it is, I’m running out of ideas.
According to my administrator, nothing on the system has changed.
To be clear: the message is IMO caused by some program that is linked statically against against a lib version that is not available on the local system - what’s puzzling though is the stated fact that between “the application works” and “the application aborts with an error”, nothing has changed on the system.
Regards,
Jens
Hi
Running ldd on the binary that calls it may glean a missing library on
the system having the issue?
–
Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890)
SLED 11 SP3 (x86_64) GNOME 2.28.0 Kernel 3.0.93-0.8-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!
The administrator discovered that it works correctly when executed as root on the trouble system, but fails with the relocation error when executed as a user. A trace is below. Does anything jump out to anyone?
Hi fischega,
(sorry for the late reply, I was unavailable for a few days)
From the strace ouput, it looks like a “parent process” was traced, that executed some child process. While the excerpt you quoted is much too small to really tell, I have the feeling that it was the child process that ran into the error - so you might want to trace that child and compare the location of the libraries loaded within the child.
Regards,
Jens
PS: Any su-ing involved? LD_LIBRARY_PATH is reset when calling programs that switch the user context (i.e. “sticky user” bit set), so it might be that although the LD_LIBRARY_PATH points to the “proper” lib when calling the parent process, it might get reset during the call to the child (or if the parent switches user context), leaving that correct library out of the search path. When calling the program as a root user, this reset does not happen so the LD_LIBRARY_PATH still points to the proper lib.
My administrator found the problem! He ran another trace and noticed that both machines were trying to open /var/run/nscd/socket. One failed and one did not.
The problem machines had the Name Service Cache Daemon (nscd) disabled. This was done because it tended to cause problems when people changed passwords.
The mechanics of how this resulted in a relocation error in libnsl is still a mystery…
Hi fischega,
[QUOTE=fischega;17675]My administrator found the problem! He ran another trace and noticed that both machines were trying to open /var/run/nscd/socket. One failed and one did not.
The problem machines had the Name Service Cache Daemon (nscd) disabled. This was done because it tended to cause problems when people changed passwords.
The mechanics of how this resulted in a relocation error in libnsl is still a mystery…[/QUOTE]
that is indeed a strange thing - kudos to your admin that he found it and thank you for reporting back.
BTW, if it’s the password caching that causes trouble with nscd, I’d suggest to try disabling only passwd (“enable-cache passwd no” in /etc/nscd.conf), rather than disabling all of nscd.
Regards,
Jens
Greg’s Admin here…
Further clarification…I changed the passwd entry in nscd.conf to “no”, restarted nscd and it fails. I then disable all maps with the exception of passwd and it works.
I certainly don’t understand this, but if leaving the passwd entry in nscd.conf fixes things, it’s a done deal.
Hi walshjl,
[QUOTE=walshjl;17718]Greg’s Admin here…
Further clarification…I changed the passwd entry in nscd.conf to “no”, restarted nscd and it fails. I then disable all maps with the exception of passwd and it works.
I certainly don’t understand this, but if leaving the passwd entry in nscd.conf fixes things, it’s a done deal.[/QUOTE]
welcome to the forums and yes, that’s how I’d handle it as well. Sometimes one simply does not have the time to dig down all the way… though your description does sound intriguing 
Regards,
Jens