I’m working with splunk support on a open file/thread limit problem we are running into. They are asking about the systemd limits, that I’m not familiar with.
We have increased the settings in /etc/security/limits.conf to nproc & nofiles to 120000 for all users but the splunk application doesn’t see this limit. It is reporting a limit 65535.
I just want to make sure that I use the right setting in /etc/systemd/system.conf or user.conf to test increasing the limits within systemd.
Or is this the wrong path entirely ?
Thanks
[QUOTE=cisaksen;58114]I’m working with splunk support on a open file/thread limit problem we are running into. They are asking about the systemd limits, that I’m not familiar with.
We have increased the settings in /etc/security/limits.conf to nproc & nofiles to 120000 for all users but the splunk application doesn’t see this limit. It is reporting a limit 65535.
I just want to make sure that I use the right setting in /etc/systemd/system.conf or user.conf to test increasing the limits within systemd.
Or is this the wrong path entirely ?
Thanks[/QUOTE]
Hi
That’s a pam thing, not a systemd thing…?
I hate to say it, but a peruse of the command; man 5 limits.conf may help clarify the needed settings?
I discovered a setting in a the service file that splunk docs recommend for systemd setup. Funny thing we ran into was that splunk is having problem running from systemd. If I run it the orginal way /opt/splunk/bin/splunk start it runs better or as it should. Something in Systemd and splunk doesn’t seem to be working quite right. If I run splunk through the systemd service file splunk keeps running into thread limits, and errors stating the OS thread limit has been reached when it clearly hasn’t. We’ll see what splunk comes back with.
This is on a physical server by the way not a VM.
[QUOTE=cisaksen;58117]I discovered a setting in a the service file that splunk docs recommend for systemd setup. Funny thing we ran into was that splunk is having problem running from systemd. If I run it the orginal way /opt/splunk/bin/splunk start it runs better or as it should. Something in Systemd and splunk doesn’t seem to be working quite right. If I run splunk through the systemd service file splunk keeps running into thread limits, and errors stating the OS thread limit has been reached when it clearly hasn’t. We’ll see what splunk comes back with.
This is on a physical server by the way not a VM.[/QUOTE]
Hi
Can you post the systemd service file and also the ouput from;
systemctl status splunk
Is /opt/splunk/bin/splunk a binary or script?
/opt/splunk/bin/splunk is a binary.
I did find the file limit that I was hitting - it was in the systemd service file. But of course that didn’t fix the problem. I tried to run the splunk procedures for running it as systemd and it wouldn’t even start. Said it needed root privileges even though it was configured to run as the user splunk (we created). splunk.com is trying to figure this out - it’s been elevated.
[QUOTE=cisaksen;58168]/opt/splunk/bin/splunk is a binary.
I did find the file limit that I was hitting - it was in the systemd service file. But of course that didn’t fix the problem. I tried to run the splunk procedures for running it as systemd and it wouldn’t even start. Said it needed root privileges even though it was configured to run as the user splunk (we created). splunk.com is trying to figure this out - it’s been elevated.[/QUOTE]
Hi
The systemd service file should contain the user to run…
For example, the username is specified;
https://github.com/malcolmlewis/systemd-htop-service/blob/master/htop.service