Cannot increase open file limit beyond 999999 on 14.04+
I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf
. But, cannot get past 999999
, anything above this value reverts to default 1024
nor I can set it to unlimited
Here is my limits.conf
* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999
The above works fine and ulimit -n
return 999999
.
But, if I make the value one step higher or to unlimited
or -1
or infinity
it doesn't work. ulimit -n
return 1024
I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage
Thanks
ulimit resource-limiting
add a comment |
I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf
. But, cannot get past 999999
, anything above this value reverts to default 1024
nor I can set it to unlimited
Here is my limits.conf
* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999
The above works fine and ulimit -n
return 999999
.
But, if I make the value one step higher or to unlimited
or -1
or infinity
it doesn't work. ulimit -n
return 1024
I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage
Thanks
ulimit resource-limiting
1
What is your real problem?
– Jos
Mar 22 '18 at 19:58
Hi Jos, let me make it simple for you. Thought the pam_limits documentation saysAll items support the values -1, unlimited or infinity indicating no limit, except for priority and nice.
I can't setunlimited
to nofile, if I do soulimit -n
returns 1024, not sure if this is an expected behaviour
– Shan
Mar 23 '18 at 13:10
Do you really need more than a million files open?
– Jos
Mar 23 '18 at 13:13
Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.
– Shan
Mar 23 '18 at 13:53
add a comment |
I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf
. But, cannot get past 999999
, anything above this value reverts to default 1024
nor I can set it to unlimited
Here is my limits.conf
* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999
The above works fine and ulimit -n
return 999999
.
But, if I make the value one step higher or to unlimited
or -1
or infinity
it doesn't work. ulimit -n
return 1024
I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage
Thanks
ulimit resource-limiting
I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf
. But, cannot get past 999999
, anything above this value reverts to default 1024
nor I can set it to unlimited
Here is my limits.conf
* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999
The above works fine and ulimit -n
return 999999
.
But, if I make the value one step higher or to unlimited
or -1
or infinity
it doesn't work. ulimit -n
return 1024
I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage
Thanks
ulimit resource-limiting
ulimit resource-limiting
edited Mar 23 '18 at 12:25
Drakonoved
7462515
7462515
asked Mar 22 '18 at 19:39
ShanShan
338136
338136
1
What is your real problem?
– Jos
Mar 22 '18 at 19:58
Hi Jos, let me make it simple for you. Thought the pam_limits documentation saysAll items support the values -1, unlimited or infinity indicating no limit, except for priority and nice.
I can't setunlimited
to nofile, if I do soulimit -n
returns 1024, not sure if this is an expected behaviour
– Shan
Mar 23 '18 at 13:10
Do you really need more than a million files open?
– Jos
Mar 23 '18 at 13:13
Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.
– Shan
Mar 23 '18 at 13:53
add a comment |
1
What is your real problem?
– Jos
Mar 22 '18 at 19:58
Hi Jos, let me make it simple for you. Thought the pam_limits documentation saysAll items support the values -1, unlimited or infinity indicating no limit, except for priority and nice.
I can't setunlimited
to nofile, if I do soulimit -n
returns 1024, not sure if this is an expected behaviour
– Shan
Mar 23 '18 at 13:10
Do you really need more than a million files open?
– Jos
Mar 23 '18 at 13:13
Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.
– Shan
Mar 23 '18 at 13:53
1
1
What is your real problem?
– Jos
Mar 22 '18 at 19:58
What is your real problem?
– Jos
Mar 22 '18 at 19:58
Hi Jos, let me make it simple for you. Thought the pam_limits documentation says
All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice.
I can't set unlimited
to nofile, if I do so ulimit -n
returns 1024, not sure if this is an expected behaviour– Shan
Mar 23 '18 at 13:10
Hi Jos, let me make it simple for you. Thought the pam_limits documentation says
All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice.
I can't set unlimited
to nofile, if I do so ulimit -n
returns 1024, not sure if this is an expected behaviour– Shan
Mar 23 '18 at 13:10
Do you really need more than a million files open?
– Jos
Mar 23 '18 at 13:13
Do you really need more than a million files open?
– Jos
Mar 23 '18 at 13:13
Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.
– Shan
Mar 23 '18 at 13:53
Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.
– Shan
Mar 23 '18 at 13:53
add a comment |
2 Answers
2
active
oldest
votes
There are three files to modify to get this to work:
/etc/security/limits.conf
/etc/pam.d/common-session
/etc/pam.d/common-session-noninteractive
You have already added the required lines in the first file, then in the other two files this line is needed:
session required pam_limits.so
Please note:
The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.
Hence if such a process exists then you will have to modify the config file to get it to work.
Source:
https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors
Thanks George. I understand all the above. The issue here is setting it tounlimted
is not working. I mean setting that in upstart, systemd scripts of the process as well.
– Shan
Mar 23 '18 at 13:07
add a comment |
I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.
* hard nofile 999999
* soft nofile 999999
root hard nofile 999999
root soft nofile 999999
After doing this you should see:
$ ulimit -n
999999
$
Your file should have always worked for root as the limits were correctly defined.
Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:
$ cat /proc/sys/fs/nr_open
1048576
$
You also have to start a new session for the limits to update.
New contributor
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1018351%2fcannot-increase-open-file-limit-beyond-999999-on-14-04%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
There are three files to modify to get this to work:
/etc/security/limits.conf
/etc/pam.d/common-session
/etc/pam.d/common-session-noninteractive
You have already added the required lines in the first file, then in the other two files this line is needed:
session required pam_limits.so
Please note:
The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.
Hence if such a process exists then you will have to modify the config file to get it to work.
Source:
https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors
Thanks George. I understand all the above. The issue here is setting it tounlimted
is not working. I mean setting that in upstart, systemd scripts of the process as well.
– Shan
Mar 23 '18 at 13:07
add a comment |
There are three files to modify to get this to work:
/etc/security/limits.conf
/etc/pam.d/common-session
/etc/pam.d/common-session-noninteractive
You have already added the required lines in the first file, then in the other two files this line is needed:
session required pam_limits.so
Please note:
The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.
Hence if such a process exists then you will have to modify the config file to get it to work.
Source:
https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors
Thanks George. I understand all the above. The issue here is setting it tounlimted
is not working. I mean setting that in upstart, systemd scripts of the process as well.
– Shan
Mar 23 '18 at 13:07
add a comment |
There are three files to modify to get this to work:
/etc/security/limits.conf
/etc/pam.d/common-session
/etc/pam.d/common-session-noninteractive
You have already added the required lines in the first file, then in the other two files this line is needed:
session required pam_limits.so
Please note:
The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.
Hence if such a process exists then you will have to modify the config file to get it to work.
Source:
https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors
There are three files to modify to get this to work:
/etc/security/limits.conf
/etc/pam.d/common-session
/etc/pam.d/common-session-noninteractive
You have already added the required lines in the first file, then in the other two files this line is needed:
session required pam_limits.so
Please note:
The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.
Hence if such a process exists then you will have to modify the config file to get it to work.
Source:
https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors
answered Mar 22 '18 at 21:35
George UdosenGeorge Udosen
20.6k94568
20.6k94568
Thanks George. I understand all the above. The issue here is setting it tounlimted
is not working. I mean setting that in upstart, systemd scripts of the process as well.
– Shan
Mar 23 '18 at 13:07
add a comment |
Thanks George. I understand all the above. The issue here is setting it tounlimted
is not working. I mean setting that in upstart, systemd scripts of the process as well.
– Shan
Mar 23 '18 at 13:07
Thanks George. I understand all the above. The issue here is setting it to
unlimted
is not working. I mean setting that in upstart, systemd scripts of the process as well.– Shan
Mar 23 '18 at 13:07
Thanks George. I understand all the above. The issue here is setting it to
unlimted
is not working. I mean setting that in upstart, systemd scripts of the process as well.– Shan
Mar 23 '18 at 13:07
add a comment |
I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.
* hard nofile 999999
* soft nofile 999999
root hard nofile 999999
root soft nofile 999999
After doing this you should see:
$ ulimit -n
999999
$
Your file should have always worked for root as the limits were correctly defined.
Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:
$ cat /proc/sys/fs/nr_open
1048576
$
You also have to start a new session for the limits to update.
New contributor
add a comment |
I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.
* hard nofile 999999
* soft nofile 999999
root hard nofile 999999
root soft nofile 999999
After doing this you should see:
$ ulimit -n
999999
$
Your file should have always worked for root as the limits were correctly defined.
Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:
$ cat /proc/sys/fs/nr_open
1048576
$
You also have to start a new session for the limits to update.
New contributor
add a comment |
I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.
* hard nofile 999999
* soft nofile 999999
root hard nofile 999999
root soft nofile 999999
After doing this you should see:
$ ulimit -n
999999
$
Your file should have always worked for root as the limits were correctly defined.
Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:
$ cat /proc/sys/fs/nr_open
1048576
$
You also have to start a new session for the limits to update.
New contributor
I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.
* hard nofile 999999
* soft nofile 999999
root hard nofile 999999
root soft nofile 999999
After doing this you should see:
$ ulimit -n
999999
$
Your file should have always worked for root as the limits were correctly defined.
Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:
$ cat /proc/sys/fs/nr_open
1048576
$
You also have to start a new session for the limits to update.
New contributor
New contributor
answered 19 mins ago
HamishHamish
1
1
New contributor
New contributor
add a comment |
add a comment |
Thanks for contributing an answer to Ask Ubuntu!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1018351%2fcannot-increase-open-file-limit-beyond-999999-on-14-04%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
What is your real problem?
– Jos
Mar 22 '18 at 19:58
Hi Jos, let me make it simple for you. Thought the pam_limits documentation says
All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice.
I can't setunlimited
to nofile, if I do soulimit -n
returns 1024, not sure if this is an expected behaviour– Shan
Mar 23 '18 at 13:10
Do you really need more than a million files open?
– Jos
Mar 23 '18 at 13:13
Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.
– Shan
Mar 23 '18 at 13:53