Cannot increase open file limit beyond 999999 on 14.04+












0















I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf. But, cannot get past 999999, anything above this value reverts to default 1024 nor I can set it to unlimited



Here is my limits.conf



* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999


The above works fine and ulimit -n return 999999.



But, if I make the value one step higher or to unlimited or -1 or infinity it doesn't work. ulimit -n return 1024



I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage



Thanks










share|improve this question




















  • 1





    What is your real problem?

    – Jos
    Mar 22 '18 at 19:58











  • Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

    – Shan
    Mar 23 '18 at 13:10













  • Do you really need more than a million files open?

    – Jos
    Mar 23 '18 at 13:13











  • Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

    – Shan
    Mar 23 '18 at 13:53
















0















I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf. But, cannot get past 999999, anything above this value reverts to default 1024 nor I can set it to unlimited



Here is my limits.conf



* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999


The above works fine and ulimit -n return 999999.



But, if I make the value one step higher or to unlimited or -1 or infinity it doesn't work. ulimit -n return 1024



I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage



Thanks










share|improve this question




















  • 1





    What is your real problem?

    – Jos
    Mar 22 '18 at 19:58











  • Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

    – Shan
    Mar 23 '18 at 13:10













  • Do you really need more than a million files open?

    – Jos
    Mar 23 '18 at 13:13











  • Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

    – Shan
    Mar 23 '18 at 13:53














0












0








0








I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf. But, cannot get past 999999, anything above this value reverts to default 1024 nor I can set it to unlimited



Here is my limits.conf



* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999


The above works fine and ulimit -n return 999999.



But, if I make the value one step higher or to unlimited or -1 or infinity it doesn't work. ulimit -n return 1024



I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage



Thanks










share|improve this question
















I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf. But, cannot get past 999999, anything above this value reverts to default 1024 nor I can set it to unlimited



Here is my limits.conf



* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999


The above works fine and ulimit -n return 999999.



But, if I make the value one step higher or to unlimited or -1 or infinity it doesn't work. ulimit -n return 1024



I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage



Thanks







ulimit resource-limiting






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 23 '18 at 12:25









Drakonoved

7462515




7462515










asked Mar 22 '18 at 19:39









ShanShan

338136




338136








  • 1





    What is your real problem?

    – Jos
    Mar 22 '18 at 19:58











  • Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

    – Shan
    Mar 23 '18 at 13:10













  • Do you really need more than a million files open?

    – Jos
    Mar 23 '18 at 13:13











  • Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

    – Shan
    Mar 23 '18 at 13:53














  • 1





    What is your real problem?

    – Jos
    Mar 22 '18 at 19:58











  • Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

    – Shan
    Mar 23 '18 at 13:10













  • Do you really need more than a million files open?

    – Jos
    Mar 23 '18 at 13:13











  • Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

    – Shan
    Mar 23 '18 at 13:53








1




1





What is your real problem?

– Jos
Mar 22 '18 at 19:58





What is your real problem?

– Jos
Mar 22 '18 at 19:58













Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

– Shan
Mar 23 '18 at 13:10







Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

– Shan
Mar 23 '18 at 13:10















Do you really need more than a million files open?

– Jos
Mar 23 '18 at 13:13





Do you really need more than a million files open?

– Jos
Mar 23 '18 at 13:13













Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

– Shan
Mar 23 '18 at 13:53





Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

– Shan
Mar 23 '18 at 13:53










2 Answers
2






active

oldest

votes


















0














There are three files to modify to get this to work:




  • /etc/security/limits.conf

  • /etc/pam.d/common-session

  • /etc/pam.d/common-session-noninteractive


You have already added the required lines in the first file, then in the other two files this line is needed:



session required pam_limits.so


Please note:




The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




Hence if such a process exists then you will have to modify the config file to get it to work.



Source:
https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors






share|improve this answer
























  • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

    – Shan
    Mar 23 '18 at 13:07



















0














I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



* hard nofile 999999
* soft nofile 999999
root hard nofile 999999
root soft nofile 999999


After doing this you should see:



$ ulimit -n
999999
$


Your file should have always worked for root as the limits were correctly defined.



Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



$ cat /proc/sys/fs/nr_open
1048576
$


You also have to start a new session for the limits to update.






share|improve this answer








New contributor




Hamish is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "89"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1018351%2fcannot-increase-open-file-limit-beyond-999999-on-14-04%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    There are three files to modify to get this to work:




    • /etc/security/limits.conf

    • /etc/pam.d/common-session

    • /etc/pam.d/common-session-noninteractive


    You have already added the required lines in the first file, then in the other two files this line is needed:



    session required pam_limits.so


    Please note:




    The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




    Hence if such a process exists then you will have to modify the config file to get it to work.



    Source:
    https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors






    share|improve this answer
























    • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

      – Shan
      Mar 23 '18 at 13:07
















    0














    There are three files to modify to get this to work:




    • /etc/security/limits.conf

    • /etc/pam.d/common-session

    • /etc/pam.d/common-session-noninteractive


    You have already added the required lines in the first file, then in the other two files this line is needed:



    session required pam_limits.so


    Please note:




    The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




    Hence if such a process exists then you will have to modify the config file to get it to work.



    Source:
    https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors






    share|improve this answer
























    • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

      – Shan
      Mar 23 '18 at 13:07














    0












    0








    0







    There are three files to modify to get this to work:




    • /etc/security/limits.conf

    • /etc/pam.d/common-session

    • /etc/pam.d/common-session-noninteractive


    You have already added the required lines in the first file, then in the other two files this line is needed:



    session required pam_limits.so


    Please note:




    The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




    Hence if such a process exists then you will have to modify the config file to get it to work.



    Source:
    https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors






    share|improve this answer













    There are three files to modify to get this to work:




    • /etc/security/limits.conf

    • /etc/pam.d/common-session

    • /etc/pam.d/common-session-noninteractive


    You have already added the required lines in the first file, then in the other two files this line is needed:



    session required pam_limits.so


    Please note:




    The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




    Hence if such a process exists then you will have to modify the config file to get it to work.



    Source:
    https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Mar 22 '18 at 21:35









    George UdosenGeorge Udosen

    20.6k94568




    20.6k94568













    • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

      – Shan
      Mar 23 '18 at 13:07



















    • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

      – Shan
      Mar 23 '18 at 13:07

















    Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

    – Shan
    Mar 23 '18 at 13:07





    Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

    – Shan
    Mar 23 '18 at 13:07













    0














    I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



    * hard nofile 999999
    * soft nofile 999999
    root hard nofile 999999
    root soft nofile 999999


    After doing this you should see:



    $ ulimit -n
    999999
    $


    Your file should have always worked for root as the limits were correctly defined.



    Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



    $ cat /proc/sys/fs/nr_open
    1048576
    $


    You also have to start a new session for the limits to update.






    share|improve this answer








    New contributor




    Hamish is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.

























      0














      I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



      * hard nofile 999999
      * soft nofile 999999
      root hard nofile 999999
      root soft nofile 999999


      After doing this you should see:



      $ ulimit -n
      999999
      $


      Your file should have always worked for root as the limits were correctly defined.



      Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



      $ cat /proc/sys/fs/nr_open
      1048576
      $


      You also have to start a new session for the limits to update.






      share|improve this answer








      New contributor




      Hamish is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.























        0












        0








        0







        I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



        * hard nofile 999999
        * soft nofile 999999
        root hard nofile 999999
        root soft nofile 999999


        After doing this you should see:



        $ ulimit -n
        999999
        $


        Your file should have always worked for root as the limits were correctly defined.



        Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



        $ cat /proc/sys/fs/nr_open
        1048576
        $


        You also have to start a new session for the limits to update.






        share|improve this answer








        New contributor




        Hamish is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.










        I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



        * hard nofile 999999
        * soft nofile 999999
        root hard nofile 999999
        root soft nofile 999999


        After doing this you should see:



        $ ulimit -n
        999999
        $


        Your file should have always worked for root as the limits were correctly defined.



        Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



        $ cat /proc/sys/fs/nr_open
        1048576
        $


        You also have to start a new session for the limits to update.







        share|improve this answer








        New contributor




        Hamish is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.









        share|improve this answer



        share|improve this answer






        New contributor




        Hamish is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.









        answered 19 mins ago









        HamishHamish

        1




        1




        New contributor




        Hamish is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.





        New contributor





        Hamish is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.






        Hamish is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Ask Ubuntu!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1018351%2fcannot-increase-open-file-limit-beyond-999999-on-14-04%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            GameSpot

            日野市

            Tu-95轟炸機