Repository navigation
resource library (getrlimit, setrlimit) interprets resource limits as signed integers instead of unsigned #137044
Description
Bug report
Bug description:
- as per System Calls Manual (man 2 setrlimit), rlim_t is an unsigned integer type
- in Python, reading the default hard limit (resource.getrlimit(resource.RLIMIT_CPU)) will return -1
- executing this:
resource.setrlimit(resource.RLIMIT_CPU, (-2, -2))
os.system("ulimit -t")returns 18446744073709551614
- executing ulimit -t 18446744073709551614, then in python3 resource.getrlimit(resource.RLIMIT_CPU), will return -2
This behaviour is very weird and will cause the following rather useful and common snippet to fail:
_, hardlimit = resource.getrlimit(resource.RLIMIT_CPU)
rlimit = (min(timelimit, hardlimit), hardlimit)
resource.setrlimit(resource.RLIMIT_CPU, rlimit)There happens to exist the following workaround:
_, hardlimit = resource.getrlimit(resource.RLIMIT_CPU)
softlimit = ctypes.c_long(min(seconds, ctypes.c_ulong(hardlimit).value)).value
rlimit = (softlimit, hardlimit)
resource.setrlimit(resource.RLIMIT_CPU, rlimit)but it just feels extremely hacky to do things like this and I believe a lot of programmers end in this trap.
Especially that when you handle the -1 case specially (as it is mentioned in man, that "RLIM_INFINITY typically is the same as -1"), it still fails with other negative values like -2.
CPython versions tested on:
3.15
Operating systems tested on:
Linux
Linked PRs
- gh-137044: Support large limit values in getrlimit() and setrlimit() #137338
- [3.14] gh-137044: Support large limit values in getrlimit() and setrlimit() (GH-137338) #137506
- [3.13] gh-137044: Support large limit values in getrlimit() and setrlimit() (GH-137338) #137507
- gh-137044: Make resource.RLIM_INFINITY always positive #137511
- gh-137044: Revert "gh-137044: Make resource.RLIM_INFINITY always positive (GH-137511)" #137938
- gh-137044: Fix test_resource on 32-bit Linux #137941
- gh-137044: To weaken the statement regarding the RLIM_INFINITY value #137954
Activity
serhiy-storchaka commented on Jul 23, 2025
This is one of the cases where the integer type is unsigned, but -1 is a special value. RLIM_INFINITY is defined as ((__rlim_t) -1). It is more convenient to get/set -1 on all platforms than 0xffffffff or 0xffffffffffffffff depending on platform.
There is no standard way to handle this. We have special converters for uid_t, gid_t and dev_t. It seems that it is also needed for rlim_t.
serhiy-storchaka commented on Jul 23, 2025
I assigned this to myself because it requires writing non-trivial bulky code, and I've already started.
pythongh-137044: Support large limit values in getrlimit() and setrli…
serhiy-storchaka commented on Aug 3, 2025
User code can depends on the fact that RLIM_INFINITY is -1 (although this is not documented). But there are reasons to represent it a large positive value, larger than any finite limit. One of them shown in the original report, but there is also the glibc feature on 32-bit platforms: if a program tried to set a resource limit to a value larger than can be represented in a 32-bit unsigned long, then the glibc setrlimit() wrapper function silently converted the limit value to RLIM_INFINITY.
So, I am planning two changes:
serhiy-storchaka commented on Aug 6, 2025
On some platforms RLIM_INFINITY is not (rlim_t)-1, but (rlim_t)-3, 2**31-1 or 2**63-1. On FreeBSD rlim_t is signed.
gh-137044: Support large limit values in getrlimit() and setrlimit() (G…
pythongh-137044: Support large limit values in getrlimit() and setrli…
pythongh-137044: Support large limit values in getrlimit() and setrli…
pythongh-137044: Make resource.RLIM_INFINITY always positive
[3.13] gh-137044: Support large limit values in getrlimit() and setrl…
encukou commented on Aug 7, 2025
There are two parts to step 2:
I don't think we should make the second change. It seems it could break an unknown number of quick & dirty platform-specific scripts.
Thinking about it, I'm -0 on making RLIM_INFINITY positive, too. If it's -1 (or -3 on some systems), I can just print it out from a script and it'll make sense. If it's a huge number, it needs the "proper" treatment (an if and print "unlimited" instead) -- too much extra work in a quick admin script.
ArturBieniek4 commented on Aug 7, 2025
Why shouldn't it just work like in C?
I also don't think passing negative integers should be disallowed - why not just allow them but interpret them as unsigned values?
9 remaining items
encukou commented on Aug 19, 2025
AFAIU, that sentence means that whenever you have RLIM_INFINITY you should consider it infinite, regardless of what the actual numeric value is.
In other words, the value doesn't matter. It should not be used for numeric comparisons.
Revert "pythongh-137044: Make resource.RLIM_INFINITY always positive (p…
pythongh-137044: Fix test_resource on 32-bit Linux
serhiy-storchaka commented on Aug 19, 2025
Even if it can be interpreted that way, I don't know of any platform where RLIM_INFINITY is not greater than any other value (except for the special values RLIM_SAVED_MAX and RLIM_SAVED_CUR). I think a lot of code depends on that, so it would be not safe to define RLIM_INFINITY any other way.
encukou commented on Aug 19, 2025
Do you have any examples?
ArturBieniek4 commented on Aug 19, 2025
The most trivial example is already given at the beginning of this issue:
I genuinely wrote this code and expected it to work and was so surprised it did not.
encukou commented on Aug 19, 2025
This would suggest that existing, working Python code does not depend on RLIM_INFINITY being a large int. (As it should not, per Posix.)
serhiy-storchaka commented on Aug 19, 2025
But a similar code in C works.
Well, I'll remove that sentence if you insist.
pythongh-137044: Support large limit values in getrlimit() and setrli…
pythongh-137044: Make resource.RLIM_INFINITY always positive (pythonG…
gh-137044: Fix test_resource on 32-bit Linux (GH-137941)
pythongh-137044: To weaken the statement regarding the RLIM_INFINITY …
gh-137044: To weaken the statement regarding the RLIM_INFINITY value (G…
[3.14] gh-137044: Support large limit values in getrlimit() and setrl…