Bug report

Bug description:

When allocating a SharedMemory block on linux, if there's insufficient shared memory on the system, creating the block works but accessing it triggers SIGBUS and the process crashes.

docker run --shm-size=3k --rm -t python:3.12.1 bash -c "python -c '
from multiprocessing.shared_memory import SharedMemory
while True:
  shm = SharedMemory(create=True, size=1024)
  print(1)
  shm.buf[0] = 0
  print(2)
print(3)
' ; echo"

# output:
1
2
1
bash: line 9:     7 Bus error               python -c '
from multiprocessing.shared_memory import SharedMemory
while True:
  shm = SharedMemory(create=True, size=1024)
  print(1)
  shm.buf[0] = 0
  print(2)
print(3)
'

Cause and Solution

A file is created under /dev/shm and its size is reserved using ftruncate() in SharedMemory.__init__.
This in fact does not allocate the memory and causes SIGBUS to be sent when accessing the memory.
Instead, a call to fallocate() would throw an error when out of memory which can be caught and handled correctly by the user.
The existing code already captures OSError when calling ftruncate() but no error is thrown in this situation.

Note that this change should only run on linux - see explanation down in references.

So my suggested solution is:

class SharedMemory:
    ...

    def __init__(self, name=None, create=False, size=0):
        ...

        if _USE_POSIX:
            ...  # set self._fd
            try:
                if create and size:
-                    os.ftruncate(self._fd, size)
+                  if _IS_LINUX:
+                      os.posix_fallocate(self._fd, 0, size)
+                  else:
+                      os.ftruncate(self._fd, size)
                stats = os.fstat(self._fd)
                size = stats.st_size
                self._mmap = mmap.mmap(self._fd, size)
            except OSError:
                self.unlink()
                raise

            ...

References

Chromium

A similar issue was found and fixed in chromium in 2019 - https://bugs.chromium.org/p/chromium/issues/detail?id=951431

Firefox

Also in mozilla core in 2016 - https://bugzilla.mozilla.org/show_bug.cgi?id=1245239

PostgreSQL

In 2017 a similar issue was fixed in postgres - postgres/postgres@899bd78

The change should only be for linux, as summarized in their commit:

Make this code #ifdef linux, because (a) there's not currently a
reason to think the same problem exists on other platforms, and (b)
applying posix_fallocate() to an FD created by shm_open() isn't very
portable anyway.

Others

• Improve error handling if /dev/shm too small llnl/UnifyFS#72

CPython versions tested on:

3.8, 3.9, 3.10, 3.11, 3.12, 3.13

Operating systems tested on:

Linux