Activity

1. Unode commented on Apr 17, 2016

   

   Unodemannequin

   on Apr 17, 2016

   MannequinAuthor

   More actions

   Hi everyone,

   I'm not really sure if this is a new issue but digging through the bug reports from the past I couldn't find an answer.
   There's http://bugs.python.org/issue1438480 but this seems to be a different issue.
   I also found http://bugs.python.org/issue9993 that addressed problems with symlinks but didn't correct the behavior reported here.

   The problem can be visualized with the following code.
   Code fails on python 2.7 as well as python 3.4+. Not tested in python <2.7 and <3.4.

       import shutil
       import os
       
       TMPDIR = "/tmp/tmpdir"
       TESTLINK = "test_dir"
       
       if not os.path.isdir(TMPDIR):
           os.mkdir(TMPDIR)
       
       if not os.path.islink(TESTLINK):
           os.symlink(TMPDIR, TESTLINK)
       
       shutil.move(TESTLINK, TMPDIR)

   When executed it gives me:

       % python3 test.py
       Traceback (most recent call last):
         File "test.py", line 14, in <module>
           shutil.move(TESTLINK, TMPDIR)
         File "/usr/lib64/python3.4/shutil.py", line 516, in move
           os.rename(src, dst)
       OSError: [Errno 18] Invalid cross-device link: 'test_dir' -> '/tmp/tmpdir'

   This happens because /tmp is:

   tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime,nodiratime)

   In the past the recommendation to handle this problem was to stop using os.rename and use shutil.move instead.
   This was even discussed in a bug report - http://bugs.python.org/issue14848

   If one searches for this exception there's plenty of advice [1][2][3][4] in the same direction.
   However, given that shutil.move uses os.rename internally, the problem returns.

   On the other end doing the equivalent action in the shell with 'mv' works fine.

   [1] - http://stackoverflow.com/a/15300474
   [2] - https://mail.python.org/pipermail/python-list/2005-February/342892.html
   [3] - http://www.thecodingforums.com/threads/errno-18-invalid-cross-device-link-using-os-rename.341597/
   [4] - pypa/pip#103
2. Unode

   added

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on Apr 17, 2016
3. Unode commented on Apr 17, 2016

   

   Unodemannequin

   on Apr 17, 2016

   MannequinAuthor

   More actions

   Also related to http://bugs.python.org/issue212317
4. SilentGhost commented on Apr 18, 2016

   

   SilentGhostmannequin

   on Apr 18, 2016

   Mannequin

   More actions

   This seems to be only triggered when moving a symlink into its destination. Assumption in the code is that when _samefile returns True, it must be due to case-insensitive filesystem, rather than this edge case.
5. SilentGhost

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Apr 18, 2016
6. MurrayWilson commented on Jul 31, 2020

   

   MurrayWilsonmannequin

   on Jul 31, 2020

   Mannequin

   More actions

   It also happens when moving a file in linux from an xfs filesystem to a NFS mounted filesystem.
7. websurfer5 commented on Aug 6, 2020

   

   websurfer5mannequin

   on Aug 6, 2020

   Mannequin

   More actions

   SilentGhost's analysis is correct. The provided example code causes the error because it is trying to move the symlink into its target when the target is a directory. Any cross-device moving issues are unrelated to this example code. Here is the relevant code in the master branch:

       if os.path.isdir(dst):
           if _samefile(src, dst):
               # We might be on a case insensitive filesystem,
               # perform the rename anyway.
               os.rename(src, dst)
               return

   shutil._samefile() considers the example link and its target to be the same. When _samefile() returns False, this code gets executed:

           real_dst = os.path.join(dst, _basename(src))
   
           if os.path.exists(real_dst):
               raise Error("Destination path '%s' already exists" % real_dst)
       try:
           os.rename(src, real_dst)
       except OSError:
           if os.path.islink(src):
               linkto = os.readlink(src)
               os.symlink(linkto, real_dst)
               os.unlink(src)

   A simple fix is to check whether src is a symlink when _samefile() returns True. The "Destination path...already exists" error isn't a problem for our symlink case because the shell mv command also returns an error.

   $ ls -l /tmp/tmpdir/
   total 0
   lrwxr-xr-x  1 jeff  staff  11 Aug  5 23:36 test_dir -> /tmp/tmpdir
   $ mv test_dir /tmp/tmpdir
   mv: test_dir and /tmp/tmpdir/test_dir are identical

   I will generate a pull request.
8. websurfer5

   added

   3.7 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   3.10 (EOL)end of lifeend of life

   on Aug 6, 2020
9. iritkatriel commented on Nov 30, 2021

   

   iritkatriel

   on Nov 30, 2021

   Member

   More actions

   Removing old versions.
10. iritkatriel

    added

    3.11only security fixesonly security fixes

    and removed

    3.7 (EOL)end of lifeend of life

    3.8 (EOL)end of lifeend of life

    on Nov 30, 2021
11. ezio-melotti

    transferred this issue fromon Apr 10, 2022
12. serhiy-storchaka

    added

    3.12only security fixesonly security fixes

    3.13only security fixesonly security fixes

    and removed

    3.10 (EOL)end of lifeend of life

    3.9 (EOL)end of lifeend of life

    on Dec 27, 2023
13. serhiy-storchaka

    closed this as completedon Dec 27, 2023