[Tutor] subprocess.Popen(..., cwd) and UNC paths

[Albert-Jan Roskam]

Hello,

Windows has the 'feature' that the CD command does not work with UNC paths.
So in the code below, I cannot use the 'cwd' parameter of subprocess.Popen.
Therefore I use pushd/popd to accomplish the same effect. Is there a better way to do this?
(other than using full path names everywhere, or os.chdir). Would it be a useful improvement
of Python itself if cwd also works with UNC path?

import sys
import os
import getpass
import subprocess

path = r'\\server\share\possibly with\space'
executable = 'blah.exe'
username = os.getenv("USERNAME")
password = getpass.getpass("Enter password: ")
infile = sys.argv[1]
outfile = sys.argv[2]
cmds = ['pushd "%s" &&' % path, executable, username, password, infile, outfile, "&& popd"]

result = subprocess.Popen(" ".join(cmds), shell=True)
error = result.stderr
if error:
raise RuntimeError(error.read())

Regards,

Albert-Jan

PS: Python 2.7 on Windows 7 32



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a

fresh water system, and public health, what have the Romans ever done for us?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Dave Angel]

Post by Albert-Jan Roskam
Hello,
Windows has the 'feature' that the CD command does not work with UNC paths.
So in the code below, I cannot use the 'cwd' parameter of subprocess.Popen.
Therefore I use pushd/popd to accomplish the same effect. Is there a better way to do this?
(other than using full path names everywhere, or os.chdir). Would it be a useful improvement
of Python itself if cwd also works with UNC path?
import sys
import os
import getpass
import subprocess
path = r'\\server\share\possibly with\space'
executable = 'blah.exe'
username = os.getenv("USERNAME")
password = getpass.getpass("Enter password: ")
infile = sys.argv[1]
outfile = sys.argv[2]
cmds = ['pushd "%s" &&' % path, executable, username, password, infile, outfile, "&& popd"]
result = subprocess.Popen(" ".join(cmds), shell=True)
error = result.stderr
raise RuntimeError(error.read())
Regards,
Albert-Jan
PS: Python 2.7 on Windows 7 32

Just a comment about Windows. There is a current directory for each
lettered drive partition. But a unc name is not necessarily on any
known drive.

And if executable, infile and outfile might have spaces within them, you
need to quote them as well.

[Peter Otten]

Post by Albert-Jan Roskam
Hello,
Windows has the 'feature' that the CD command does not work with UNC
paths. So in the code below, I cannot use the 'cwd' parameter of
subprocess.Popen. Therefore I use pushd/popd to accomplish the same
effect. Is there a better way to do this? (other than using full path
names everywhere, or os.chdir). Would it be a useful improvement of Python
itself if cwd also works with UNC path?

What is the error you get? I don't have Windows to verify, but a quick look
into the subprocess source suggests that it uses CreateProcess().
Googling for that finds

<https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx>

which states

"""
lpCurrentDirectory [in, optional]
The full path to the current directory for the process. The string can also
specify a UNC path.
"""

Post by Albert-Jan Roskam
import sys
import os
import getpass
import subprocess
path = r'\\server\share\possibly with\space'
executable = 'blah.exe'
username = os.getenv("USERNAME")
password = getpass.getpass("Enter password: ")
infile = sys.argv[1]
outfile = sys.argv[2]
cmds = ['pushd "%s" &&' % path, executable, username, password, infile, outfile, "&& popd"]
result = subprocess.Popen(" ".join(cmds), shell=True)
error = result.stderr
raise RuntimeError(error.read())
Regards,
Albert-Jan
PS: Python 2.7 on Windows 7 32
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine,
public order, irrigation, roads, a
fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
_______________________________________________
https://mail.python.org/mailman/listinfo/tutor

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Albert-Jan Roskam]

-----------------------------

Post by Peter Otten

Post by Albert-Jan Roskam
Hello,
Windows has the 'feature' that the CD command does not work with UNC
paths. So in the code below, I cannot use the 'cwd' parameter of
subprocess.Popen. Therefore I use pushd/popd to accomplish the same
effect. Is there a better way to do this? (other than using full path
names everywhere, or os.chdir). Would it be a useful improvement of Python
itself if cwd also works with UNC path?

What is the error you get? I don't have Windows to verify, but a quick look
into the subprocess source suggests that it uses CreateProcess().
Googling for that finds
<https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx>
which states
""
lpCurrentDirectory [in, optional]
The full path to the current directory for the process. The string can also
specify a UNC path.
""

Hmmm, that sounds pretty convincing indeed (makes it even stranger that CD works the way it works). I believe it threw a WindowsError, indicating that the file(s) could not be found, because the dir was not changed. I actually first ran into this problem with this script, so with my current script I immediately refrained from using cwd: http://code.activestate.com/recipes/578883-git-pre-commit-hook-to-reject-large-files-using-py/
Git gave a fatal error in windows and the pushd/popd fixed it.

Post by Peter Otten

Post by Albert-Jan Roskam
import sys
import os
import getpass
import subprocess
path = r'\\server\share\possibly with\space'
executable = 'blah.exe'
username = os.getenv("USERNAME")
password = getpass.getpass("Enter password: ")
infile = sys.argv[1]
outfile = sys.argv[2]
cmds = ['pushd "%s" &&' % path, executable, username, password, infile,
outfile, "&& popd"]
result = subprocess.Popen(" ".join(cmds), shell=True)
error = result.stderr
raise RuntimeError(error.read())
Regards,
Albert-Jan
PS: Python 2.7 on Windows 7 32

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[eryksun]

On Wed, Apr 29, 2015 at 11:54 AM, Albert-Jan Roskam

Post by Albert-Jan Roskam
Hmmm, that sounds pretty convincing indeed (makes it even stranger that CD works the way it works).
I believe it threw a WindowsError, indicating that the file(s) could not be found, because the dir was
not changed. I actually first ran into this problem with this script, so with my current script I
http://code.activestate.com/recipes/578883-git-pre-commit-hook-to-reject-large-files-using-py/
Git gave a fatal error in windows and the pushd/popd fixed it.

I don't see why you'd need shell=True. Windows supports UNC paths in
the working directory, but the cmd.exe shell (being a crusty relic of
the 1980s) does not. So just use the default value, shell=False.
(However, on POSIX systems, unlike Windows, using shell=False requires
`cmd` to be a list.) BTW, there's no need to explicitly pass
cwd=os.getcwd(). The default behavior is to inherit the working
directory of the current process.
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alan Gauld]

Post by eryksun
the working directory, but the cmd.exe shell (being a crusty relic of
the 1980s) does not.

Actually cmd.exe is fine with UNC paths. It's only when you
combine them with Windows /option style that it has issues
but even then putting the path in quotes will usually do
the trick.

It's the old MS-DOS COMMAND.COM that can't cope.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[eryksun]

Post by Alan Gauld

Post by eryksun
the working directory, but the cmd.exe shell (being a crusty relic of
the 1980s) does not.

Actually cmd.exe is fine with UNC paths. It's only when you
combine them with Windows /option style that it has issues
but even then putting the path in quotes will usually do
the trick.
It's the old MS-DOS COMMAND.COM that can't cope.

cmd.exe was developed for OS/2 (1987) to replace COMMAND.COM (1981).

cmd.exe cannot use a UNC path as the current directory. What it can do
is `pushd` a UNC path, which mounts it as a drive letter. If you do
try to start cmd.exe with a UNC path as the working directory, it
instead sets the working directory to the Windows directory. For
example:

(test) C:\Temp>python
Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:16:31)
[MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

Post by Alan Gauld

Post by eryksun

import os, subprocess
os.chdir(r'\\127.0.0.1\C$')
subprocess.call('cmd')

'\\127.0.0.1\C$'
CMD.EXE was started with the above path as the current directory.
UNC paths are not supported. Defaulting to Windows directory.
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

(test) C:\Windows>
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[eryksun]

Post by eryksun
cmd.exe was developed for OS/2 (1987) to replace COMMAND.COM (1981).

Actually, I stand corrected about the reason being cmd's 1980s
crustiness. Per [KB156276][1] this check was added to NT 4.0 (1996) to
address a vaguely described problem ("a UNC name may cause problems
with child processes launched from such a console when that console is
exited or halted"), which may not even be a problem nowadays (Windows
7+) since the console was moved out of CSRSS.EXE (the Windows session
server) into instances of conhost.exe that run in the security context
of the client. Try adding the registry setting to disable this check.
Probably nothing bad will happen. Here's a reg.exe command to disable
the check for the current user:

reg add "HKCU\Software\Microsoft\Command Processor"
/v DisableUNCCheck /t REG_DWORD /d 1

After disabling the check, my previous example should work fine:

(test) C:\Temp>python
Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:16:31) [MSC
v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

Post by eryksun

import os, subprocess
os.chdir(r'\\127.0.0.1\C$')
subprocess.call('cmd')

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

(test) \\127.0.0.1\C$>

[1]: https://support.microsoft.com/en-us/kb/156276
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[eryksun]

Except it doesn't accept paths relative to a UNC working directory:

(test) \\127.0.0.1\C$>cd Temp
The system cannot find the path specified.

And the cd command appears to ignore the registry setting:

(test) \\127.0.0.1\C$>cd \\127.0.0.1\C$\Temp
'\\127.0.0.1\C$\Temp'
CMD does not support UNC paths as current directories.

Otherwise it works fine. ;-)
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alan Gauld]

Post by eryksun

Post by Alan Gauld
Actually cmd.exe is fine with UNC paths.

cmd.exe cannot use a UNC path as the current directory.

Oops, my mistake. I got my POSIX and UNC mixed up.
I was thinking about forward slashes etc not network names.

Apologies for not reading the message properly.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[eryksun]

I used a str for cmd because I found it more readable that way. I could do cmd.split().

Don't use cmd.split(). That just splits on whitespace without
respecting how the shell tokenizes the command. Use shlex.split(cmd)
instead.

So os.getcwd() == os.path.dirname(os.path.abspath(__file__)) here?

No, a Python script could be started from any directory, while the RHS
there is specifically the directory of the script. In terms of a
crusty old cmd script we're talking about the difference between %cd%
and %~dp0 (where d=drive and p=path).
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor