[Tutor] try and file existence

[Clayton Kirkwood]

try:
fp = open( user_preferences )
except( PermissionError ):
else:
with open(user_preferences ) as f:

I originally only had the bottom open statement. Ran but file didn't exist,
and my run failed with file doesn't exist. I figured I'd check to see if the
file existed. This is one of those situations where a search of
documentation for fd_exist (which I thought I'd seen once), or exist turns
up either nothing or nothing relevant. I finally found that the try: clause
with the open statement might help and I copied the snippet to my code. I am
getting an indentation error: expected an indent block. What is wrong, and
what is the best way to find out if a file exists?

TIA,

Clayton

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Cameron Simpson]

Post by Clayton Kirkwood
fp = open( user_preferences )
I originally only had the bottom open statement. Ran but file didn't exist,
and my run failed with file doesn't exist. I figured I'd check to see if the
file existed. This is one of those situations where a search of
documentation for fd_exist (which I thought I'd seen once), or exist turns
up either nothing or nothing relevant. I finally found that the try: clause
with the open statement might help and I copied the snippet to my code. I am
getting an indentation error: expected an indent block. What is wrong, and
what is the best way to find out if a file exists?

In purely syntactic terms you need some code in the suite under the "except"
clause, and you don't want the brackets:

try:
fp = open( user_preferences )
except PermissionError as e:
print("open(%r) fails: %s" % (user_preferences, e))
else:
with open(user_preferences ) as f:

In logical terms, the "with" is not wanted - you're opening the file again.
Leaving aside the logical issue there, this structure (test then operate) is
also racy: suppose the file has its attributes changed or is removed between
the first open and the second.

Next: you're catching PermissionError. That normally means that you have not
got rights for opening the file; you will get a different exception if the file
does not exist. You're being too precise if you want both.

But maybe you don't. You need to decide

Finally, the usual Python pattern is not to catch exceptions _at all_ unless
you have a deliberate polciy plan for what to do. Normally you would have some
function looking like this:

def read_prefs(filename):
prefs = {}
with open(filename) as fp:
... read preferences, save in prefs dict for example ...
return prefs

If the file is missing or you have no permissions, that will raise an
exception. Let it!

Then in an outer layer of your program you might catch the exception, where you
can make a policy decision because you have a wider view of what is going on:

try:
prefs = read_prefs(prefs_filename)
except FileNotFoundError as e:
print("warning: file not found: %r: %s" % (prefs_filename, e))
# proceed with empty preferences, not an error
prefs = {}

This bit of code catches _only_ FileNotFoundError on the (presumed) policy that
a missing preferences file is ok - your program will proceed with default
behaviours - but any _other_ kind of exception is not expected - let your
program abort! Do not proceed!

Cheers,
Cameron Simpson <***@zip.com.au>

Capitalism is the extraordinary belief that the nastiest of men, for the
nastiest of reasons, will somehow work for the benefit of us all.
- John Maynard Keynes
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Mark Lawrence]

Post by Clayton Kirkwood
fp = open( user_preferences )

You need a pass statement here if you don't intend doing anything with
the error, but see my comments at the bottom.

Post by Clayton Kirkwood
I originally only had the bottom open statement. Ran but file didn't exist,
and my run failed with file doesn't exist. I figured I'd check to see if the
file existed. This is one of those situations where a search of
documentation for fd_exist (which I thought I'd seen once), or exist turns
up either nothing or nothing relevant. I finally found that the try: clause
with the open statement might help and I copied the snippet to my code. I am
getting an indentation error: expected an indent block. What is wrong, and
what is the best way to find out if a file exists?
TIA,
Clayton

There's nothing to stop you using multiple except statements with one
try. So something like this is how I'd go about it.

try:
with open(user_preferences) as f:
do_something()
except PermissionError:
whatever()
except FileNotFoundError:
oh_heck()
etc.

Seee this for an explanation of exception handling
https://docs.python.org/3/tutorial/errors.html#handling-exceptions. A
full list of the exceptions you'd need to consider is here
https://docs.python.org/3/library/exceptions.html#os-exceptions
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Steven D'Aprano]

Post by Clayton Kirkwood
fp = open( user_preferences )

try:
fp = open(user_preferences)
except (IOError, OSError) as e:
handle_error()
else:
with fp as f:
handle_file()


[...]

Post by Clayton Kirkwood
what is the best way to find out if a file exists?

Try to open it and see what happens. If the open() succeeds, then the
file exists and can be read. If it fails, then either the file doesn't
exist, or it can't be read. Inspect the error to find out which.

There is also os.path.exists(filename), but you should avoid using that
if possible. The problem is this:

if os.path.exists(filename):
# file exists *right now*
# but a millisecond later, some other program deletes it...
# and now it doesn't exist any more
with open(filename) as f: # gives an error
...



This is called a "time of check to time of use" bug, and it is a major
cause of security bugs in software. Remember, even if you're the only
*person* using your computer, there could be hundreds of other programs
running in the background, and one of those might delete the file after
you've checked its existence.

Also, just because the file *exists* doesn't mean you can open it.
Perhaps the file is locked, unreadable, you don't have permissions, or
maybe even the disk is faulty and the file is corrupt and unreadable.

[Mark Lawrence]

Post by Steven D'Aprano

Post by Clayton Kirkwood
fp = open( user_preferences )

fp = open(user_preferences)
handle_error()
handle_file()

I'll just point out that you can catch finer grained errors owing to
https://www.python.org/dev/peps/pep-3151/. There is a handy little
table here
https://docs.python.org/3/library/exceptions.html#exception-hierarchy
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[boB Stepp]

Post by Steven D'Aprano

Post by Clayton Kirkwood
what is the best way to find out if a file exists?

Try to open it and see what happens. If the open() succeeds, then the
file exists and can be read. If it fails, then either the file doesn't
exist, or it can't be read. Inspect the error to find out which.
There is also os.path.exists(filename), but you should avoid using that
# file exists *right now*
# but a millisecond later, some other program deletes it...
# and now it doesn't exist any more
with open(filename) as f: # gives an error
...

I understand your points, but wonder then what is the intended use for
os.path.exists()? That is, in what types of circumstances would it be
both appropriate and safe to use?

boB
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Laura Creighton]

Post by boB Stepp
I understand your points, but wonder then what is the intended use for
os.path.exists()? That is, in what types of circumstances would it be
both appropriate and safe to use?
boB

If you want to locate dangling symlinks, os.path.exists
will return False, so the symlink is there, but the file it pointed to
is long gone.

Laura

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Clayton Kirkwood]

-----Original Message-----
Behalf Of Laura Creighton
Sent: Saturday, August 15, 2015 2:49 PM
Subject: Re: [Tutor] try and file existence

Post by boB Stepp
I understand your points, but wonder then what is the intended use for
os.path.exists()? That is, in what types of circumstances would it be
both appropriate and safe to use?
boB

If you want to locate dangling symlinks, os.path.exists will return

False, so

the symlink is there, but the file it pointed to is long gone.

Can't you do that with os.path.open() and get a value in os.path.status? (I
think that is the thing to call)
crk

Laura
_______________________________________________
https://mail.python.org/mailman/listinfo/tutor

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Cameron Simpson]

Post by Clayton Kirkwood

Behalf Of Laura Creighton

[..]

Post by Clayton Kirkwood

Post by boB Stepp
I understand your points, but wonder then what is the intended use for
os.path.exists()? That is, in what types of circumstances would it be
both appropriate and safe to use?

If you want to locate dangling symlinks, os.path.exists will return False, so
the symlink is there, but the file it pointed to is long gone.

Can't you do that with os.path.open() and get a value in os.path.status? (I
think that is the thing to call)

Open does more that os.stat (which is what os.path.exists uses underneath).

There are plenty of times you will want to know a file exists but not have
permission to open it. Also, open can have side effects if the target file is a
device or a socket/pipe.

Always use the smallest thing you can to achieve an effect: stat is smaller
than open.

Cheers,
Cameron Simpson <***@zip.com.au>

Q: How many user support people does it take to change a light bulb?
A: We have an exact copy of the light bulb here and it seems to be
working fine. Can you tell me what kind of system you have?
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Laura Creighton]

Post by Laura Creighton

If you want to locate dangling symlinks, os.path.exists will return

False, so

the symlink is there, but the file it pointed to is long gone.

Can't you do that with os.path.open() and get a value in os.path.status? (I
think that is the thing to call)
crk

Laura

There is no os.path.open
I assume you are thinking of os.open? or open the builtin? or maybe
os.stat?

If what you really want to do is open the file, if it exists, then
trying to open it and then if that fails handle whatever problem you
get is most often the way to go.

But often you never wanted to open it in the first place. Maybe it is
a directory. Maybe it is a lockfile --- oops, I will come back later.
There are a lot of times when the sense you really want is not
'check if this valued file exists' but rather 'check for cruft you
don't want to find'.

Open is also slow, which again isn't a problem if you need to open the
file anyway, but will matter if all you want to do is check your entire
large filesystem for files named 'core'.

Laura

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Steven D'Aprano]

Post by boB Stepp

Post by Steven D'Aprano
There is also os.path.exists(filename), but you should avoid using that
# file exists *right now*
# but a millisecond later, some other program deletes it...
# and now it doesn't exist any more
with open(filename) as f: # gives an error
...

I understand your points, but wonder then what is the intended use for
os.path.exists()? That is, in what types of circumstances would it be
both appropriate and safe to use?

def print_file_names(possible_names):
print("List of file names checked")
print("--------------------------"
for name in possible_names:
if os.path.exists(name):
print(name)
else:
print("missing:", name)

[boB Stepp]

Post by Steven D'Aprano

Post by boB Stepp
I understand your points, but wonder then what is the intended use for
os.path.exists()? That is, in what types of circumstances would it be
both appropriate and safe to use?

print("List of file names checked")
print("--------------------------"
print(name)
print("missing:", name)

<Chuckle!>

Your example, giving about the most benign possible uses, is for emphasis?

boB
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Steven D'Aprano]

Post by boB Stepp

Post by Steven D'Aprano

Post by boB Stepp
I understand your points, but wonder then what is the intended use for
os.path.exists()? That is, in what types of circumstances would it be
both appropriate and safe to use?

print("List of file names checked")
print("--------------------------"
print(name)
print("missing:", name)

<Chuckle!>
Your example, giving about the most benign possible uses, is for emphasis?

Well, not really. I was trying to think of a case where you want to
check whether a file exists but not actually open the file (or at
least, not open the file *yet*).