[Tutor] Stacks and Stack underflow/Stack overflow

[Joe Batt]

Hi All
Could some kind soul please explain why you get a stack underflow and a stack overflow.
I am getting the following error in Python 3
Traceback (most recent call last): File "/Users/joebatt/Desktop/python/pickling puzzle 5.py", line 39, in <module> a=pickle.load(file)_pickle.UnpicklingError: unpickling stack underflow
when I am running the following
import picklefile=open('///Users/joebatt/Desktop/banner.p.webarchive','rb')a=pickle.load(file)file.close()print (a)
Now I am very much a learner so please correct and explain my misunderstanding. I am visualising my program as taking the 'banner.p.webarchive' and and pushing it onto the stack byte by byte until it reaches the end of the file i.e. A B C ---> push C then B then A
A B C ---> A B C
Then the program pops the stack C does what it needs to do to unpickle it then pops B does what it needs to unpickle then A.
A ----->C B ABC
My understanding is that the stack underflow means that it is trying to pop from the top of the stack and there is nothing to pop i.e. it is empty. Why though if the stack has been loaded with the file 'banner.p' in my program does it say there is nothing to pop and thus a stack underflow?
Also my understanding of the stack overflow is that the stack itself is a finite size and when it has tried to push the file to the stack it didn't fit because it was too big, i.e. in my example if the stack was only big enough for 2 letters and I tried to push ABC it would give a stack overflow because I was trying to push 3. How would I deal with this? Can I make the stack bigger or a way to just push A B pop AB then push C and pop it.
Thank you guys Im sorry for probably what are very silly basic questions.
Joe

[Terry Carroll]

Hi All 
Could some kind soul please explain why you get a stack underflow and a stack overflow.
I am getting the following error in Python 3
  File "/Users/joebatt/Desktop/python/pickling puzzle 5.py", line 39, in
<module>
    a=pickle.load(file)
_pickle.UnpicklingError: unpickling stack underflow
when I am running the following
import pickle
file=open('///Users/joebatt/Desktop/banner.p.webarchive','rb')
a=pickle.load(file)
file.close()
print (a)

When you created the pickle file, did you create it in binary form (with
"wb")?

[Walter Prins]

Hi,

Post by Joe Batt
I am getting the following error in Python 3
File "/Users/joebatt/Desktop/python/pickling puzzle 5.py", line 39, in
<module>
a=pickle.load(file)
_pickle.UnpicklingError: unpickling stack underflow
when I am running the following
import pickle
file=open('///Users/joebatt/Desktop/banner.p.webarchive','rb')
a=pickle.load(file)
file.close()
print (a)

OK, there's basically 2 things you can infer from the error message: 1) The
unpickling failed (to which you should ask yourself, why?), and 2) The
picking/unpickling logic must use some sort of stack internally. The first
is the most important bit as far as you're concerned, and the second is for
the most part irrelevant from your point of view, since you're not really
interested in the inner workings of pickling and unpickling. So why might
the unpickling be failing? Broadly speaking, it must be one of 2 things --
either you're doing it wrong, or you're putting the wrong stuff in, so to
speak. The first seems unlikely, so it's likely the second.

So, I'm with Terry on this one, I'd guess that the input that you're
feeding into pickle.load() is not a valid pickle file, probably because
pickle files are binary files and you probably opened it as a text file.
See here: http://docs.python.org/py3k/library/functions.html#open

Cheers,

Walter

[Steven D'Aprano]

Post by Walter Prins
pickle files are binary files and you probably opened it as a text file.
See here: http://docs.python.org/py3k/library/functions.html#open

Not so. Pickle protocol 0 is text, and it happens to be the default
protocol used if you don't instruct it different. So by default, pickles
are text unless you specifically use a binary protocol.

[Alan Gauld]

Post by Steven D'Aprano

Post by Walter Prins
pickle files are binary files and you probably opened it as a text file.
See here: http://docs.python.org/py3k/library/functions.html#open

Not so. Pickle protocol 0 is text, and it happens to be the default
protocol used if you don't instruct it different. So by default, pickles
are text unless you specifically use a binary protocol.

Now that's interesting. All the examples I've sen use binary files.
And even the examples on the documentation pages use binary mode.
I just assumed therefore that pickles would, by their nature, be binary...

[Hugo Arts]

Post by Alan Gauld

Post by Steven D'Aprano

Post by Walter Prins
pickle files are binary files and you probably opened it as a text file.
See here: http://docs.python.org/py3k/library/functions.html#open

Not so. Pickle protocol 0 is text, and it happens to be the default
protocol used if you don't instruct it different. So by default, pickles
are text unless you specifically use a binary protocol.

Now that's interesting. All the examples I've sen use binary files.
And even the examples on the documentation pages use binary mode.
I just assumed therefore that pickles would, by their nature, be binary...

If you open the file in text mode and it happens to use a binary
protocol, things won't work because the binary might incorporate stuff
interpreted as newlines by the file object, with newline conversion
and such. If you open the file in binary it'll always work because the
file object doesn't touch the contents. So you always open the files
in binary mode to be on the safe side.

Hugo

[Walter Prins]

Hi Steve/Joe,

Post by Walter Prins
pickle files are binary files and you probably opened it as a text file.

See here: http://docs.python.org/py3k/**library/functions.html#open<http://docs.python.org/py3k/library/functions.html#open>

Not so. Pickle protocol 0 is text, and it happens to be the default
protocol used if you don't instruct it different. So by default, pickles
are text unless you specifically use a binary protocol.

Well I stand corrected, thanks! :) For some reason I thought they were
binary by default too, but hey ho... will try to remember that for future
reference.

To Joe, another hint: Note that you can /directly/ pass the /file-like/
object returned by urllib directly to pickle....

So, pickle can effectively unpickle data read directly from the website
(via a file-like object that actually reads from the webiste, not a real
file), and you then needn't care what format the pickle file (on the
website) is in... Neat eh?

This by the way, is a small demonstration of "duck typing" -- Pickle
doesn't really care whether what you pass into pickle.load() method is
actually *really* a file object -- As long as it walks like a file, and
quacks like a file, sufficiently well for pickle's purposes then everyone's
happy and you get back an unpickled object... ;)

Walter

PS You can get the unpickled object in Python for this puzzle with between
1 and 3 lines of code, depending on how much nesting of calls you use...

[Alan Gauld]

Post by Joe Batt
File "/Users/joebatt/Desktop/python/pickling puzzle 5.py", line 39, in
<module>
a=pickle.load(file)
_pickle.UnpicklingError: unpickling stack underflow
when I am running the following
import pickle
file=open('///Users/joebatt/Desktop/banner.p.webarchive','rb')

I don't know why the error, it suggests the data file is corrupted.
You may need to fetch another copy. Make sure you save the file rather
than try to open it so that it downloads in binary format or it may be
truncated...


However, having downloaded it I'm not sure why you are specifying the
filename with /// at the front. It looks like you are on Linux? You
should only need

file=open('/Users/joebatt/Desktop/banner.p.webarchive','rb')

But that shouldn't really make any difference to the end result.

[Steven D'Aprano]

Hi All Could some kind soul please explain why you get a stack
underflow and a stack overflow.

I don't, because I never make misteaks.

*grin*

Stack underflow and stack overflow are generic errors. You get a stack
underflow when you have (say) five items on a stack and try to retrieve
six: if you try to retrieve more items than are actually there, the
stack underflows.

Stack overflow occurs when you fill a stack. Many, but not all,
applications set their stack to have a maximum size, to prevent it
growing without bounds and using all your memory. So if the maximum size
is (say) 1000 items, and you push 1001 items onto it, the stack will be
filled and you get an overflow.

*Specifically* related to pickle, you get an stack underflow when either
your pickle file is corrupt, or because you aren't using it correctly.
Or (very unlikely) pickle has a bug.

You can't manually change the stack. You shouldn't need to change the
stack. An application could have dozens of stacks, for all you know.

(Sometimes people talk about "the stack", meaning the stack used by the
operating system for executing code. You don't want to touch that, even
if you could, which fortunately you can't.)

Without seeing how you created your pickle file, and what you did to it
in the meantime, it is impossible to say why it is corrupt or how it
became corrupted. But my guess is that you may have opened the file in
Notepad or something, and hit save, and Notepad "helpfully" changed the
contents somehow. Result: a corrupt file that pickle can no longer read
correctly.

Please run this code and confirm that pickle works correctly for you:

# Create some data.
a = {'bb': None, 'cc': [1, 2, 3], 'dd': 42}
b = [None, (23, 24, 25), "spam"]

# Save it to a pickle file.
import pickle
f = open('///Users/joebatt/Desktop/tmp12345', 'wb')
pickle.dump(a, f)
pickle.dump(b, f)
f.close()

# Read it back in.
f = open('///Users/joebatt/Desktop/tmp12345', 'rb')
c = pickle.load(f)
d = pickle.load(f)
f.close()

# Confirm the values are correct.
assert a == c
assert b == d