[Tutor] FrozenDict

[Albert-Jan Roskam]

Hi,
I wanted to create a read-only dict to hold some constants. I looked around on the internet and created two implementations:-FrozenDict (derives from collections.mapping)-ChillyDict (derives from dict, which seems more obvious to me)
The code can be found here: http://pastebin.com/QJ3V2mSK
Some questions:1. one doctest from FrozenDict fails: fd.keys() returns an empty list. Why?2. Is FrozenDict the way to use collections.mapping (aside from the error!). I just discovered this and i seems quite cool (pun intended)3. Which implementation is better, and why? I like ChillyDict better because it is more straightforward and shorter.
The read-only dict does not need to be very fast, it just needs to give some reasonable protection against mutating values.It also needs to work under Python 2.7 and 3.3+.
Thank you!
Albert-Jan



_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Albert-Jan Roskam]

> From: ***@hotmail.com
> To: ***@python.org
> Date: Wed, 7 Oct 2015 16:10:20 +0000
> Subject: [Tutor] FrozenDict
>
> Hi,
> I wanted to create a read-only dict to hold some constants. I looked around on the internet and created two implementations:-FrozenDict (derives from collections.mapping)-ChillyDict (derives from dict, which seems more obvious to me)
> The code can be found here: http://pastebin.com/QJ3V2mSK
> Some questions:1. one doctest from FrozenDict fails: fd.keys() returns an empty list. Why?

ok, the correct way to write __iter__ appears to be:
def __iter__(self):
return iter(self.__kwargs)
Now at least all doctests pass.

2. Is FrozenDict the way to use collections.mapping (aside from the error!). I just discovered this and i seems quite cool (pun intended)3. Which implementation is better, and why? I like ChillyDict better because it is more straightforward and shorter.
> The read-only dict does not need to be very fast, it just needs to give some reasonable protection against mutating values.It also needs to work under Python 2.7 and 3.3+.
> Thank you!
> Albert-Jan

Just in case, I am also pasting the code here, with the slightly modified__iter__:

from collections import Mapping

class FrozenDict(Mapping):
"""A dictionary that does not support item assignment after initialization
>>> fd = FrozenDict(a=1, b=2)
>>> fd["a"]
1
>>> fd["a"] = 777 # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
TypeError: 'FrozenDict' object does not support item assignment
>>> sorted(fd.items())
[('a', 1), ('b', 2)]
>>> sorted(fd.keys())
['a', 'b']
>>> fd
FrozenDict('a'=1, 'b'=2)
"""
def __init__(self, **kwargs):
self.__kwargs = kwargs
self.__next__ = self.next
self.__counter = 0
self.__init__ = None
def __repr__(self):
kwargs = ["%r=%r" % (k, v) for k, v in sorted(self.__kwargs.items())]
return "%s(%s)" % (self.__class__.__name__, ", ".join(kwargs))
def __getitem__(self, key):
return self.__kwargs[key]
def __iter__(self):
return iter(self.__kwargs)
def next(self):
try:
value = self.__kwargs.items()[self.__counter][0]
except IndexError:
raise StopIteration
self.__counter += 1
return value
def __len__(self):
return len(self.__kwargs)

class ChillyDict(dict):
"""A dictionary that does not support item assignment after initialization
>>> cd = ChillyDict(a=1, b=2)
>>> cd["a"]
1
>>> cd["a"] = 777 # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
NotImplementedError: 'ChillyDict' object does not support item assignment
>>> sorted(cd.items())
[('a', 1), ('b', 2)]
>>> sorted(cd.keys())
['a', 'b']
>>> cd
ChillyDict('a'=1, 'b'=2)
"""
def __init__(self, **kwargs):
self.__kwargs = kwargs
super(ChillyDict, self).__init__(**self.__kwargs)
self.__init__ = None
def __repr__(self):
kwargs = ["%r=%r" % (k, v) for k, v in sorted(self.__kwargs.items())]
return "%s(%s)" % (self.__class__.__name__, ", ".join(kwargs))
def __not_implemented(self, *args, **kwargs):
msg = "'%s' object does not support item assignment"
raise NotImplementedError, msg % self.__class__.__name__
__setitem__ = __delitem__ = update = pop = popitem = clear = __not_implemented

if __name__ == "__main__":
import doctest
doctest.testmod()


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alexandre Chabot-Leclerc]

Hi Albert-Jan,
As far as I know, the recommended object to subclass when subclassing a `dict` is `UserDict`. In Python 3, it's in `collections.UserDict` and in Python 2 is in `UserDict.UserDict`.

Here's an basic example of how it would work:

try:
from collections import UserDict
except ImportError:
from UserDict import UserDict

class FrozenDict(UserDict):
def __setitem__(self, key, item):
raise TypeError("'FrozenDict' object does not support item assignment")

According to the Fluent Python book (by Luciano Ramalho, which I recommend wholeheartedly), subclassing built-in types is tricky because: "the code of the built-ins (written in C) does not call special methods overridden by user-defined classes." Therefore, other methods of `dict`, like `update` or `__init__` will *not* call your special `__setitem__` method.

However, be aware that although the FrozenDict above is read-only, it's not *really* frozen, i.e., it cannot be used as a key in another dict. In order to do that, you would need to define the `__hash__` method.

This StackOverflow answer, which you might have seen, provide an implementation of a FrozenDict that could be used as a dict.
http://stackoverflow.com/questions/2703599/what-would-a-frozen-dict-be

Cheers,
Alex

On Wed, Oct 7, 2015, at 18:10, Albert-Jan Roskam wrote:
> Hi,
> I wanted to create a read-only dict to hold some constants. I looked around on the internet and created two implementations:-FrozenDict (derives from collections.mapping)-ChillyDict (derives from dict, which seems more obvious to me)
> The code can be found here: http://pastebin.com/QJ3V2mSK
> Some questions:1. one doctest from FrozenDict fails: fd.keys() returns an empty list. Why?2. Is FrozenDict the way to use collections.mapping (aside from the error!). I just discovered this and i seems quite cool (pun intended)3. Which implementation is better, and why? I like ChillyDict better because it is more straightforward and shorter.
> The read-only dict does not need to be very fast, it just needs to give some reasonable protection against mutating values.It also needs to work under Python 2.7 and 3.3+.
> Thank you!
> Albert-Jan
>
>
>
> _______________________________________________
> Tutor maillist - ***@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Albert-Jan Roskam]

----------------------------------------
> From: ***@alexchabot.net
> To: ***@python.org
> Date: Wed, 7 Oct 2015 18:54:42 +0200
> Subject: Re: [Tutor] FrozenDict
>
> Hi Albert-Jan,
> As far as I know, the recommended object to subclass when subclassing a `dict` is `UserDict`. In Python 3, it's in `collections.UserDict` and in Python 2 is in `UserDict.UserDict`.
>
> Here's an basic example of how it would work:
>
> try:
> from collections import UserDict
> except ImportError:
> from UserDict import UserDict
>
> class FrozenDict(UserDict):
> def __setitem__(self, key, item):
> raise TypeError("'FrozenDict' object does not support item assignment")
>
> According to the Fluent Python book (by Luciano Ramalho, which I recommend wholeheartedly), subclassing built-in types is tricky because: "the code of the built-ins (written in C) does not call special methods overridden by user-defined classes." Therefore, other methods of `dict`, like `update` or `__init__` will *not* call your special `__setitem__` method.
>
> However, be aware that although the FrozenDict above is read-only, it's not *really* frozen, i.e., it cannot be used as a key in another dict. In order to do that, you would need to define the `__hash__` method.
>
> This StackOverflow answer, which you might have seen, provide an implementation of a FrozenDict that could be used as a dict.
> http://stackoverflow.com/questions/2703599/what-would-a-frozen-dict-be
>
> Cheers,
> Alex
>

Aha, that's useful to know. So it's a no-no to subclass *any* builtin?

I checked collections.UserDict and it indeed looks promising. Caveat: it;s Python 3 only (not sure what versionit was introduced).

Best wishes,
Albert-Jan

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alexandre Chabot-Leclerc]

> Aha, that's useful to know. So it's a no-no to subclass *any* builtin?

I don't think it's a no-no, I just think it comes with a few problems that are solved if you subclass the classes that are *meant* to be subclassed, like UserDict, UserList, or UserString.

> I checked collections.UserDict and it indeed looks promising. Caveat: it;s Python 3 only (not sure what versionit was introduced).

It's in both Python 2 and 3, but they're not in the same module. That's why I used the following construct to import it. In both cases, UserDict is imported correctly and the module where it came from does not matter.

try:
from collections import UserDict
except ImportError:
from UserDict import UserDict

Here are the docs:

- Python 2: https://docs.python.org/2/library/userdict.html
- Python 3: https://docs.python.org/3/library/collections.html#collections.UserDict

Regards,
Alex
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Steven D'Aprano]

On Wed, Oct 07, 2015 at 04:10:20PM +0000, Albert-Jan Roskam wrote:
> Hi,
> I wanted to create a read-only dict to hold some constants. I looked around on the internet and created two implementations:-FrozenDict (derives from collections.mapping)-ChillyDict (derives from dict, which seems more obvious to me)
> The code can be found here: http://pastebin.com/QJ3V2mSK

> Some questions:1. one doctest from FrozenDict fails: fd.keys() returns
> an empty list. Why?

No it doesn't.

py> fd = FrozenDict(a=1, b=2)
py> fd.keys()
['a', 'b']


That's under Python 2.7.

In 3.3, you will have a problem that FrozenDict is not a proper
iterator. You can't set self.__next__ = self.next, that won't work.
Dunder methods have to be on the class, not on the instance, so instead
of making the assignment in the __init__ method, put this in the body of
your class:

def next(self):
# Python 2 method
...

__next__ = next # Python 3 method.


Unfortunately that's not enough to get it working in Python 3. I need
more time to think about that.


> 2. Is FrozenDict the way to use collections.mapping (aside from the
> error!). I just discovered this and i seems quite cool (pun intended)

I think the error is quite significant...


> 3. Which implementation is better, and why? I like ChillyDict better
> because it is more straightforward and shorter.

I dislike the ChillyDict implementation because it looks like it
supports item assignment:

hasattr(ChillyDict, '__setitem__')

will return True, but it actually doesn't. That could make it risky in
code that assumes that the existence of __setitem__ means you can set
items.



--
Steve
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Oscar Benjamin]

On 8 October 2015 at 01:47, Steven D'Aprano <***@pearwood.info> wrote:
> In 3.3, you will have a problem that FrozenDict is not a proper
> iterator. You can't set self.__next__ = self.next, that won't work.
> Dunder methods have to be on the class, not on the instance, so instead
> of making the assignment in the __init__ method, put this in the body of
> your class:
>
> def next(self):
> # Python 2 method
> ...
>
> __next__ = next # Python 3 method.
>
>
> Unfortunately that's not enough to get it working in Python 3. I need
> more time to think about that.

There shouldn't be a __next__ method on the FrozenDict class. __iter__
should return a distinct iterator. Albert has already fixed this by
using:

def __iter__(self):
return iter(self.__kwargs)

--
Oscar
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Steven D'Aprano]

On Thu, Oct 08, 2015 at 12:03:28PM +0100, Oscar Benjamin wrote:
> On 8 October 2015 at 01:47, Steven D'Aprano <***@pearwood.info> wrote:
> > In 3.3, you will have a problem that FrozenDict is not a proper
> > iterator. You can't set self.__next__ = self.next, that won't work.
> > Dunder methods have to be on the class, not on the instance, so instead
> > of making the assignment in the __init__ method, put this in the body of
> > your class:
> >
> > def next(self):
> > # Python 2 method
> > ...
> >
> > __next__ = next # Python 3 method.
> >
> >
> > Unfortunately that's not enough to get it working in Python 3. I need
> > more time to think about that.
>
> There shouldn't be a __next__ method on the FrozenDict class. __iter__
> should return a distinct iterator. Albert has already fixed this by
> using:
>
> def __iter__(self):
> return iter(self.__kwargs)


That's one solution, but it is certainly possible for the class to be
its own iterator, in which case it needs to follow two rules:

(1) self.__next__() needs to return the next value, or raise
StopIteration;

(2) self.__iter__() needs to return self;


and of course like all dunder methods __next__ and __iter__ need to be
defined on the class itself, not on the instance.


Just returning iter(values) is probably much easier, but not as much fun
:-)



--
Steve
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[eryksun]

On 10/8/15, Steven D'Aprano <***@pearwood.info> wrote:
>
> That's one solution, but it is certainly possible for the class to be
> its own iterator, in which case it needs to follow two rules:
>
> (1) self.__next__() needs to return the next value, or raise
> StopIteration;
>
> (2) self.__iter__() needs to return self;
>
> and of course like all dunder methods __next__ and __iter__ need to be
> defined on the class itself, not on the instance.

Except this is generally a bad way to iterate a reiterable. Without a
separate iterator, there's no simple way to maintain state for
concurrent iterations.

file types are an exception. A file is reiterable (i.e. by seeking
back to 0), but the OS file pointer makes a file naturally an
iterator. Thus getting concurrent iterations of a disk file requires
separate file objects that have separate OS file pointers.

FrozenDict.next is an example of what not to do:

def next(self):
try:
value = self.__kwargs.items()[self.__counter][0]
except IndexError:
raise StopIteration
self.__counter += 1
return value

In Python 2 this iterates the dict's keys by creating a list of (key,
value) tuples -- every time next() is called. In Python 3, you'd have
to explicitly create the list using list(self.__kwargs.items()). The
design also lacks support for concurrent iterations, and not even
multiple iterations since __counter isn't reset in __iter__.

The simple approach is to have __iter__ return an instance of the
wrapped dict's iterator. There's no reason to reinvent the wheel. Plus
in the non-frozen case, the built-in dict iterators are smart enough
to raise an error when the dict is modified, since it's assumed that
any modification to the hash table invalidates the iteration.
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Albert-Jan Roskam]

----------------------------------------
> Date: Fri, 9 Oct 2015 01:14:05 -0500
> From: ***@gmail.com
> To: ***@python.org
> Subject: Re: [Tutor] FrozenDict
>
> On 10/8/15, Steven D'Aprano <***@pearwood.info> wrote:
>>
>> That's one solution, but it is certainly possible for the class to be
>> its own iterator, in which case it needs to follow two rules:
>>
>> (1) self.__next__() needs to return the next value, or raise
>> StopIteration;
>>
>> (2) self.__iter__() needs to return self;
>>
>> and of course like all dunder methods __next__ and __iter__ need to be
>> defined on the class itself, not on the instance.
>
> Except this is generally a bad way to iterate a reiterable. Without a
> separate iterator, there's no simple way to maintain state for
> concurrent iterations.
>
> file types are an exception. A file is reiterable (i.e. by seeking
> back to 0), but the OS file pointer makes a file naturally an
> iterator. Thus getting concurrent iterations of a disk file requires
> separate file objects that have separate OS file pointers.
>
> FrozenDict.next is an example of what not to do:
>
> def next(self):
> try:
> value = self.__kwargs.items()[self.__counter][0]
> except IndexError:
> raise StopIteration
> self.__counter += 1
> return value
>
> In Python 2 this iterates the dict's keys by creating a list of (key,
> value) tuples -- every time next() is called. In Python 3, you'd have
> to explicitly create the list using list(self.__kwargs.items()). The
> design also lacks support for concurrent iterations, and not even
> multiple iterations since __counter isn't reset in __iter__.

Hi Erysun, Steven and others.

Eww, I see now that this indeed horribly inefficient.

>
> The simple approach is to have __iter__ return an instance of the
> wrapped dict's iterator. There's no reason to reinvent the wheel. Plus
> in the non-frozen case, the built-in dict iterators are smart enough
> to raise an error when the dict is modified, since it's assumed that
> any modification to the hash table invalidates the iteration.


I don't understand what you mean with this. It reads like super(FrozenDict, self).__iter__(), but that can't be correct because the supper class is an abstract class.
Anyway, I rewrote the code (__repr__ is omitted for brevity):

from collections import Mapping
import sys

class FrozenDict(Mapping):
    """A dictionary that does not support item assignment after initialization
   >>> fd = FrozenDict(a=1, b=2)
   >>> fd["a"]
    1
   >>> fd["a"] = 777  # doctest: +IGNORE_EXCEPTION_DETAIL
    Traceback (most recent call last):
    ...
    TypeError: 'FrozenDict' object does not support item assignment
   >>> sorted(fd.items())
    [('a', 1), ('b', 2)]
   >>> sorted(fd.keys())
    ['a', 'b']
    """
    def __init__(self, **kwargs):
        self.__kwargs = kwargs
        self.__init__ = None
    def __getitem__(self, key):
        return self.__kwargs[key]
    def __iter__(self):
        return self.__kwargs.iterkeys()
    if sys.version_info.major> 2:
        __iter__ = lambda self: iter(self.__kwargs.keys())   # Python 3: I am assuming this approach is more efficient than using try-except.
    def __len__(self):
        return len(self.__kwargs)

Curious what you think about this code. By the way, why is a TyperError raised when one tries to set an item? An AttributeError seems more logical.

if __name__ == "__main__":
    import doctest
    doctest.testmod()


As always, thanks a lot!

Best wishes,
Albert-Jan
 

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://m

[Albert-Jan Roskam]

----------------------------------------
> Date: Thu, 8 Oct 2015 11:47:41 +1100
> From: ***@pearwood.info
> To: ***@python.org
> Subject: Re: [Tutor] FrozenDict
>
> On Wed, Oct 07, 2015 at 04:10:20PM +0000, Albert-Jan Roskam wrote:
>> Hi,
>> I wanted to create a read-only dict to hold some constants. I looked around on the internet and created two implementations:-FrozenDict (derives from collections.mapping)-ChillyDict (derives from dict, which seems more obvious to me)
>> The code can be found here: http://pastebin.com/QJ3V2mSK
>
>> Some questions:1. one doctest from FrozenDict fails: fd.keys() returns
>> an empty list. Why?
>
> No it doesn't.
>
> py> fd = FrozenDict(a=1, b=2)
> py> fd.keys()
> ['a', 'b']
>
>
> That's under Python 2.7.
>
> In 3.3, you will have a problem that FrozenDict is not a proper
> iterator. You can't set self.__next__ = self.next, that won't work.
> Dunder methods have to be on the class, not on the instance, so instead
> of making the assignment in the __init__ method, put this in the body of
> your class:
>
> def next(self):
> # Python 2 method
> ...
>
> __next__ = next # Python 3 method.
>
>
> Unfortunately that's not enough to get it working in Python 3. I need
> more time to think about that.
>
>
>> 2. Is FrozenDict the way to use collections.mapping (aside from the
>> error!). I just discovered this and i seems quite cool (pun intended)
>
> I think the error is quite significant...
>
>
>> 3. Which implementation is better, and why? I like ChillyDict better
>> because it is more straightforward and shorter.
>
> I dislike the ChillyDict implementation because it looks like it
> supports item assignment:
>
> hasattr(ChillyDict, '__setitem__')


Thanks. I had not considered that. I usually use try-except (forgiveness) rather than hasattr (permission)
I will post another implementation shortly.

>
> will return True, but it actually doesn't. That could make it risky in
> code that assumes that the existence of __setitem__ means you can set
> items.
>
>
>
> --
> Steve
> _______________________________________________
> Tutor maillist - ***@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor