[Tutor] Are the methods in a class copied or just linked to?

[Jim Mooney Py3.4.3winXP]

When an instance uses a class method, does it actually use the method that
is in the class object's memory space, or is the method copied to the
instance?

[Danny Yoo]

On Thu, Jul 2, 2015 at 12:30 PM, Jim Mooney Py3.4.3winXP

Post by Jim Mooney Py3.4.3winXP
When an instance uses a class method, does it actually use the method that
is in the class object's memory space, or is the method copied to the
instance?

Unsure. How would it be observable?



[The following is not beginner material, and mostly
implementation-specific experimentation. Skip if you are a beginner.]


We know that bound functions are more than code: they need to include
references to the otherwise free variables. But I think the
underlying function code is shared. It's not logically required, but
it makes engineering sense. Otherwise, making bound functions would
be significantly more expensive than the obvious approach of sharing
the part that doesn't change.

As an example:

#########################################
... def g():
... return name
... return g
...

Post by Jim Mooney Py3.4.3winXP

d = f("danny")
j = f("jim")
d()

'danny'

Post by Jim Mooney Py3.4.3winXP

j()

'jim'

Post by Jim Mooney Py3.4.3winXP

id(d)

4407845544

Post by Jim Mooney Py3.4.3winXP

id(j)

4407847344

Post by Jim Mooney Py3.4.3winXP

d.__code__

<code object g at 0x106ba1cb0, file "<stdin>", line 2>

Post by Jim Mooney Py3.4.3winXP

j.__code__

<code object g at 0x106ba1cb0, file "<stdin>", line 2>
#########################################

Here, 'd' and 'j' may be different values, but they share the same
underlying __code__, the compiled code object that defines 'g'.



__code__ and __func__ are special variables used to access the
underlying function values. They're described in:
https://docs.python.org/3/reference/datamodel.html

The following section in the reference are immediately relevant to
your question:

""""
When an instance method object is created by retrieving a user-defined
function object from a class via one of its instances, its __self__
attribute is the instance, and the method object is said to be bound.
The new method’s __func__ attribute is the original function object.

When a user-defined method object is created by retrieving another
method object from a class or instance, the behaviour is the same as
for a function object, except that the __func__ attribute of the new
instance is not the original method object but its __func__ attribute.

When an instance method object is created by retrieving a class method
object from a class or instance, its __self__ attribute is the class
itself, and its __func__ attribute is the function object underlying
the class method.
""""


From my reading of this, the language reference is guaranteeing that
__func__ is shared.
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://m

[Peter Otten]

Post by Danny Yoo
On Thu, Jul 2, 2015 at 12:30 PM, Jim Mooney Py3.4.3winXP

Post by Jim Mooney Py3.4.3winXP
When an instance uses a class method, does it actually use the method
that is in the class object's memory space, or is the method copied to
the instance?

Unsure. How would it be observable?

[snip implementation details]

My understanding of the question is less involved. I'd suggest:

Create a class Foo with a method bar(), say, instantiate it, replace foo in
the class and then invoke foo in the previously created instance.

If the old method is executed the instance must keep a copy, if the new
method is executed there is probably no copy.
... def bar(self): print("original bar")
...

Post by Danny Yoo

Post by Jim Mooney Py3.4.3winXP

foo = Foo()

... print("modified bar")
...

Post by Danny Yoo

Post by Jim Mooney Py3.4.3winXP

Foo.bar = modified_bar
foo.bar()

modified bar

Conclusion: no copy.

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alan Gauld]

Post by Jim Mooney Py3.4.3winXP
When an instance uses a class method, does it actually use the method that
is in the class object's memory space, or is the method copied to the
instance?

As soon as you mention memory space in relation to how something
works in Python you know you are barking up the wrong tree.
Memory space is a concept that only applies at the
implementation level in Python, never in the concepts.

Python objects (including classes, which are just another object)
are not necessarily constrained to a single memory space, Classes
are conglomerations of other objects (data and functions) and
linked to other classes (via inheritance). This is not like C++
(or even Java).

The method definitions inside a class define method objects
that live in their own conceptual space as values in the dictionary that
defines the class structure. Two methods of the same class
could live in wildly different memory locations or they could all be
part of the same memory area. That's an implementation detail and likely
to be very different in CPython, Jython and IronPython etc..

So the question should be whether Python objects access the references
to those methods by linking through the class object or whether they
get copies of the references internally.

I think Danny's answer covers that aspect.

But remember that questions of where Python objects live
in memory are nearly always the wrong question to ask.

Finally, I'm guessing that when you said a "class method" you
actually meant a "method in a class", ie an instance method? You
did not mean an actual class method as defined using the
@classmethod decorator?
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Steven D'Aprano]

Post by Jim Mooney Py3.4.3winXP
When an instance uses a class method, does it actually use the method that
is in the class object's memory space, or is the method copied to the
instance?

The first. And it is not just an implementation detail, it is part of
Python's programming model.

When you define a class in you define methods and other attributes in
the class namespace:

class C:
attr = 23
def method(self, arg):
pass

'attr' and 'method' exist inside C.__dict__. That is, I believe, more
than just implementation, I believe that is considered part of the
... def method(self): pass
...

Post by Jim Mooney Py3.4.3winXP

C.__dict__['method']

<function method at 0x000000000000002B>


even though classes in IronPython are based on a completely different
technology (the .Net CLR) from the standard CPython implementation.

When you create an instance, under normal circumstances it gets its own
__dict__, but that does *not* get a copy of the method, nor even a

Post by Jim Mooney Py3.4.3winXP

c = X()
c.__dict__ != C.__dict__ # Separate dictionaries?

True

Post by Jim Mooney Py3.4.3winXP

'method' in c.__dict__ # in the instance?

False

Post by Jim Mooney Py3.4.3winXP

'method' in C.__dict__ # in the class?

True


When you call a method, c.method(), Python does a name look-up.
Simplified, it looks in c.__dict__, then C.__dict__, then the __dict__s
of each of C's parents (if any).

The upshot of this is that you can actually override the method for a
specific instance. In Ruby, this technique is called "singleton method",
in Python it doesn't have a standard name, but it can be useful. It only
works on classes with an instance __dict__, so it doesn't work on ints,
floats, strings and other built-in types, and for technical reasons it
doesn't work on magic __dunder__ methods, and it is a little fiddly to
... def method(self):
... return "shared"
...

Post by Jim Mooney Py3.4.3winXP

c = C()
d = C()
from types import MethodType
c.method = MethodType(lambda self: 'overridden', c)
c.method()

'overridden'

Post by Jim Mooney Py3.4.3winXP

d.method()

'shared'

[Alan Gauld]

Post by Steven D'Aprano

Post by Jim Mooney Py3.4.3winXP
When an instance uses a class method, does it actually use the method that
is in the class object's memory space, or is the method copied to the
instance?

The first. And it is not just an implementation detail, it is part of
Python's programming model.

Pythons implementation model does not dictate the memory space
of the method. The method is in the class name space and referenced
via the internal dict. But the actual memory space of the method
object can be outside the memory space of the class. That's what
I mean by implementation detail. You cannot for example do a
memcopy of a class and expect to get a fully functioning
independant clone of the class in the new memory space. Nor an you
reliably calculate a method's memory location by adding an offset
to the class's location as you can in say C++ (a common way
of gaining access to private methods in early C++ code).

Post by Steven D'Aprano
When you define a class in you define methods and other attributes in

But this is true and the mechanism for looking up the method is defined
in the Python model and it goes via the class. But where the class
stores its methods is up to the implementation.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Steven D'Aprano]

Post by Alan Gauld

Post by Steven D'Aprano

Post by Jim Mooney Py3.4.3winXP
When an instance uses a class method, does it actually use the method that
is in the class object's memory space, or is the method copied to the
instance?

The first. And it is not just an implementation detail, it is part of
Python's programming model.

Pythons implementation model does not dictate the memory space
of the method. The method is in the class name space and referenced
via the internal dict. But the actual memory space of the method
object can be outside the memory space of the class. That's what
I mean by implementation detail. You cannot for example do a
memcopy of a class and expect to get a fully functioning
independant clone of the class in the new memory space. Nor an you
reliably calculate a method's memory location by adding an offset
to the class's location as you can in say C++ (a common way
of gaining access to private methods in early C++ code).

Oh. Right, I agree with this. Classes, and instances, in Python can be
spread over multiple chunks of heap memory. The specific details of
where the various parts live in memory are not defined by the language,
and will differ from implementation to implementation.

All you know is that methods (by default) will be found in the class
__dict__, but not where the methods actually are located, of even where
the class __dict__ will be located.

Post by Alan Gauld

Post by Steven D'Aprano
When you define a class in you define methods and other attributes in

But this is true and the mechanism for looking up the method is defined
in the Python model and it goes via the class. But where the class
stores its methods is up to the implementation.

Methods are stored in the __dict__, but the location of the __dict__ is
a mystery :-)

[Oscar Benjamin]

Post by Steven D'Aprano

Post by Jim Mooney Py3.4.3winXP
When an instance uses a class method, does it actually use the method that
is in the class object's memory space, or is the method copied to the
instance?

The first. And it is not just an implementation detail, it is part of
Python's programming model.
When you define a class in you define methods and other attributes in
attr = 23
pass

[snip]

Post by Steven D'Aprano
When you call a method, c.method(), Python does a name look-up.
Simplified, it looks in c.__dict__, then C.__dict__, then the __dict__s
of each of C's parents (if any).

To expand on this slightly. The expression c.method() is calculated as
(c.method)(). So the expression c.method must be evaulated first. To
evaluate this the name lookup occurs and a bound method object is
created. The bound method object wraps together the instance with the
method that was found during lookup. Having evaluated the expression
c.method the resulting bound method is then called to complete the
evaluation of the expression c.method(). Subsequently replacing the
method in the class does not affect the bound method since the lookup
... def foo(self):
... print('Original foo')
...

Post by Steven D'Aprano

Post by Jim Mooney Py3.4.3winXP

a = A()
afoo = a.foo
A.foo = lambda self: print('New foo')
afoo

<bound method A.foo of <__main__.A object at 0x7fefd773cc50>>

Post by Steven D'Aprano

Post by Jim Mooney Py3.4.3winXP

afoo()

Original foo

Post by Steven D'Aprano

Post by Jim Mooney Py3.4.3winXP

a.foo()

New foo

--
Oscar
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor