Python Points:

Memory Management
Memory management in Python involves a private heap containing all Python objects and data
structures. The management of this private heap is ensured internally by the Python memory
manager. The Python memory manager has different components which deal with various dynamic
storage management aspects, like sharing, segmentation, preallocation or caching.
The PYTHONMALLOC environment variable can be used to configure the memory allocators used
by Python.

Set the family of memory allocators used by Python:

• default: use the default memory allocators.

• malloc: use the malloc() function of the C library for all domains
(PYMEM_DOMAIN_RAW, PYMEM_DOMAIN_MEM, PYMEM_DOMAIN_OBJ).

• pymalloc: use the pymalloc allocator for PYMEM_DOMAIN_MEM and

PYMEM_DOMAIN_OBJ domains and use the malloc() function for the
PYMEM_DOMAIN_RAW domain.

The PYTHONMALLOCSTATS environment variable can be used to print statistics of the pymalloc
memory allocator every time a new pymalloc object arena is created, and on shutdown.

Monkey Patching in Python (Dynamic Behavior)

In Python, the term monkey patch refers to dynamic (or run-time) modifications of a class or
module. In Python, we can actually change the behavior of code at run-time.
# monk.py
class A:
     def func(self):
          print ("func() is being called")

We use above module (monk) in below code and change behavior of func() at run-time by assigning
different value.
import monk
def monkey_f(self):
     print ("monkey_f() is being called")
   
# replacing address of "func" with "monkey_f"
monk.A.func = monkey_f
obj = monk.A()
  
# calling function "func" whose address got replaced
# with function "monkey_f()"
obj.func()

Examples:
Output :monkey_f() is being called

pickle — Python object serialization

The pickle module is used for implementing binary protocols for serializing and de-serializing a
Python object structure. 
 
• Pickling: It is a process where a Python object hierarchy is converted into a byte stream. 
 
• Unpickling: It is the inverse of Pickling process where a byte stream is converted into an
object hierarchy. 
 
Module Interface : 
 

• dumps() – This function is called to serialize an object hierarchy.

• loads() – This function is called to de-serialize a data stream.
Constants provided by the pickle module : 
 

1. pickle.HIGHEST_PROTOCOL 
This is an integer value representing the highest protocol version available. This is
considered as the protocol value which is passed to the functions dump(), dumps(). 
 
2. pickle.DEFAULT_PROTOCOL 
This is an integer value representing the default protocol used for pickling whose value may
be less than the value of the highest protocol. 
 
Functions provided by the pickle module : 
 
1. pickle.dump(obj, file, protocol = None, *, fix_imports = True) 
This function is equivalent to Pickler(file, protocol).dump(obj). This is used to write a
pickled representation of obj to the open file object file.
The optional protocol argument is an integer that tells the pickler to use the given protocol.
The supported protocols are 0 to HIGHEST_PROTOCOL. If not specified, the default is
DEFAULT_PROTOCOL. If a negative number is specified, HIGHEST_PROTOCOL is
selected.
If fix_imports is true and protocol is less than 3, the pickle will try to map the new Python 3
names to the old module names used in Python 2, so that the pickle data stream is readable
with Python 2. 
Python - Magic or Dunder Methods
Magic methods in Python are the special methods that start and end with the double underscores.
They are also called dunder methods. Magic methods are not meant to be invoked directly by you,
but the invocation happens internally from the class on a certain action. For example, when you add
two numbers using the + operator, internally, the __add__() method will be called.

Built-in classes in Python define many magic methods. Use the dir() function to see the number
of magic methods inherited by a class. For example, the following lists all the attributes and
methods defined in the int class.

>>> dir(int)
['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__',
'__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__',
'__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__',
'__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__',
'__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__',
'__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__',
'__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__',
'__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__',
'__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'bit_length', 'conjugate',
'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']

__new__() method
Languages such as Java and C# use the new operator to create a new instance of a class. In Python
the __new__() magic method is implicitly called before the __init__() method. The
__new__() method returns a new object, which is then initialized by __init__().

Example: __new__()
class Employee:
def __new__(cls):
print ("__new__ magic method is called")
inst = object.__new__(cls)
return inst
def __init__(self):
print ("__init__ magic method is called")
self.name='Satya'

The above example will produce the following output when you create an instance of the
Employee class.
>>> emp = Employee()
__new__ magic method is called
__init__ magic method is called
Thus, the __new__() method is called before the __init__() method.

__str__() method
Another useful magic method is __str__(). It is overridden to return a printable string
representation of any user defined class. We have seen str() built-in function which returns a
string from the object parameter. For example, str(12) returns '12'. When invoked, it calls the
__str__() method in the int class.
>>> num=12
>>> str(num)
'12'
>>> #This is equivalent to
>>> int.__str__(num)
'12'

Important Magic Methods

The following tables list important magic methods in Python 3.

Initialization and Construction Description

__new__(cls, other) To get called in an object's instantiation.
__init__(self, other) To get called by the __new__ method.
__del__(self) Destructor method.
Unary operators and functions Description
__pos__(self) To get called for unary positive e.g. +someobject.
__neg__(self) To get called for unary negative e.g. -someobject.
__abs__(self) To get called by built-in abs() function.
__invert__(self) To get called for inversion using the ~ operator.
__round__(self,n) To get called by built-in round() function.
__floor__(self) To get called by built-in math.floor() function.
__ceil__(self) To get called by built-in math.ceil() function.
__trunc__(self) To get called by built-in math.trunc() function.

Type Conversion Magic

Description
Methods
__int__(self) To get called by built-int int() method to convert a type to an int.
__float__(self) To get called by built-int float() method to convert a type to float.
To get called by built-int complex() method to convert a type to
__complex__(self)
complex.
__oct__(self) To get called by built-int oct() method to convert a type to octal.
To get called by built-int hex() method to convert a type to
__hex__(self)
hexadecimal.
To get called on type conversion to an int when the object is used in
__index__(self)
a slice expression.
__trunc__(self) To get called from math.trunc() method.
String Magic Methods Description
To get called by built-int str() method to return a string representation of a
__str__(self)
type.
To get called by built-int repr() method to return a machine readable
__repr__(self)
representation of a type.
To get called by built-int unicode() method to return an unicode string of a
__unicode__(self)
type.
String Magic Methods Description
__format__(self, To get called by built-int string.format() method to return a new style of
formatstr) string.
__hash__(self) To get called by built-int hash() method to return an integer.
__nonzero__(self) To get called by built-int bool() method to return True or False.
To get called by built-int dir() method to return a list of attributes of a
__dir__(self)
class.
To get called by built-int sys.getsizeof() method to return the size of an
__sizeof__(self)
object.

Underscore (_) in Python

Following are different places where _ is used in Python:
1.Single Underscore:
•In Interpreter
•After a name
•Before a name
2.Double Underscore:
•__leading_double_underscore
•__before_after__
Single Underscore
In Interpreter:
_ returns the value of last executed expression value in Python
Prompt/Interpreter
>>> a = 10
>>> b = 10
>>> _
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '_' is not defined
>>> a+b
20
>>> _
20
>>> _ * 2
40
>>> _
40
>>> _ / 2
20
For ignoring values:
Multiple time we do not want return values at that time assign those
values to Underscore. It used as throwaway variable.
# Ignore a value of specific location/index
for _ in range(10)
    print ("Test")
  
# Ignore a value when unpacking
a,b,_,_ = my_method(var1)

After a name
Python has their by default keywords which we can not use as the variable
name. To avoid such conflict between python keyword and variable we use
underscore after name

Example:
>>> class MyClass():
...     def __init__(self):
...             print ("OWK")
  
>>> def my_defination(var1 = 1, class_ = MyClass):
...     print (var1)
...     print (class_)
  
>>> my_defination()
1
__main__.MyClass
>>>

Before a name
Leading Underscore before variable/function/method name indicates to
programmer that It is for internal use only, that can be modified whenever
class want.
Here name prefix by underscore is treated as non-public. If specify from
Import * all the name starts with _ will not import. Python does not
specify truly private so this ones can be call directly from other modules if
it is specified in __all__, We also call it weak Private
class Prefix:
...     def __init__(self):
...             self.public = 10
...             self._private = 12
>>> test = Prefix()
>>> test.public
10
>>> test._private
12
Python class_file.py
def public_api():
    print ("public api")
  
def _private_api():
    print ("private api")

Calling file from Prompt

>>> from class_file import *
>>> public_api()
public api
  
>>> _private_api()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '_private_api' is not defined
  
>>> import class_file
>>> class_file.public_api()
public api
>>> class_file._private_api()
private api

Double Underscore(__)
__leading_double_underscore
Leading double underscore tell python interpreter to rewrite name in order
to avoid conflict in subclass.Interpreter changes variable name with class
extension and that feature known as the Mangling.
testFile.py
class Myclass():
    def __init__(self):
        self.__variable = 10

Calling from Interpreter

Private Variables in Python

Prerequisite: Underscore in Python

In Python, there is no existence of “Private” instance variables that cannot

be accessed except inside an object. However, a convention is being
followed by most Python code and coders i.e., a name prefixed with an
underscore, For e.g. _geek should be treated as a non-public part of the
API or any Python code, whether it is a function, a method, or a data
member. While going through this we would also try to understand the
concept of various forms of trailing underscores, for e.g., for _ in
range(10), __init__(self). 
 
Mangling and how it works

In Python, there is something called name mangling, which means that

there is limited support for a valid use-case for class-private members
basically to avoid name clashes of names with names defined by
subclasses. Any identifier of the form __geek (at least two leading
underscores or at most one trailing underscore) is replaced with
_classname__geek, where classname is the current class name with a
leading underscore(s) stripped. As long as it occurs within the definition of
the class, this mangling is done. This is helpful for letting subclasses
override methods without breaking intraclass method calls. 
Let’s look at this example and try to find out how this underscore works: