More on Modules and Packages

In the Python eBook, you learned about the meaning of a module. Modular programming helps in organizing the code base in a more manageable way from both a maintenance and usability perspective.

A module is imported into a program using the import keyword. So to import random module you would execute the below command:

import random

This command will import random and all of the modules attributes can be accessed using the 'random' namespace. However, you can change this namespace by defining the new namespace to be used in your program by the below statement

import random as rd

Once an alias 'rd' is given for 'random' namespace, you can access all its attributes using 'rd' instead.

Now let us understand a bit on how modules are searched when you import a module. When you import any module, it searches for the .py file with that name. So when we import 'random' module it searches for random.py file and the list of directories it searches can be found from the sys module and here is the code to find all the directories that your program will search in:

import sys
sys.path

The result of this on the Colab is

'',
 '/content',
 '/env/python',
 '/usr/lib/python37.zip',
 '/usr/lib/python3.7',
 '/usr/lib/python3.7/lib-dynload',
 '/usr/local/lib/python3.7/dist-packages',
 '/usr/lib/python3/dist-packages',
 '/usr/local/lib/python3.7/dist-packages/IPython/extensions',
 '/root/.ipython'

So in brief the list of directories are:

  • first it searches for the file in the current directory of the running program or the interpreter.
  • if PYTHONPATH environment variable is set, then it searches for the file in that path
  • list of directories that are configured during Python installation

On Colab you can find the PYTHONPATH that is set by running the below command:

!echo $PYTHONPATH

And the result of this on Colab is:

/env/python

While you can tweak the PYTHONPATH to add additional directories, you can also add paths during runtime by using the command below

sys.path.append(r'C:\user\myprogram') on windows and equivalently it would be sys.path.append(r'/user/myprogram') on MacOS, Linux, Unix etc..

Once imported, you can actually find the filepath by running the below command

random.__file__

When you run the above statement on Colab, you will see that the full path of random module:

/usr/lib/python3.7/random.py

Module Import Variations

You have already been exposed to the three ways of importing a module:

  • import random - import the module into the current program and to access any of its attributes, use the full name of the module, like random.randint
  • import random as rd - import the module by giving it a different namespace inside of your program. Then access the attributes using the new namespace 'rd.randint'
  • 'from random import randint' - import only the randint function into the existing namespace. In this case, you don't need to use any prefix as randint is in the current namespace. In this method you have chances of polluting your namespace by overwriting the existing attributes with the ones imported and vice-versa.

dir() function

If you are curious to find all the defined names (attributes) available in the current namespace available in your runtime, you can call the dir() function, which returns a list of all the names.

From the interpreter if you run this command, you may see a list as below:

['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']

As new names are added during execution, they get added to this list

the main module

When a Python file is run directly, its __name__ attribute is set to '__main__'. Rest of all the important modules, will have the __name__ set to the file name. So, a file's __name__ attribute will be either '__main__' or its module name, depending on how it is executed.

So if you would like to do some testing on specific functionality in each of the modules, which you would test by running that module alone, then you can have a code piece something like below:

def add(a, b):
    return a+b

if (__name__ == '__main__'):
    import sys
    if len(sys.argv) > 2:
      x = float(sys.argv[1])
      y = float(sys.argv[2])
      print(add(x, y))

When this program is imported as a module, it will not enter the 'if' block. However, when you run this program from the command prompt directly, then the 'if' block is executed which prints out the result for you to check the 'add' function

Packages in Python

If your application ends up having a large number of modules, then packaging them into separate packages might be easy to manage.

A package is essentially a folder or directory. So by creating a folder and then dropping your .py files into the folder will arrange the programs in a package.

Note: For a directory to be recognized as a package by Python, it must contain a file named __init__.py. This file could be empty with just comment lines, for example.

Reference: https://docs.python.org/3/tutorial/modules.html#packages