I was trying to get a virtual environment set up on Python 3 using mkvirtualenv but somehow the virtual environment was getting created on Python 2.7 (my system python).

If you already know about virtual environments and why they are useful, you may skip the next two paragraphs. I came to know about virtual environments only recently. Virtual environment helps you create an isolated space wherein whatever packages you install won't have an impact outside the environment. To give you a better understanding I will tell you why I use virtual environment.

I have two virtual environments in my computer - one for office work (say office_space) and the other for my personal learnings (personal_space). Now, office_space has all the packages that I need to get my daily office coding done. This has packages particular to the recommendation engines that we have built. Whereas the personal_space environment has just normal packages - numpy, pandas, luigi and others.

Virtual environment makes it easier to work on more than one project at a time without introducing conflicts in their dependencies. It helps to manage the package versions without much hassle. In a realistic scenario, many a times different projects may require different versions of the same package and a change in the version of a package may give erratic results. I have learnt the hard way to always have your package version pinned. Always, make sure the version of the package you are using in the production code is the right one and pin that version. Virtual environments help you to easily meet and manage different package versions for different projects.

Python 2 or 3 or may be both

With Python making a change to Python 3, we always have this question - which version should I use. I started learning Python recently and one of my seniors recommended learning Python 3. You decide what you want to learn. My recommendation is Python 3. Having said that, you can easily have both the versions in your system. Comes to the rescue pyenv. Pyenv is a python version management tool. It lets you to switch from one version to other with just a single command.

If you are on mac, consider installing with Homebrew package manager by typing:

brew update
brew install pyenv

You can read more on how to install, use and other details here.

Create a virtual env on python3

You need to install virtualenvwrapper package using:

pip install virtualenvwrapper

virtualenvwrapper helps creating and deleting virtual environments and I have already explained why we need virtual environments.

Once you have it installed, you can easily create an environment say ml_learn_env by typing

mkvirtualenv ml_learn

This will activate your virtual environment. You can now install packages in this environment that you need. Any package that you install in this env won't have any impact on anything outside this space.

Many times, you may have to pass the path of the python version along with mkvirtualenv to ensure the virtual env is created over that python particular version.

If I do pyenv versions on my terminal, this is what I get.

system
* 3.4.3 (set by /Users/manishb-imac/.python-version)
3.5.2

There are three versions of Python on my machine: system - the default Python provided by Mac which is Python 2.7. And then I have Python 3.4.3 (marked by asterisk meaning the current active version) and Python 3.5.2.

If I want to make a virtual env corresponding to a particular python version, say 3.4.3 I can pass the path explicitly like

mkvirtualenv -p /Users/Manish/.pyenv/versions/3.4.3/bin/python ml_learn_env

This will definitely have 3.4.3 as the python version on your virtual env - ml_learn_env.

Deactivate and delete virtual environment

If you want to delete a virtual env. (say tmp_venv) you need to first get out of the virtual env. by deactivating it using: deactivate tmp_venv. This will get you out of the virtual env. and now if you want to delete it, type rmvirtualenv tmp_venv.

Create a Jupyter kernel over virtual env.

Once your virtual environment is created, we want to install packages specific to the project you created the virtual environment for. When writing the code on Jupyter notebook for this project we want to use these packages.

To do so we need to install a package called ipykernel that allows us to use these packages in our jupyter notebook. We then create a kernel over this virtual environment by typing the below code.

pip install ipykernel
python -m ipykernel install --user --name ml_learn_env --display-name 'ml_learn_kernel'

You should now able to switch to ml_learn_kernel by clicking on change kernel under the Kernel tab of Jupyter notebook.

This post is not complete yet. I will keep updating this as and when I learn more on virtual environments.