deep_learning_miniconda_project

This is an old revision of the document!


deep_learning_miniconda_project

# file: Setting up virtual environments using .md

The Miniconda installer installs a smaller version of the conda package manager to your home directory as will as well as it's own version of Python. Once Miniconda is installed, you can use the conda command to create additional Python environments which you can customize as you like. You can create additional environments that run “any” version of Python and any other Python programs you install to the virtual environments.

Benefits

- You can easily create, modify, activate and deactivate mulitiple custom Python environments. - You can create and run virtual Python environments using almost “any” version of Python. - Each Python environement can be customized as you wish.

Keep in mind

It is easy to accidentally customize the Python environment that the miniconda installer installs for you and you probably do not want to do this.

Create a new, hosted, Python (virtual) environment for each of your project(s). This way your host Python installation, the one that supports the conda program, remains 'clean'. This allows you to freely experiment and easily discard or duplicate and or exchange virtual environment contained Python projects without disturbing host Python version.

Requirements

The user on the system must have sudo access to the following commands:

- apt-get update or apt update

- apt-get install or apt install

Options

The checksums comparison in this document makes use of html2text

  sudo apt update
  sudo apt install html2text

Install miniconda

  conda --version
  mkdir -p ~/sys/sw/ubuntu/16.04/miniconda
  cd ~/sys/sw/ubuntu/16.04/miniconda
  wget https://repo.continuum.io/miniconda/Miniconda2-4.2.12-Linux-x86_64.sh
  chmod +x Miniconda2-4.2.12-Linux-x86_64.sh

checksums

  wget https://repo.continuum.io/miniconda/ -O checksums.html
  html2text -width 500 checksums.html > md5sums
  md5sum Miniconda2-4.2.12-Linux-x86_64.sh
  cat md5sums | grep Miniconda2-4.2.12-Linux-x86_64.sh

Installation

  bash Miniconda2-4.2.12-Linux-x86_64.sh -b

Add to ~/.bashrc

  nano ~/.bashrc

Content example:

  # BEGIN ANSIBLE MANAGED BLOCK
  #
  # added to /etc/skel/.bashrc. The Miniconda2 4.2.12
  # installer (and other versions?) usually add to a
  # users home directory.
  # set PATH to include '$HOME/miniconda2/bin'
  #
  if [ -d PATH="$HOME/miniconda2/bin" ] ; then
     PATH="$HOME/miniconda2/bin:$PATH"
  fi
  # END ANSIBLE MANAGED BLOCK

This way we can run conda without restarting our terminal

  export PATH="$HOME/miniconda2/bin:$PATH"

test

  conda --version

List current conda stuff

  conda list
  conda update conda

Use the conda create command, followed by any name you wish to call it:

  conda create --name deep python=2

output example

Might be add source commands to acemenu and / or .bash_aliases

  #
  # To activate this environment, use:
  # > source activate deep
  #
  # To deactivate this environment, use:
  # > source deactivate deep
  #

Activate the environment

If you run which python before activating your new Python environment you will see that the Python environment that ships with miniconda is the active Python at the moment:

  which python

Output example:

  /home/users/mary/miniconda2/bin/python

Activate your new Python environment:

  source activate deep

Run which python again an you whould see some thing like this:

  /home/users/mary/miniconda2/envs/deep/bin/python

Notice your prompt

When you activate a conda created Python environement the name you gave the environment prepended to your command prompt. In this example you will see that the prompt now begins with (deep) telling us that the custom python 2 environment we created earlier called deep is now the active Python for our user.

  (deep) csteel@ace-ws-30:~/sys/sw/linux/miniconda$ 

Install some stuff

Now we are going to install our pip packages to our custom virtual python environment. Notice that we do not need to use sudo or the root user as this installation takes place in our home directory.

Deep project learning example:

  pip install numpy matplotlib tensorflow

Create another custom Python environment

Here we create a second Python enviroment as an example. In this case we create a Python 3 environment called data.

  conda create --name data python=3

IMPORTANT: Notice that the Python environment deep is still active. If you install anything Pythonic now it would be installed to the deep Python environment. You will want to deactivate deep and then activate data before you can install to your Python 3 environment:

  source deactivate deep

you will notice that the (deep) command prompt prefix is no longer visible. Next we will activate our new Python 3 environment:

  source activate data

Now you are ready to install to the deep Python 3 environment. Note that you can switch environments without deactivating as activating a new environment automatically deactivates the currently active environment. In other words we could have just run source activate data .

Install your stuff

  pip install ipython notebook

Which pythons are available

You set up Python environments with (almost?) any version of Python available. Have a look:

  conda search python

List your current conda environments:

  conda info --envs

output example:

  # conda environments:
  #
  data                  *  /home/vagrant/miniconda2/envs/data
  deep                     /home/vagrant/miniconda2/envs/deep
  root                     /home/vagrant/miniconda2

Hint:

Notice the environment named root. This is the Python environment which hosts the conda command. You will probably don't want to install to your root Python so ensure that you make one of your custom Pythons is active before installing programs.

Removing environments

You can probably just delete any non-root Python environments you have created using the conda command like this:

  rm -R /home/vagrant/miniconda2/envs/data

but you probably want to check out the manual first eh?

Completely uninstalling miniconda

If you destory your Miniconda installation you can get rid of it very easily. Keep in mind that this will also delete any associated environments as well.

  rm -rf ~/miniconda
  rm -rf ~/.condarc ~/.conda ~/.continuum

remove PATH environment variable

  edit ~/.bash_profile
  • deep_learning_miniconda_project.1489181741.txt.gz
  • Last modified: 2024/03/26 13:52
  • (external edit)