Python on TUNI Linux computers

TUNI Linux computers meet Python requirements in several ways. This page describes who to use Python on TUNI Linux computers.

Python versions provided by the distribution and the IT Helpdesk

• The version of Python in the RHEL7 distribution is 2.7.5. It is installed on all computers and starts with the command python.
  • Many standard modules are packaged as separately installable components. Not all of these are installed by default, or even provided at all.
  • You can't remove this default version of Python since it is used by the system scripts!
• IT Helpdesk offers Python 3.6.x and 3.7.x. These are targeted towards courses, but others are welcome to use them. Python 3.6.x is installed on remote desktops and the workstation classroom TC217.
  • installation on your personal workstation: pkcon install lintutfi-python36 or pkcon install lintutfi-python37
  • both packages cannot be installed on a single computer at the same time
  • use with the commands python3 and pip3
  • standard modules are included in the package
  • it is intended to avoid version upgrades while courses are running.
• There are some other versions of Python in the channels for various reasons, but we don't recommend them. Some are not even on the default path, which makes them complicated to use.

In addition to the interpreters themselves the administration offers the PyCharm Python IDE. This too is targeted towards courses, but others are welcome to use it. It is installed on remote desktops and the workstation classroom TC217.

• installation on your personal workstation: pkcon install pycharm

Python versions outside distributed software

If you need an advanced Python programming environment on TUNI Linux computer, we suggest to install it yourself. With this method you can choose the version of Python and libraries you need, and they will be updated on the schedule of your choice. There is at least two different options for this.

It is, however, your responsibility to ensure all security updates are installed. 

Miniforge

Miniforge provides installer for conda package manager and environment management system for Python. Miniforge provided conda uses free of charge conda-forge channel as the default and only channel.

• Go to https://github.com/conda-forge/miniforge and download Miniforge installer version you need. You should not save it in your home directory but somewhere under /worktmp/username/ .
• Run the installer through the shell using your own account, something like this: bash /worktmp/${USERNAME}/Miniforge3-Linux-x86_64.sh .
  • Follow the instructions and accept the license.
  • The installer suggests to install conda (and by default all packages you install with conda) under your home directory. You should not install it under your home directory, since there is no need to backup something you can easily install again and home directory space is limited because of the backups. Give it another path, for example /worktmp/username/miniforge3 .
  • Allow the installer to add its path into your shell initialization files if you want to. Doing this means you will effectively use anaconda as Python in all shells opened afterwards.

Abandoning Anaconda and Miniconda

This page previously recommended Anaconda, Inc.'s Python distribution named Anaconda as the basis for an advanced Python installation. Miniconda is an installation tool also from the same company, which by default utilizes Python add-ons from Anaconda’s repository. The licensing terms for Anaconda have changed, and both the tool itself and the add-ons offered from its repositories now require a paid license. You need to obtain a license through the IT Helpdesk for their usage.

If Anaconda was previously used on the computer and has been removed, a configuration file may have been left behind, causing Miniforge to use Anaconda’s paid add-ons. The conda tool creates the configuration file which is typically located at /home/username/.condarc, but it may also be located e.g. in a Python project’s own directory in some cases.

Disabling Anaconda's paid add-ons

Here’s how to ensure that Miniforge does not use Anaconda’s paid add-ons:

• Enter the command conda info to display the conda settings on the screen.
• Check if the section "channel URLs" section lists addresses with the server "repo.anaconda.com". This is the address offering paid add-ons, and it needs to be removed. "conda.anaconda.org", on the other hand, is Miniforge’s default channel and offers free content.
• If paid add-on channel is use, it is likely due to an add-on channel named "defaults" that remains from the Anaconda installation. This setting is typically in configuration file /home/username/.condarc. Deleting the file is the easiest way to fix the issue. Running conda again will create a new and clean .condarc file.

If you want to preserve other conda setting changes that you may have done, and don't want to delete .condarc, you can edit your .condarc file with text editor and remove "defaults" or modify the add-on channels using conda command:

• Check configured channels with the command conda config --show channels
• Remove the unwanted channel "defaults" with command: conda config --remove channels defaults
• If Miniforge’s default free add-on channel "conda-forge" is not in use, add it with the command: conda config --add channels conda-forge
• Finally, verify with command conda info that "repo.anaconda.com" is no longer listed in the add-on channel addresses.

Compiling a specific version of Python

If you need a specific version of Python, you can compile it from source code. For example:

mkdir /worktmp/${USERNAME}
cd /worktmp/${USERNAME}
wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz
tar xvJf Python-3.7.0.tar.xz
cd Python-3.7.0
./configure --prefix=/worktmp/${USERNAME}/mypython
make -j4
make install

To test this, run command /worktmp/${USERNAME}/mypython/bin/python3 --version .

If you want to add this to your default PATH, you can add following on your bash init scripts (ie. /home/${USERNAME}/.bashrc file):

PATH=/worktmp/${USERNAME}/mypython/bin:${PATH}
export PATH

Please note! Normally you should NOT add anything before the default PATH. Only do this when you are sure that EVERY binary in this directory are something you want to use instead of the default ones!

Installation of extra modules; pip and conda

The major versions (meaning the first two numbers of the version) of Python do not share modules with each other. This means that modules installed for Python 2.7.5 or 3.4.9 are not available to Python 3.6.6, but those installed for Python 3.6.1 should be. Various modules may have dependencies or version demands, so when installing one required module it may be necessary to install ten others. Python's own package manager pip can resolve dependencies and fetch necessary modules.

• When using pip or pip3 with a Python provided by the operating system or the IT Helpdesk, it's necessary to give it the --user flag.
  • The modules then install into the home directory, for example under /home/${USERNAME}/.local/lib/python2.7 or /home/${USERNAME}/.local/lib/python3.6 . Python should find them from here without extra steps.
  • The modules consume some space from the home directory. Some of it can be reclaimed by clearing pip's cache afterwards like this: rm /home/${USERNAME}/.cache/pip 
• If you have compiled Python by yourself, you can install extra modules on top of it without giving pip the --user flag. The same applies when using conda package manager installed with Miniforge installer, but it would probably be better to use it's own conda command instead of pip.