2017-06-16

User-site-installed python packages, and path modification

PEP 370 allows you to have user-local installs of packages. You're supposed to run pip --user install whatever.

If you want to do that by default, so any future "pip install" does install to the user-site, you can add a pip configuration file telling pip to do so. In MacOS, the file must be located at ~/Library/Application Support/pip/pip.conf, and contains
[global]
user = true
(BEWARE! If you use virtual environments, note that thanks to another instance of Python brain damage/unfinished business, as of Python 3.5.3 and pip 9 the --user argument causes problems with installs in virtual environments of both venv and virtualenv varieties... and looks like the problem comes rather from the argument, not from the functionality! A Quick&Dirty fix is to do something like export PIP_CONFIG_FILE=/dev/null inside the virtual environment. This could be automated with hooks in virtualenvwrapper, but it starts to feel like a pile-up of hacks...)

OK, so now you have user-local installs by default. But their commands are not in your PATH, so you need to set that too.

You could go the naïve way and just add something like
PATH="~/Library/Python/3.5/bin/:$PATH"
to your .profile. But, note: here we're hardwiring python 3.5 (or any other version). Is that what you want? Is the rest of your system so hardwired? Unless you specifically did so, the answer probably is: no, it isn't. So you might end up mixing Python installs/environments.


I couldn't find an official solution (or even acknowledgement) to this problem. However, Ruby does offer a very nice FAQ example for their case [1]. Following their example, I ended up with this Python snippet:
PATH="$(python -c 'import site; print(site.USER_BASE)')/bin:$PATH"
 ... which asks the current python interpreter for its user-site directory.

But this is not enough yet, because if you use something like MacPorts you can perfectly be in a situation where you have activated different Python versions for the interpreter and for PIP. So this can cause a situation where your "python" command runs python 2.7, while "pip" runs the pip from python 3.5, and any pip-installed packages/commands are therefore for 3.5. That would be, at the very least, confusing.

So, again following Ruby's example, you can add some guarding to make sure that at least both versions are the same.
PYTHON_WISHED_VERSION=35
if port select --show python | grep $PYTHON_WISHED_VERSION > /dev/null &&
   port select --show pip | grep $PYTHON_WISHED_VERSION > /dev/null ; then
    PATH="$(python -c 'import site; print(site.USER_BASE)')/bin:$PATH"
else
    echo Python and PIP not at version $PYTHON_WISHED_VERSION: not adding the user-site PATH
fi
Of course here we're hardwiring Python 3.5 again :P. However, at least now 1) it's self-consistent, 2) if it wasn't it will let you know instead of going on obliviously. And well, there's a single number to change to make it work in any other version.

I guess that instead of mandating a version number you could make it auto-detect the python version, check that pip is at the same version, and then use the corresponding path; but my use case doesn't involve such frequent/flexible version changes.

Anyway, perhaps this could be done more elegantly in other ways, but again Python is not my workhorse and I only care here about CLI environment sanity. If you can improve this, let me know!




[1] It's interesting how every time I peek a bit into Ruby it looks so much nicer than Python. I really need to get into it.

No comments

Post a Comment