Some of my Python projects rely on a variety of packages, most of which install flawlessly via PIP. Recently I’ve been setting up a new machine running Ubuntu 16.04 and I ran into a few issues installing CFFI via PIP. Essentially CFFI requires some packages that are not installed by default on Ubuntu 16.04, the following should help you get past those issues.
When running pip install cffi the first failure you might see is something like this:
building '_cffi_backend' extension creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/c x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python3.5m -I/home/janis/.virtualenvs/virt_env/include/python3.5m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.5/c/_cffi_backend.o c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 ---------------------------------------- Failed building wheel for cffi
This happens because Python development headers are not installed (as hinted at by the missing Python.h include file), to fix that we must install:
sudo apt-get install python3-dev
Or if for some reason you are running Python2 and the packages were not already installed:
sudo apt-get install python-dev
After installing that there’s a second fail you might encounter:
running build_ext building '_cffi_backend' extension creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/c x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python3.5m -I/home/janis/.virtualenvs/virt_env/include/python3.5m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.5/c/_cffi_backend.o c/_cffi_backend.c:13:17: fatal error: ffi.h: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
This is again a result of missing packages, this time libffi-dev:
sudo apt-get install libffi-dev
After installing that I got a successful CFFI install via PIP.
aadarsh@aadarsh-pc:~$ sudo pip install cffi
The directory ‘/home/aadarsh/.cache/pip/http’ or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo’s -H flag.
The directory ‘/home/aadarsh/.cache/pip’ or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo’s -H flag.
Collecting cffi
Exception:
Traceback (most recent call last):
File “/usr/lib/python2.7/dist-packages/pip/basecommand.py”, line 209, in main
status = self.run(options, args)
File “/usr/lib/python2.7/dist-packages/pip/commands/install.py”, line 317, in run
requirement_set.prepare_files(finder)
File “/usr/lib/python2.7/dist-packages/pip/req/req_set.py”, line 360, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File “/usr/lib/python2.7/dist-packages/pip/req/req_set.py”, line 512, in _prepare_file
finder, self.upgrade, require_hashes)
File “/usr/lib/python2.7/dist-packages/pip/req/req_install.py”, line 273, in populate_link
self.link = finder.find_requirement(self, upgrade)
File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 442, in find_requirement
all_candidates = self.find_all_candidates(req.name)
File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 400, in find_all_candidates
for page in self._get_pages(url_locations, project_name):
File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 545, in _get_pages
page = self._get_page(location)
File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 648, in _get_page
return HTMLPage.get_page(link, session=self.session)
File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 757, in get_page
“Cache-Control”: “max-age=600”,
File “/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py”, line 480, in get
return self.request(‘GET’, url, **kwargs)
File “/usr/lib/python2.7/dist-packages/pip/download.py”, line 378, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File “/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py”, line 468, in request
resp = self.send(prep, **send_kwargs)
File “/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py”, line 576, in send
r = adapter.send(request, **kwargs)
File “/usr/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/adapter.py”, line 46, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File “/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/adapters.py”, line 376, in send
timeout=timeout
File “/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/connectionpool.py”, line 610, in urlopen
_stacktrace=sys.exc_info()[2])
File “/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/util/retry.py”, line 228, in increment
total -= 1
TypeError: unsupported operand type(s) for -=: ‘Retry’ and ‘int’
Ubuntu’s packaging of pip and requests come with modifications, when you start using pip to install globally you can start to run into issues like the one you see here. I suspect this is relevant:
https://github.com/requests/requests/issues/3560
Generally speaking a good approach to use is to do all your development work in virtualenvironments, this will save you from these mismatches between apt-get packages and pip installed packages.
So I almost never will have a case where I would do something like “sudo pip install cffi” I’d always be installing this into a virtualenvironment. The headers found in python-dev are a bit of a different situation because they are installing non python packages (C headers) hence the alternative install route.
There are of course other approaches to this but I have found this to be the path of least friction on Ubuntu.
Hope that helps!