Resolving CFFI install issues on Ubuntu

Resolving CFFI install issues on Ubuntu

Some of my Python projects rely on a variety of packages, most of which install flawlessly via PIP. Recently I’ve been setting up a new machine running Ubuntu 16.04 and I ran into a few issues installing CFFI via PIP. Essentially CFFI requires some packages that are not installed by default on Ubuntu 16.04, the following should help you get past those issues.

When running pip install cffi the first failure you might see is something like this:

building '_cffi_backend' extension
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/c
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python3.5m -I/home/janis/.virtualenvs/virt_env/include/python3.5m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.5/c/_cffi_backend.o
c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Failed building wheel for cffi

This happens because Python development headers are not installed (as hinted at by the missing Python.h include file), to fix that we must install:

sudo apt-get install python3-dev

Or if for some reason you are running Python2 and the packages were not already installed:

sudo apt-get install python-dev

After installing that there’s a second fail you might encounter:

running build_ext
building '_cffi_backend' extension
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/c
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python3.5m -I/home/janis/.virtualenvs/virt_env/include/python3.5m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.5/c/_cffi_backend.o
c/_cffi_backend.c:13:17: fatal error: ffi.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

This is again a result of missing packages, this time libffi-dev:

sudo apt-get install libffi-dev

After installing that I got a successful CFFI install via PIP.

Liked it? Take a second to support Janis on Patreon!

2 Comments

  1. aadarsh@aadarsh-pc:~$ sudo pip install cffi
    The directory ‘/home/aadarsh/.cache/pip/http’ or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo’s -H flag.
    The directory ‘/home/aadarsh/.cache/pip’ or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo’s -H flag.
    Collecting cffi
    Exception:
    Traceback (most recent call last):
    File “/usr/lib/python2.7/dist-packages/pip/basecommand.py”, line 209, in main
    status = self.run(options, args)
    File “/usr/lib/python2.7/dist-packages/pip/commands/install.py”, line 317, in run
    requirement_set.prepare_files(finder)
    File “/usr/lib/python2.7/dist-packages/pip/req/req_set.py”, line 360, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
    File “/usr/lib/python2.7/dist-packages/pip/req/req_set.py”, line 512, in _prepare_file
    finder, self.upgrade, require_hashes)
    File “/usr/lib/python2.7/dist-packages/pip/req/req_install.py”, line 273, in populate_link
    self.link = finder.find_requirement(self, upgrade)
    File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 442, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
    File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 400, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
    File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 545, in _get_pages
    page = self._get_page(location)
    File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 648, in _get_page
    return HTMLPage.get_page(link, session=self.session)
    File “/usr/lib/python2.7/dist-packages/pip/index.py”, line 757, in get_page
    “Cache-Control”: “max-age=600”,
    File “/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py”, line 480, in get
    return self.request(‘GET’, url, **kwargs)
    File “/usr/lib/python2.7/dist-packages/pip/download.py”, line 378, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
    File “/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py”, line 468, in request
    resp = self.send(prep, **send_kwargs)
    File “/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py”, line 576, in send
    r = adapter.send(request, **kwargs)
    File “/usr/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/adapter.py”, line 46, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
    File “/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/adapters.py”, line 376, in send
    timeout=timeout
    File “/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/connectionpool.py”, line 610, in urlopen
    _stacktrace=sys.exc_info()[2])
    File “/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/util/retry.py”, line 228, in increment
    total -= 1
    TypeError: unsupported operand type(s) for -=: ‘Retry’ and ‘int’

    1. Ubuntu’s packaging of pip and requests come with modifications, when you start using pip to install globally you can start to run into issues like the one you see here. I suspect this is relevant:
      https://github.com/requests/requests/issues/3560

      Generally speaking a good approach to use is to do all your development work in virtualenvironments, this will save you from these mismatches between apt-get packages and pip installed packages.

      So I almost never will have a case where I would do something like “sudo pip install cffi” I’d always be installing this into a virtualenvironment. The headers found in python-dev are a bit of a different situation because they are installing non python packages (C headers) hence the alternative install route.

      There are of course other approaches to this but I have found this to be the path of least friction on Ubuntu.

      Hope that helps!

Leave a Reply

Your email address will not be published. Required fields are marked *