Skip to content

pip install cryptography broken on OS X El Capitan #2692

Closed
@ctismer

Description

@ctismer

The simple pip install cryptography command does not work on OS X (only tested on El Capitan).

What I tried:

brew install openssl
pip install cryptography

results in

build/temp.macosx-10.9-intel-2.7/_openssl.c:425:10: fatal error: 'openssl/aes.h' file not found

#include <openssl/aes.h>

         ^

1 error generated.

error: command 'clang' failed with exit status 1

The following workaround helped:

$ ARCHFLAGS="-arch x86_64" LDFLAGS="-L/usr/local/opt/openssl/lib" \
   CFLAGS="-I/usr/local/opt/openssl/include" pip install cryptography

Activity

reaperhulk

reaperhulk commented on Jan 28, 2016

@reaperhulk
Member

Upgrade your pip as you appear to be using a version that predates support for binary wheels. What does pip --version give you?

ctismer

ctismer commented on Jan 28, 2016

@ctismer
Author
$ pip --version
pip 8.0.2 from /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (python 2.7)

No, I use wheel support myself since a long time. There seems to be a reason that prevents
the application of a wheel.
Also I don't think it is correct if it enforces wheels.

reaperhulk

reaperhulk commented on Jan 28, 2016

@reaperhulk
Member

Perhaps an explanation of why cryptography prefers wheels would be helpful here.

cryptography ships with an OpenSSL backend. This backend needs to be compiled and linked against libssl and libcrypto. OS X historically has shipped with OpenSSL, but Apple made the decision to deprecate and strongly recommend against its use starting with OS X 10.7. While we previously supported linking against it, this support is being dropped as the 0.9.8 series is no longer supported by the OpenSSL project and is also so old that major features necessary for modern TLS support are not present. Since we need to compile against a newer version then an alternate version of OpenSSL must be installed (typically by either homebrew or macports). Unfortunately, since Apple still ships the 0.9.8 libraries (but not the headers as of OS X El Capitan) these newer OpenSSL libraries can't safely be put in the default lib search path. So, our options were to either force anyone who wanted to install cryptography to install openssl separately and understand the C toolchain enough to provide the proper include/linker flags, or build a static wheel that would allow pip install cryptography to work with no compiler required at all.

Of course, we do still document how to build it on OS X using LDFLAGS and CFLAGS to dynamically link against a user-installed OpenSSL, so it's not correct to state that we require them. When calling pip install it preferentially uses them because it dramatically simplifies the install process for the vast majority of our OS X and Windows users.

In general this binary wheel resolution process is pretty reliable, but sometimes edge cases occur. We currently upload two OS X wheels for each Python we support (a 10_6 SDK version and a 10_10). pip 8 altered its wheel compatibility code to prefer the newest compatible wheel. In your case it appears you're running on Mavericks, so the 10_6 SDK wheel should have been downloaded. To test I just booted a Mavericks VM and using pip 8.0.2 I got the 10_6 wheel as expected with pip install cryptography, so I'm not sure what's going on. You may want to open an issue with pip.

ctismer

ctismer commented on Jan 28, 2016

@ctismer
Author

Thanks for the explanation. No, as stated on top, I used El Capitan, which is 10.11.2 .
With which python do you try on OS X?
There is a big difference between a downloaded Python from Python.org and a homebrew python.
They are not compatible, because they use different models, and when I build packages,
I always have to build them twice, because the one or the other will complain (like me :) ).
Don't remember exactly right now, but I think it was universal build (python.org) vs. simple build from homebrew.

Of course, we do still document how to build it on OS X using LDFLAGS and CFLAGS to dynamically link against a user-installed OpenSSL, so it's not correct to state that we require them.

The setup script should configure itself and find needed arguments. The problem is that due
to common practice in other packages, cryptography simply gets installed as a dependancy, and
that then fails miserably.

This was what actually happened, when I tried

pip install --pre gitgub3.py
ctismer

ctismer commented on Jan 28, 2016

@ctismer
Author

I think I was causing the problem by using another python version (stackless python).
Sorry about the confusion.
The statement still holds: The build should configure itself, when it claims to be installable
with pip.

Cheers - Chris

alex

alex commented on Jan 28, 2016

@alex
Member

There's no way for the build to configure itself, if OpenSSL's headers aren't on your platform's include path, there's nothing we can do.

ctismer

ctismer commented on Jan 28, 2016

@ctismer
Author

@alex the installation paths of homebrew et.al. are a de-facto standard,
so I think it should at least be possible to give a useful hint instead of crashing.

zopyx

zopyx commented on Aug 22, 2016

@zopyx

I have a similar issue with installing Plone 5.0 and some add-ons depending on the cryptography module. I see the same include problem running buildout on MacOSX. However the workaround here is to "pip install crypography" inside the local virtual env that is used to run buildout.

carn1x

carn1x commented on Nov 9, 2016

@carn1x

Upgrading from pip==1.3.1 to pip==9.0.1 fixed this for me

hanxinhisen

hanxinhisen commented on Dec 20, 2016

@hanxinhisen

pip install --upgrade pip

alexislg2

alexislg2 commented on Dec 22, 2016

@alexislg2

Thanks @hanxinhisen that worked ;)

iolloyd

iolloyd commented on Jan 6, 2017

@iolloyd
dmitrytokarev

dmitrytokarev commented on Jan 16, 2017

@dmitrytokarev

brew to the rescue:

brew install openssl

then brew gives a hint:

Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries

Generally there are no consequences of this for you. If you build your
own software and it requires this formula, you'll need to add to your
build variables:

    LDFLAGS:  -L/usr/local/opt/openssl/lib
    CPPFLAGS: -I/usr/local/opt/openssl/include
    PKG_CONFIG_PATH: /usr/local/opt/openssl/lib/pkgconfig

So just export these env vars and rerun your favorite pip install (or better to avoid havoc in future add following to your ~/.bash_profile:

export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"
export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"

This worked for me like a charm!

Jay54520

Jay54520 commented on Jan 18, 2017

@Jay54520

@dmitrytokarev This worked for me

python3.6/site-packages/wheel/bdist_wheel.py", line 155, in get_tag
      assert tag == supported_tags[0]
  AssertionError
  
  ----------------------------------------
  Failed building wheel for cryptography
  Running setup.py clean for cryptography
Failed to build cryptography
Installing collected packages: cryptography
  Running setup.py install for cryptography ... done
Successfully installed cryptography-1.6


pip list | grep cry
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
cryptography (1.6)
pycrypto (2.6.1)
reaperhulk

reaperhulk commented on Jan 18, 2017

@reaperhulk
Member

cryptography 1.7 has a py36 wheel.

22 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @alex@walles@reaperhulk@leonsmith@mahmudahsan

        Issue actions

          `pip install cryptography` broken on OS X El Capitan · Issue #2692 · pyca/cryptography