Skip to content
This repository was archived by the owner on Jun 9, 2021. It is now read-only.
This repository was archived by the owner on Jun 9, 2021. It is now read-only.

Instructions to install TensorFlow in a Conda Environment #153

Open
@mwidjaja1

Description

@mwidjaja1

This is not so much an issue as opposed to a 'How To' if you'd like to install this version of Tensorflow in Conda.

Prerequisites: You must be on macOS Big Sur

If you have an Apple Silicon Mac, this is a freebie, you're already on Big Sur. If you're on an Intel Mac, the Intel versions of TensorFlow are Big Sur only.

Sanity Check before Proceeding: To ensure you're on the right version of macOS, run sw_vers -productVersion in your terminal. If it's not version 11.##, you're not on Big Sur and must upgrade to it from the macOS App Store.

Prerequisites: Install XCode Command Line Tools

Install Xcode Command Line tools if you haven't. To do so, run this in your terminal: xcode-select --install

Sanity Check before Proceeding: To ensure installation worked, run which xcrun in your terminal and you should get a path like /usr/bin/xcrun. If you haven't, you did not install it correctly.

Prerequisites: Install Miniforge

Where to download Miniforge from
Miniforge, is a 'lightweight' Python interpreter that has full access to the Conda ecosystem. You can download Miniforge from https://github.com/conda-forge/miniforge#miniforge3. You can use Anaconda if you're on Intel, but note that this guide will be written from the perspective of using miniforge.

Sanity Check before Proceeding:

  • Run file $(which python) in your terminal (thanks to @lebigot for this shortcut!). Please make sure that you got:
    • This path implies you're running your miniforge version of Python. It'll probably be <your home dir>/miniforge3/bin/python.
    • If you have an Apple Silicon Mac, it should also say Mach-O 64-bit executable arm64. If you have an Intel Mac, it should also say Mach-O 64-bit executable x86_64.
  • Run which pip in your terminal and it too should resolve to some path that implies you're using miniforge3.

If any of those sanity checks failed, you must redo this section. Please ensure that you downloaded the correct Miniforge for your system architecture and installed it. If you did all that, set your environment paths to Miniforge's Python Installation. To do that, you need to figure out where conda was installed to (it's probably ~/miniforge3/condabin/conda) and then run ~/miniforge3/condabin/conda init in your terminal.

Apple Silicon Only Warning: You CANNOT use Anaconda

This warning only applies to Apple Silicon Macs. Anaconda comes with many Python packages included, some of which are not Apple Silicon (i.e. ARM) compatible and thus Anaconda is not ARM compatible. You can use Anaconda if you're using an Intel Mac though.

If you were planning to use Anaconda on ARM, please scroll back up and install Miniforge. Miniforge has Conda, which means you can install many of the packages you want such as Pandas, Scipy, and Numpy -- unlike Anaconda, you just have to do the install manually by running conda install mypackagenamehere.

Intel Only Warning: Python Bugs in Big Sur

This warning only apply to Intel Macs. For Intel, both Anaconda and MiniForge have a Python Bug which prevents you from running Python correctly in some instances on macOS Big Sur. Until the Python community fixes this, each time prior to loading Python, you must run export SYSTEM_VERSION_COMPAT=0. You could also add this to your .bash_profile or other shell environment file if you have one, to do this automatically for you.

Installing TensorFlow

Attached to this Issue is a YAML file which will help you create a Conda Environment with TensorFlow, along with all the prerequisites you need from the ARM conda-forge channel.

  1. Download environment.yml, which contains the instructions to create a Python environment with the dependencies you need -- we'll install TensorFlow afterwards. Some browsers insist on adding .txt to the end of the file -- do not let your browser do that. [thanks to @isuruf for streamlining this file to be all Conda]
  2. In your terminal run this command, replacing the uppercase variables with the path to your environment.yml file and your desired name for this environment: conda env create --file=PATH_TO_ENVIRONMENT.YML --name=YOUR_ENV_NAME_HERE.
  3. Activate that environment by running this command, replacing the uppercase variable with your environment's name: conda activate YOUR_ENV_NAME_HERE
  4. Pip install the TensorFlow wheels by running the commands below. By the way, the URLs for the TensorFlow wheel files came from the Releases page, so you can swap these wheel files out with a prior version of TensorFlow as needed.

For X86 as of 03/11/2021:
Thanks to @edwin-yan for the updated commands

pip install --upgrade --force --no-dependencies https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha3/tensorflow_macos-0.1a3-cp38-cp38-macosx_11_0_x86_64.whl https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha3/tensorflow_addons_macos-0.1a3-cp38-cp38-macosx_11_0_x86_64.whl 

For Apple Silicon as of 03/11/2021:

pip install --upgrade --force --no-dependencies https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha3/tensorflow_macos-0.1a3-cp38-cp38-macosx_11_0_arm64.whl https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha3/tensorflow_addons_macos-0.1a3-cp38-cp38-macosx_11_0_arm64.whl
  1. Finally, give it a spin. Run python and try importing tensorflow.

Example Commands

In this below example, I'm installing & running the ARM version of tensorflow from an environment I've named test. The yml file is placed in the same directory I'm running this command from, which is my home directory (i.e. ~)

conda env create --file=environment.yml --name=test
conda activate test
pip install --upgrade --force --no-dependencies https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha2/tensorflow_addons_macos-0.1a2-cp38-cp38-macosx_11_0_arm64.whl https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha2/tensorflow_macos-0.1a2-cp38-cp38-macosx_11_0_arm64.whl
python
import tensorflow

Screen Shot 2021-02-02 at 10 19 26 PM

Troubleshooting for importing TensorFlow

  • Type in which python and then which pip in your terminal. Both paths should point to a Python that is inside the environment you created in Step 2. If it doesn't, you may not have installed Miniforge correctly, ran Step 2 correctly, and/or may not have ran Step 3.
  • Run python --version and it should be version 3.8. If it isn't, you most likely did not create or activate your environment correctly, as per Steps 2 & 3. Do those again.
  • If python is correctly pointed to the right environment but you cannot import tensorflow, consider running step 5 again just to make sure you installed Tensorflow in the appropriate environment.
  • If you are using Intel and got a not a supported wheel on this platform error, run export SYSTEM_VERSION_COMPAT=0 in your terminal and try again. If this works, you'll need to do this everytime you use Python until a Python Bug is resolved.
  • Please verify that you did ALL of the Sanity Checks from the previous section and that they resolve appropriately before posting your issue here. If you do post your issue, please provide the terminal outputs from those steps and bonus points if you share the results of your Sanity Check and run pip with a -v flag for additional logging. Remember I'm just a volunteer -- I'll try to help but there's only so much I can help with.

Troubleshooting for setting up TensorFlow

  • For those having issues with tf.keras.models.load_model about a failed to decode error: Try downgrading to h5py to the 2.10.0 wheel file that was packaged with this alpha release (pip install ~/path to h5py.whl). Thanks to @ramicaza.

Activity

changed the title [-]Installing TensorFlow in a Conda Environment[/-] [+]Instructions to install TensorFlow in a Conda Environment[/+] on Feb 3, 2021
thomastiotto

thomastiotto commented on Feb 3, 2021

@thomastiotto

I couldn't install the wheels without renaming them to read "...-macosx_10_16_..." instead of "...-macosx_11_0_..."

mwidjaja1

mwidjaja1 commented on Feb 3, 2021

@mwidjaja1
Author

@Tioz90 I didn't have that problem. Could you tell me what your Mac's version is? That's Apple (in the Menu Bar) > About my Mac. Mine is Version 11.2.

thomastiotto

thomastiotto commented on Feb 3, 2021

@thomastiotto

@mwidjaja1 MacOs 11.1

mwidjaja1

mwidjaja1 commented on Feb 3, 2021

@mwidjaja1
Author

@Tioz90 You may want to create a new issue then about this. I'm not on the team working on this, I just wrote up these instructions as a 'side contributor' with their help.

thomastiotto

thomastiotto commented on Feb 3, 2021

@thomastiotto

I think it's a known issue with pip

mwidjaja1

mwidjaja1 commented on Feb 3, 2021

@mwidjaja1
Author

That's an interesting thought @Tioz90 -- My pip version is 20.2.4 which I checked by running pip --version. If yours isn't that version, could you get that version and try again?

thomastiotto

thomastiotto commented on Feb 3, 2021

@thomastiotto

Mine is 21.0.1 and it still does not work..

mwidjaja1

mwidjaja1 commented on Feb 3, 2021

@mwidjaja1
Author

@Tioz90 I cannot reproduce your issue. I've upgraded to the same pip version you had, trashed my environment, and was able to successfully rebuild it. I have a hunch somewhere in your Python installation, you're accidentally linking to some other Python that you did not intend to use but I wouldn't know how to begin figuring that out. Glad you got it to work by renaming it though

hoytak

hoytak commented on Feb 4, 2021

@hoytak
Collaborator

There's a few things that could make this issue a mess. I haven't found versions of pip later than 20.2.4 to work here; version 20.3 and later actually broke some of the logic. Also, the logic inside of pip is duplicated across setuptools and packaging, both of which are baked into the core python distribution; upgrading these might help, but your mileage may vary.

Stanfording

Stanfording commented on Feb 4, 2021

@Stanfording

Does it mean that installing TensorFlow in a Conda Environment allows m1 chip to use tensorflow with SciPy and dependent packages?

thomastiotto

thomastiotto commented on Feb 4, 2021

@thomastiotto
jettpark

jettpark commented on Feb 5, 2021

@jettpark

I got this Error
ERROR: tensorflow_macos-0.1a2-cp38-cp38-macosx_11_0_x86_64.whl is not a supported wheel on this platform.
This is my System Information:

  • python: /Users/gyuannn/miniforge3/envs/test/bin/python, 3.8.6
  • pip: /Users/gyuannn/miniforge3/envs/test/bin/pip, 20.2.4
  • System: MacOS11.2
  • Laptop: MBP16,2019
  • Run command on Conda Env builded from environment.yml

I'm Confused... 

Run python -m pip debug --verboseand
find pip not support cp38-cp38-macosx_11_0_x86_64 only support cp38-cp38-macosx_10_16_x86_64

Indeed:
Python 3.8 from the system:

/usr/bin/python3 -c "import platform; print(platform.platform())"
macOS-11.0.1-x86_64-i386-64bit

Python 3.9 installed with homebrew:

/usr/local/bin/python3 -c "import platform; print(platform.platform())"
macOS-11.0.1-x86_64-i386-64bit

Python 3.8 from a conda-forge env:

/Users/ogrisel/miniforge3/envs/py38/bin/python -c "import platform; print(platform.platform())"
macOS-10.16-x86_64-i386-64bit

So conda-forge's Python is indeed pretending to be installed on a fake 10.16 version of macOS to bypass this problem with pip 20.2.4.

91 remaining items

johnnynunez

johnnynunez commented on May 17, 2021

@johnnynunez

tensorflow v2.5.0 is out. Is it compatible with m1?
Support native python 3.9...

JakobLS

JakobLS commented on May 24, 2021

@JakobLS

Failing with installing Miniforge on Mac M1 using brew

If any of those sanity checks failed, you must redo this section. Please ensure that you downloaded the correct Miniforge for your system architecture and installed it. If you did all that, set your environment paths to Miniforge's Python Installation. To do that, you need to figure out where conda was installed to (it's probably ~/miniforge3/condabin/conda) and then run ~/miniforge3/condabin/conda init in your terminal.

My installation was failing on this part. It turned out that Miniforge had been installed to /opt/homebrew/Caskroom/miniforge, so by running /opt/homebrew/Caskroom/miniforge/base/condabin/conda init followed by source .bash_profile solved the problem for me.

mwidjaja1

mwidjaja1 commented on May 26, 2021

@mwidjaja1
Author

Hey @JakobLS, while I'm glad you figured it out, I'm super curious how your Homebrew got installed to caskroom. I... don't think that was possible. Do you recall manually requiring miniforge to save there or did it just happen?

Glad you figured it out though, that is the absolute right step! Thanks for sharing

JakobLS

JakobLS commented on May 26, 2021

@JakobLS

Hi @mwidjaja1. Thank you so much for supporting the community. I don't recall the exact details, but I'm pretty sure I didn't manually specify miniforge to be installed at a certain location, nor with Homebrew. I followed your steps in the top, but chose to install miniforge using Homebrew purely for convenience (rather than downloading it first).

And this tutorial was very useful to install Homebrew with.

I had to add export PATH=$PATH:/opt/homebrew/bin and export PATH="/opt/homebrew/Caskroom/miniforge/base:$PATH" to .bash_profile for Homebrew and miniforge respectively though.

Running on MacOS Big Sur 11.3.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @lebigot@hoytak@mathieuboudreau@solie@gosstyla

        Issue actions

          Instructions to install TensorFlow in a Conda Environment · Issue #153 · apple/tensorflow_macos