Skip to content

py_available() always returns FALSE although Python is installed #385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MTamPham opened this issue Nov 15, 2018 · 12 comments
Open

py_available() always returns FALSE although Python is installed #385

MTamPham opened this issue Nov 15, 2018 · 12 comments

Comments

@MTamPham
Copy link

MTamPham commented Nov 15, 2018

Hi

I'm using Mac OS 10.13.6 and I installed Python 3.7.1 by using pyenv. I already switched to that version by pyenv global 3.7.1. So it returns 3.7.1 when I check by $ python --version and returns two Python directories namely '/Users/tampm/.pyenv/shims/python' and '/usr/bin/python' when I run $ which -a python.
But unfortunately, when I call py_available() method on R console, it always returns FALSE although I can see two Python directories mentioned above when I call py_discover_config() method.
Would anyone mind helping me to solve that issue?

Update if I try to call import a Python library on R console and call py_available() method again, it returns TRUE. I did

library(reticulate)
os <- import("os")
os$listdir(".")

It is quite wierd, isn't it? Did I do something wrong in my Python installation?

Thanks

@ghuls
Copy link

ghuls commented Nov 16, 2018

We also experienced the same problem:
py_available() always returns FALSE
with R and python on Clearlinux.

The problem is that on Clearlinux LIBDIR is set to /usr/lib while libpython is located in /usr/lib64

I temporarily fixed it with this by hardcoding the value:

reticulate/inst/config/config.py

try:
  import sysconfig
  if not platform.system() == 'Windows':
+    sysconfig._CONFIG_VARS['LIBDIR'] = '/usr/lib64'
    sys.stdout.write('\nLIBPL: ' + sysconfig.get_config_vars('LIBPL')[0])
    sys.stdout.write('\nLIBDIR: ' + sysconfig.get_config_vars('LIBDIR')[0])
  sys.stdout.write('\nPREFIX: ' + sysconfig.get_config_vars('prefix')[0])
  sys.stdout.write('\nEXEC_PREFIX: ' + sysconfig.get_config_vars('exec_prefix')[0])
except Exception:
pass

The code in JuliaPy (on which this was based according to https://github.com/rstudio/reticulate/blob/24805aa1af2a6e59d22c5e088f2c358fff9e2b3b/inst/config/config.py) works correctly (finds the library in LD_LIBRARY_PATH):
https://raw.githubusercontent.com/JuliaPy/PyCall.jl/master/deps/find_libpython.py

$ python ~/HOME/find_libpython.py
/usr/lib64/libpython3.6m.so.1.0

@MTamPham
Copy link
Author

Apologize but I can't catch you. Do you mean that you're using Clear Linux OS and you fixed the problem manually by adding the LIBDIR = "/usr/lib64" in reticulate/inst/config/config.py?

Can you show me where the config.py is? I don't use JuliaPy and I'm new to Python as well.

@ghuls
Copy link

ghuls commented Nov 28, 2018

The following should give the base dir of the reticulate package:

library('reticulate')
path.package('reticulate')

Then find libpython3.7:

find / -name 'libpython3.7*'

@protodonata
Copy link

Sorry that I am new to everything.
I am also facing the same problem. My Mac is using 2.7 (the very original one). How can I find the path of the python? Should I install the latest by the way?
Many thanks.

@mnr
Copy link

mnr commented Feb 12, 2019

I'm trying to trace this down. Here's what I find...

  1. py_available() returns false
  2. py_discover_config()
> py_discover_config()
python:         C:\Users\markn\AppData\Local\Programs\Python\Python38\python.exe
libpython:      C:/Users/markn/AppData/Local/Programs/Python/Python38/python38.dll
pythonhome:     C:\Users\markn\AppData\Local\Programs\Python\Python38
version:        3.8.0a1 (tags/v3.8.0a1:e75eeb00b5, Feb  3 2019, 20:47:39) [MSC v.1916 64 bit (AMD64)]
Architecture:   64bit
numpy:           [NOT FOUND]

python versions found: 
 C:\Users\markn\AppData\Local\Programs\Python\Python38\python.exe
 C:\Users\markn\AppData\Local\Programs\Python\PYTHON~1\\python.exe
 C:\Users\markn\AppData\Local\Programs\Python\Python38\\python.exe
  1. debug(py_available)
  2. py_available(initialize = TRUE) gets to ensure_python_initialized() which returns:
Error in py_initialize(config$python, config$libpython, config$pythonhome,  : 
  Py_CompileString - The specified procedure could not be found.
  1. Change line 12 of libpython.h to:
#define LIBPYTHON_EXTERN

removes the error related to Py_CompileString, but py_available() still returns FALSE

  1. resume debugging without the line 12 change in libpython.h.
  2. Step into ensure_python_initialized
  3. !is_python_initialized() returns TRUE and we go into the if... statement
  4. !is.null(.globals$delay_load_module) returns FALSE
  5. initialize_python(required_module, use_environment) returns:
Error in py_initialize(config$python, config$libpython, config$pythonhome,  : 
  Py_CompileString - The specified procedure could not be found.

Question: Should the parameters to initialize_python be defined as something? Why are they just listed in the function call without some sort of "=" ?

  1. step into initialize_python(). (Note that config$numpy$version < "1.6" returns logical(0). Was it expecting FALSE?)

  2. step into py_initialize(config$python, config$libpython, config$pythonhome, config$virtualenv_activate, config$version >= "3.0", interactive(), numpy_load_error)

Here is a dump of the variables being passed to py_initialize...

config
python:         C:\Users\markn\AppData\Local\Programs\Python\Python38\python.exe
libpython:      C:/Users/markn/AppData/Local/Programs/Python/Python38/python38.dll
pythonhome:     C:\Users\markn\AppData\Local\Programs\Python\Python38
version:        3.8.0a1 (tags/v3.8.0a1:e75eeb00b5, Feb  3 2019, 20:47:39) [MSC v.1916 64 bit (AMD64)]
Architecture:   64bit
numpy:           [NOT FOUND]

python versions found: 
 C:\Users\markn\AppData\Local\Programs\Python\Python38\python.exe
 C:\Users\markn\AppData\Local\Programs\Python\PYTHON~1\\python.exe
 C:\Users\markn\AppData\Local\Programs\Python\Python38\\python.exe

Browse[4]> config$virtualenv_activate
[1] ""

Browse[4]> config$version
[1] "3.8"

Browse[4]> interactive()
[1] TRUE

Browse[4]> numpy_load_error
[1] "installation of Numpy >= 1.6 not found"

  1. invisible(.Call(_reticulate_py_initialize, python, libpython, pythonhome, virtualenv_activate, python3, interactive, numpy_load_error))

... and we exit py_available with return FALSE

@ncoppersmith-r7
Copy link

I'm not sure what the difference is, but I was experiencing this issue using Python 3.8 on Windows 10. I was able to solve it by installing Anaconda for Windows and changing my Sys.setenv(RETICULATE_PYTHON = PATH) to use Anaconda python instead of the regular python binary. Not sure what the underlying issue is, but it would appear that reticulate just doesn't like standard python on Windows.

@Jiefei-Wang
Copy link

Jiefei-Wang commented Nov 15, 2019

I have the same issue using Python 3.8 on Windows 10. Everything is newly installed, so it is not caused by outdated software. I will try @ncoppersmith-r7 's suggestion. Please fix it...

@kevinushey
Copy link
Collaborator

It would help if you shared the error you were seeing.

@kevinushey
Copy link
Collaborator

Here's what I see with the CRAN version of reticulate:

> reticulate::use_python("~/AppData/Local/Programs/Python/Python38/python.exe", required = TRUE)
> reticulate::py_config()
Error in py_initialize(config$python, config$libpython, config$pythonhome,  : 
  Py_CompileString - The specified procedure could not be found.

And with the development version of reticulate:

> reticulate::use_python("~/AppData/Local/Programs/Python/Python38/python.exe", required = TRUE)
> reticulate::py_config()
python:         C:/Users/kevinushey/AppData/Local/Programs/Python/Python38/python.exe
libpython:      C:/Users/kevinushey/AppData/Local/Programs/Python/Python38/python38.dll
pythonhome:     C:/Users/kevinushey/AppData/Local/Programs/Python/Python38
version:        3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:37:50) [MSC v.1916 64 bit (AMD64)]
Architecture:   64bit
numpy:           [NOT FOUND]

NOTE: Python version was forced by use_python function

Assuming that you're seeing the same issue, this is fixed in the development version of reticulate. Hopefully we'll be able to push an update to CRAN by the end of the year.

@Landau1994
Copy link

try use python 3.6 instead, it works for me

@setgree
Copy link

setgree commented Jul 14, 2020

Have this issue as well. R 4.0, Python 3.7, miniconda3 recently installed, OS X Catalina
Prelim note: my .Rprofile for this project consists of:

Sys.setenv(RETICULATE_PYTHON = '~/miniconda3/bin/python')
Sys.setenv(RETICULATE_MINICONDA_PATH = '~/miniconda3/')

I'm running the following code:

library(reticulate)
library(rprojroot)
# conda_create(envname = 'r-reticulate') 
use_python(python = conda_python())
use_condaenv(condaenv = 'r-reticulate', required = TRUE)
# conda_list() # This lists two conda environments, miniconda3 (which should be called 'base'?) and r-reticulate
reticulate:::miniconda_meta_path()
[1]  "/path/to/Library/Application Support/r-reticulate/miniconda.json"
use_miniconda()
conda_version()
[1] "conda 4.8.3"
conda_python()
[1] "/path/to/miniconda3/envs/r-reticulate/bin/python"
reticulate::conda_binary()
[1] "/path/to/miniconda3/bin/conda"
reticulate::py_available() 
[1] FALSE
system('which python') # /usr/bin/python

Finally fixed this by running py_available(initialize = TRUE) (got a warning mesage about it requesting the python in the r-reticulate environment but using the python in the regular environment.

Anyway I figured it out -- maybe this is something documentation could focus on? I notice the video on the Rstudio website focuses on virtualenv but a lot of the actual reticulate functions focus on conda... thanks for all your help!

@304243504
Copy link

@setgree That worked for me, really appreciate it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants