PyCharm doesn't resolve anything under tensorflow.keras #53144

markorakita · 2021-11-20T21:13:03Z

System information

OS Platform and Distribution: Windows 10
TensorFlow installed from: binary
TensorFlow version: 2.7.0
Python version: 3.9
Installed using: Conda
CUDA/cuDNN version: 11.2/8.1

Describe the problem
I've upgraded tensorflow from 2.5 to 2.7 and now PyCharm doesn't resolve anything under tensorflow.keras.

Other modules of tensorflow work, it's only keras that's problematic. I believe this has something to do with the change in TF 2.6 where keras has been split into a separate PIP package.

People have also been reporting this problem to JetBrains (PyCharm developers): https://youtrack.jetbrains.com/issue/PY-50318

Provide the exact sequence of commands / steps that you executed before running into the problem

Create the conda environment:
~ conda create --name tensorflow27
Activate it:
~ conda activate tensorflow27
Install Python:
~ conda install python=3.9
Install TensorFlow:
~ pip install tensorflow
Create new project in PyCharm
Write this code:

import tensorflow as tf
tf.keras.preprocessing.image_dataset_from_directory()

When you move mouse to image_dataset_from_directory function there will be no autocomplete.
Writing tf.keras. yields no modules/functions

Any other info / logs
/

The text was updated successfully, but these errors were encountered:

mohantym · 2021-11-22T02:29:11Z

Hi @sanatmpa1! Could you please look at this issue?

sanatmpa1 · 2021-11-23T15:59:46Z

@markorakita,

Can you clarify if you mean that autocomplete is not working in Pycharm, or you're not even able to load the modules/functions in tf.keras?

google-ml-butler · 2021-11-30T16:30:17Z

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

markorakita · 2021-11-30T17:08:07Z

Sorry I thought I replied to this.

You can write code and run it and it works, but you don't have autocomplete nor you can go to declaration of any keras function. It's like writing code in notepad :)

cpuimage · 2021-12-03T03:12:13Z

hi, @markorakita
It's a naming conflict issue.
try modify the file site-packages/tensorflow/__init__.py near line 387.

before:

_keras_module = "keras.api._v2.keras"
keras = _LazyLoader("keras", globals(), _keras_module)
_module_dir = _module_util.get_parent_dir_for_name(_keras_module)
if _module_dir:
  _current_module.__path__ = [_module_dir] + _current_module.__path__
setattr(_current_module, "keras", keras)

after:

import typing as _typing
if _typing.TYPE_CHECKING:
  from keras.api._v2 import keras
else:
  _keras_module = "keras.api._v2.keras"
  keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", keras)

it work for me.

sanatmpa1 · 2021-12-03T05:49:33Z

@cpuimage,

Thanks for sharing your workaround.

@markorakita,

Can you try as per the suggestion by @cpuimage and let us know if it helps in resolving the auto completion problem?

markorakita · 2021-12-03T15:55:04Z

@cpuimage Thank you for finding the cause of the problem in the source code!

@sanatmpa1 I've tried the workaround and it works, but I don't think it's the complete fix since some of the keras classes are still not visible. For example tensorflow.keras.layers.CenterCrop and tensorflow.keras.layers.Rescaling layers cannot be resolved, this gets highlighted in red with the message "Cannot find reference 'CenterCrop' in __init__.py":
from tensorflow.keras.layers import CenterCrop

I hope someone in your team could take a deeper look and apply the proper fix for the next tensorflow update?

markorakita · 2021-12-03T21:38:17Z

@cpuimage Thanks, they can be found in that path but that is the old path, in TF 2.7 they were moved from experimental to tensorflow.keras.layers (see API docs) and those new paths should work in IDE. Maybe the same bug that is in site-packages/tensorflow/__init__.py is also in site-packages/tensorflow/keras/__init__.py

These are just the classes that I noticed at a first glance that cannot be resolved, but there might be more, somebody with domain knowledge should really look into this and devise full and proper fix.

cpuimage · 2021-12-04T00:52:10Z

@markorakita Everything is stirred up, for the time being, only part of it can be solved first, and I hope that someone will solve the problem as soon as possible.

birjinia · 2022-08-18T10:23:50Z

hi, @markorakita It's a naming conflict issue. try modify the file site-packages/tensorflow/__init__.py near line 387.

before:

_keras_module = "keras.api._v2.keras"
keras = _LazyLoader("keras", globals(), _keras_module)
_module_dir = _module_util.get_parent_dir_for_name(_keras_module)
if _module_dir:
  _current_module.__path__ = [_module_dir] + _current_module.__path__
setattr(_current_module, "keras", keras)

after:

import typing as _typing
if _typing.TYPE_CHECKING:
  from keras.api._v2 import keras
else:
  _keras_module = "keras.api._v2.keras"
  keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", keras)

it work for me.

This solution worked for me. Thanks for sharing it.

ashwin8121 · 2023-05-22T11:39:25Z

Just copy the folder keras from site-packages/tensorflow/python to site-packages/tensorflow. And the auto completion will start working perfectly

King-Of-Knights · 2023-06-12T09:22:27Z

Just copy the folder keras from site-packages/tensorflow/python to site-packages/tensorflow. And the auto completion will start working perfectly

Thanks! It is a simple and prefect way!

xiaolv7 · 2023-06-13T03:21:41Z

Just copy the folder keras from site-packages/tensorflow/python to site-packages/tensorflow. And the auto completion will start working perfectly

Thank you for your idea! However, it only works for some. It looks like site-packages/tensorflow/python/keras is different from site-packages/keras.

After applying your method, I have the following results:

set_random_seed is not resolved when calling from the copied folder

romerojhh · 2023-07-13T05:43:22Z

To everyone that's still couldn't make it work. Try this, I just tweaked the popular solution on this thread:

modify the file site-packages/tensorflow/__init__.py near line 387
From:

_keras_module = "keras.api._v2.keras"
keras = _LazyLoader("keras", globals(), _keras_module)
_module_dir = _module_util.get_parent_dir_for_name(_keras_module)
if _module_dir:
  _current_module.__path__ = [_module_dir] + _current_module.__path__
setattr(_current_module, "keras", keras)

To:

Note that I used _keras instead of keras

import typing as _typing
if _typing.TYPE_CHECKING:
  from keras.api._v2 import keras
else:
  _keras_module = "keras.api._v2.keras"
  _keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", _keras)

And then make sure to copy folder
/site-packages/keras/ into /site-packages/tensorflow/

After all of that, invalidate and restart your IDE

This works on my setup using Conda

dhruv1710 · 2023-07-17T06:18:19Z

It worked on my anaconda also! . Thanks for all your help code completion and that mini documentation is very important for productivity.

ChienKangLu · 2023-07-28T01:28:14Z

To everyone that's still couldn't make it work. Try this, I just tweaked the popular solution on this thread:

modify the file site-packages/tensorflow/__init__.py near line 387 From:
_keras_module = "keras.api._v2.keras"
keras = _LazyLoader("keras", globals(), _keras_module)
_module_dir = _module_util.get_parent_dir_for_name(_keras_module)
if _module_dir:
  _current_module.__path__ = [_module_dir] + _current_module.__path__
setattr(_current_module, "keras", keras)
To:

Note that I used _keras instead of keras
import typing as _typing
if _typing.TYPE_CHECKING:
  from keras.api._v2 import keras
else:
  _keras_module = "keras.api._v2.keras"
  _keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", _keras)
And then make sure to copy folder /site-packages/keras/ into /site-packages/tensorflow/

After all of that, invalidate and restart your IDE

This works on my setup using Conda

It worked perfectly on PyCharm IDE with TensorFlow 2.13.0! Finally from tensorflow.keras.layers import Dense, Flatten, Conv2D is successfully recognized by IDE.

With a small change, instead of copy folder /site-packages/keras/ into /site-packages/tensorflow/, I use symbolic link ln -s <full-path-to-site-package>/site-packages/keras <full-path-to-site-package>/tensorflow/keras

chottuthejimmy · 2024-05-12T12:32:15Z

To everyone that's still couldn't make it work. Try this, I just tweaked the popular solution on this thread:

modify the file site-packages/tensorflow/__init__.py near line 387 From:
_keras_module = "keras.api._v2.keras"
keras = _LazyLoader("keras", globals(), _keras_module)
_module_dir = _module_util.get_parent_dir_for_name(_keras_module)
if _module_dir:
  _current_module.__path__ = [_module_dir] + _current_module.__path__
setattr(_current_module, "keras", keras)
To:

Note that I used _keras instead of keras
import typing as _typing
if _typing.TYPE_CHECKING:
  from keras.api._v2 import keras
else:
  _keras_module = "keras.api._v2.keras"
  _keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", _keras)
And then make sure to copy folder /site-packages/keras/ into /site-packages/tensorflow/

After all of that, invalidate and restart your IDE

This works on my setup using Conda

I am facing this super weird issue.

I am on version 2.16.1 First of my site-packages/tensorflow/__init__.py does not contain any such method.

And this is how the file looks like near line 387.

I still copied folder /site-packages/keras/ into /site-packages/tensorflow/ . But still no luck

Very interesting thing is that I can access the definition of __version__ in VS Code insiders but not on PyCharm.

At the same time I can go to Dense via from tensorflow.keras.layers import Dense on PyCharm, but not on VS Code insiders, also I am not able to access Input which also has the same parent package of Dense.

And ya, both of them are running on the same global interpreter 3.12.0

One thing I figured out was PyCharm was saving these packages at its own end.

Whereas, VS Code insiders was doing it from the /site-packages/tensorflow/keras. I was able to access tensorflow.keras after copying the folder, but not tensorflow.keras.layers.

HeisenbergKnocks · 2024-06-05T05:29:17Z

To everyone that's still couldn't make it work. Try this, I just tweaked the popular solution on this thread:
modify the file site-packages/tensorflow/__init__.py near line 387 From:
_keras_module = "keras.api._v2.keras"
keras = _LazyLoader("keras", globals(), _keras_module)
_module_dir = _module_util.get_parent_dir_for_name(_keras_module)
if _module_dir:
  _current_module.__path__ = [_module_dir] + _current_module.__path__
setattr(_current_module, "keras", keras)
To:
Note that I used _keras instead of keras
import typing as _typing
if _typing.TYPE_CHECKING:
  from keras.api._v2 import keras
else:
  _keras_module = "keras.api._v2.keras"
  _keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", _keras)
And then make sure to copy folder /site-packages/keras/ into /site-packages/tensorflow/
After all of that, invalidate and restart your IDE
This works on my setup using Conda
I am facing this super weird issue.

I am on version 2.16.1 First of my site-packages/tensorflow/__init__.py does not contain any such method. And this is how the file looks like near line 387.

I still copied folder /site-packages/keras/ into /site-packages/tensorflow/ . But still no luck

Very interesting thing is that I can access the definition of __version__ in VS Code insiders but not on PyCharm.

At the same time I can go to Dense via from tensorflow.keras.layers import Dense on PyCharm, but not on VS Code insiders, also I am not able to access Input which also has the same parent package of Dense.

And ya, both of them are running on the same global interpreter 3.12.0

One thing I figured out was PyCharm was saving these packages at its own end. Whereas, VS Code insiders was doing it from the /site-packages/tensorflow/keras. I was able to access tensorflow.keras after copying the folder, but not tensorflow.keras.layers.

I have the same problem. It looks like they've changed the code so the old workarounds are no longer relevant. Has anybody found a fix for the newest versions of Tensorflow?

aimfeld · 2024-06-07T10:45:26Z

I have the same problem. It looks like they've changed the code so the old workarounds are no longer relevant. Has anybody found a fix for the newest versions of Tensorflow?

Same issue here with tf 2.16.1

jbed94 · 2024-06-07T10:49:37Z

So far I haven't seen anyone propose this solution, so I'm passing it on. If PyCharm does not suggest the code to you, but the code itself executes (i.e. if you run the code, e.g. tf.keras.layers.LayerNormalization, which the IDE does not suggest, executes correctly), you can install the package:

pip install types-tensorflow

which contains "stubs" compatible with the corresponding versions of TensorFlow. If you use TensorFlow 2.16.x, you can install, for example, types-tensorflow==2.16.0.20240606 (see which version of TensorFlow these definitions are for).

pip install types-tensorflow==2.16.0.20240606

PyCharm, after refreshing the indexes, should correctly suggest classes contained in TensorFlow, unfortunately it will not suggest documentation, but it always makes writing code easier.

aimfeld · 2024-06-07T13:55:24Z

So far I haven't seen anyone propose this solution, so I'm passing it on. If PyCharm does not suggest the code to you, but the code itself executes (i.e. if you run the code, e.g. tf.keras.layers.LayerNormalization, which the IDE does not suggest, executes correctly), you can install the package:

pip install types-tensorflow

which contains "stubs" compatible with the corresponding versions of TensorFlow. If you use TensorFlow 2.16.x, you can install, for example, types-tensorflow==2.16.0.20240606 (see which version of TensorFlow these definitions are for).

pip install types-tensorflow==2.16.0.20240606

PyCharm, after refreshing the indexes, should correctly suggest classes contained in TensorFlow, unfortunately it will not suggest documentation, but it always makes writing code easier.

The stubs are nice, but documentation is what I need most. I decided to downgrade to 2.12.1 to get documentation working again. Now code is linked correctly and documentation works, but GPU support is broken, because I use Cuda 12.3 which requires a newer tensorflow version.

steveepreston · 2024-08-28T13:34:13Z

this issue still exist in PyCharm 2024.2 with TensorFlow 2.16.1

sana33 · 2024-09-15T20:27:00Z

hi, @markorakita It's a naming conflict issue. try modify the file site-packages/tensorflow/__init__.py near line 387.

before:

_keras_module = "keras.api._v2.keras"
keras = _LazyLoader("keras", globals(), _keras_module)
_module_dir = _module_util.get_parent_dir_for_name(_keras_module)
if _module_dir:
  _current_module.__path__ = [_module_dir] + _current_module.__path__
setattr(_current_module, "keras", keras)

after:

import typing as _typing
if _typing.TYPE_CHECKING:
  from keras.api._v2 import keras
else:
  _keras_module = "keras.api._v2.keras"
  keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", keras)

it work for me.

I tried this wih Python 3.9 and tensorflow 2.11, though nothing changed!?

jithinvijayan · 2024-10-20T07:57:17Z

Just copy the folder keras from site-packages/tensorflow/python to site-packages/tensorflow. And the auto completion will start working perfectly

This worked for me as well..

steveepreston · 2024-10-20T09:40:17Z

This worked for me as well

Works but that is a dirty way, and I think it need to be repeated after each update
Should fixed on source side

steveepreston · 2024-10-20T09:42:37Z

Since this issue is a duplicate as #56231, I am going to close this one, so that everyone can track on same issue.

@qlzh727 Both issues are closed and problem is not fixed yet
Please re-open this

qlzh727 · 2024-10-20T16:49:12Z

Sorry for the very late reply. Both me and haifeng has moved to some new project for sometime. Let me cc @mattdangerw and @hertschuh here.

steveepreston · 2024-10-20T20:48:03Z

@qlzh727 Thank you!

simond07 · 2024-11-19T15:36:09Z

Same problem in 2.18.0.

hertschuh · 2024-12-06T20:00:39Z

Hi,

as of Tensorflow 2.16, the preferred way of using keras is to import keras and use it directly:

import keras

inputs = keras.Input(...)

tf.keras is only provided for backwards compatibility.

My experience is that code completion for keras works in PyCharm with import keras.

negsi · 2025-01-25T16:26:12Z

My experience is that code completion for keras works in PyCharm with import keras.

Working like a charm.

Also I am using this:

from keras.api.datasets import mnist
from keras.api.models import Sequential
from keras.api.layers import Dense, Flatten
from keras.api.utils import to_categorical

gabemorris12 · 2025-03-07T20:54:48Z

Hi,

as of Tensorflow 2.16, the preferred way of using keras is to import keras and use it directly:

import keras

inputs = keras.Input(...)
tf.keras is only provided for backwards compatibility.

My experience is that code completion for keras works in PyCharm with import keras.

I don't like having to resolve to importing directly from keras. Doing from tensorflow.keras.models import Sequential is much cleaner as it enforces the use of the tensorflow backend. Is there still no fix for this in March of 2025?

hertschuh · 2025-03-07T22:00:14Z

@gabemorris12

Doing from tensorflow.keras.models import Sequential is much cleaner as it enforces the use of the tensorflow backend.

The use of tensorflow.keras vs. import keras does not enforce the use of the TensorFlow backend.

import os
os.environ["KERAS_BACKEND"] = "jax"

import tensorflow as tf

type(tf.keras.ops.ones(()))

jaxlib.xla_extension.ArrayImpl

kpilkk · 2025-03-29T10:37:17Z

Has any solution worked? I'm using tensorflow version 2.17.0.

markorakita added the type:build/install label Nov 20, 2021

google-ml-butler bot assigned mohantym Nov 20, 2021

mohantym added TF 2.7 comp:keras type:bug subtype:windows and removed comp:keras type:bug labels Nov 21, 2021

mohantym assigned sanatmpa1 and unassigned mohantym Nov 22, 2021

sanatmpa1 added the stat:awaiting response label Nov 23, 2021

google-ml-butler bot added the stale label Nov 30, 2021

google-ml-butler bot removed stale stat:awaiting response labels Nov 30, 2021

markorakita mentioned this issue Dec 1, 2021

Code completion of Keras Module no longer work since 2.6 #53271

Closed

sanatmpa1 added stat:awaiting response comp:keras type:bug and removed type:build/install subtype:windows labels Dec 3, 2021

tensorflowbutler removed the stat:awaiting response label Dec 6, 2021

potentend mentioned this issue Jun 20, 2024

tensorflow/keras not found Nuitka/Nuitka#2941

Closed

qlzh727 assigned hertschuh and mattdangerw and unassigned qlzh727 and haifeng-jin Oct 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PyCharm doesn't resolve anything under tensorflow.keras #53144

PyCharm doesn't resolve anything under tensorflow.keras #53144

markorakita commented Nov 20, 2021

mohantym commented Nov 22, 2021

sanatmpa1 commented Nov 23, 2021

google-ml-butler bot commented Nov 30, 2021

markorakita commented Nov 30, 2021

cpuimage commented Dec 3, 2021 •

edited

Loading

sanatmpa1 commented Dec 3, 2021

markorakita commented Dec 3, 2021

markorakita commented Dec 3, 2021

cpuimage commented Dec 4, 2021

birjinia commented Aug 18, 2022

ashwin8121 commented May 22, 2023

King-Of-Knights commented Jun 12, 2023

xiaolv7 commented Jun 13, 2023

romerojhh commented Jul 13, 2023 •

edited

Loading

dhruv1710 commented Jul 17, 2023

ChienKangLu commented Jul 28, 2023 •

edited

Loading

chottuthejimmy commented May 12, 2024

HeisenbergKnocks commented Jun 5, 2024 •

edited

Loading

aimfeld commented Jun 7, 2024 •

edited

Loading

jbed94 commented Jun 7, 2024

aimfeld commented Jun 7, 2024

steveepreston commented Aug 28, 2024 •

edited

Loading

sana33 commented Sep 15, 2024

jithinvijayan commented Oct 20, 2024

steveepreston commented Oct 20, 2024

steveepreston commented Oct 20, 2024 •

edited

Loading

qlzh727 commented Oct 20, 2024

steveepreston commented Oct 20, 2024

simond07 commented Nov 19, 2024

hertschuh commented Dec 6, 2024

negsi commented Jan 25, 2025 •

edited

Loading

gabemorris12 commented Mar 7, 2025

hertschuh commented Mar 7, 2025

kpilkk commented Mar 29, 2025

PyCharm doesn't resolve anything under tensorflow.keras #53144

PyCharm doesn't resolve anything under tensorflow.keras #53144

Comments

markorakita commented Nov 20, 2021

mohantym commented Nov 22, 2021

sanatmpa1 commented Nov 23, 2021

google-ml-butler bot commented Nov 30, 2021

markorakita commented Nov 30, 2021

cpuimage commented Dec 3, 2021 • edited Loading

sanatmpa1 commented Dec 3, 2021

markorakita commented Dec 3, 2021

markorakita commented Dec 3, 2021

cpuimage commented Dec 4, 2021

birjinia commented Aug 18, 2022

ashwin8121 commented May 22, 2023

King-Of-Knights commented Jun 12, 2023

xiaolv7 commented Jun 13, 2023

romerojhh commented Jul 13, 2023 • edited Loading

dhruv1710 commented Jul 17, 2023

ChienKangLu commented Jul 28, 2023 • edited Loading

chottuthejimmy commented May 12, 2024

HeisenbergKnocks commented Jun 5, 2024 • edited Loading

aimfeld commented Jun 7, 2024 • edited Loading

jbed94 commented Jun 7, 2024

aimfeld commented Jun 7, 2024

steveepreston commented Aug 28, 2024 • edited Loading

sana33 commented Sep 15, 2024

jithinvijayan commented Oct 20, 2024

steveepreston commented Oct 20, 2024

steveepreston commented Oct 20, 2024 • edited Loading

qlzh727 commented Oct 20, 2024

steveepreston commented Oct 20, 2024

simond07 commented Nov 19, 2024

hertschuh commented Dec 6, 2024

negsi commented Jan 25, 2025 • edited Loading

gabemorris12 commented Mar 7, 2025

hertschuh commented Mar 7, 2025

kpilkk commented Mar 29, 2025

cpuimage commented Dec 3, 2021 •

edited

Loading

romerojhh commented Jul 13, 2023 •

edited

Loading

ChienKangLu commented Jul 28, 2023 •

edited

Loading

HeisenbergKnocks commented Jun 5, 2024 •

edited

Loading

aimfeld commented Jun 7, 2024 •

edited

Loading

steveepreston commented Aug 28, 2024 •

edited

Loading

steveepreston commented Oct 20, 2024 •

edited

Loading

negsi commented Jan 25, 2025 •

edited

Loading