Skip to content

'spacy' is a package and cannot be directly executed #2514

Closed
@DSamuylov

Description

@DSamuylov

How to reproduce the behaviour

As described in the documentation, I run the following command to install spaCy:

conda install -c conda-forge spacy

However, if I call:

python -m spacy download en

It fails with the following error message:

/opt/conda/bin/python: No module named spacy.__main__; 'spacy' is a package and cannot be directly executed

Your Environment

  • Operating System: Ubuntu 18.04
  • Python Version Used: Python 3.6 (Anaconda distribution)
  • spaCy Version Used:
  • Environment Information: I also tried to install it in inside Docker image (FROM continuumio/anaconda3), and it fails. A few weeks ago, it worked without any problem.

Activity

added
installInstallation issues
condaconda package manager
on Jul 5, 2018
ines

ines commented on Jul 5, 2018

@ines
Member

Thanks for the report. I'm pretty sure this is related to #2462 – for some reason in some situations, users who install via conda end up with an old version of spaCy. Like, a really really old one, the one that the official Anaconda distribution used to ship and that didn't have CLI commands yet.

If you still have your install logs, could you check if there's some hint at that? Maybe conda didn't actually install the latest version from conda-forge, because your local Anaconda distribution already had spaCy included (but an old version)? And does conda update -c conda-forge spacy make a difference?

drussellmrichie

drussellmrichie commented on Jul 12, 2018

@drussellmrichie

@DSamuylov, not sure if you solved this, but the solution @BBCMax alluded to in his second comment of the thread @ines refers to worked for me. I'm also on Ubuntu 18.04 and py3.6 via anaconda. Although now I'm having this problem, though that seems solvable. :-)

ines

ines commented on Jul 12, 2018

@ines
Member

@drussellmrichie Do you still have your install logs and could you check if there's anything in them that looks like an old version of spaCy was used? Like, which command did you run and what exactly was conda doing there that resulted in you ending up with an old version?

I'd love to get to the bottom of this and see if there's anything we can do to prevent this. This is such a weird problem and unfortunately, it's mostly been hitting new users which is extra frustrating.

Btw, about the download timeout problem: If you want full control over how it's downloaded, I'd suggest finding the direct link to the archive file via the model releases. The model details also list the file size, so you can decide how you want to download it and what works best for you.

drussellmrichie

drussellmrichie commented on Jul 12, 2018

@drussellmrichie

@ines, happy to help if I can. Not sure if I have install logs (how would I find those?), but I just did history | grep conda at the command line, and it seems I did the following:

   20  bash Anaconda3-5.2.0-Linux-x86_64.sh 
   27  conda list
   38  anaconda
   43  export PATH=~/anaconda3/bin:$PATH
   44  conda
   45  conda list
   48  conda --version
   55  conda
   62  bash Anaconda3-5.2.0-Linux-x86_64.sh 
   86  conda list
   87  conda install spacy

I think that last line must have been how I got the old version of spacy on my machine (which I only got a few weeks ago).

ines

ines commented on Jul 13, 2018

@ines
Member

@drussellmrichie Thanks so much! (And not sure if conda saves the logs – I was mostly thinking about that detailed table it prints that tells you what it's going to install and update, but that might only be available in the scrollback.)

So in this case, we can rule out an old Anaconda distribution then. My other theory was that maybe you have some other package installed (maybe a dependency of something else) that conflicts with one of spaCy's dependencies. So instead of updating all of the other dependencies, conda then just serves you an old version of spaCy that's compatible with the constraints.

Do you remember which packages you installed explicitly since getting the new machine?

drussellmrichie

drussellmrichie commented on Jul 13, 2018

@drussellmrichie

Through some more greping through my terminal history, it seems that, after installing spacy, I installed tqdm and gensim through conda, and textblob-de and msgpack through pip.

ines

ines commented on Jul 15, 2018

@ines
Member

@drussellmrichie Thanks! (And sorry for asking you all these random questions, but I really want to get to the bottom of this issue and I feel like we're very close here.)

Do you remember if you installed anything before you installed spaCy?

drussellmrichie

drussellmrichie commented on Jul 15, 2018

@drussellmrichie

No worries about the questions. This is the absolute least I can do after using spaCy for free. :-)

Here's what I installed through the command-line (history | grep install) before spaCy:

    3  sudo apt install nvidia-381
    6  sudo ubuntu-drivers autoinstall
   12  sudo ubuntu-drivers autoinstall
   15  sudo ubuntu-drivers autoinstall
   29  sup apt install vim
   30  sudo apt install vim
   36  sudo apt install google-chrome-stable
   41  sudo apt install gnome-usage
   59  sudo apt-get install fluxgui

I also installed pycharm, atom, dropbox, and skype, I think through some installers (not through commandline), but not sure if I did those before or after installing spacy.

ines

ines commented on Jul 17, 2018

@ines
Member

@drussellmrichie Thanks! There's nothing here that looks problematic. Basically, I'm trying to figure out if there could have been something in your Anaconda environment before you installed spaCy that caused conda to download and install an old version of spaCy (e.g. to ensure compatibility).

A similar problem also just came up in #2542. The user didn't have access to the install log anymore, but shared the update log. Copying it over here for further investigation:

Solving environment: done

## Package Plan ##

  environment location: /root/anaconda3

  added / updated specs: 
    - spacy==2.0.11


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cytoolz-0.8.2              |           py36_0         1.1 MB  conda-forge
    preshed-1.0.0              |   py36hfc679d8_0         226 KB  conda-forge
    termcolor-1.1.0            |             py_2           6 KB  conda-forge
    thinc-6.10.1               |   py36hd61447b_0         1.5 MB
    pathlib-1.0.1              |             py_1          15 KB  conda-forge
    libgcc-7.2.0               |       h69d50b8_2         304 KB
    regex-2017.11.09           |           py36_0         670 KB  conda-forge
    ca-certificates-2018.4.16  |                0         139 KB  conda-forge
    openssl-1.0.2o             |                0         3.5 MB  conda-forge
    murmurhash-0.28.0          |   py36hfc679d8_0          39 KB  conda-forge
    tqdm-4.23.4                |             py_0          35 KB  conda-forge
    msgpack-numpy-0.4.3        |             py_0          10 KB  conda-forge
    spacy-2.0.11               |           py36_0        41.9 MB  conda-forge
    dill-0.2.8.2               |           py36_0         109 KB  conda-forge
    ------------------------------------------------------------
                                           Total:        49.6 MB

The following NEW packages will be INSTALLED:

    dill:            0.2.8.2-py36_0         conda-forge
    libgcc:          7.2.0-h69d50b8_2                  
    msgpack-numpy:   0.4.3-py_0             conda-forge
    pathlib:         1.0.1-py_1             conda-forge
    regex:           2017.11.09-py36_0      conda-forge
    termcolor:       1.1.0-py_2             conda-forge
    tqdm:            4.23.4-py_0            conda-forge

The following packages will be REMOVED:

    anaconda:        5.2.0-py36_3                      

The following packages will be UPDATED:

    ca-certificates: 2018.03.07-0                       --> 2018.4.16-0           conda-forge
    murmurhash:      0.26.4-py36_0          conda-forge --> 0.28.0-py36hfc679d8_0 conda-forge
    openssl:         1.0.2o-h20670df_0                  --> 1.0.2o-0              conda-forge
    preshed:         0.46.4-py36_0          conda-forge --> 1.0.0-py36hfc679d8_0  conda-forge
    spacy:           0.101.0-py36_0                     --> 2.0.11-py36_0         conda-forge
    thinc:           5.0.8-py36_0                       --> 6.10.1-py36hd61447b_0            

The following packages will be DOWNGRADED:

    cytoolz:         0.9.0.1-py36h14c3975_0             --> 0.8.2-py36_0          conda-forge

Proceed ([y]/n)?
drussellmrichie

drussellmrichie commented on Jul 17, 2018

@drussellmrichie

Unfortunately, I don't think I have a way to look at the output from when I ran the update days ago. =( But my output must have looked similar to that.

Let me know if there's anything else I can do.

10 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    condaconda package managerinstallInstallation issues

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @dsp-de@blkfin@DSamuylov@drussellmrichie@ines

        Issue actions

          'spacy' is a package and cannot be directly executed · Issue #2514 · explosion/spaCy