Closed
Description
As Django docs (https://docs.djangoproject.com/en/1.7/ref/applications/#namespace-packages-as-apps-python-3-3) said, Django supports a _NamespacePath app.
But when I used _NamespacePath with django-celery, I got some errors:
/Users/dj/.virtualenvs/3.4.3/lib/python3.4/site-packages/djcelery/loaders.py in find_related_module(app, related_name)
197
198 try:
--> 199 imp.find_module(related_name, app_path)
200 except ImportError:
201 return
/Users/dj/.virtualenvs/3.4.3/lib/python3.4/imp.py in find_module(name, path)
269 # Backwards-compatibility
270 raise RuntimeError("'list' must be None or a list, "
--> 271 "not {}".format(type(name)))
272
273 if path is None:
RuntimeError: 'list' must be None or a list, not <class 'str'>
I know celery itself also not support it, will support ?
Thanks
Activity
kmollee commentedon Mar 13, 2015
This error will occur when use
autodiscover_task
in celery and one of these app's folder did't have__init__.py
filepretty hard to find out what is going on.. debug information is lack
aztrock commentedon Jan 9, 2016
Traceback (most recent call last):
File "/opt/kibox/bin/celery", line 11, in
sys.exit(main())
File "/opt/kibox/local/lib/python3.4/site-packages/celery/main.py", line 30, in main
main()
File "/opt/kibox/local/lib/python3.4/site-packages/celery/bin/celery.py", line 81, in main
cmd.execute_from_commandline(argv)
File "/opt/kibox/local/lib/python3.4/site-packages/celery/bin/celery.py", line 770, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/opt/kibox/local/lib/python3.4/site-packages/celery/bin/base.py", line 311, in execute_from_commandline
return self.handle_argv(self.prog_name, argv[1:])
File "/opt/kibox/local/lib/python3.4/site-packages/celery/bin/celery.py", line 762, in handle_argv
return self.execute(command, argv)
File "/opt/kibox/local/lib/python3.4/site-packages/celery/bin/celery.py", line 694, in execute
).run_from_argv(self.prog_name, argv[1:], command=argv[0])
File "/opt/kibox/local/lib/python3.4/site-packages/celery/bin/worker.py", line 179, in run_from_argv
return self(_args, *_options)
File "/opt/kibox/local/lib/python3.4/site-packages/celery/bin/base.py", line 274, in call
ret = self.run(_args, *_kwargs)
File "/opt/kibox/local/lib/python3.4/site-packages/celery/bin/worker.py", line 213, in run
).start()
File "/opt/kibox/local/lib/python3.4/site-packages/celery/worker/init.py", line 95, in init
self.app.loader.init_worker()
File "/opt/kibox/local/lib/python3.4/site-packages/celery/loaders/base.py", line 128, in init_worker
self.import_default_modules()
File "/opt/kibox/local/lib/python3.4/site-packages/celery/loaders/base.py", line 116, in import_default_modules
signals.import_modules.send(sender=self.app)
File "/opt/kibox/local/lib/python3.4/site-packages/celery/utils/dispatch/signal.py", line 166, in send
response = receiver(signal=self, sender=sender, *_named)
File "/opt/kibox/local/lib/python3.4/site-packages/amqp/utils.py", line 42, in call
self.set_error_state(exc)
File "/opt/kibox/local/lib/python3.4/site-packages/amqp/utils.py", line 39, in call
*_dict(self.kwargs, **kwargs) if self.kwargs else kwargs
File "/opt/kibox/local/lib/python3.4/site-packages/celery/app/base.py", line 330, in _autodiscover_tasks
self.loader.autodiscover_tasks(packages, related_name)
File "/opt/kibox/local/lib/python3.4/site-packages/celery/loaders/base.py", line 251, in autodiscover_tasks
related_name) if mod)
File "/opt/kibox/local/lib/python3.4/site-packages/celery/loaders/base.py", line 272, in autodiscover_tasks
return [find_related_module(pkg, related_name) for pkg in packages]
File "/opt/kibox/local/lib/python3.4/site-packages/celery/loaders/base.py", line 272, in
return [find_related_module(pkg, related_name) for pkg in packages]
File "/opt/kibox/local/lib/python3.4/site-packages/celery/loaders/base.py", line 294, in find_related_module
_imp.find_module(related_name, pkg_path)
File "/opt/kibox/lib/python3.4/imp.py", line 271, in find_module
"not {}".format(type(name)))
RuntimeError: 'list' must be None or a list, not <class 'str'>
the funny thing is that it was working fine, just kill the process kill -9 celery, as I have cast several times, but this time it stopped working
It is a common mistake, it is almost impossible to find, there is no solution?
aztrock commentedon Jan 9, 2016
a little help, know where to look.
vim /opt/kibox/lib/python3.4/imp.py
line: 271
Backwards-compatibilit
raise RuntimeError("'list' must be None or a list, "
"not {}".format(type(name)))
to
Backwards-compatibilit
print (path, list)
raise RuntimeError("'list' must be None or a list, "
"not {}".format(type(name)))
you will see something like
_NamespacePath(['/opt/kibox/kibox/apps/apprest']) <class 'list'>
rm /opt/kibox/kibox/apps/apprest/init.py
rm /opt/kibox/kibox/apps/apprest/init.pyc
touch /opt/kibox/kibox/apps/apprest/init.py
ask commentedon Jan 12, 2016
The development version (celery master branch) uses
app,autodiscover_tasks()
with an empty list instead of specifying INSTALLED_APPS, and the Django support populates it with apps.get_apps() instead, which may resolve the issue?aztrock commentedon May 24, 2016
app.autodiscover_tasks (), if full app.autodiscover_tasks manually ([ "apps.example", "apps.example2"]), works perfect, app.autodiscover_tasks (settings.INSTALLED_APPS), also works well, app.autodiscover_tasks (lambda : settings.INSTALLED_APPS), works well,
The error detecting achieve, is that if one does not have my files, get the error.
apps / examples / *without py files .py
fails to get more information, lack of time and so far we could migrate to django 1.9
rbreu commentedon Dec 15, 2016
What's the status of this?
I am having the same problem with celery 4.0.1 and Django 1.10.1.
The issue is that Django allows namespace packages (see link in the OP), i.e. packages without
__init__.py
file, but celery's autodiscover_tasks fails on those packages with the error above.I've tracked it down to the call of _imp.find_module here. imp.find_module is deprecated since Python 3.3.
I'm not sure what find_module is supposed to be doing that importlib.import_module can't do? Because when I replace the last bit of that code, starting from line 269, with
the loading seems to work fine for me. But as I said, I don't know what imp.find_module is supposed to be covering here (or why there's an import_module after that without catching ImportError), so I might be overlooking something.
rixx commentedon Aug 18, 2017
I don't think you are overlooking anything here, @rbreu, so I'm creating a PR to fix this. And if the PR is rejected, at least we will have an explanation.
Fix modules missing __init.py__ files
Use huey queue (#496)