Skip to content

New version of redis package is the cause of the Error #1868

New issue
New issue
Closed
Closed
New version of redis package is the cause of the Error#1868
Labels
bugneeds further review
@egregors

Description

@egregors
egregors
opened on Nov 18, 2018
Contributor

What happened?

After redeploy, we got redis==3.0.1 instead redis==2.10.6. So, when you try to work with cache for example, we got the exception:

DataError: Invalid input of type: 'CacheKey'. Convert to a byte, string or number first.
  File "django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "django/core/handlers/base.py", line 128, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "django/core/handlers/base.py", line 126, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "python3.6/contextlib.py", line 52, in inner
    return func(*args, **kwds)
  File "django/utils/decorators.py", line 134, in _wrapped_view
    result = middleware.process_request(request)
  File "django/middleware/cache.py", line 133, in process_request
    cache_key = get_cache_key(request, self.key_prefix, 'GET', cache=self.cache)
  File "django/utils/cache.py", line 360, in get_cache_key
    headerlist = cache.get(cache_key)
  File "django_redis/cache.py", line 32, in _decorator
    return method(self, *args, **kwargs)
  File "django_redis/cache.py", line 81, in get
    client=client)
  File "django_redis/client/default.py", line 202, in get
    value = client.get(key)
  File "redis/client.py", line 1207, in get
    return self.execute_command('GET', name)
  File "redis/client.py", line 754, in execute_command
    connection.send_command(*args)
  File "redis/connection.py", line 619, in send_command
    self.send_packed_command(self.pack_command(*args))
  File "redis/connection.py", line 659, in pack_command
    for arg in imap(self.encoder.encode, args):
  File "redis/connection.py", line 124, in encode
    "byte, string or number first." % typename)

The problem exactly in the redis==3.0.1, as we may see there: https://stackoverflow.com/questions/53331405/django-compress-error-invalid-input-of-type-cachekey

It is because of this pkg sets in the requiriments/base.txt as redis>=2.10.5 # https://github.com/antirez/redis

What should've happened instead?

All should work well

Steps to reproduce

Just rebuild compose.

(Ubuntu 18.04, Docker 17.05.0-ce)

Activity

egregors
mentioned this on Nov 18, 2018
egregors

egregors commented on Nov 18, 2018

@egregors
egregors
on Nov 18, 2018
ContributorAuthor

This is all about this issue: jazzband/django-redis#342

browniebroke

browniebroke commented on Nov 19, 2018

@browniebroke
browniebroke
on Nov 19, 2018
Member

Hi, thanks for the report. I believe it was fixed upstream in django-redis. I've just merged #1870 which should solve this.

browniebroke
added
needs further review
bug
on Nov 19, 2018
egregors

egregors commented on Nov 20, 2018

@egregors
egregors
on Nov 20, 2018
ContributorAuthor

@browniebroke Unfortunately, I still get Error, even with django-redis==4.10.0 and redis==3

Do you still think pinning redis to 2.10.4, <3 is a bad idea? At least as a temporary solution.

celeryworker_1  | [2018-11-20 13:30:07,838: WARNING/MainProcess] celery@cd4e42f8b190 ready.
celeryworker_1  | [2018-11-20 13:30:07,927: ERROR/MainProcess] Unrecoverable error: AttributeError("'str' object has no attribute 'items'",)
celeryworker_1  | Traceback (most recent call last):
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/worker/__init__.py", line 206, in start
celeryworker_1  |     self.blueprint.start(self)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 123, in start
celeryworker_1  |     step.start(parent)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 374, in start
celeryworker_1  |     return self.obj.start()
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer.py", line 280, in start
celeryworker_1  |     blueprint.start(self)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 123, in start
celeryworker_1  |     step.start(parent)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer.py", line 884, in start
celeryworker_1  |     c.loop(*c.loop_args())
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/worker/loops.py", line 76, in asynloop
celeryworker_1  |     next(loop)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/kombu/async/hub.py", line 340, in create_loop
celeryworker_1  |     cb(*cbargs)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py", line 1019, in on_readable
celeryworker_1  |     self._callbacks[queue](message)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/__init__.py", line 534, in _callback
celeryworker_1  |     self.qos.append(message, message.delivery_tag)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py", line 146, in append
celeryworker_1  |     pipe.zadd(self.unacked_index_key, delivery_tag, time()) \
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2263, in zadd
celeryworker_1  |     for pair in iteritems(mapping):
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/redis/_compat.py", line 123, in iteritems
celeryworker_1  |     return iter(x.items())
celeryworker_1  | AttributeError: 'str' object has no attribute 'items'
browniebroke

browniebroke commented on Nov 20, 2018

@browniebroke
browniebroke
on Nov 20, 2018
Member

Well, the error is different. Technically, it is now coming from Celery/Kombu, not from django-celery. They did a similar fix: celery/kombu#947 but it's not yet released.

browniebroke
closed this as completedin #1869on Nov 20, 2018
rolep
mentioned this on Mar 12, 2019
angerrp

angerrp commented on Feb 1, 2021

@angerrp
angerrp
on Feb 1, 2021

@browniebroke Unfortunately, I still get Error, even with django-redis==4.10.0 and redis==3

Do you still think pinning redis to 2.10.4, <3 is a bad idea? At least as a temporary solution.

celeryworker_1  | [2018-11-20 13:30:07,838: WARNING/MainProcess] celery@cd4e42f8b190 ready.
celeryworker_1  | [2018-11-20 13:30:07,927: ERROR/MainProcess] Unrecoverable error: AttributeError("'str' object has no attribute 'items'",)
celeryworker_1  | Traceback (most recent call last):
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/worker/__init__.py", line 206, in start
celeryworker_1  |     self.blueprint.start(self)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 123, in start
celeryworker_1  |     step.start(parent)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 374, in start
celeryworker_1  |     return self.obj.start()
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer.py", line 280, in start
celeryworker_1  |     blueprint.start(self)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 123, in start
celeryworker_1  |     step.start(parent)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer.py", line 884, in start
celeryworker_1  |     c.loop(*c.loop_args())
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/celery/worker/loops.py", line 76, in asynloop
celeryworker_1  |     next(loop)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/kombu/async/hub.py", line 340, in create_loop
celeryworker_1  |     cb(*cbargs)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py", line 1019, in on_readable
celeryworker_1  |     self._callbacks[queue](message)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/__init__.py", line 534, in _callback
celeryworker_1  |     self.qos.append(message, message.delivery_tag)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py", line 146, in append
celeryworker_1  |     pipe.zadd(self.unacked_index_key, delivery_tag, time()) \
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2263, in zadd
celeryworker_1  |     for pair in iteritems(mapping):
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/redis/_compat.py", line 123, in iteritems
celeryworker_1  |     return iter(x.items())
celeryworker_1  | AttributeError: 'str' object has no attribute 'items'

@egregors did u find a proper fix for this? having the same issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugneeds further review

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @browniebroke@egregors@angerrp

        Issue actions
          New version of redis package is the cause of the Error · Issue #1868 · cookiecutter/cookiecutter-django