Skip to content
This repository was archived by the owner on Oct 23, 2023. It is now read-only.
This repository was archived by the owner on Oct 23, 2023. It is now read-only.

Prevents access to request raw data in Flask #457

Open
@miracle2k

Description

@miracle2k

This all seems to be a bit of a mess, but if you want to access the raw request data in Flask regardless of the content type (I can't control the client), you need to call

 request.get_data()

and you need to do this before accessing request.data or request.form, because both of these will call get_data() in such a way that the stream will be exhausted, parsed into the form dict, and the raw data is never cached.

This doesn't seem documented except in the comments: https://github.com/mitsuhiko/werkzeug/blob/master/werkzeug/wrappers.py#L440

Should the sentry extension call request.get_data() first to make sure the raw data is cached before using request.form?

Activity

groveriffic

groveriffic commented on Mar 4, 2016

@groveriffic

I've run into this issue also recently.

It makes sense that Flask doesn't want to cache the unparsed body. This could be really large in the case of file uploads. Triggering the caching is probably not best for all scenarios.

This is working for me right now

# Workaround to allow unparsed request body to be be read from cache
# This is required to validate a signature on webhooks
# This MUST go before Sentry integration as sentry triggers form parsing
@app.before_request
def enable_form_raw_cache():
  if request.path.startswith('/redacted'):
    if request.content_length > 1024 * 1024:  # 1mb
      abort(413)  # Payload too large
    request.get_data(parse_form_data=False, cache=True)

if config.SENTRY_DSN:
  sentry = Sentry(app, dsn=config.SENTRY_DSN, logging=True, level=logging.ERROR)

There might still be a better, more general solution. I'm not sure if the code that captures request.form could just be moved into an after_request and still have the same results?

mitsuhiko

mitsuhiko commented on Mar 9, 2016

@mitsuhiko
Contributor

I don't think the raven client should force buffering of the body. That seems like a terrible idea because this could be (as mentioned) huge.

davidhariri

davidhariri commented on Aug 4, 2016

@davidhariri

Just ran into this. It took me a long time to realize that this is what was happening. @mitsuhiko could we perhaps add this issue to the docs in Sentry?

jcerjak

jcerjak commented on Feb 28, 2017

@jcerjak

Please add a big fat warning in the Sentry docs, also got bitten by this :/

added a commit that references this issue on Mar 2, 2017
b0c882f
added 2 commits that reference this issue on Oct 24, 2017
2d7c579
4ce2176
added a commit that references this issue on Nov 13, 2017
59f212e
added a commit that references this issue on Feb 19, 2018
564b477
jeteon

jeteon commented on Sep 25, 2018

@jeteon

Also just ran into this. Why does the Sentry SDK need to access request.data or request.form? If the SDK is already accessing the data (and triggering it being parsed into a data structure that is attached the request) then I don't understand how it would be doing much for memory/performance concerns by not cacheing it.

mitsuhiko

mitsuhiko commented on Sep 25, 2018

@mitsuhiko
Contributor

For what it's worth the new sentry-python SDK makes this configurable and generally is less invasive for accessing this information.

added a commit that references this issue on Mar 18, 2022

2 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @mitsuhiko@miracle2k@groveriffic@jcerjak@xordoquy

      Issue actions

        Prevents access to request raw data in Flask · Issue #457 · getsentry/raven-python