Skip to content

[gatsby-remark-images-contentful] intermittent build errors with/to base64 #11867

New issue
New issue
Closed
Closed
[gatsby-remark-images-contentful] intermittent build errors with/to base64#11867
Labels
topic: GraphQLRelated to Gatsby's GraphQL layertype: bugAn issue or pull request relating to a bug in Gatsby
@floriangaechter

Description

@floriangaechter
floriangaechter
opened on Feb 18, 2019

Description

We're having an issue during the build phase of our project: sometimes the run graphql queries... step finishes without any issues and the site builds fine. But since recently the build process gets stuck at the ⡀ run graphql queries — 123/125 46.15 queries/second step and keeps either spinning forever or, if it finishes, takes a very long time (it doesn't always stop at 123. Sometimes it stops at 7/125 or other numbers).

Steps to reproduce

  1. delete .cache folder
  2. run gatsby build

Expected result

The build process should finish without issues.

Actual result

The build process gets stuck at the "run graphql queries..." phase.

Environment

  System:
    OS: macOS 10.14.3
    CPU: (8) x64 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 11.10.0 - ~/.nvm/versions/node/v11.10.0/bin/node
    npm: 6.8.0 - ~/.nvm/versions/node/v11.10.0/bin/npm
  Languages:
    Python: 2.7.15 - /usr/local/bin/python
  Browsers:
    Chrome: 72.0.3626.109
    Firefox: 65.0.1
    Safari: 12.0.3
  npmPackages:
    gatsby: ^2.1.4 => 2.1.4 
    gatsby-image: ^2.0.29 => 2.0.29 
    gatsby-plugin-manifest: ^2.0.17 => 2.0.17 
    gatsby-plugin-offline: ^2.0.23 => 2.0.23 
    gatsby-plugin-postcss: ^2.0.5 => 2.0.5 
    gatsby-plugin-react-helmet: ^3.0.6 => 3.0.6 
    gatsby-plugin-remove-trailing-slashes: ^2.0.7 => 2.0.7 
    gatsby-plugin-robots-txt: ^1.4.0 => 1.4.0 
    gatsby-plugin-sharp: ^2.0.20 => 2.0.20 
    gatsby-plugin-sitemap: ^2.0.5 => 2.0.5 
    gatsby-plugin-svgr: ^2.0.1 => 2.0.1 
    gatsby-remark-embed-video: ^1.7.0 => 1.7.0 
    gatsby-remark-images-contentful: ^2.0.8 => 2.0.8 
    gatsby-remark-responsive-iframe: ^2.0.9 => 2.0.9 
    gatsby-source-contentful: ^2.0.29 => 2.0.29 
    gatsby-source-filesystem: ^2.0.20 => 2.0.20 
    gatsby-source-hire-with-google: ^1.3.0 => 1.3.0 
    gatsby-transformer-remark: ^2.2.5 => 2.2.5 
    gatsby-transformer-sharp: ^2.1.13 => 2.1.13 
  npmGlobalPackages:
    gatsby-cli: 2.4.10

Activity

wardpeet

wardpeet commented on Feb 18, 2019

@wardpeet
wardpeet
on Feb 18, 2019 · edited by wardpeet
Contributor

@floriangaechter it seems to me that this could be a case like #11747. I'll be closing it in favour of that one please tell us as much as you can about this issue (maybe share your repo).

If it does not, feel free to re-open 😄

wardpeet
closed this as completedon Feb 18, 2019
floriangaechter

floriangaechter commented on Feb 18, 2019

@floriangaechter
floriangaechter
on Feb 18, 2019
Author

@wardpeet thanks for your response. I’m not sure if those two are related as in our case, deleting the cache folder actually makes the issue worse instead of better. I’ll leave the issue closed for now and follow the other topic but might reopen it later if it turns out that the issues are not related.

floriangaechter

floriangaechter commented on Feb 18, 2019

@floriangaechter
floriangaechter
on Feb 18, 2019
Author

Turns out the issue is the _tracedSVG part of the images we're using. If we just query for "normal" images it works just fine.

wardpeet

wardpeet commented on Feb 18, 2019

@wardpeet
wardpeet
on Feb 18, 2019
Contributor

lets re-open. Could you maybe try to create a small reproduction? ^^

wardpeet
reopened this on Feb 18, 2019
floriangaechter

floriangaechter commented on Feb 19, 2019

@floriangaechter
floriangaechter
on Feb 19, 2019 · edited by floriangaechter
Author

Sure, check out https://github.com/floriangaechter/tracedsvgtest

I hard-coded the api keys for contentful, since it seems to only happen when the images are stored there.

If you have a look at the query in https://github.com/floriangaechter/tracedsvgtest/blob/master/src/pages/index.js#L37 and remove the _tracedSVG part, the build process runs without any issues. But with the _tracedSVG part, it gets stuck during the run graphql queries step.

wardpeet
added
type: bugAn issue or pull request relating to a bug in Gatsby
on Feb 19, 2019
floriangaechter

floriangaechter commented on Feb 19, 2019

@floriangaechter
floriangaechter
on Feb 19, 2019 · edited by floriangaechter
Author

Some new information – sometimes I'm getting the following error:

error UNHANDLED REJECTION


  Error: read ETIMEDOUT

  - stream_base_commons.js:162 TLSWrap.onStreamRead
    internal/stream_base_commons.js:162:27

Here's the stack trace:

{ Error: read ETIMEDOUT
      at TLSWrap.onStreamRead (internal/stream_base_commons.js:162:27)
    errno: 'ETIMEDOUT',
    code: 'ETIMEDOUT',
    syscall: 'read',
    config:
     { adapter: [Function: httpAdapter],
       transformRequest: [Object],
       transformResponse: [Object],
       timeout: 0,
       xsrfCookieName: 'XSRF-TOKEN',
       xsrfHeaderName: 'X-XSRF-TOKEN',
       maxContentLength: -1,
       validateStatus: [Function: validateStatus],
       headers: [Object],
       method: 'get',
       responseType: 'arraybuffer',
       url:
        'https://images.ctfassets.net/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40',
       data: undefined },
    request:
     Writable {
       _writableState: [WritableState],
       writable: true,
       _events: [Object],
       _eventsCount: 2,
       _maxListeners: undefined,
       _options: [Object],
       _redirectCount: 0,
       _redirects: [],
       _requestBodyLength: 0,
       _requestBodyBuffers: [],
       _onNativeResponse: [Function],
       _currentRequest: [ClientRequest],
       _currentUrl:
        'https://images.ctfassets.net/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40' },
    response: undefined } }

and:

{ Error: read ETIMEDOUT
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:162:27)
  errno: 'ETIMEDOUT',
  code: 'ETIMEDOUT',
  syscall: 'read',
  config:
   { adapter: [Function: httpAdapter],
     transformRequest: { '0': [Function: transformRequest] },
     transformResponse: { '0': [Function: transformResponse] },
     timeout: 0,
     xsrfCookieName: 'XSRF-TOKEN',
     xsrfHeaderName: 'X-XSRF-TOKEN',
     maxContentLength: -1,
     validateStatus: [Function: validateStatus],
     headers:
      { Accept: 'application/json, text/plain, */*',
        'User-Agent': 'axios/0.18.0' },
     method: 'get',
     responseType: 'arraybuffer',
     url:
      'https://images.ctfassets.net/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40',
     data: undefined },
  request:
   Writable {
     _writableState:
      WritableState {
        objectMode: false,
        highWaterMark: 16384,
        finalCalled: false,
        needDrain: false,
        ending: false,
        ended: false,
        finished: false,
        destroyed: false,
        decodeStrings: true,
        defaultEncoding: 'utf8',
        length: 0,
        writing: false,
        corked: 0,
        sync: true,
        bufferProcessing: false,
        onwrite: [Function: bound onwrite],
        writecb: null,
        writelen: 0,
        bufferedRequest: null,
        lastBufferedRequest: null,
        pendingcb: 0,
        prefinished: false,
        errorEmitted: false,
        emitClose: true,
        autoDestroy: false,
        bufferedRequestCount: 0,
        corkedRequestsFree: [Object] },
     writable: true,
     _events:
      [Object: null prototype] {
        response: [Function: handleResponse],
        error: [Function: handleRequestError] },
     _eventsCount: 2,
     _maxListeners: undefined,
     _options:
      { protocol: 'https:',
        maxRedirects: 21,
        maxBodyLength: 10485760,
        path:
         '/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40',
        method: 'get',
        headers: [Object],
        agent: undefined,
        auth: undefined,
        hostname: 'images.ctfassets.net',
        port: null,
        nativeProtocols: [Object],
        pathname:
         '/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif',
        search: '?w=40' },
     _redirectCount: 0,
     _redirects: [],
     _requestBodyLength: 0,
     _requestBodyBuffers: [],
     _onNativeResponse: [Function],
     _currentRequest:
      ClientRequest {
        _events: [Object],
        _eventsCount: 6,
        _maxListeners: undefined,
        outputData: [],
        outputSize: 0,
        writable: true,
        _last: true,
        chunkedEncoding: false,
        shouldKeepAlive: false,
        useChunkedEncodingByDefault: false,
        sendDate: false,
        _removedConnection: false,
        _removedContLen: false,
        _removedTE: false,
        _contentLength: 0,
        _hasBody: true,
        _trailer: '',
        finished: true,
        _headerSent: true,
        socket: [TLSSocket],
        connection: [TLSSocket],
        _header:
         'GET /i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40 HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nUser-Agent: axios/0.18.0\r\nHost: images.ctfassets.net\r\nConnection: close\r\n\r\n',
        _onPendingData: [Function: noopPendingOutput],
        agent: [Agent],
        socketPath: undefined,
        timeout: undefined,
        method: 'GET',
        path:
         '/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40',
        _ended: false,
        res: null,
        aborted: false,
        timeoutCb: null,
        upgradeOrConnect: false,
        parser: null,
        maxHeadersCount: null,
        _redirectable: [Circular],
        [Symbol(isCorked)]: false,
        [Symbol(outHeadersKey)]: [Object] },
     _currentUrl:
      'https://images.ctfassets.net/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40' },
  response: undefined }

I think it has to do with the way the images are pulled from Contentful…

ColeTownsend

ColeTownsend commented on Feb 22, 2019

@ColeTownsend
ColeTownsend
on Feb 22, 2019
Contributor

@floriangaechter We've been running into similar issues using Contentful with @robinpowered

dirtyredz

dirtyredz commented on Feb 24, 2019

@dirtyredz
dirtyredz
on Feb 24, 2019 · edited by dirtyredz

I am having a similar issue, which was confusing to me since my latest build on Netlify was working perfectly fine.
I tried multiple troubleshooting steps but one in particular worked. I dont remember why I thought to try disabling my antivirus but after I did everything worked again without issue.
Digging deeper it was my HIPS (Host Intrusion Protection System, Whenever this protection is activated I get this error:

> gatsby develop

error UNHANDLED EXCEPTION


  Error: read EINVAL

  - stream_base_commons.js:162 TTY.onStreamRead
    internal/stream_base_commons.js:162:27

But with HIPS disabled I am able to perform gatsby develep without any issues. I made sure to delete both the public and .cache dir after each test.

I dont know exactly what this means for Gatsby or why, but I hope this helps.

Edit:
It seems this was being blocked:
capture

claudiobmgrtnr

claudiobmgrtnr commented on Mar 11, 2019

@claudiobmgrtnr
claudiobmgrtnr
on Mar 11, 2019

We have the same problem. A fix would be appreciated.

floriangaechter

floriangaechter commented on Mar 11, 2019

@floriangaechter
floriangaechter
on Mar 11, 2019
Author

It really seems to be an issue with the images coming from Contentful. If I add the _noBase64 suffix, the build seems to succeed every time.

Maybe @Khaledgarbaya has an idea what the issue could be?

Khaledgarbaya

Khaledgarbaya commented on Mar 11, 2019

@Khaledgarbaya
Khaledgarbaya
on Mar 11, 2019
Contributor

Hey Folks,
I think when the plugin tries to load the image and generate a base64 out of it times out could be this part

But I am not sure where exactly this is failing. Is it happening only with GIF images?

floriangaechter

floriangaechter commented on Mar 12, 2019

@floriangaechter
floriangaechter
on Mar 12, 2019
Author

Yes, I think it has something to do with the generation of either the base64 or tracedSVG variant of the image. Because if I remove both of them and tell Gatsby to generate neither base64 or tracedSVG the build succeeds each time. And it happens to all images, not just GIF files.

30 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic: GraphQLRelated to Gatsby's GraphQL layertype: bugAn issue or pull request relating to a bug in Gatsby

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @coreyward@floriangaechter@itsmylife@ColeTownsend@wardpeet

        Issue actions
          [gatsby-remark-images-contentful] intermittent build errors with/to base64 · Issue #11867 · gatsbyjs/gatsby