Skip to content

Support for asynchronous loading (and not packing everything in one file) #58

Closed
@arnaud-lb

Description

@arnaud-lb

Packing everying in one file causes some problems:

  • You have to re-pack the whole thing everytime you modify a single file
  • When an error is reported during execution, it does not reports the line number and file name of the real source file
  • Multiple browserified modules can't share dependencies: they are packed independently in each module; as a result they are downloaded twice.

This is a feature request to add support (as a browserify option) for using asynchronous loading instead of packing everything in a single file.

This would solve all the problems described above: dev becomes easier, bandwidth is saved, and pages load faster (because scripts are loaded asynchronously).

Most node modules are currently using synchronous loading, but it would be possible to convert a script like that:

var http = require('http');

function doRequest() {
    var url = require('url');
    ....
}

exports.doRequest = doRequest;

To something like this:

define(['http', 'url'], function(http, url) {
    function doRequest() {
        ....
    }

    exports.doRequest = doRequest;
    return exports;
});

Or even easier (just wrap the unmodified script with a header and footer):

define(['http', 'url'], function() {

    // now that http and url modules are loaded, require('http') and require('url') can be used directly
    // no need to modify the script :-)

    var http = require('http');

    function doRequest() {
        var url = require('url');
        ....
    }

    exports.doRequest = doRequest;
    return exports;
});

Optimization

Such asynchronous modules can still be packed together in a single javascript file for reducing the number of HTTP requests in production.

This is done by just adding the module name as first argument of define (define('moduleName', ['dependency'], func)) and concatenating all modules.

Activity

ghost

ghost commented on Aug 20, 2011

@ghost

1: wrapping a whole file in a function block is ugly

2: node modules use synchronous requires

3: browserify's goal is to let code written for node run in the browser

ghost closed this as completedon Aug 20, 2011
arnaud-lb

arnaud-lb commented on Aug 21, 2011

@arnaud-lb
Author

1: wrapping a whole file in a function block is ugly

That's a quite common pattern in Javascript. This avoids poluting the global namespace, this allows to close some variables private to the module.

That's called the module pattern.

2: node modules use synchronous requires

That's not a problem:

Once a module is loaded you can synchronously require() it (as per the spec):

// this loads 'foo' and 'bar' before calling the function
define(['foo', 'bar'], function() {
    // this synchronous require() works because 'foo' is already loaded:
   var foo = require('foo');
});

See the example code below "Or even easier (just wrap the unmodified script with a header and footer)"

So a node module using synchronous requires can be made asynchronous by just wrapping it in a define([dependencies...], functino() { ... })

3: browserify's goal is to let code written for node run in the browser

In the browser, that's why the ability to make them asynchronous at the same time would be awesome

ghost

ghost commented on Aug 21, 2011

@ghost

This issue completely misses the point of browserify. If you want that stuff go use requirejs.

arnaud-lb

arnaud-lb commented on Aug 21, 2011

@arnaud-lb
Author
  1. Nodejs modules are CommonJS modules.
  2. There is already a way to load CommonJS modules in the browser, called asynchronous module definition.
  3. It is very easy to convert synchronous modules to asynchronous modules.
  4. Multiple javascript loaders for the browser use this spec to do asynchronous loading, and are able to load CommonJS modules. requirejs is only one of them.

So converting nodejs modules to asynchronous modules is a quite obvious way to load them in the browser. Doing something else is completely missing the point.

ghost

ghost commented on Aug 21, 2011

@ghost

We accept pull requests.

arnaud-lb

arnaud-lb commented on Aug 21, 2011

@arnaud-lb
Author

Great! Could you reopen the issue then ?

ghost reopened this on Aug 21, 2011
medikoo

medikoo commented on Aug 24, 2011

@medikoo

@arnaud-ib
CommonJS modules (1.x) are not asynchronous
Spec you are referring is probably AMD which conceptually is more about loading packages than modules (if you look from node.js point of view)

Why don't you use requirejs ?

arnaud-lb

arnaud-lb commented on Aug 24, 2011

@arnaud-lb
Author

CommonJS modules (1.x) are not asynchronous

They can by defined in a way that they can be asynchronously loaded (see http://wiki.commonjs.org/wiki/Modules/AsynchronousDefinition ).

As I shown in the issue, converting a synchronous module to an asynchronous one is very easy.

Spec you are referring is probably AMD

Yes, this is a draft that now is maintained on the AMD project, which stands for Asynchronous Module Definition:

From the page: The Asynchronous Module Definition (AMD) API specifies a mechanism for defining modules such that the module and its dependencies can be asynchronously loaded.

This is particularly well suited for the browser environment where synchronous loading of modules incurs performance, usability, debugging, and cross-domain access problems.

which conceptually is more about loading packages than modules (if you look from node.js point of view)

It is a way to reliably load multiple inter-dependent javascript files asynchronously.

Why don't you use requirejs ?

requirejs only provides define() and require() implementations; it's only an asynchronous javascript loader, it does not rewrites scripts. Script have to use asynchronous module definition.

medikoo

medikoo commented on Aug 24, 2011

@medikoo

They can by defined in a way that they can be asynchronously loaded (see http://wiki.commonjs.org/wiki/Modules/AsynchronousDefinition ).

It's not part of Modules 1.x

You also said:

Most node modules are currently using synchronous loading

Fact is, that modules in Node are only synchronous and there are no plans even in far future for asynchronous support . Browserify is closely tied to Node and share same approach.

Running modules asynchronously will just slow down page load and will still need kind of file wrapping, I'm not sure what's the added value.

arnaud-lb

arnaud-lb commented on Aug 24, 2011

@arnaud-lb
Author

Please read earlier answers :)

It's not part of Modules 1.x

Why is this a problem ? All script loaders conform to this spec.

Even node implements it. Try it yourself:

// foo.js

define(function() {
    return {yes: "node implements asynchronous module definition too"};
});

// bar.js

foo = require('./foo');
console.log(foo.yes);

Browserify is closely tied to Node and share same approach

What is node's approach ? To concatenate the world in a single file ? To do things synchronously ?

Node's approch is to load scripts on demand. The let's load scripts synchronously approch is based on the fact that node can do it, and with guarantees on the order of execution of required scripts. BTW that's probably the only synchronous by default thing in node.

Browserify is porting node modules to the web.

In a web environment you can't load multiple scripts synchronously in a portable manner, and most importantly, you can't guarantee the order of execution. That means you can't guarantee that module A is executed before module B, and this is bad if module B depends on module A.

That's why CommonJS for the web uses asynchronous loading.

That's still follows node's approach to load scripts on demand, and to keep modules in a separate file.

Browserify wraps and packs everything in a single file. Nothing avoid it from converting everything to asynchronously-loadable modules. That's just cleaner and follows node's philosophy.

Running modules asynchronously will just slow down page load

  • Asynchronous means that the browser can continue loading and rendering the page even if the scripts aren't loaded yet.
  • Synchronous loading means that when the browser sees a <script> tag, it stops rendering the page until the script is downloaded and executed.

So the page will actually appear to load faster to the user with asynchronous loading.

This also means that you can load some modules only when needed, for example you can load your form validation module only when the user actually uses a form.

Any optimisation tips you read these days will recommend you to load your scripts asynchronously.

and will still need kind of file wrapping, I'm not sure what's the added value.

  • The page will load faster.
  • Your scripts can load other modules on demand, just like you do in node, and they are not downloaded until you ask it (if I understand browserify loads everything, everytime)

With current browserify, you get the following problems:

  • If you have two separate browserified modules each depending on the same modules, you download the modules twice.
  • You have to re-pack the whole thing everytime you modify a single file
  • Packing all in a single file changes line-numbers; this doesn't makes debugging easy
arnaud-lb

arnaud-lb commented on Aug 24, 2011

@arnaud-lb
Author
itay

itay commented on Aug 24, 2011

@itay
Contributor

I don't think the "async loading" vs "all in one file" approach is at all settled. Look at the discussion here:

https://github.com/paulirish/html5-boilerplate/issues/28

In general, the browserify approach is great for what it tries to do. I use it not to emulate node.js modules, but rather to allow for me to package my node.js module (so I get code sharing) as something that is suitable to the browser.

Given what you want, it seems that browserify isn't the project for you. You'd be better served by something like requireJS/yabble/etc.

arnaud-lb

arnaud-lb commented on Aug 25, 2011

@arnaud-lb
Author

You'd be better served by something like requireJS/yabble/etc.

Doesn't anyone read my answers before replying ?

This is what I want ! Using browserify to produce asynchronously defined modules, and using requireJS/yabble/etc to load the produced modules.

Producing scripts and loading them are two different things.

8 remaining items

ghost closed this as completedon Mar 11, 2012
azer

azer commented on Feb 7, 2013

@azer

For those who needs this feature; https://github.com/azer/onejs#multiple

fresheneesz

fresheneesz commented on Aug 1, 2013

@fresheneesz

@substack can you please reopen this issue? I see agreement that this is:

  • a minimally invasive feature
  • would be incredibly helpful for people who don't want to have a package that holds all of their javascript (which is obviously suboptimal)

I assume you closed this thinking that there was no traction. Well this is still a problem despite the fact that no one has fixed it (I'd hope this is obvious ; ). I'd also like to hear from @arnaud-lb why he went dark, if he's willing to revisit this issue, and/or how he worked around it in his projects.

ghost

ghost commented on Aug 1, 2013

@ghost

I don't care if there is traction or not. browserify is for loading node-style modules in the browser. Asynchronous loading is not something that browserify does. If you want that, there are lots of alternatives such as requirejs that do asynchronous loading. This is not something that browserify will ever support.

fresheneesz

fresheneesz commented on Aug 2, 2013

@fresheneesz

What changed your mind since you originally reopened the issue after saying "We accept pull requests."?

jaydson

jaydson commented on Sep 12, 2013

@jaydson

browserify is for loading node-style modules in the browser

Well, the fact is that node ecosystem is completely different from the Browser, maybe the Browserify "mission" is just nice, but doesn't work for the web.
Don't get me wrong, i really like Browserify, and i'm using in production, but i would like if Browserify support the asynchronous way to load dependencies.
If you have a huge application, and have to deal with a large amount of JavaScript files, this "node-style" is not what you want, but at the same time, you'll need a tool like the excelent Browserify.
The truth is that we don't have(yet) an optimal solution for modules in the Browser.
Browserify will help some kind of applications, AMD will help others, but we still need to workaround.

esamattis

esamattis commented on Sep 12, 2013

@esamattis
Contributor

we still need to workaround

Here's mine https://github.com/epeli/browserify-externalize

fresheneesz

fresheneesz commented on Sep 12, 2013

@fresheneesz

I've been working on my own solution as well: https://github.com/fresheneesz/asyncify

Its purpose is to make commonJS code just work with browsers, as long as you have control over the server-side (which you probably do if you care about this kind of thing). Browsers can request a set of modules, and the server will package up those modules and all their nested dependencies that haven't been already requested by the browser (no load duplication) and sends em down. It uses detective (just like browserify) to find dependencies and will also use browser-builtins to serve browser-side versions of node.js builtins.

Its pretty close to being in a working state, and I'll probably try to put up a working version sometime in the next month (i'm pretty busy at the moment). Let me know if anyone is interested in helping me speed up this process.

sokra

sokra commented on Sep 12, 2013

@sokra

Here's mine: https://github.com/webpack/webpack

It uses CommonJS and AMD. CommonJS and AMD define are loaded sync (packed into the same file), AMD require is loaded async (packed into another file). This way the code base is automatically splitted into multiple chunks, which are loaded on demand. It generates static files so no server-side component is required.

thlorenz

thlorenz commented on Sep 13, 2013

@thlorenz
Collaborator

Although you should think twice about asynchronously loading anything from a remote url, etc., you can use bromote for that use case.
It integrates with browserify and is very simple to setup.

Using this together with browserify --standalone also allows you to externalize part of your dependencies and load them asynchronously (on demand).

However I agree 100% that this functionality should not be part of browserify itself.

jhnns

jhnns commented on Sep 14, 2013

@jhnns

I'm using webpack because it offers a bundle-feature that allows me to split the entire code into separate bundles that can be loaded on demand. They can also be cached more efficiently.

Don't get me wrong, I don't want to advertise another module in this issue. But I think that this feature is very useful.

fresheneesz

fresheneesz commented on Feb 27, 2014

@fresheneesz

I just wanted to note that I've since tried webpack and have to say it handles asynchronous loading exceptionally well using the commonjs require.ensure proposal. Webpack is definitely the best of require.js with the best of browserify.

tnrich

tnrich commented on Mar 4, 2015

@tnrich

+1 to @arnaud-lb's ideas

ghost locked and limited conversation to collaborators on Mar 4, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @noonat@itay@azer@jaydson@medikoo

        Issue actions

          Support for asynchronous loading (and not packing everything in one file) · Issue #58 · browserify/browserify