My first idea on the way this would work would be like the files autoloader, a new preload autoloading type. That way every package can declare file(s) that must be preloaded, and we generate a single file with all the includes. Those files can then have more smart logic on what to preload. I'd expect symfony would for example preload according to the container config and whatnot.

I don't think preloading all classes always is a good idea, that'll just blow up memory usage for no reason.

This is also very cheap to generate much like the files autoloading it's simply dumping an array of files in preload.php so it can be built always no need to be optional like optimized autoloader.

I see. That's a nice idea too!

I don't think preloading all classes always is a good idea, that'll just blow up memory usage for no reason.

I agree, it makes no sense to load all the classes if not all of them are used. However, as the developer of an app I would like to be able to optimize this myself so I don't think it makes sense to give the responsibility to define the classes that need to be preloaded to the developer of the library. Let's take the Symfony VarDumper component as an example: In 99.5% of all cases, this is used for debugging purposes only so it likely makes no sense to preload classes of it. But what if somebody builds an API service that uses this component to style some output? Preloading would make sense there then.
In other words: Whether or not preloading a class makes sense depends on how it's being used and more often than not, the developer of the library cannot tell how it's going to be used.

I also wondered how much RAM usage we're talking about. So here are some stats for everybody:

• I reset my opcode cache. It uses 18.07 MB by default (not sure where this default usage comes from though but everytime I reset it, it was at 18.07 again so I guess it's fair to take this as starting point)
• I ran composer create-project symfony/skeleton and called the welcome page it in prod environment (debug set to true).
• After that the memory usage was at 19.56 MB (157 cached files).

Then I ran composer dump-autoload -o --no-dev and edited my index.php to do some simple preloading based on the autoload_classmap.php. So I really just used the _preload() function from the RFC and added this to my index.php:

$classmap = include './../vendor/composer/autoload_classmap.php';
$classmap = array_unique($classmap); // Needed because we have multiple classes in some files
_preload($classmap);

I reset the cache and visited the welcome page again: Memory usage increased to 27.74 MB (919 cached files).

So yes, memory usage does increase but is it really that significant? I mean, if you enable preloading in the first place you're after good performance, right? Is it a problem then that your app uses a fair amount of RAM constantly?

All I'm trying here is really to just throw in some numbers so we can weigh up the pros and cons.

Maybe there are more variants? 😄

Here's the numbers of a bigger project using ApiPlatform, Doctrine, Guzzle, Enqueue, Symfony Translator, Redis, Symfony Console etc.):

Index page(docs endpoint) without preloading everything: 33.36 MB (15.29 MB net)
Index page(docs endpoint) with preloading everything: 75.31 (57.24 MB net)

So you need more then double amount of ram. In other words you can serve only less then half the users in comparison to non-preloaded with the same server

Well, that's not a fair comparison. I called just one endpoint. I would need to call the other endpoints that use other components one after the other if we wanted to find out the percentage of "useless cached files". I think I'd get a lot closer to the 57.24 MB if I did that 😊

Still a lot of overhead. More then I would like to pay to call the feature usefull when loading all the things

IMO loading everything only works for small apps

I appreciate your enthusiasm, buuut I don't think any of this is composer's responsibility. I fully agree that it's most likely app specific what you want to preload, but there are two things to consider in my proposal:

• the preload autoload doesn't have to be defined by every package. Typically I'd expect symfony to define that, or maybe some crypto lib that highly benefits from preloading assuming we get JIT optimizations in there.
• you don't have to point your php.ini to the preload.php file that composer generates. If you want a custom one, build a custom one. Frameworks definitely should offer tools for that.

Lastly regarding the option of preloading the whole classmap, that's a 3-line foreach loop that you can write as your own preload script loading all classes from composer's classmap if you are so inclined to waste memory ;) It could also be offered as a package toflar/preload-all-the-things that has a preload autoload which then goes and includes all files from the classmap.

So my position at least for now is to either do some very simple thing to facilitate things and "standardize" on a vendor/preload.php file, or alternatively we do nothing at all.

I tend to agree with @Seldaek here.

The preloading-everything strategy is already straightforward once you generate a full classmap for the optimized autoloader (which you should do anyway if you care about performance, and you don't need preloading if you don't care).
Any smarter algorithm would have to rely on the structure of the project, and so it might be hard to deal with that in Composer.
In Symfony, it would probably make no sense to have a config in the package deciding which Symfony files are preloaded for all Symfony projects. But Symfony could be generating a preload files for projects using it (and so with class coming from Symfony, but also from other dependencies or from the project itself) based on some heuristic. this is being suggested in symfony/symfony#29105 (comment) (I took the example of Symfony here, because I know about the current state of the discussion here, but the same could apply for other frameworks of course).

BTW: I'm not inclined to waste memory at all. It was just one variant that came to my mind and so I elaborated on it. I think it's good to consider multiple approaches, also for the people that read the issue later on. I'm perfectly fine with having the bad ones ruled out 😄

We could allow the preload section only in the root composer.json (so project specific) and that's where you specify the files you like to be included. Whether or not you use other files that are dynamically generated by e.g. Symfony is your business then. But the important feature here would be Composer that lets you aggregate out of the box (and maybe we can get the default php.ini setting to be set to vendor/preload.php in php itself which would just be ignored if it doesn't exist 😄).

@Toflar if it is root-only, why asking to put them in composer.json so that composer requires them in another file that you can then reference in your php.ini ? You could reference your own file directly in the php.ini.

I know 😄 The only thing it would do is somewhat "standardizing" the way to do it. Nothing else 😊

I would think that creating levels for autoloading like logging levels would allow for some sort of "automated" control with the library providing the files/classes that would be appropriate for the level. An uber level like "all" could include all files including the files for libraries not supporting the levels. Sounds like something that might fit into the PHP-FIG realm of discussion as well.