Is anyone able to give me some help with this please? I am using npm 5.3.0 and I need my production environment to be the same overtime, which is why I was originally using the package-lock.json. Please can someone give me some information on how I should now have my project set-up as package-lock.json is now updated every time and not locked at all.

The concept of the package-lock.json is either utterly confusing or not really thought through yet.

This document states that npm would install dependencies as they are specified in the lock file. However, as you observed, this is not true at all and I can reproduce the exact same behaviour with npm 5.3.0.

As a current workaround, I pin my dependencies in package.json, so I write 1.2.0 instead of ~1.2.0 or ^1.2.0. I commit the auto-generated package-lock.json and keep it up to date, but I’m not exactly sure what npm does with it under the hood. At least that strategy seems to yield reproducible builds then.

@jakeNiemiec instead of just voting this question down it would be nice if you would provide any helpful explanation or guidance instead. Lock files are anything else then transparent at the moment and infact behave fundamentally different as they do on other platforms, such as PHP/composer for instance. The confusion here is more then understandable.

Btw., see this Stack Overflow discussion, which also raises more questions than it solves: https://stackoverflow.com/questions/45022048/why-does-npm-install-rewrite-package-lock-json

Thank you @jotaen, I have been trying to highlight this issue for over a week now. Currently I have had to rollback to npm 5.0.3 as this issue doesn't seem to occur there. I looked at the change log and saw that when npm 5.1.0 was released it allowed the package.json to trump the lock file, so it seems suspicious that the issue started happening after that.
Currently I have no way to create a static, reproducible production environment for testing, because every time I do a fresh npm install with version 5.3.0 it ignores the lock file and installs the newest versions of packages.

Thanks for pointing to the Changelog: the root cause is #16866, which is the proof that the current behaviour is indeed intended. There was already a lengthy discussion happening in this PR, however, I doubt that the documentation was properly updated after the change being applied.

In my eyes #16866 is not a good decision, because the introduced behaviour is not predictable and transparent to the user. Also apparently, the naming lock doesn’t reflect the actual purpose of the file anymore, because it’s no longer a lock file, but rather just a dependency snapshot that is taken into consideration in particular cases.

I doubt that this design decision is reverted, so I guess we now have to deal with it (or switch to yarn or whatever). At least the documentation should be updated properly to allow users to understand the current behaviour.

Last note: #16866 is a breaking, non-backward compatible change, which shouldn’t have been introduced as minor version upgrade.

@jotaen this is indeed what I have been trying to highlight with this question here and my posts on the other feed. The fact that now the lock file no longer locks means it has no purpose in terms of locking packages. A suggestion to have an additional flag (i.e. npm install ---lock-file) to enforce an installation ONLY from the lock file is a decent one, but I do not know where to officially make this request.
Currently version 5.3.0 is unusable when trying to create a static, repeatable environment.

@kidd3 This might be helpful:
https://github.com/npm/npm/blob/latest/CONTRIBUTING.md

I am neither a collaborator, nor an employee of npm. But based on my experience you can either:

1. make the changes yourself and make a pull request (if you have the time and feel comfortable enough to change the code).
2. Or wait for a collaborator or a npm employee to do so. But considering the number of issues and threads on this repo, it would be very unlikely for the latter to happen, unless a huge number of people are experiencing the same issue. Which in this case, unfortunately, I don't think is happening.

So your best chance might be to make the changes in the code yourself, reference it in this issue, and hope the PR is accepted.

P.S: Just as a pointer, you will want to start with making your changes to this file, if you decide to go with the DIY option:
https://github.com/npm/npm/blob/latest/lib/install.js

Thanks for pointing that out. The only thing that is a bit unfortunate though, is that such a PR would basically be a revert of #16866, which makes it quite unlikely to be accepted. This whole topic has already been discussed back and forth in that PR.

The missing bit for me is now to update the documentation accordingly, which is misleading if not incorrect. On the page about package locks it says:

The presence of a package lock changes the installation behavior such that:

1. The module tree described by the package lock is reproduced. This means reproducing the structure described in the file, using the specific files referenced in "resolved" if available, falling back to normal package resolution using "version" if one isn't.

This is not true, as the generated module tree might be (as of v5.1.0) a combined result of both package.json and package-lock.json.

Also the documentation should point out a way how to perform truly reproducible builds. That was the reason why package locks have been introduced in the first place and that was also what a lot of people obviously had relied on since package-lock.json had been introduced.

@jotaen Very true. That exact part of the documentation is a bit misleading. It might be a good idea to change that as well.
But this PR does not need to be a revert of #16866 since we are not changing the default behavior of npm install.
npm install can still do exactly what it is doing right now, so this PR won't affect anyone who is happily using it now. We can just add another option (E.g. --lock or --read-only or --lock-file) that reads the tree from lock file and does not heed the differences between the lock file and package.json, whatsoever.

@Maziar-Fotouhi that makes sense, thanks for clarifying.

Maybe it’s an idea to extract the outcome of the discussion here into separate issues. I doubt that this issue here deserves the right attention, since the title is rather suggesting a support request.

I opened #18103 for changing/updating the documentation.

@kidd3 Maybe you could also open a new issue in which you present the idea of introducing some sort of CLI option. (Or, if you have the time and feel confident, you could directly work on a PR.) If you think that your needs are covered thereby, this issue here could be closed then.

@Maziar-Fotouhi thanks for the information, but unfortunately I don't have enough time right now to attempt to fix this issue myself, I have already had to rollback to v5.0.3 to continue my own work.
Considering it didn't exist prior to #16866 I can only assume people haven't encountered this issue as they aren't needing to have a repeatable, static install for an environment for testing. Otherwise I really don't understand why #16866 was allowed to happen, as it completely breaks npm and the lock file - which as @jotaen has pointed out, no longer matches the documentation which is clearly still a version of docs from before v5.1.0.
As it currently stands, npm v5.3.0 is completely unusable if you are attempting to use the package-lock.json file as an actual lock file. So I would really hope someone in the npm community can add a fix to the workflow soon, or at least give some indication of a timeframe for a fix.

I wouldn’t go so far and term it “completely unusable” (it somehow used to work prior to npm 5 as well). But it’s definitely a big problem and also a regression that I don’t think people are aware of enough.

Doesn’t it do the trick for you to pin your dependencies in package.json to fixed versions (without ~ or ^)? Because that seems to be a suitable workaround for me at the moment.

Regarding your idea of the additional CLI option I think it’s still worth it to open a separate issue with a feature request specifically asking for that. Maybe that’s heard better by the community or the maintainers.

Strange that the other thread doesn't have the simplest solution... Which updates package.json and package-lock.json

npm install package@version --save

The change in 16866 broke my CI builds today...

I thought I was getting this problem but when I try and reproduce it in isolation (npm 5.3.0), it behaves as expected. For example:

mkdir npmtest
cd npmtest
npm init --yes
npm install --save-exact debug@2.1

At this point package.json, package-lock.json and node_modules/debug/package.json all show 2.1.3. Now I edit package.json to change it to a semver range that is compatible with the version in the lock file, say ^2.0.0:

rm -rf node_modules
npm install

This makes no changes: package.json shows ^2.0.0 and package-lock.json and node_modules/debug/package.json still show 2.1.3.

If I change the version in package.json to be incompatible with the version in package-lock.json, say ~2.3.0:

npm install

Now the lock file and installed package get updated to 2.3.3.

This is all as I would expect. Provided package.json hasn't been edited (in an inconsistent way) since the generation of package-lock.json then the versions installed will be exactly as listed in package-lock.json.

Am I missing something?

@tamlyn Yes, what you describe is exactly the way it currently works. However, it’s not the way it used to work prior to 5.1.0 and it’s also not the way that is currently documented.

The difference between you and the issue creator is --save-exact, which kidd3 not did (he uses fuzzy dependencies in his question). Note that in your example once the package changes from 2.3.0 to, say, 2.3.1, your dependency would be automatically upgraded upon next npm i once the new version is available (since you made the version fuzzy by adding ~ to ~2.3.0). This would also update package-lock.json then.

This wasn’t the case prior to npm 5.1.0, where package-lock.json never was automatically overwritten, no matter what versions you changed in package.json. People are now asking in this thread what the correct way is to reliably reproduce the package-lock.json tree. (Currently the answer to this seems to be using exact versions in package.json, which is, however, not optimal).

Wow.

Every time a developer does npm install, a file of 13k lines (YMMV) is updated. Why?

Hey y'all, here's some notes:

1. Making it so npm install after editing package.json updates package-lock.json is working as intended, and has been intended that way since 5.0. The fact that it didn't at first was a bug, and we have fixed it accordingly. package.json is an authoritative manifest file, and package-lock.json is considered to be a manifestation of a particular package.json at a point in time.

2. There is no option/flag to freeze installs to force them to ignore package.json. There is also no intention of implementing this into npm. We're actively changing the way the CLI interacts with and thinks about manifests, lockfiles, and node_modules, and npm5 is a bit of a middle state. Part of these changes is to hopefully remove some level of configurability from the CLI itself to reduce the tool's complexity.

3. Because the use case of "I wanna make sure this is locked down" is still valid, instead of adding that --freeze option, we're working on getting cipm out the door, because we believe this solves the use case of wanting to make damn sure your CI or production builds are obeying package-lock.json. We are still planning on making it error when package.json does not correspond to package-lock.json, though. Note that cipm has the added benefit of being 2-3x faster than npm itself because of how much smaller and focused it is.

4. If you're running into any cases where, without modification of package.json, a newer version is installed than is specified in package-lock.json, please try again using the latest npm (npm@5.4.0 as of this writing), and if you can still reproduce it, file an issue with a repro case. It is a bug if this happens.

Thanks for responding @zkat. Your list is exactly how I expect the lock file to work.

What I've been seeing is a lot of churn just from npm install, both on my and my (strictly backend) co-workers. This could be because we work in both MacOS and Ubuntu environments, but whatever it is we need the lock file to be the source of truth and not need to be committed anytime someone works anywhere in the repo (npm excepting). Additionally, installing a single package npm install <packagename> seems to go through an entire npm install which hasn't been the case in the past.

I'll give 5.4.0 a shot and see if that resolves what we're seeing. I really do appreciate the update on the state of things.

@greggb if you're on different platforms, you'll have to wait a wee bit because we're still pending a patch to make lockfile generation consistent across them.

Ah, got it

So package-lock.json is only useful when you declare exact dependencies in your package.jsonfile?

@laserus Ok, that's weird. I thought I had reproduced this with a new clean project, but apparently I haven't.

So I looked a bit closer at the project that consistently has this issue (i.e. delete node_modules + npm install ends up with a different set of modules + modified lock file). Looks like its package-lock.json contains some packages where the .tgz URL that's should be in resolved is instead in the version property. Unfortunately I don't know what version of npm was used to make that happen :(

Thank you for trying to reproduce this.

I think the best path forward is to install according to the lock file except for in a few different scenarios.

1. If there is no lock file resolve everything according to package.json
2. If the semver range for a package in package.json has changed re-resolve that dependency on the next npm i
3. If a package is removed in package.json remove it from the lock file on the next npm i.

Notably this is how Bundler and CocoaPods work. The package–lock.json file needs to contain the semver ranges of top level dependencies so that npm can determine at install whether the user has changed any of the packages ranges and take appropriate action.

There should be an option to install, but not update the package-lock.json. As it stands I can't do a deploy and be sure that the deps installed are the same ones I have.

There should be an option to install, but not update the package-lock.json. As it stands I can't do a deploy and be sure that the deps installed are the same ones I have.

@markwellis npm i --no-save to the rescue

@SleepWalker that only doesn't update the package-lock.json file, it does not make sure that the versions installed are the ones declared in package-lock.json i think

@daften correct. It's the equivalent of doing npm i and then just revert the changes to package-lock. Therefore not a solution

@ameenross but if you'll use the exact versions in package.json, it will, actually install the correct versions and won't mess up your package-lock.json in deploy environment.

So in dev environment you are using: npm i -SE or npm i -DE (or save-exact = true in .npmrc)
And then in deploy environment: npm i --no-save

It won't install wrong dependencies.

@SleepWalker The reason why every package manager has 2 separate files and uses semver is so you wouldn't need to put exact versions in the main dependencies file. Using a package.json with exact numbers defeats the purpose in a large part. Package locking on npm is just broken, and we're switching to yarn because of this.

@SleepWalker well that might scratch this itch, but introduces other problems. If you then want to upgrade your dependencies you have to change all of the version definitions in package.json. Also, the file is basically a lie then, because it says "this project is compatible with foo/foo 1.2.3" as opposed to "this project is compatible with foo/foo 1.x, and guaranteed tested and working on 1.2.3".

My original issue was when there are ranges specified within the package.json file, as running a npm install causes the package-lock.json file to be automatically updated every time a newer version of a package gets released, meaning there was no control over what gets installed and impossible to keep it "locked" (for which the lock file, by it's name, should be doing).
It is correct that if you didn't specify ranges in the package.json file, then the lock file would only get updated when the package.json file got updated (manually).
I have yet to try the newest version of NPM to see if this issue still exists or not, I stuck to using npm version 5.0.3 as this version actually worked as expected.

I'm with @daften here. I think there's a lot of value in going with the path of least surprise and emulating the behaviour of other package managers in other ecosystems that use the manifest/lock file approach.

See

• Bundler
• CocoaPods
• Cargo
• Composer

@shellscape nah I was wrong. npm behaves as I expect it to:

The following should modify the lockfile: npm update, npm i packageName, npm i but only if you manually edited package.json. That is, the only time package-lock is updated on npm i is if the lockfile couldn't possibly be generated by the package.json. npm i doesn't modify lockfile if there happens to be a newer version that matches the range in package.json but is newer than what's in lockfile. So in that sense, it's a proper lockfile. They just added the ux case of someone manually editing package.json and expecting npm i to work. I support this. And I'm going to delete my previous comment because it's noise.

@kidd3 That is literally what the newest versions fix.

I'm going to close this as the now current behavior is AFAIK correct:

1. If you have a package.json and you run npm i we generate a package-lock.json from it.
2. If you run npm i against that package.json and package-lock.json, the latter will never be updated, even if the package.json would be happy with newer versions.
3. If you manually edit your package.json to have different ranges and run npm i and those ranges aren't compatible with your package-lock.json then the latter will be updated with version that are compatible with your package.json. Further runs of npm i will be as with 2 above.

You can see 3 in action if you use a package-lock locally but don't commit it. When you do a pull, you'll get an updated package.json which will be incompatible with your package-lock.json. (In older npms changes made directly a package.jsonwould NEVER propagate to apackage-lock.json/npm-shrinkwrap.json` without regenerating it from scratch.)

On the horizon is cipm which will only read your package-lock.json or npm-shrinkwrap.json and install from that. It will be entirely readonly when it comes to your package metadata, even when that metadata is internally inconsistent. So for folks who want to, for instance, mangle their package-lock.json to point at versions incompatible with their package.json will be able to use cipm to install that. (Maybe to be known as npm ci, branding tbd.) It's also much-much faster in the "I started with no node_modules" case and so will be preferable for CI and deployment even without it's greater immutability guarantees.

If you do run into a case where npm@^5.4.2 mutates a package-lock.json that was otherwise compatible with the paired package.json please open a new issue. This sort of thing would constitute a high priority bug.

(I'm also locking this issue because comments on closed issues are by-and-large not seen by the npm team. This doesn't mean we don't want to hear from you, in fact, the opposite. We want you to talk to us in a way we can hear, which is why I'd like you to either open a new issue or chat with us over at WeAllJS's #npm channel.))