I also find this behavior problematic. For now I'm working around it by changing my npm install command to rm -f package-lock.json && npm install. Obviously, that's not an elegant solution, and somewhat defeats the purpose of having a lockfile in the first place.

Same problem here with npm 5.0.0 and node 7.3.0 on macOS.
Each time I add a dependency in my package.json file, I run npm i and I've got this :

$ npm i 
removed 3 packages in 5.953s

My new dependencies are not installed after this. I had to delete the package-lock.json and re-run npm i and this time my new dependencies are correctly installed.

If you run npm update package-lock.json gets updated and the new dependency is installed.

I think adding npm update command's new option to update only package-lock.json will be a better solution. And then npm install would be better to throw an error or show alerts of the conflicts of the two files.

npm update does not solve it for me.
For some reason it keeps thinking that some package versions are invalid when I try to use npm update.
This is super annoying and a huge downside to npm@5. I'm really disappointed at npm!
npm should not be that hard to use. It was much better when a simple npm install did the job.
Now I have a good reason to switch to yarn!

Yea, this has been causing me serious grief the last couple of days. The "speed up" isn't worth the constant requirement to delete my package-lock file every time I need to update a dependency... Changing my package.json never updates my package-lock, and npm update after changing the version doesn't update it either.

@zone-ghost thats exactly my point

we are updating our dependencies automated every moring ... since two days we have a up-to-date package.json file ... but we dont get the new npm cuz the package-lock.json points to the wront version.

to delete the package-lock.json is an absolute useless step ... but fix it.

is there a way to update the package-lock.json??

Put "preinstall": "npm config set package-lock false" in your scripts to disable package-lock.json.

If we wanted yarn, we would have used that. This half-hearted attempt helps no one.

Edit: I merely wanted to highlight that npm should not try and be yarn at the expense of npm. They are both solid package managers built to solve different problems.

@jakeNiemiec It helps the people who had their builds sped up by whole minutes due to package-lock.json

I don't get it. So what are you supposed to do after you update your package.json? Run npm update? I'm getting the same "invalid" messages as @Maziar-Fotouhi does when I try that.

Do we just need a npm lock command like we had npm shrinkwrap?

This just generates the shrinkwrap with current dependencies (so has the effect of updating the lock if you've added a new dependency to package.json).

@fxck after reading @BigstickCarpet 's comment, I added this to my package.json in the scripts section:
"freshtall": "rm -rf node_modules && rm -rf package-lock.json && npm install",
Now I just need to run npm run freshtall and I will get a fresh install of every package.
P.S: "freshtall" was just a cheesy name I came up with, you can name yours whatever else you want if you want to change it!
But I am using yarn right now since I am only using npm for installing packages.
@jakeNiemiec This really depends on the project that you are working on but what is the point if you don't use the lock file?! This is just sweeping everything under the carpet. I prefer to keep an unbiased opinion about this issue and use yarn (which by the way easily utilizes its lock file and is much faster), until this is fixed in npm.

@Maziar-Fotouhi don't forget a npm cache clean --force in there too.

what is the point if you don't use the lock file

I only do this because updating 'individual packages' was removed. I can't: npm update @me/myCustomDep anymore.

@jakeNiemiec That makes sense. I change the version numbers directly in package.json. Maybe that can help.
Thanks for the hint! Why do we need to clean the cache though? The npm documentation says:

it should never be necessary to clear the cache for any reason other than reclaiming disk space, thus why clean now requires --force to run.

@Maziar-Fotouhi Thanks for the hint! Why do we need to clean the cache though?

#16665 crash installing: "Error: Argument #2: Expected string or string but got null"

This nasty error that intertwines with this issue, at least for me. I have found that removing node_modules can have weird aftereffects with npm5.

@jakeNiemiec Thank you for sharing that! Sounds like you had a lot of fun with this issue!
So here is the script that I have added to the scripts section in package.json and using it through npm run freshtall:
"freshtall": "npm cache clean --force && rm -rf node_modules && rm -f package-lock.json && npm install",

@jakeNiemiec Thanks, following your advice, i simply added this line to my .npmrc file.
package-lock = false

Removing node_modules is good as a workaround but it is overkill in my opinion. It would also not be viable for large projects that have 5+ min install times due to native build steps.

Deleting package-lock.json each time seems kludgy and creates problems for automated tools like npm-check-updates, which I maintain. Users of npm-check-updates as well as users who hand edit their package.json need a way to regenerate the package-lock.json file without having to reinstall all modules.

Isn't the purpose of package-lock.json to block changes in package.json. It makes sense that the node_modules don't update until you wipe/invalidate package-lock.json when making edits to package.json.

I do agree that this can be bothersome if you're not expecting this behavior by default. I wouldn't, however, say that this is a bug or anything.

@Ketamin I initially did this, but this won't work for anyone else using your package (in my case: co-workers). I was hoping it was temporary, but after seeing #16991 it's probably better to downgrade to npm4 and wait for npm5 to stabilize.

@kapadiamush I would expect that package.json is treated as the prime source of truth...package-lock.json must validate against package.json. If found to be invalid, update only the needed libs and reflect those updates in package-lock.json.

Use case:
package.json

• myCoolLib@~1.0.0
• reactOrAngularSuperLib@~2.2.0

package-lock.json

• myCoolLib@1.0.4
• ReactOrAngularSuperLib@2.2.13

I want to update myCoolLib to 1.0.5. My choices:

• ❌ In package.json set myCoolLib@~1.0.5 and run npm install. You would expect package-lock.json to update the new locked version.

• ❌ npm update myCoolLib This worked in npm4, it would be ideal since it only updates the item I want. I would expect the entry in package.lock to updated as well.

• ❌ npm update This will update myCoolLib along with everything else (very bad for large libs). This is OK, but it negates the usefulness of the lock file since it doesn't change hence this issue.

• ✅ rm -f package-lock.json && npm install Same intention as above except package-lock.json gets updated.

At least, thats how I imagined everything should work from personally working with other package locking systems.

@jakeNiemiec Yeah, it's the other way around.. sorta. It's my understanding that the package-lock.json is meant to prevent exactly what you're trying to do.

You don't need to remove node_modules to update the package-lock.json -- rm package-lock.json && npm install should do the trick (and be significantly faster).

@kapadiamush I disagree with your interpretation... there are multiple installs possible for a given package.json. package-log.json is meant to lock node_modules to a particular installation for a given package.json. package-log.json is not meant to lock the package.json itself, which is obviously in source control and not changing arbitrarily.

The problem arises from the fact that the package.json can be changed before npm install is run, leaving the package-lock.json in an invalid state. Since they are separate files, this is somewhat inevitable, but there should be a way to update the lock file. If this can be done without re-installing all modules, even better.

@raineorshine That actually makes a lot more sense regarding the package-lock.json locking the node_modules in place.

You should still be able to update the package-lock.json without completely removing all of the node_modules. From what I've seen while playing around with it, removing package-lock.json and running npm install should "update" the package-lock.json without reinstalling all the node_modules.

I have over 750MB of dependencies, so removing node_modules is not a good idea.
Now, what is package-lock.json? well, it is not intuitive. I kinda got an idea now that I read about it, but new team members will not know about it.
What are the improvements of npm@5? I'm not sure. yarn worked well from scratch (and it is really faster and more intuitive) but I don't really need it... so I'm not gonna use it.

For the time being, I'm going back to npm@4 at least until version 5 becomes the de-facto standard in the industry.

I am absolutely on board with @jakeNiemiec 's interpretation. package.json should be the source of truth, and the lock file should be validated against that. If I update my package.json, it is very unintuitive that npm refuses to install what is in that file.

@albertosantini That is not what I am seeing.

npm --version
5.3.0

package.json:

{
  "name": "npm-test",
  "version": "1.0.0",
  "dependencies": {
    "eslint": "^4.2.0"
  }
}

If I npm install I get 4.3.0 installed, 4.3.0 in the lock file, and the package.json stays at ^4.2.0.
If the intention is that package.json should be rewritten I 1) have a bug, and 2) strongly disagree with the design choice. Having install rewrite package.json would be like having a compiler rewriting my source code.

If I npm update or explicitly npm install eslint, the package.json is bumped to ^4.3.0 as expected. (Side note: If install updates and rewrites the package.json, what is the difference between install and update?)

@wulfsberg I am glad someone else is understanding the problem now, npm install is causing the newest version of a package to be installed, ignoring the lock file. In my eyes this completely defeats the point of having a lock file, and I might as well not include it as it's not locking anything.
@Maziar-Fotouhi I am fully aware of how semver works, I am including the range in my package.json file for when I actually want to do an npm update it will get the newest version of the package based on the range. But as I say, this would be WHEN I choose to do an npm update. Not an npm install which ignores the lock file. Including a range shouldn't matter IF the lock file was actually locking the version specified.
As @wulfsberg put it, what is the difference between install and update if they are both ignoring the lock file and installing the newest version every time?
My original question and still the source of all my issues is, how can I create a stable environment for production testing when every time npm install is run for a fresh install it installs the newest versions of packages?

@wulfsberg is the npm install command (without any other options), changing your package.json content?! If so, that is a bug and very weird!
@kidd3 You are absolutely right. In your case, npm install and npm update are doing the exact same thing. That's why I personally never use a range for the version numbers.

All in all, I think you have a valid point as right now there is no way (at least that I know of!) to just read from the lock files and not the package.json. Maybe a new option can be added to the npm install command in future (E.g. --lock or --read-only) to just read the lock file and disregard the package.json.

But this is not the right thread for that. I believe a new feature request must be created for that.

For now, you can do what I am doing. Only use exact version numbers. This means another thing to do for you as you will have to update your packages manually, but it also gives you much more control over the details which is always good. Another command that can be useful if you decide to do this, is npm outdated.

@Maziar-Fotouhi No, it is not. I was referring to (and disagreeing with) @albertosantini's #16866 (comment) case 1 and #16866 (comment) which state it should.

@Maziar-Fotouhi thank you for your suggestion. Having a way to create a static and repeatable installation is essential for testing as currently with version 5.3.0 I can not do this. A new flag such as --lock or --read-only would allow npm install to ONLY install from the lock file, and would then give a repeatable installation every time. How can I request this? It is something I feel is incredibly important as currently version 5.3.0 is completely unusable in this situation.

@kidd3 I agree with you. Let's continue this discussion in #17979 that you created.

Glad we were able to sort that out, but I feel really bad for those coming from Google and seeing this mess of (mostly true) information on package locks. @zkat or another maintainer might consider locking this issue.

It seems to me that there's a lot of differing use cases and personal preferences for the package lock file. Generally people like it, but the default behavior doesn't work for everyone. I'm highly doubtful we'll ever be able to find a default behavior that works for everyone.

So why not make the package lock configurable and manually controllable?

For example, why not have a configurable field in .npmrc that lets users disable package-lock? Or another option that you require explicit updates to package-lock.json? (this latter approach is my personal preference)

And then provide devs tools to manually run the package-lock algorithm. Like

npm update-lock mycoolmodule

Those of us who opt non-default behavior will still need a way to configure our package-lock to our liking.

What do you think of this two pronged proposal?

1. Make package-lock configurable via .npmrc or some other way
2. Provide low level commands to update and set package-lock

Oh, good point that package-lock.json can already be turned off in .npmrc. But that is the only configuration possible. On or off. It needs to be more tunable. For instance, a setting to make package-lock only update when explicitly updated:

package-lock=explicit-only

And then commands to tune the package lock, such as:

npm update-lock mycoolmodule

Tldr, the fact that .npmrc lets us turn package-lock on or off doesn't get us nearly close enough to making the package-lock workflow customizable.

@scottmas please see this comment #17979 (comment) by @iarna.

should we turn off the package-lock in .npmrc or not to avoid install errors?

Has there been any consensus here? I participated in this thread months ago but today got completely ass-bitten by package-lock. Are there any behaviour changes between npm 5.0.x and 5.5.x?

@NathanCH It looked like things were fixed in 5.4.x

What happened?

See #18286 for the addition of a --from-lock-file (or similar) flag that would install only from the package-lock.json without modifying it.

I can confirm that clearing the package lock file, the issue is solved

If anyone needs to support Windows, this appears to work for our team:

{
  "scripts": {
    "clean": "rm -Rf node_modules/ && rm -f ./package-lock.json && npm cache clean -f",
    "clean_windows": "IF EXIST node_modules rd /s /q node_modules && IF EXIST package-lock.json DEL package-lock.json && npm cache clean -f",
    "rebuild": "npm run clean && npm i",
    "rebuild_windows": "npm run clean_windows && npm i"
  }
}

After running npm i <some-package>, those on Windows machines use npm run rebuild_windows to ensure the application's dependencies are actually updated.

I understand that manually updating package.json won't update package-lock.json.

It is for issues like this that we need package-lock.json removed. Yarn uses yarn.lock and won't stand in our way if we modified package.json

Manually updating a package.json should be updating package-lock.json as of several versions ago. If you're not seeing these updates, please first update to the latest npm version before submitting a new issue with specifics. In the meantime, I'm going to lock this thread because I think it'll only be confusing to others if there's folks saying otherwise due to misunderstanding or older versions.