Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WSL 2 consumes massive amounts of RAM and doesn't return it #4166

Open
LordMonoxide opened this issue Jun 17, 2019 · 355 comments
Open

WSL 2 consumes massive amounts of RAM and doesn't return it #4166

LordMonoxide opened this issue Jun 17, 2019 · 355 comments
Assignees
Labels
wsl2 Issue/feature applies to WSL 2

Comments

@LordMonoxide
Copy link

LordMonoxide commented Jun 17, 2019

  • Your Windows build number: 18917

  • What's wrong / what should be happening instead: WSL 2 starts using huge amounts of RAM after a while, just using it like normal. At the moment I'm using phpstorm, and did a dump/load of a database. Vmmem is using 7 GB of my 16 GB of RAM and not returning any, even though Ubuntu is actually using much less. I have seen it grow until nearly 100% of my system memory is in use, and it will not release it until I shut down the WSL 2 VM.

This may or may not be related to #4159

corey@Corey-Laptop:/mnt/c/WINDOWS/system32$ vmstat -s
     15235516 K total memory
       920348 K used memory
      1886048 K active memory
      6434312 K inactive memory
      6606548 K free memory
        76280 K buffer memory
      7632340 K swap cache
            0 K total swap
            0 K used swap
            0 K free swap
       163729 non-nice user cpu ticks
          298 nice user cpu ticks
        13177 system cpu ticks
     68988300 idle cpu ticks
         8962 IO-wait cpu ticks
            0 IRQ cpu ticks
        10022 softirq cpu ticks
            0 stolen cpu ticks
      1481417 pages paged in
      6792976 pages paged out
            0 pages swapped in
            0 pages swapped out
      1079177 interrupts
      5131981 CPU context switches
   1560599814 boot time
         8772 forks
@Drakota
Copy link

Drakota commented Jun 17, 2019

Same here with Docker running on Ubuntu 18.04
image

@benhillis
Copy link
Member

Thanks for opening the issue. We have a fix for this in the works.

@mithunshanbhag
Copy link

mithunshanbhag commented Jun 19, 2019

After moving to WSL2, VmMem seems to be constantly pegging my CPU. Please see attached screenshots.

Anything I can do to help your team troubleshoot?

Relevant details:

  • Ubuntu 18.04 and Debian on WSL2, but none are running anything at the moment.
  • Windows insider build 18917.1000 on Macbook Air (8 GB, 4 proc).

image
image

@crispinb
Copy link

crispinb commented Jul 2, 2019

After moving to WSL2, VmMem seems to be constantly pegging my CPU. Please see attached screenshots.

I'm experiencing this too. It happens every time I run wsl after some elapsed time (varying from minutes to hours). It continues regardless of what's running in wsl, and I need to issue a wsl --shutdown to stop it.

Windows insider build: 18922.1000
Linux distro: 18.04

I assume this really belongs in a new issue, but wanted to group my comment with that of @mithunshanbhag so I'll leave it here.

@md0x
Copy link

md0x commented Jul 22, 2019

Hi,
Same problem here.
Lots of memory consumption with WSL2, nodejs app and VS Code with Remote - WSL while very low consumption in WSL1 with the same tasks.

@liyo
Copy link

liyo commented Aug 1, 2019

Same issue on Win build 18950

@zachChilders
Copy link

@benhillis - Do we have a rough ETA on the fix? I'm actually hitting OOM issues on a machine with 32 gigs while just running a normal ninja build that should succeed.

Build 18955

@benhillis
Copy link
Member

@zachChilders - ETA on this one is a bit hard unfortunately. Certainly before WSL2 ships to non-insiders.

@avafloww
Copy link

Can confirm this on 18963 as well. WSL 2 ate most (29GB+ of 32GB) of my RAM after less than 15 minutes of uptime for no good reason; all I've done is a rsync from the host NTFS volume to the ext4 volume, and an apt-get update.

@david-dumke
Copy link

I can confirm the same issue on Microsoft Windows [Version 10.0.18963.1000]

C:\Users\david_d>wsl -l -v
NAME STATE VERSION

  • Ubuntu-18.04 Running 2
    Ubuntu Stopped 1

image

I at the time was running gdb

@rob-solana
Copy link

rob-solana commented Aug 22, 2019

how about a workaround? this just started happening for me (with 18965) and it has effectively killed my WSL 2 environment

@mbc07
Copy link

mbc07 commented Aug 22, 2019

Depending of your usage, nocache utility can be used as a workaround to greatly reduce WSL 2 memory consumption, especially if whatever you're running inside it does a lot of filesystem operations...

@rob-solana
Copy link

does nocache follow children? I could start my shell with "nocache"...

@mbc07
Copy link

mbc07 commented Aug 22, 2019

Haven't tried, but if I understood how nocache works correctly, it should...

@rob-solana
Copy link

thanks! nocache has changed my env from "completely broken" to "merely sucks"

@Beretta1979
Copy link

Beretta1979 commented Aug 27, 2019

thanks! nocache has changed my env from "completely broken" to "merely sucks"

Hi, could you explain how nocache should be used?

@willbuilds
Copy link

Hi @benhillis , any update on when a fix for this might be deployed to insiders? My 8gb of ram is really getting hammered by this issue... :(

@rob-solana
Copy link

I wrap compilations in nocache to keep limping along. wrapping nocache around a shell works, but breaks emacs. I've also ordered more RAM :-(

@benhillis
Copy link
Member

Sorry no firm timeline, certainly before WSL2 is out of Insider builds though.

@fabienheureux
Copy link

fabienheureux commented Aug 29, 2019

Hi @benhillis , do you have a workaround you could recommend though ?

@fabienheureux
Copy link

Well...I need it all day long and the memory consumption goes crazy after only a few minutes (using docker and a quite heavy stack: elastic search, lots of workers, some databases etc). So...not an option unfortunately. Maybe go back to linux while it is not fixed 😎

@weberdominik
Copy link

I am facing the same issue. Is there any workaround (except shutting wsl down) for this @benhillis ?

@microsoft microsoft deleted a comment from md0x Aug 29, 2019
@benhillis
Copy link
Member

Not currently. For some additional context this change requires some changes to the Linux kernel that are in the process of being upstreamed. We will be integrating these changes into the WSL2 kernel as soon as we can.

@apostolos
Copy link

Workaround: Create a %UserProfile%\.wslconfig file in Windows and use it to limit memory assigned to WSL2 VM.

Example

[wsl2]
memory=6GB
swap=0
localhostForwarding=true

This will still consume the entire 6GBs regardless of Linux memory usage, but at least it'll stop growing more than that.

Supported settings are documented here.

@squareloop1
Copy link

We've turned on autoMemoryReclaim=dropCaches by default for the latest WSL version and updated the inactivity time to 10 minutes to help with this issue. This will make WSL by default return its used memory after 10 minutes of inactivity.
If you're seeing this issue, please can you upgrade to the latest with wsl --update --pre-release and try out the new behaviour and let us know your feedback!

Tested this over the period of a couple days, this seems to release the memory in most cases after 10 minutes, however, running Docker seems to prevent memory being released, even after exiting and waiting out the 10 minute inactivity period.

100% agree on this. Docker when using WSL2 as backend is a huge pain as regards memory consumption. When not running docker however, I observed the same as others have already mentioned. Memory is freed relatively quickly. Maybe this now relate to the Docker dev team but both teams have to maybe communicate better on this...

You can drop Docker Desktop for Windows all together and use native docker engine in WSL - which will be more efficient and less ram hungry. I did it more then 6 month ago and not looking back.

Does this have any negative side effects to be aware of?

@yohosuff
Copy link

yohosuff commented Mar 4, 2024

We've turned on autoMemoryReclaim=dropCaches by default for the latest WSL version and updated the inactivity time to 10 minutes to help with this issue. This will make WSL by default return its used memory after 10 minutes of inactivity.
If you're seeing this issue, please can you upgrade to the latest with wsl --update --pre-release and try out the new behaviour and let us know your feedback!

Tested this over the period of a couple days, this seems to release the memory in most cases after 10 minutes, however, running Docker seems to prevent memory being released, even after exiting and waiting out the 10 minute inactivity period.

100% agree on this. Docker when using WSL2 as backend is a huge pain as regards memory consumption. When not running docker however, I observed the same as others have already mentioned. Memory is freed relatively quickly. Maybe this now relate to the Docker dev team but both teams have to maybe communicate better on this...

You can drop Docker Desktop for Windows all together and use native docker engine in WSL - which will be more efficient and less ram hungry. I did it more then 6 month ago and not looking back.

This right here. Also, this: https://github.com/jesseduffield/lazydocker

@gund
Copy link

gund commented Mar 4, 2024

We've turned on autoMemoryReclaim=dropCaches by default for the latest WSL version and updated the inactivity time to 10 minutes to help with this issue. This will make WSL by default return its used memory after 10 minutes of inactivity.
If you're seeing this issue, please can you upgrade to the latest with wsl --update --pre-release and try out the new behaviour and let us know your feedback!

Tested this over the period of a couple days, this seems to release the memory in most cases after 10 minutes, however, running Docker seems to prevent memory being released, even after exiting and waiting out the 10 minute inactivity period.

100% agree on this. Docker when using WSL2 as backend is a huge pain as regards memory consumption. When not running docker however, I observed the same as others have already mentioned. Memory is freed relatively quickly. Maybe this now relate to the Docker dev team but both teams have to maybe communicate better on this...

You can drop Docker Desktop for Windows all together and use native docker engine in WSL - which will be more efficient and less ram hungry. I did it more then 6 month ago and not looking back.

Does this have any negative side effects to be aware of?

For me it did not cause any unexpected issues at all.
It did however change how networking works, to match how it's working natively on linux, for example in my compose files I had to explicitly specify what is host.docker.internal, as it's available in docker desktop out of the box but not in docker engine.
Ah also when I was on Docker Desktop my Devcontainers networking was not working properly, and switching to docker engine just fixed them to run smoothly without any bugs.

@knowhatamine
Copy link

This is working as intended.

Microsoft is unable to deliver performance when integrating different filesystems.

So they decided to keep everything that is aaccessed in memory... Surprised this isn't common knowledge.

@scruel
Copy link

scruel commented Mar 20, 2024

image
So cool! :)

Windows 11 Work Statison 23H2 10.0.22631.3296
Ubuntu 22.04 jammy(on the Windows Subsystem for Linux) x86_64 Linux 5.15.133.1-microsoft-standard-WSL2
WSL 2.0.9.0, 5.15.133.1-1
WSLg 1.0.59

@luciorq
Copy link

luciorq commented Mar 20, 2024

image
So cool! :)

Don't Panic! Your ram is fine!

This is working as intended, you have 8GB of unused RAM and Linux is using it as cache, this is a feature in Linux, not a bug. This makes the system faster, not slower.

What you need to test is when Windows requests the RAM back if WSL2 is releasing it.

And that is what the configuration wsl2.autoMemoryReclaim=drop is supposed to allow.

@sarim
Copy link

sarim commented Mar 20, 2024

This is working as intended, you have 8GB of unused RAM and Linux is using it as cache, this is a feature in Linux, not a bug. This makes the system faster, not slower.

Nah. People don't have time to understand your logical answer. Whats more likely is its a conspiracy by microsoft to eat your RAM and do evil things with it. 😉

@jinzzasol
Copy link

We've turned on autoMemoryReclaim=dropCaches by default for the latest WSL version and updated the inactivity time to 10 minutes to help with this issue. This will make WSL by default return its used memory after 10 minutes of inactivity.

If you're seeing this issue, please can you upgrade to the latest with wsl --update --pre-release and try out the new behaviour and let us know your feedback!

When will this be in a stable release? Just a bit cautious about using pre-release. My .wslconfig is using autoMemoryReclaim=dropcache.

@scruel
Copy link

scruel commented Mar 20, 2024

When will this be in a stable release? Just a bit cautious about using pre-release. My .wslconfig is using autoMemoryReclaim=dropcache.

Should already be released in 2.1.4

@jinzzasol
Copy link

jinzzasol commented Mar 20, 2024

When will this be in a stable release? Just a bit cautious about using pre-release. My .wslconfig is using autoMemoryReclaim=dropcache.

Should already be released in 2.1.4

Thanks. My WSL version is 2.1.5 already, so that means I don't need a .wslconfig file to set this option? Can I just remove or disable it to avoid any conflict?

@SebastianSaipSenacor
Copy link

I'm on 2.1.5 (completly uninstalled wsl and re-installed) and wsl still eats up all the memory up to my configured limit (8 GB) in .wslconfig, although my docker containers only need 3 GB. No memory is returned.

@sarim
Copy link

sarim commented Mar 20, 2024

I'm on 2.1.5 (completly uninstalled wsl and re-installed) and wsl still eats up all the memory up to my configured limit (8 GB) in .wslconfig, although my docker containers only need 3 GB. No memory is returned.

probably because wsl doesn't see your VM as being idle. It has a internal setting <X cpu usage for Y min then trigger memory reclaim. So if your processes still using cpu it won't trigger, as expected.

@craigloewen-msft
Copy link
Member

This is in stable release now at 2.1.5! For the folks who are on this release and are still not seeing memory be returned can you help us troubleshoot by doing these things:

  • Stop using your machine (Or just WSL) for 15 minutes so that it is for sure 'idle'
  • If vmmem didn't go down then:
  • Open htop in WSL and take a screenshot of it
  • Check to see if your CPU usage was still high or if you had any other background tasks running
  • Check to see if you had any applications open that were using CPU
  • Post to here your screenshot of htop, as well as the list of any open applications or services so we can try and reproduce this on the WSL team

Thank you!

@ingentismg
Copy link

I am a bit confused.

  1. The documentation still lists the default for autoMemoryReclaim as being disabled: https://learn.microsoft.com/en-us/windows/wsl/wsl-config#experimental-settings
    Maybe this has not yet been updated?
  2. The documentation lists gradual and dropcache as the available options for autoMemoryReclaim. There is no drop as mentioned in the release notes.

Manually setting autoMemoryReclaim=dropcache works. Not setting anything does not seem to do anything for the cache. The only application running/idling in both cases is Docker Desktop for Windows (besides the Ubuntu background tasks).
I should be up to date, WSL version is shown as 2.1.5.0.

@sarim
Copy link

sarim commented Mar 22, 2024

Seeing not many taking @craigloewen-msft 's offer to scientifically showcase microsoft's secret formula (J.K.) for unnecessarily eating RAM, let me take the opportunity to showcase auto memory reclaim working as intended and releasing RAM after using. I'm using gradual setting, and it takes about ~30s to release the RAM. The other setting might result in more aggressive outcome.

Here's the video evidence.
https://youtu.be/P6p6-P6HfzI
stress-ng is used to stress 15GB memory for 10sec. In windows task manager we see memory usage of vmmemwsl process jump to ~16GB memory usage. After stress-ng finishes we see within ~30s WSL successfully reclaims the memory.

Here's my completely subjective personal opinion comment, those who still think WSL2 has RAM eating bug, make sure it is actually a bug, not just user error/skill issue ☮️

@ninbura
Copy link

ninbura commented Mar 22, 2024

Here's my completely subjective personal opinion comment, those who still think WSL2 has RAM eating bug, make sure it is actually a bug, not just user error/skill issue ☮️

The remaining issue is that the cache dropping detection method is global WSL resource utilization. If you're running a process that's constantly utilizing resources within WSL, the memory will never be reclaimed; regardless of if the cached data is actually being accessed by anything.

@Ali-thepro
Copy link

I deleted docker recently, but vmmem is still using a lot of memory, couple days ago it was fine now after fresh boot its using 3 gb of memory. I just have 2 wsl terminals open. If I close them it will go down to 1.5.
image

This is what I get from htop
image

I have 32 gb ram on my laptop but its getting annoying, its taking too much storage. I tried to limit it in .wslconfig but it causes perfomance issues if I set it too low.

@sarim
Copy link

sarim commented Mar 28, 2024

@Ali-thepro the system distro aka wslg is also running a wayland server & stuffs. You can disable wslg if free memory is that important to you. Also linux could use free memory as cache. You can tweak those too, but those are all linux quarks/features, not really WSL issue.

I also have 32GB ram in desktop and gave 16GB in .wslconfig. I can't remember facing a error with 32GB full ram. Seeing your task manager screenshot saying overall memory usage is 50% I'd its very okey, why worry?

@Ali-thepro
Copy link

@sarim

I know it's fine but a couple days ago it wasn't this much. I know right now it's only 3 gb but couple hours ago I shutdown my laptop and as soon as I opened it 6gb was being used.

@Ali-thepro
Copy link

after running wsl-shutdown vmmem goes to 1.5 gb but just opening vscode in wsl it goes back up to 3 gb, this issue never happened before.

@wolf99
Copy link

wolf99 commented Mar 28, 2024

after running wsl-shutdown vmmem goes to 1.5 gb but just opening vscode in wsl it goes back up to 3 gb, this issue never happened before.

IME , Running wsl-shutdown should stop vmmem completely.
If you still see 1.5 GB usage then perhaps your machine is running something else aside from WSL that is impacting this?

@Ali-thepro
Copy link

@wolf99

My mistake, I meant after running wsl after shutting it down it takes 1.5gb

@Ali-thepro
Copy link

image

just ran a small django and front end app on docker 👍, nice stuff wsl team, vmmem was using 10 gigs at one point as well.

@eboraks
Copy link

eboraks commented Apr 12, 2024

Something is not right with WSL2 with Windows version 10.0.22631, Build 22631. After a fresh restart of the system and nothing running on WSL2, it's consuming 8GB. I was able to reduce it to 4GB by running sudo sync; echo 3 | sudo tee /proc/sys/vm/drop_caches

@KoalaBear84
Copy link

KoalaBear84 commented Apr 12, 2024

Yes, I did have something similar, nothing super special about the containers, but this time it did fill up the whole 48GB RAM, by using 14GB or 24GB or so. Don't know anymore. I tried updating Windows, wsl --shutdown etc and rebooting. Rebooting didn't help for long. Now I've finally made the %userprofile%.wslconfig file.

# Settings apply across all Linux distros running on WSL 2
[wsl2]

# Limits VM memory to use no more than 4 GB, this can be set as whole numbers using GB or MB
memory=8GB

[experimental]
autoMemoryReclaim=gradual

Seems for work for the last days. Currently the vmmem 'process' is using only 2774 MB.

@wasuaje
Copy link

wasuaje commented Apr 13, 2024

I'm having the same issue, I'm trying this for a while

# Settings apply across all Linux distros running on WSL 2
[wsl2]

# Limits VM memory to use no more than 4 GB, this can be set as whole numbers using GB or MB
memory=8GB

[experimental]
autoMemoryReclaim=gradual

@moracabanas
Copy link

moracabanas commented Apr 19, 2024

Same issuse as mentioned here:

I deleted docker recently, but vmmem is still using a lot of memory, couple days ago it was fine now after fresh boot its using 3 gb of memory. I just have 2 wsl terminals open. If I close them it will go down to 1.5. image

This is what I get from htop image

I have 32 gb ram on my laptop but its getting annoying, its taking too much storage. I tried to limit it in .wslconfig but it causes perfomance issues if I set it too low.

My fresh ubuntu WSL with autoMemoryReclaim=dropcache

image

My exact same WSL fresh instance afeter just opening vscode:

image

The first time I started to use WSL2 when it came out, it was a breeze. Then Docker Desktop integration was a dream to work with.
Finally, today I cannot just run a single container without taking 90% RAM of my 16Gb and start noticing massive performance drop because Swap start kicking in. Not even to mention about building simple docker images.
Suddenly your .vhdx distro virtual disk will eat your space if you work with large files, python environment and that wont ever be reclaimed back. You can try to compress the .vhdx with DISKPART googling some tutorials, or try the next .wslconfig:

[experimental]
sparseVhd=true

But when that experimental feature came out, it just destroyed my windows system, corrupting random files across my computer. I was forced to format once.
I am giving it a last chance to work with WSL and sparseVhd. But my productivity is hold by a thin rope because of this issue.

I finally uninstalled Docker Desktop and installed Docker CE on WSL2 but the problem persist with VSCode as a RAM hoarder.
Today, after years of recommending wsl to my teammates, my dream development environment is a nightmare and I cannot fix theirs.

I am forced to dual boot again as the last workaround.

@italocjs
Copy link

Same issue:

  • fresh install of win 11 23H2
  • WSL2 + ubuntu (default)
  • installed docker desktop (followed official documentation)
  • 32GB of ram in host, wsl using 8-12gb just by opening a vscode devcontainer.
    code --version 1.88.1 e170252f762678dec6ca2cc69aba1570769a5d39 x64

Before reinstalling windows (just did a fresh reinstall today) i rarely hit 2gb, unfortunately i did not register the exact versions

@craigloewen-msft
Copy link
Member

@moracabanas When you had autoMemoryReclaim=dropCache turned on, did you find it wasn't dropping cache? Even when your machine was idle and you were using it?

We turned that feature on by default in latest WSL pre-release builds and our aim is for it to help tackle the 'WSL holds onto lots of my RAM' problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wsl2 Issue/feature applies to WSL 2
Projects
None yet
Development

No branches or pull requests