Memory leak #957
Description
Describe the bug
Over the last few weeks I've been trying to figure out lage memory usage and why it uses so much memory. Currently for us lage uses close to 3 GB of RAM (the RSS of the process) even when every step is skipped.
I added some logging to the lage run and found that the RSS grows steadily throughout the run. In the earlier tasks it is using ~500MB and then the RSS grows by some number of MB per task that is run.
Since we have several hundred tasks, this adds up to a lot of memory overall, in small steps.
To Reproduce
Steps to reproduce the behavior:
- Have a monorepo with a lot of tasks
- Run a build to populate the cache fully
- Run another build that skips everything, monitoring the memory usage (may want to patch lage to do some more logging)
- Observe memory usage grows continuously through the build process
Expected behavior
When all tasks are skipped and no new workers are being launched, the memory usage should be stable.