Address /nix/store recursive lookup/syscall overhead

[ilsubyeega]

I'm writing this as "umbrella"/mega/tracking issue in order to address entire nixos ecosystem. the content will be modified frequently.

---

As /nix/store is fragmented with lots of directories, Finding single resource usually lookup multiple directories fed by huge environment variables¹.

This makes application to perform ton of filesystem calls to locate resources (more than O(n)), resulting in spam of stat/openat related syscalls where 90% return ENOENT/EINVAL etc.

While this might be somehow trivial for high-end workstations, on low-power hardware(such like laptop), causing startup delays exceeding 4 seconds even in battery balanced preset.² I believe this is resource-wasting task and more worse when using battery eco preset.

---

Background

I've been personally tracking this issue since KDE Plasma environment was slow³, and found out that it has a lot of syscalls to load each resources. I'm using a desktop right now, so i could ignore this, but soon I must use laptop while staying at univ. This didn't happen for me when using Arch which is FHS-compliant distro few years ago before switching to NixOS. Today I became aware of past discussions(2016) and started writing about this because it has not been actively discussed recent.

Link

• This issue (might) covers following nixpkgs issues/PRs: KDE 5 applications are slow #21345 Link dynamic libraries more granularly to Java #21676 Link to libraries through absolute paths? #24844 Excessively long environment variables in kde plasma #126590 Speed up application startup by using ld.so.cache #207061 KDE applications are slow in Plasma #363068 Remove patchelf from stdenv #362353 RFC: Slow dynamic linker resolution of shared libraries #367683
• fzakaria made some research for addressing this (2022):
  • https://github.com/fzakaria/nix-harden-needed, https://github.com/fzakaria/shrinkwrap
  • Mapping Out the HPC Dependency Chaos: https://arxiv.org/abs/2211.05118
• RFC may cover some portions about this (2025): [RFC 0189] Contracts rfcs#189

Some scenarios

WIP. This is not a complete list of scenarios, Going to post later when i got time and will update when found new scenarios.

strace -f dolphin |& tee out
cat out | grep -F .so | wc -l
# results 4954 (loading libraries via ld and also qt's plugin)
cat out | grep -F qt.qm | wc -l
# results 7576

Fontconfig overhead

Adding more fonts makes significantly more loading time. some programs like coppwr do not affect from this as it seems to use static font (by embedded?). The log below is running strace and line counts when installed 20 fonts in nixos system(home manager installed but its configured at nixos side).

strace -f zeditor |& tee out
cat out | grep fontconfig | grep EINVAL | wc -l
# results 318 items
cat out | grep fonts | wc -l
# results 2539 items

rm out
strace -f dolphin |& tee out
cat out | grep share/fonts | grep ENOENT | wc -l
# results 390

Note

I believe there is no oneshot fix to mitigate these but it can satisfy some users which uses potato pc like me.
I use NixOS with home-manager, usually guis are installed in home-manager, just for sure.

Mention ilsubyeega@catgirl.cloud on matrix if needed, i might able to chat.

From my knowledge, i can propose this issue by multiple preset (not much ideal):

• energy efficient: aggressive usage of link farming for low cpu and io usage when runtime, but it trades with build time and disk inode usage.
• storage efficient: rely on some custom dynamic linker or fs cache with negative cache support.
• more security: no changes from now, there is dedicated hardening NixOS documentation.

Footnotes

1. Environment variables like XDG_DATA_DIRS, QT_PLUGIN_PATH, LD_LIBRARY_PATH. also wrapper feed these. ↩

2. Thinkpad T14 Gen1 (Intel), i5-10210U, 16+32GB, samsung nvme ssd(evo), FDE. ↩

3. https://www.reddit.com/r/kde/comments/1h7xr5n/how_can_i_speed_up_of_launching_kdeqt_based/
   KDE applications are slow in Plasma #363068 ↩

[mightyiam]

Full Time Nix | Stable linking with Farid Zakaria