Add buildx disk usage logging after buildkit is ready #44
Conversation
- Add logBuildxDiskUsage() function to log docker buildx du --verbose output - Displays all cache entries in build order to correlate with Dockerfile structure - Includes 5-second timeout to prevent hanging - Shows formatted sizes and identifies shared cache entries - Helps identify cached layers for debugging and optimization
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| }; | ||
|
|
||
| return value * (multipliers[unit] || 1); | ||
| } |
There was a problem hiding this comment.
Bug: Disk Size Parsing Fails Without 'B' Suffix
The parseSizeToBytes function's regex allows size units like 'K', 'M', 'G', or 'T' without a 'B' suffix, but the multipliers object only includes units with 'B' (e.g., 'KB'). This mismatch means sizes from docker buildx du output are incorrectly treated as bytes, leading to miscalculated disk usage.
|
@eltonkl this is an example run let me know if theres a better way to output this data https://github.com/FastActions/fa/pull/2385 |
| for (const line of lines) { | ||
| // Parse lines that contain cache IDs and sizes. | ||
| // Expected format: ID SIZE [SHARED] DESCRIPTION | ||
| const match = line.match(/^(\S+)\s+(\d+(?:\.\d+)?[KMGT]?B?)\s+(.*?)$/); | ||
| if (match) { | ||
| const [, id, sizeStr, description] = match; | ||
| const isShared = description.includes("shared"); |
There was a problem hiding this comment.
The parsing logic doesn't work as is.
On the version of buildx we install by default (v0.23.0), docker buildx du --verbose outputs
ID: x4eh4xn1wx4s5ypq2qs0fmkva
Parent: vvkxylkn6pgzuochbi3jp6d75
Created at: 2025-09-23 18:41:43.506943453 +0000 UTC
Mutable: false
Reclaimable: true
Shared: false
Size: 359.2MB
Description: pulled from docker.io/library/golang:1.21@sha256:4746d26432a9117a5f58e95cb9f954ddf0de128e9d5816886514199316e4a2fb
Usage count: 1
Last used: 2 weeks ago
Type: regular
Starting from v0.29.0, it uses this template:
https://github.com/docker/buildx/blob/beaebcbf39b9f61cda762669083450c65470ec00/commands/diskusage.go#L38-L60
| stateHelper.setTmpDir(Context.tmpDir()); | ||
| }, | ||
| // post action - cleanup | ||
| async () => { |
There was a problem hiding this comment.
Post action it would be useful to log again using https://docs.docker.com/reference/cli/docker/buildx/du/#filter using --until= + the time difference between the start of the job and the current time. That should indicate exactly which records were used by image builds during this job
There was a problem hiding this comment.
very nice idea, will add this in
2 participants
This PR adds logging for
docker buildx du --verboseafter buildkit accepts connections to help identify cached layers.Changes
logBuildxDiskUsage()function that runsdocker buildx du --verbosewith a 5-second timeoutBenefits
Note
Adds
logBuildxDiskUsage()and invokes it post-builder setup to report cached layer usage with parsed, formatted output and timeout.src/main.ts):logBuildxDiskUsage()to print Buildx cache disk usage.src/setup_builder.ts):logBuildxDiskUsage()to rundocker buildx du --verbosewith 5s timeout, parse entries, sum total size, and mark shared caches.parseSizeToBytes()andformatBytes()for size parsing/formatting.Written by Cursor Bugbot for commit a9aabc0. This will update automatically on new commits. Configure here.