fix: add timeout to buildx export operation #1395

adityamaru · 2025-08-04T22:48:13Z

Summary

Add 30-second timeout to buildxHistory.export() operation
Prevent workflow jobs from hanging when buildx dial-stdio crashes
Continue with build reporting even if export fails

Problem

Customer reported workflow jobs timing out with "broken pipe" errors when the buildx backend process becomes unresponsive during the export phase.

Solution

Implemented a timeout wrapper using Promise.race() that:

Times out after 30 seconds if the export operation hangs
Logs a warning but continues with build reporting
Gracefully handles export failures without blocking the entire build

Test plan

Build completes successfully
Linting passes
TypeScript compilation succeeds
Test with actual failing buildx export scenario

🤖 Generated with Claude Code

1. Checks we have buildx installed 2. Configures a remote builder if we get an address back 3. Uses the already configured builder if we don't get an address back This change does not plumb the dockerfile path through as the entity, and does not differentiate a failed build from a succesful to report to anvil in the post step yet.

…llback

*: basic scaffolding for build-push-action

* tls * set up tls while creating the remote builder

This change teaches the build push action to request a stickydisk every time it runs. Once the SD is hotloaded the VM will mount the buildkit root dir and starts buildkitd.

src: add a retry with backoff to combat 429s when downloading buildkit

*: allow users to pass in a buildx version

action: add missing option string

The remote builder was hardcoded to use --platform linux/amd64 regardless of user input or runner architecture. This caused performance issues on ARM runners and cache inefficiencies. Now properly uses the platforms input or detects host architecture to avoid unnecessary QEMU emulation and improve build performance. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

The test was hardcoded to expect arm64 platform, causing failures on AMD runners. Now checks actual host architecture dynamically. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

fix: use correct platform when creating remote buildx builder

Signed-off-by: Aditya Maru <[email protected]>

src: use BLACKSMITH prefixed VM ID env var

src: only prune if buildkitd was spun up

- Remove buildkitd startup and configuration logic - Remove buildkitd shutdown and cleanup from both main and post actions - Remove buildkitd-related imports and helper functions - Update startBlacksmithBuilder to check for existing builder from setup-docker-builder - Keep sticky disk setup and build reporting functionality intact BREAKING CHANGE: This action now requires setup-docker-builder to be run first to manage the Docker builder lifecycle

- Remove sticky disk mounting and unmounting logic - Remove sticky disk commit logic from both main and post actions - Replace setupStickyDisk with reportBuildStart to only report build start - Update build completion reporting to not depend on exposeId - Keep build tracking and reporting functionality intact The sticky disk lifecycle is now fully managed by setup-docker-builder

This commit completes the refactoring of build-push-action to focus solely on Docker build reporting and metrics, with all infrastructure management moved to the separate setup-docker-builder action. Changes: - Remove all setupOnly references from context.ts, main.ts, and state-helper.ts - Rename startBlacksmithBuilder to reportBuildMetrics to better reflect its purpose - Remove exposeId from all function signatures and state management - Remove sticky disk commit logic from reporter.ts - Update tests to match new function names and signatures - Clean up unused imports and fix linting issues The action now assumes that a Docker builder has already been configured (either via setup-docker-builder or existing setup) and focuses only on: - Running Docker builds with the configured builder - Reporting build metrics and status to Blacksmith API - Managing build outputs and metadata 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

This commit completes the cleanup of build-push-action by removing all the buildkit and sticky disk setup logic that has been moved to the separate setup-docker-builder action. Changes: - Delete setup_builder.ts which contained 380+ lines of unused code - Create new build-reporter.ts with only the reportBuildStart function - Update all imports to use the new build-reporter module - The new file name better reflects its single responsibility The action is now cleaner and more focused, with infrastructure setup logic properly separated into the setup-docker-builder action. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

Inline the reportBuildStart function directly into main.ts since it was just a thin wrapper around reporter.reportBuild. This removes an unnecessary abstraction layer and makes the code simpler. Changes: - Delete build-reporter.ts file - Inline the reportBuild logic directly in reportBuildMetrics function - Update tests to mock reporter.reportBuild directly - Fix test expectations to match the new error messages The code is now cleaner with one less file and abstraction layer. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

Since setup-docker-builder handles all infrastructure setup, build-push-action no longer needs to: - Install buildx (just assert it's available) - Manage temporary directories (handled by actions toolkit) Changes: - Replace buildx installation with simple availability assertion - Remove tmpDir state management entirely - Remove buildx-version input parameter - Clean up unused imports and functions The action now assumes buildx is already configured by setup-docker-builder or another setup action, making it simpler and more focused. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

Remove the nofallback input parameter as it's no longer needed. The action now assumes that a builder is already configured and available. Changes: - Remove nofallback from action.yml inputs - Remove nofallback from Inputs interface and getInputs function - Update tests to remove nofallback references - Also remove setup-only and buildx-version from action.yml (already removed from code) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

- Fix axios-retry type compatibility issues in reporter.ts - Remove unused imports (Context, Metric_MetricType) - Update test expectations to match current implementation - Fix ESLint errors and apply formatting 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

- Rename reportBuildMetrics to reportBuildStart for clarity - Check if builder name contains 'blacksmith' before reporting - Log warning when not using a Blacksmith builder - Skip build start and completion reporting for non-Blacksmith builders - Update tests to reflect function rename 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

- Move builder variable declaration to outer scope - Use optional chaining for builder.name in buildRef call - Fixes runtime error where builder was undefined outside its declaration scope 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

@ts-expect-error

- The @ts-expect-error directive was causing CI failures - npm dependencies don't have the type mismatch that pnpm dependencies had - Verified all tests and builds pass with npm commands matching CI 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

- v2 is a breaking version requiring setup-docker-builder to be called first - Commenting out workflow to prevent accidentally bumping v1 tags to breaking changes - Workflow kept as reference for future tag bumping procedures 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

…anagement refactor: split build-push-action to separate infrastructure setup

Add a 30-second timeout to the buildxHistory.export() call to prevent workflow jobs from hanging indefinitely when the buildx dial-stdio process crashes or becomes unresponsive. The build will continue with reporting even if the export fails or times out. Fixes issue where docker run export-build command would hang with "broken pipe" errors when buildx backend is unavailable. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

adityamaru and others added 30 commits September 11, 2024 20:08

src: add logic to report differently on success and failure

fca077e

plumb through the dockerfile path when creating a build_task

cb250fe

configure a local builder using the docker-container driver as a fa…

595f5a7

…llback

Merge pull request #1 from useblacksmith/wireup-remote-docker-build

c2b089e

*: basic scaffolding for build-push-action

log specific warning when no builder instances are available (#2)

79167f5

point to anvil staging environment (#3)

fd041da

log anvil url (#4)

ea26987

npm run build (#5)

d135977

point to fly staging (#6)

84699d1

add a fallback input (#7)

294f759

fix typo (#8)

30b096e

log the submitted build task id

2d18077

additional logging

242068a

retry ephemeral anvil errors

bd6be25

misc cleanup

f261a98

improve error logs

b15cec1

log abandonment url

2b966d1

plumb task id through to abandon call

828fa88

include e2e builder launch time the complete request payload (#9)

cb07435

normalize dockerfile paths

5158225

log the prefix task id since its more user readable

c675e14

revert back to logging task id

717de04

retry on ephemeral http errors (#10)

2fe866c

only retry build task submission on ephemeral server errors (#11)

9c81a9e

improve error logging

6960b24

tls (#12)

3f6698f

* tls * set up tls while creating the remote builder

rely on GITHUB_REPOSITORY to get the repo name

6f17c30

point staging runs to staging anvil

6f9ad79

*: teach action to hot load sticky disks

3b1df39

This change teaches the build push action to request a stickydisk every time it runs. Once the SD is hotloaded the VM will mount the buildkit root dir and starts buildkitd.

adityamaru and others added 28 commits May 18, 2025 16:29

Merge pull request #115 from useblacksmith/buildx-ratelimit

e09a088

src: add a retry with backoff to combat 429s when downloading buildkit

*: allow users to pass in a buildx version

28c2447

Merge pull request #116 from useblacksmith/bump-buildkitd

68105fc

*: allow users to pass in a buildx version

action: add missing option string

d2acb06

Merge pull request #117 from useblacksmith/add-missing-option

6fe3b1c

action: add missing option string

.github: add docker workflow

51bef8c

Merge pull request #118 from useblacksmith/builder-misconfig

4fac798

fix: use correct platform when creating remote buildx builder

.github: add v1.2 to bump_tags_to_master.yaml

574eb0e

Signed-off-by: Aditya Maru <[email protected]>

src: use BLACKSMITH prefixed VM ID env var

a037e6f

Merge pull request #120 from useblacksmith/change-env

4e97fd3

src: use BLACKSMITH prefixed VM ID env var

src: only prune if buildkitd was spun up

f9f71c9

Merge pull request #119 from useblacksmith/grpc-fallbakc

ac765fe

src: only prune if buildkitd was spun up

Merge pull request #127 from useblacksmith/refactor/remove-buildkit-m…

c213746

…anagement refactor: split build-push-action to separate infrastructure setup

github-actions bot assigned adityamaru Aug 4, 2025

adityamaru closed this Aug 4, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: add timeout to buildx export operation #1395

fix: add timeout to buildx export operation #1395

adityamaru commented Aug 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fix: add timeout to buildx export operation #1395

fix: add timeout to buildx export operation #1395

Conversation

adityamaru commented Aug 4, 2025

Summary

Problem

Solution

Test plan

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants