pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!

dstandish · 2026-03-23T22:07:12Z

The Compile / format / lint UI pre-commit hook runs pnpm install --frozen-lockfile on every invocation, even when package.json and pnpm-lock.yaml haven't changed. This is the main source of slowness in the hook.

This PR adds a hash-based guard: SHA256 of package.json + pnpm-lock.yaml is stored in .build/ui/pnpm-install-hash.txt (consistent with the caching pattern already used in compile_ui_assets.py). When the hash matches and node_modules/ exists, pnpm install is skipped entirely.

Benchmarks

Measured on macOS Darwin 25.3.0, Apple Silicon, pnpm 9.9.0:

Scenario	Wall clock	Notes
Old: `pnpm install` (cold pnpm store)	~22 s	First-ever install, downloading from registry
Old: `pnpm install` (warm pnpm store)	~1.0–1.5 s	Typical dev scenario — "Already up to date" fast path
PR: cache miss (hash mismatch / no hash file)	~1.0–1.5 s + 0.08 s	Negligible overhead; still runs install
PR: cache hit (hash matches, node_modules exists)	~0.07 s	Pure Python hash check, skips install entirely

Typical dev savings: ~1.0–1.4 s per commit (~14–20x faster on the hot path) for contributors who frequently touch UI files without changing deps — the common case.

Cold-store savings (~22 s) apply on a fresh checkout. Cache miss adds only ~0.08 s of overhead before falling through to the normal install path.

The hash covers both package.json and pnpm-lock.yaml, so any direct or transitive dep change correctly invalidates the cache.

Was generative AI tooling used to co-author this PR?

Yes — Claude Sonnet 4.6

Generated-by: Claude Sonnet 4.6 following the guidelines

Hashing package.json + pnpm-lock.yaml and caching the result in .build/ui/pnpm-install-hash.txt allows the hook to skip the expensive pnpm install step when neither file has changed and node_modules already exists. This mirrors the hash-based caching pattern already used in compile_ui_assets.py.

dstandish · 2026-03-24T16:52:40Z

@potiuk not a huge time savings but 1s is something

bugraoz93

It looks good! Only added a nit on utilising the common processes to leverage the same file rather than creating additional one although it is not blocking

potiuk · 2026-03-24T23:50:23Z

@potiuk not a huge time savings but 1s is something

Oh absolutely - anythind we can shave-off from a thing that is executed 1000s of times but 100s of people is good.

Copilot

Pull request overview

This PR speeds up the ts-compile-lint-ui prek hook by avoiding repeated pnpm install runs when UI dependencies have not changed, using a stored SHA256 hash of package.json + pnpm-lock.yaml as a cache key.

Changes:

Compute a SHA256 hash of airflow-core/src/airflow/ui/package.json and pnpm-lock.yaml.
Skip pnpm install --frozen-lockfile when the stored hash matches and node_modules/ exists.
Persist the computed hash in .build/ui/pnpm-install-hash.txt after a successful install.

Copilot · 2026-04-02T00:48:16Z

+    hash_file = AIRFLOW_ROOT_PATH / ".build" / "ui" / "pnpm-install-hash.txt"
+    node_modules = dir / "node_modules"
+    current_hash = hashlib.sha256(
+        (dir / "package.json").read_bytes() + (dir / "pnpm-lock.yaml").read_bytes()
+    ).hexdigest()
+    stored_hash = hash_file.read_text().strip() if hash_file.exists() else ""
+
+    if node_modules.exists() and current_hash == stored_hash:
+        print("pnpm deps unchanged — skipping install.")
+    else:
+        run_command(["pnpm", "config", "set", "store-dir", ".pnpm-store"], cwd=dir)
+        run_command(["pnpm", "install", "--frozen-lockfile", "--config.confirmModulesPurge=false"], cwd=dir)


The cache-hit condition can become a false positive if node_modules/ is modified outside this hook (e.g., running pnpm install manually or via another prek hook) without updating .build/ui/pnpm-install-hash.txt. In that case current_hash == stored_hash can still be true while node_modules no longer corresponds to the current lockfile, causing the hook to skip the needed install and then run eslint/tsc against mismatched deps. Consider adding an extra validity check tied to the actual node_modules state (e.g., compare timestamps, or read pnpm’s node_modules/.modules.yaml lockfile checksum) before skipping.

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

Copilot · 2026-04-10T19:57:34Z

+    current_hash = hashlib.sha256(
+        (dir / "package.json").read_bytes() + (dir / "pnpm-lock.yaml").read_bytes()
+    ).hexdigest()


Hashing by simple byte concatenation can (in rare but real cases) produce the same combined byte string for different (package.json, pnpm-lock.yaml) pairs (e.g., if bytes are shifted between files), causing a false cache hit and skipping a required install. To make the hash unambiguous, include a delimiter and/or the length of each file, or hash each file separately and combine the digests.

Suggested change

current_hash = hashlib.sha256(

(dir / "package.json").read_bytes() + (dir / "pnpm-lock.yaml").read_bytes()

).hexdigest()

package_json_bytes = (dir / "package.json").read_bytes()

pnpm_lock_bytes = (dir / "pnpm-lock.yaml").read_bytes()

current_hash_hasher = hashlib.sha256()

current_hash_hasher.update(f"package.json:{len(package_json_bytes)}:".encode("utf-8"))

current_hash_hasher.update(package_json_bytes)

current_hash_hasher.update(f"pnpm-lock.yaml:{len(pnpm_lock_bytes)}:".encode("utf-8"))

current_hash_hasher.update(pnpm_lock_bytes)

current_hash = current_hash_hasher.hexdigest()

Copilot · 2026-04-10T19:57:35Z

+        hash_file.parent.mkdir(parents=True, exist_ok=True)
+        hash_file.write_text(current_hash)


Writing the hash file non-atomically can leave a truncated/partial hash if the process is interrupted mid-write (or if multiple runs overlap), leading to unnecessary reinstalls or inconsistent behavior. Prefer an atomic write pattern (write to a temp file in the same directory, then replace/rename) to make the cache robust.

Copilot · 2026-04-10T19:57:35Z

+    ).hexdigest()
+    stored_hash = hash_file.read_text().strip() if hash_file.exists() else ""
+
+    if node_modules.exists() and current_hash == stored_hash:


node_modules.exists() will also be true if there’s a file (or unexpected path type) named node_modules. Using node_modules.is_dir() (or checking exists() + is_dir()) makes the guard more precise and avoids incorrectly skipping installs in edge cases.

Suggested change

if node_modules.exists() and current_hash == stored_hash:

if node_modules.is_dir() and current_hash == stored_hash:

boring-cyborg Bot added area:dev-tools backport-to-v3-1-test labels Mar 23, 2026

dstandish marked this pull request as ready for review March 24, 2026 16:38

dstandish requested review from amoghrajesh, ashb, bugraoz93, gopidesupavan, jason810496, jscheffl and potiuk as code owners March 24, 2026 16:38

bugraoz93 approved these changes Mar 24, 2026

View reviewed changes

Comment thread scripts/ci/prek/ts_compile_lint_ui.py

potiuk approved these changes Mar 24, 2026

View reviewed changes

kaxil requested a review from Copilot April 2, 2026 00:43

Copilot started reviewing on behalf of kaxil April 2, 2026 00:44 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

pierrejeambrun removed the backport-to-v3-1-test label Apr 8, 2026

kaxil requested a review from Copilot April 10, 2026 19:55

Copilot AI reviewed Apr 10, 2026

View reviewed changes

pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Skip pnpm install in UI pre-commit hook when deps are unchanged#64124

Skip pnpm install in UI pre-commit hook when deps are unchanged#64124
dstandish wants to merge 1 commit into
apache:mainfrom
astronomer:worktree-skip-pnpm-install-when-deps-unchanged

dstandish commented Mar 23, 2026 •

edited

Loading

Uh oh!

dstandish commented Mar 24, 2026

Uh oh!

bugraoz93 left a comment

Uh oh!

Uh oh!

potiuk commented Mar 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 2, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Copilot AI Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.

-    current_hash = hashlib.sha256(
-        (dir / "package.json").read_bytes() + (dir / "pnpm-lock.yaml").read_bytes()
-    ).hexdigest()
+    package_json_bytes = (dir / "package.json").read_bytes()
+    pnpm_lock_bytes = (dir / "pnpm-lock.yaml").read_bytes()
+    current_hash_hasher = hashlib.sha256()
+    current_hash_hasher.update(f"package.json:{len(package_json_bytes)}:".encode("utf-8"))
+    current_hash_hasher.update(package_json_bytes)
+    current_hash_hasher.update(f"pnpm-lock.yaml:{len(pnpm_lock_bytes)}:".encode("utf-8"))
+    current_hash_hasher.update(pnpm_lock_bytes)
+    current_hash = current_hash_hasher.hexdigest()

		hash_file.parent.mkdir(parents=True, exist_ok=True)
		hash_file.write_text(current_hash)

	if node_modules.exists() and current_hash == stored_hash:
	if node_modules.is_dir() and current_hash == stored_hash:

pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!

Conversation

dstandish commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Was generative AI tooling used to co-author this PR?

Uh oh!

dstandish commented Mar 24, 2026

Uh oh!

bugraoz93 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

potiuk commented Mar 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.

dstandish commented Mar 23, 2026 •

edited

Loading