Node.js Ecosystem Deep Dive / Chapter 1

AI Tech /

E00_The_Lockfile_Was_the_Whole_Point

# The Lockfile Was the Whole Point > `package-lock.json` exists because `npm install` is not a pure function — and treating it as one is the most expensive mistake you can make in JavaScript supply-chain management. ## Key Takeaways - `package.json` is a *range*; `package-lock.json` is a *point*. Treat them as different artifacts, not redundant ones. - A transitive publish can change your `node_modules` tree without your `package.json` changing at all. The lockfile is the only thing standing between you and silent resolution drift. - `npm install` is for development; `npm ci` is for build. They are not synonyms. The CI you wrote six months ago is probably wrong. - The npm/pnpm/yarn/bun split is a *design-philosophy* split, not a feature shoot-out. Pick by what your project actually does, not by who's fastest this week. - The lockfile was the whole point. Everything else in the package-management layer — workspaces, registries, semantic-version constraints — exists to make lockfile reproducibility meaningful. --- Imagine you ship a service today. Your `package.json` declares a dependency on `B` with the range `<0.1.0`. `B` declares a dependency on `C` with the range `<0.1.0`. Today, `B@0.0.1` and `C@0.0.1` are the only versions of those packages on the registry. You run `npm install`, your tree resolves, you ship. Three weeks later you ship a hotfix. Your `package.json` is *byte-identical* to the one from three weeks ago. Your `node_modules` is not. B published `0.0.2` the day after your first release, and your second install pulled it. C is still `0.0.1`. The shape of the tree changed. Your code now runs against a transitive dependency you never named. That is the bug. That is *the* bug. ```mermaid flowchart TD A["package.json<br/>A: 0.1.0<br/>deps: { B: <0.1.0 }"] B["package.json<br/>B: 0.0.1<br/>deps: { C: <0.1.0 }"] B2["package.json<br/>B: 0.0.2<br/>deps: { C: <0.1.0 }"] C["package.json<br/>C: 0.0.1"] A --> B A -.fresh install after B@0.0.2 published.-> B2 B --> C B2 --> C ``` This is the example the npm documentation itself uses to introduce the lockfile. I cite it because it cuts through a fog that the JavaScript community has tolerated for a decade. The fog says: "`package.json` is the source of truth for my dependencies." The fog is wrong. `package.json` is the source of truth for the *ranges* you will accept. It is not the source of truth for the *versions* you will get. There is no version in `package.json` that cannot be silently renegotiated by a fresh `npm install`, and there is no test you can write that catches the renegotiation before it hits production. ## The pure-function ideal that npm keeps failing The npm docs put the point as plainly as I have ever seen a piece of infrastructure documentation put anything: "In an ideal world, npm would work like a pure function: the same `package.json` should produce the exact same `node_modules` tree, any time." That sentence is followed by an enumeration of the four ways reality refuses the ideal — npm version drift, new published versions of direct deps, new published versions of transitive deps, and registry mutation. The lockfile is the escape hatch from non-determinism. Without it, `npm install` is a coin flip every time. This is the lesson I missed for most of my career. I treated the lockfile as a generated artifact, the way you treat `.DS_Store` or `tsconfig.tsbuildinfo` — something to `.gitignore` because it "shouldn't be committed." I am not alone. There is an entire generation of JavaScript engineers whose mental model of package management has been: "`package.json` is committed, lockfiles are not, CI re-resolves." That model is wrong, and the cost of being wrong is paid in incidents. You can see the community beginning to internalize this — the latest npm guidance is that lockfiles are required reading. The popular curated catalog of Node packages, sindresorhus/awesome-nodejs, lists four package managers — `npm`, `pnpm`, `yarn`, `bun` — but the supporting tooling around them (`np`, `npm-name`, `npm-home`, `npm-hub`, `npm-check-updates`, `patch-package`, `semver`, `npm semver calculator`, `David`, `awesome-npm`) all assume the lockfile is the canonical record. The whole metascrubber exists to manage the gap between the range and the point. The gap is the lockfile's reason to exist. ## The lockfile's actual rules The lockfile wins when its resolved versions satisfy the `package.json` ranges. When they don't, `package.json` wins and the lockfile is rewritten. The rules look bureaucratic on a first read. They are not bureaucratic. They are the precise definition of which artifact is authoritative in which conflict. ``` if lockfile satisfies package.json ranges: use lockfile else: resolve fresh, write new lockfile ``` The point is asymmetry. The lockfile is a *cache of a previous resolution*. The cache is invalidated only when the manifest moves outside the range it captured. This is the same idea as a content-addressed cache in any other system — the cache is invalidated by changes to inputs that move the output outside the cached key. npm's contribution is to make the cache content a separate file (`package-lock.json`) that you commit, instead of an implicit filesystem state you don't. ## `npm install` versus `npm ci` Here is the test for whether you understand the lockfile. If your CI runs `npm install`, you don't. If your CI runs `npm ci`, you probably do. The two commands look sim

Chapter 1 of 2 10m Article Audio Video Learning path

The Lockfile Was the Whole Point

package-lock.json exists because npm install is not a pure function — and treating it as one is the most expensive mistake you can make in JavaScript supply-chain management.

Key Takeaways

  • package.json is a *range*; package-lock.json is a *point*. Treat them as different artifacts, not redundant ones.
  • A transitive publish can change your node_modules tree without your package.json changing at all. The lockfile is the only thing standing between you and silent resolution drift.
  • npm install is for development; npm ci is for build. They are not synonyms. The CI you wrote six months ago is probably wrong.
  • The npm/pnpm/yarn/bun split is a *design-philosophy* split, not a feature shoot-out. Pick by what your project actually does, not by who's fastest this week.
  • The lockfile was the whole point. Everything else in the package-management layer — workspaces, registries, semantic-version constraints — exists to make lockfile reproducibility meaningful.

---

Imagine you ship a service today. Your package.json declares a dependency on B with the range <0.1.0. B declares a dependency on C with the range <0.1.0. Today, B@0.0.1 and C@0.0.1 are the only versions of those packages on the registry. You run npm install, your tree resolves, you ship. Three weeks later you ship a hotfix. Your package.json is *byte-identical* to the one from three weeks ago. Your node_modules is not. B published 0.0.2 the day after your first release, and your second install pulled it. C is still 0.0.1. The shape of the tree changed. Your code now runs against a transitive dependency you never named.

That is the bug. That is *the* bug.

flowchart TD
    A["package.json<br/>A: 0.1.0<br/>deps: { B: <0.1.0 }"]
    B["package.json<br/>B: 0.0.1<br/>deps: { C: <0.1.0 }"]
    B2["package.json<br/>B: 0.0.2<br/>deps: { C: <0.1.0 }"]
    C["package.json<br/>C: 0.0.1"]
    A --> B
    A -.fresh install after B@0.0.2 published.-> B2
    B --> C
    B2 --> C

This is the example the npm documentation itself uses to introduce the lockfile. I cite it because it cuts through a fog that the JavaScript community has tolerated for a decade. The fog says: "package.json is the source of truth for my dependencies." The fog is wrong. package.json is the source of truth for the *ranges* you will accept. It is not the source of truth for the *versions* you will get. There is no version in package.json that cannot be silently renegotiated by a fresh npm install, and there is no test you can write that catches the renegotiation before it hits production.

The pure-function ideal that npm keeps failing

The npm docs put the point as plainly as I have ever seen a piece of infrastructure documentation put anything: "In an ideal world, npm would work like a pure function: the same package.json should produce the exact same node_modules tree, any time." That sentence is followed by an enumeration of the four ways reality refuses the ideal — npm version drift, new published versions of direct deps, new published versions of transitive deps, and registry mutation. The lockfile is the escape hatch from non-determinism. Without it, npm install is a coin flip every time.

This is the lesson I missed for most of my career. I treated the lockfile as a generated artifact, the way you treat .DS_Store or tsconfig.tsbuildinfo — something to .gitignore because it "shouldn't be committed." I am not alone. There is an entire generation of JavaScript engineers whose mental model of package management has been: "package.json is committed, lockfiles are not, CI re-resolves." That model is wrong, and the cost of being wrong is paid in incidents.

You can see the community beginning to internalize this — the latest npm guidance is that lockfiles are required reading. The popular curated catalog of Node packages, sindresorhus/awesome-nodejs, lists four package managers — npm, pnpm, yarn, bun — but the supporting tooling around them (np, npm-name, npm-home, npm-hub, npm-check-updates, patch-package, semver, npm semver calculator, David, awesome-npm) all assume the lockfile is the canonical record. The whole metascrubber exists to manage the gap between the range and the point. The gap is the lockfile's reason to exist.

The lockfile's actual rules

The lockfile wins when its resolved versions satisfy the package.json ranges. When they don't, package.json wins and the lockfile is rewritten. The rules look bureaucratic on a first read. They are not bureaucratic. They are the precise definition of which artifact is authoritative in which conflict.

if lockfile satisfies package.json ranges:
    use lockfile
else:
    resolve fresh, write new lockfile

The point is asymmetry. The lockfile is a *cache of a previous resolution*. The cache is invalidated only when the manifest moves outside the range it captured. This is the same idea as a content-addressed cache in any other system — the cache is invalidated by changes to inputs that move the output outside the cached key. npm's contribution is to make the cache content a separate file (package-lock.json) that you commit, instead of an implicit filesystem state you don't.

npm install versus npm ci

Here is the test for whether you understand the lockfile. If your CI runs npm install, you don't. If your CI runs npm ci, you probably do. The two commands look similar and do entirely different things.

npm install is *reconciling*. It compares package.json and package-lock.json, resolves new versions if the ranges have changed, updates the lockfile if necessary, and writes to node_modules. It is allowed to modify package-lock.json. It is allowed to do an HTTP roundtrip to the registry to see what's new.

npm ci is *enforcing*. It deletes node_modules outright, reads the lockfile as gospel, and installs exactly what the lockfile says. It refuses to write to package-lock.json. It refuses to write to package.json. It fails if the two disagree.

I used to think of npm ci as "the strict version of npm install." That framing buries the point. npm ci is what npm install should be by default. npm install is the *development* command — it is allowed to be approximate because you, the developer, are present to look at the diff and decide whether the resolution change is welcome. npm ci is the *build* command — it is not allowed to be approximate because the build server has no judgment.

If your CI uses npm install, every CI run is a coin flip. You can lose the coin flip in three different ways: a transitive dep publishes a new minor, the npm CLI version on your build runner upgrades, or your npm proxy's view of the registry drifts. None of those are visible to you until the build fails or the deploy does.

Imagine you're shipping a release tomorrow

You spend the day before the release tightening your deploy scripts. You commit your final package.json change at 5 PM. The release train runs at 7 AM. Between 5 PM and 7 AM, a transitive dependency publishes a patch. Your CI runs npm install and pulls the patch. Your service starts. It crashes an hour into the post-release smoke test. The postmortem takes two days. The transitive publish that caused the crash was benign; the crash was caused by a real bug in *your* code that the new transitive version happened to expose. But the point is moot, because you cannot reproduce the prod state. Your local tree is fine. Your CI tree is fine. The release-train tree is the one that mattered, and it is gone.

Switch npm install to npm ci in that CI script. The release train now installs exactly what your local tree has, exactly what your last npm install resolved, exactly what the lockfile captured the last time you ran it on your laptop. The transitive publish is now irrelevant to this release. The bug in your code still exists — the lockfile does not fix your bugs. But the lockfile ensures that *whatever* your code is doing, it is doing it on top of a known dependency graph.

That is the actionable change. If you change nothing else from this chapter, change that.

The four package managers are a taxonomy of design trade-offs

The sindresorhus/awesome-nodejs list names four package managers and only four: npm, pnpm, yarn, bun. The names look interchangeable. They are not. They are four different answers to four different questions about what a package manager should optimize.

npm is the default. It comes with every Node install. It does nothing exotic. It writes a node_modules tree that nests duplicates when the dep graph has version conflicts, and hoists singletons when it doesn't. It is universal, slow, and exactly what the documentation assumes you mean when you say "the package manager."

pnpm is content-addressable. It stores each unique package version once on disk in a content-addressed store, and materializes a node_modules tree of symlinks into that store. The result is that two projects depending on the same react@18.2.0 share the same on-disk bytes. This is what the awesome-list's "disk space efficient" tag is referring to. The trade-off is that some tools that walk node_modules expecting a real tree get confused by the symlinks; the ecosystem has largely caught up, but legacy build tools occasionally need --shamefully-hoist.

yarn is monorepo-first. Classic yarn (1.x) was a faster npm with a lockfile. Berry (2.x+) introduced plug-n-play, which skips node_modules entirely and lets Node resolve dependencies from a central cache at require-time. The trade-off is the same kind of legacy-tool breakage as pnpm; the upside is that a Yarn-workspaces monorepo is the cleanest monorepo story in the JavaScript ecosystem.

bun is the all-in-one. Runtime, bundler, transpiler, package manager, test runner — all in a single Zig-built binary that aims to be a drop-in Node replacement. The package manager is built for speed. The trade-off is that it is young; the ecosystem compatibility surface is large and bun install does not always agree with npm install about edge cases.

You do not pick by who is fastest this week. You pick by what your project does. A monorepo with shared internal packages: yarn. A polyrepo CI with strict disk budgets: pnpm. A small service that just needs to ship: npm. An all-in experiment where you're rewriting the toolchain anyway: bun. The lockfile is the common interface; the four implementations all emit one. That is what makes them interchangeable at the boundary, even when they disagree inside.

What the lockfile does *not* do

A lockfile is not a security mechanism. It pins *versions*; it does not pin *contents*. A package's tarball can be re-uploaded at the same version with a different payload, and the lockfile will accept it, because npm identifies packages by <name>@<version>, not by hash. Hash-pinning is a separate concern, handled by tools like npm audit signatures (which checks signatures, not hashes), sigstore integrations, or — for the truly paranoid — reproducible-build systems that hash the resulting tree after install and compare.

A lockfile is also not a substitute for review. A lockfile diff is the most under-reviewed diff in most codebases. The package-lock.json PR often contains hundreds of changed lines; nobody reads it; everyone approves it. The right discipline is to read the diff the way you read any other diff: what changed, why, and whether the change was intentional. Tools that surface human-readable lockfile diffs (npm install --package-lock-only followed by a custom diff, or third-party lockfile viewers) help, but the discipline is the point.

The deeper lesson

I started this analysis believing package management was a solved problem. The transitive-dependency trap changed my mind. The trap is not a corner case; it is the central case. Almost every Node project in production has at least one layer of indirection between "what I named" and "what runs in my process." The lockfile is the only mechanism that makes that indirection visible and stable. Without the lockfile, you are running code against whatever the registry feels like giving you today.

If you take one thing from this chapter, take the lockfile seriously. Commit it. Diff it. Review it. Treat your CI's npm install as a latent bug. Run npm ci everywhere you used to run npm install for builds. And when a colleague says "we don't need to commit the lockfile, it slows down the repo," show them the A→B→C example and ask them how their release will behave when B publishes 0.0.2.

The package manager's hard problem is non-determinism. The lockfile is its answer. Everything else in the layer — workspaces, registries, semver ranges, the four-way npm/pnpm/yarn/bun split — exists to make lockfile reproducibility *meaningful*. The lockfile was the whole point.

There is another non-determinism problem in the JavaScript runtime, and it is harder. The package manager locks the install. The event loop has to lock the run. That story is harder, and it starts at a cost-of-I/O table that is older than Node itself.

---

References: