CCF Part 2: ESM Vendor

2025-12-29

One thing I'm very excited about is using ESM in the browser. For the first time it is possible to just write js, without any extra steps in between. It has been a long road for javascript. I think the Rails project has always done the "most correct" way to use javascript, and so the history of Javascript in Rails also serves as a good history of Javascript in general:

AI Slop - Rails JS

Though Rails generally considers the "correct" way to use js is to try to avoid it's use entirely, or at least obfuscate it. I think the history can be generalized into these phases:

Do nothing. Make js files, link them in the <head> tag, or wherever. Or inline.
Asset concatenation. Take many js files, and concatenate them, as well as some minimal post-processing like minification. Lots of tools existed to help automate this, such as Grunt and Bower, or Jammit. Though, none of these were a complete solution or offered some functionality, but not others. Some plugins for tools for Grunt (and similar e.g. gulp.js) may have even attempted some early form of bundling, but was probably more closer to transpilation of CommonJS into UMD follow by concatenation.
Bundling. Crawl and transpile imports, concatenate the result. JavaScript applications were becoming more complex, a global scope was starting to cause problems, managing transpilation tools like babel and typescript became challenging, and people were standardizing on proper package distribution on npm. This created the need for a system more sophisticated that could handle common JS modules and dependencies. Honorable mentions: Browserify, Webpack.
Do nothing, win. The browser now natively supports modules as part of the ECMAScript standard. If <script type="module"> is used, the js code will not have a global scope, and will be able to use the import function or keyword to trigger HTTP requests to fetch other js files, and parse them as modules.

However, there is not yet a simple solution to effectively manage packages that are ESM first. In most cases, the recommended solution for distributing and installing ESM packages is by using a CDN link. There are a lot of reasons a person may not want to do this, but for me, the main problem is preservation. If I build a website, I want it to be a complete package that has no external dependencies that could go offline or become unavailable during a 3rd party outage. My outages will be my own, thank you very much.

I've researched this quite a bit and I have not found something that really thoroughly solves this problem. If anyone is aware of something, I'd love to know about it. Primarily, I'm focused only on solving the problem of vendoring ESM modules. I'm not interested in attempting to transpile code that is in common JS into ECMAScript modules.

Here's what I found so far:

JSPM seems like a tangential solution, and doesn't seem to provide a way to download modules, unless node_modules is used.
Deno can be used for this, using its vendoring feature, but it is inflexible, and also is not really the purpose of this tool (it's specifically for vendoring your dependencies when running or packaging server-side deno code)
GitHub - donseba/go-importmap: Golang importmap generator: seems promising, but I couldn't get it to work. Also, I think it is pretty narrow in scope (does not "crawl" the linked package). Some people may prefer this to what I attempt below, due to lack of tree-shaking.

Given all of this, I'm going to attempt to create a tool to accomplish this myself. Here's the prompt I'm going to start with and see how far it gets us:

Prompt

Create a cli in go for caching ECMAScript modules given a CDN URL. Use the esbuild package to accomplish this. The script should output an import map which can be used in the browser to properly load the cache files. The script should take two inputs, the CDN URL to be cached, as well as an output folder.

After some arguing over whether esbuild or a random collection of regex is the better approach (it got frustrated at first and reverted to regex), this eventually got me here, which is able to work rendering a simple D3 graph:

Modify import caching logic for relative imports · quinn/esm-experiments@4308f4f · GitHub

This works fine for a single dependency, but obviously can't handle multiple. Next, I'm going to refactor it to support any config files input that specifies multiple packages that should be used. I could at some point instead extend a package JSON config, but since this is not necessarily intended to be used with an application that's already using an NPM style package system, I think I will go with a proprietary config format for now.

Prompt

Let's refactor this to support multiple named dependencies but a single vendor directory and single import map. Let's use a config file instead of CLI arguments for this.

Implement ESM module vendoring with config · quinn/esm-experiments@62b8abb · GitHub

Next, I'd like to make some adjustments to the defaults and to the config file:

Prompt

Let's use YAML instead of JSON for the config. Let's add a top-level key that wraps the entire config called esm-vendor in the config.

Alright, it's now in a place I'm pretty happy with:

Switch from JSON to YAML config and update related code · quinn/esm-experiments@a7d697b · GitHub

OK, one more thing. I need to implement a template feature, or at least it would be convenient to have a template feature so that the user of this tool does not need to manually inject the generated JSON into where it eventually should go:

Add template feature · quinn/esm-experiments@0bc9ec2 · GitHub

Alright, I think this is ready. Here's a demo:

ESM Vendor Test

I've packaged this into a cli, documentation for that is here:

Tool - CCF - ESM Vendor

This is my Proof of Concept, I think an ideal solution would be capable of tree shaking, but I don't need that yet for any of my immediate use cases. Also, this only will mirror stuff from a CDN that uses ESM, and won't work with ESM on NPM.