Module cdn's and workflows
Exploring some es-module based CDN's

In this blogpost, let us discsuss about the state of current CDN based unbundled application development looks like. I recently started working with esm based cdn's and there are some short comings because of how the modules are being published to npm. Most of the modules are being published using cjs format and a esm entry with module flag in package.json file which is again a non nodejs standard.
So, i recently started working on something, where to mix both esm and bundling. Like a hybrid version of cdn which servers packages from npm but bundled. It started out as a fork from packd and over the time evolved and we removed the support for umd fallback. I will try to point out a few obseervation and try to explain then, a full unbundled development is just a few steps away.
How traditional UNPKG is different from Skypack and JSPM
UNPKG can be considered as "mirror" for npm packages. It basically parses the route you are asking for and serves the code. It doesn't apply any bundling, transpiling or compressing. But unpkg has a new flag
?module
which looks for the module entry point instead.PIKA and JSPM does transpiles the code from
cjs
toesm
and then remaps all the imports to a statically transpiled ahead of time packages.Both
pika
andjspm
are doing a great job in handling all the problems that are caused because of mis-configured packages.
Then why we need a new one ?
As i was referring earlier that i was working on a version on my own. Let's see the "why" and "what" and "how" here.
We need a pure
esm
based one, so we can't rely onunpkg
. Yes, it does have?module
flag. But it doesn't remap your imports.These
cdn's
mostly rely on exports-map which is a standard in node. But, most of the libraries which are shipped to npm today don't actully specify theexports-map
. They usemodule
field inpackage.json
. So, these cdn's need to do some static-analysis of the package to serve them for clients.
In the course of time, we had ran into several issues. Here, i am trying to explain a few of them.
Missing Dependencies !
When a package author misses defining a package in the package.json
.
When using a package we mainly look at direct dependencies from package.json
that the package is using.
But, we need to be careful about the sub-dependencies
of the direct-dependencies
the main package is using.
Here is a example, antd@4.6.1
is using rc-image
and rc-image
in return is using rc-util
. And rc-util
uses react
directly in it's files . But the package defines react
and reat-dom
as devDependency
but not as peerDependency
.
According to npm documentation, it's clear that devDependencies are using only in local development and testing. Any package that is required by your application should go under dependecies or peerDependencies.
When we are using such packages to bundle or build, we usually use the command that we had below.
npm install
which unstalls both deependencies and devDependencies but when we use
npm install --production
It install's only dependencies and not devDependencies. So, even if we manualy install peerDependencies. react
is not installed as it is missing from both depepdencies
and peerDependencies
. If we serve such package from a cdn
service, they tend to fail.
Since the package is missing from both the places. So, cdn's
fail to reolve to fetch any meta
like package version etc. And few cdn's resolve to the latest version of the package in that case.
https://github.com/react-component/util/pull/138
Package exporting cjs under main !
The most common approach that package authors take is, exporting cjs
using main
field in package.json. And esm
using module
field.
This is a pattern that is encouraged and followed mainly on the bundlers userland.
But in node environment you need to export using only main
field. Since, module
attribute is not supported or a standard in node.
Instead we have a another field called, type : module
which is a node standard to define if the code we are exporting is a es mdoule or cjs.
And for esm
modules we the file extension is .mjs
and not .js
. But the bundlers userland will accept even if we use the .js
extension for modules.
To overcome this issue, cdn
's like jspm
does a ahead of time static analysis. This makes then understand the package structure and the files and exports.
But during this step, they mainly rely on exports
field which is a node
standard. And which is again not used by most of the pacakge authors, since bundlers are not using them at the moment.
In such instances, the static analysis fails and cdn's
faile to serve the packages when request a specific module.
https://github.com/chakra-ui/chakra-ui/pull/1464 https://github.com/rollup/plugins/issues/572
Change in sub-dependencies !!
If you are using a sub-package called b
and a
as your main package and a
is at version number 10.0.0
.
b
is initally at 5.0.6
and you built your project around it and things are fine. But then b
had a new
release of 5.0.7
which have a breaking change.
"This happens when semver is not properly followed. Because breaking changes shouldn't go under patch or minor released"
Currently the cdn's are not pinning down this so, in future you might pull 5.0.7
and the package will stop working.
JSPM is solving this issue in their future release of jspm-cli
version 3, which exports a import-map
of every dependency that is used.
For more details please visit the discussion in here
Loading multiple versions of the same library
It's difficult to pin down the version of library that you are using withou scopes in importmaps.
For example if you are using react
and your dependencies
are also using react of various versions. Then react is
loaded multiple times since you are serving unbundled and imports resolve w.r.t to your packages deps
or peerDeeps
are loaded.
Loading multiple versions of react sometimes breaks your project. Here is a jsfiddle example
If you can run the example and inspect the console, you can see react is loaded with 16.0.8
and 16.13.1
So, currently cdn's
load the latest version by default. Here is a unpkg
link
of @material-ui/core@4.11.0
. You can see the peerDeependency
of react is 16.8.0
. But when loaded from cdn,
it loads 16.13.1
. Cdn link --> https://cdn.skypack.dev/-/antd@v4.6.1-chaSqPebG9J1riQcD3Q8/dist=es2020/antd.js
And, from the jsfiddle example we can understand that, there is a unspoken vendor effect
.
Right now, we have jspm
and skypack
and each have their own approaches in solving these issues. So, when you project is loading from multiple vendors, make sure you are
not running into problems like these. Or they have the potential to break the application.
Then why we don't run into issues like this with our projects when build locally ?
Yes, it's completly logically to think in that way. Then we need to look at how the bundlers work in more details below.
Traditional Bundlers
The main source of truth for bundlers is node_modules
. So, in most of the cases they don't care about where
the dependency is mentioned (deps, peerDeps, devDeps). As we run
npm install
which installs both dependencies
and decDependencies
They just check the package and load it from node_modules
at the time of bundling.
Here is a issues https://github.com/saleehk/lowdb-google-cloud-storage-adapter/issues/1 which expains it more clearly.
If you check the PR, i was mentioning to move babel-polyfill
from devDependency
to peerDependency
or dependency
.
Because, if we look at the code
The package is being used, but when checked in pacakge.json
it is present under devDependencies https://github.com/saleehk/lowdb-google-cloud-storage-adapter/blob/master/package.json#L17
So, if any x
package from your project uses babel-polyfill
then during npm install
this is resolved.
And the bundlers don't throw any error, since all they care if babel-polyfill
is in node_modules
.But when you
using it in project where no other package loads babel-polyfill
, this does breaks your build. As, babel-polyfill
is not resolved during installation.
The Future !!
I persoanlly like approach that deno
has taken. Betting on esmodules
is the future i am looking forward.
- Loading code from any source and not relying on
node_modules
or some CDN service to do the heavy lifting of transpilation for us. - Or waiting for a package author to provide a format that we want to load.
- Loading only a specific module from a specific pacakge using
export
maps, gives us tree-shaking out of the box. Yes, the bundlers which we use gives us thetree-shaking
support but it has it's limitations.
And i am excited about the jspm version 3
release.
If anyone is looking for a package, which follows all these standards and can serve as a good learning point. Please check preact
Conclusion
To have a quick solution for solving the above issues, the best way is have a hybrid version of cdn.
- Transpile the pacakges to
esm
- Bundle it with
dependencies
andpeerDependencies
. So, we are not dynamically relying on anything or any change in sub-packages etc.
Initial iteration of the hybrid version of cdn-packer teleport-registry-packer
I would highly recomend to watch the talk by @guybedford in youtube, where he explains about the future vision of package managers and upcoming releases for jspm.