In this blogpost, let us discsuss about the state of current CDN based unbundled application development looks like. I recently started working with esm based cdn's and there are some short comings because of how the modules are being published to npm. Most of the modules are being published using cjs format and a esm entry with module flag in package.json file which is again a non nodejs standard.
So, i recently started working on something, where to mix both esm and bundling. Like a hybrid version of cdn which servers packages from npm but bundled. It started out as a fork from packd and over the time evolved and we removed the support for umd fallback. I will try to point out a few obseervation and try to explain then, a full unbundled development is just a few steps away.
How traditional UNPKG is different from Skypack and JSPM
UNPKG can be considered as "mirror" for npm packages. It basically parses the route you are asking for and serves the code. It doesn't apply any bundling, transpiling or compressing. But unpkg has a new flag
?modulewhich looks for the module entry point instead.
PIKA and JSPM does transpiles the code from
esmand then remaps all the imports to a statically transpiled ahead of time packages.
jspmare doing a great job in handling all the problems that are caused because of mis-configured packages.
Then why we need a new one ?
As i was referring earlier that i was working on a version on my own. Let's see the "why" and "what" and "how" here.
We need a pure
esmbased one, so we can't rely on
unpkg. Yes, it does have
?moduleflag. But it doesn't remap your imports.
cdn'smostly rely on exports-map which is a standard in node. But, most of the libraries which are shipped to npm today don't actully specify the
exports-map. They use
package.json. So, these cdn's need to do some static-analysis of the package to serve them for clients.
In the course of time, we had ran into several issues. Here, i am trying to explain a few of them.
Missing Dependencies !
When a package author misses defining a package in the
When using a package we mainly look at direct dependencies from
package.json that the package is using.
But, we need to be careful about the
sub-dependencies of the
direct-dependencies the main package is using.
Here is a example,
firstname.lastname@example.org is using
rc-image in return is using
directly in it's files . But the package defines
devDependency but not as
According to npm documentation, it's clear that devDependencies are using only in local development and testing. Any package that is required by your application should go under dependecies or peerDependencies.
When we are using such packages to bundle or build, we usually use the command that we had below.
which unstalls both deependencies and devDependencies but when we use
npm install --production
It install's only dependencies and not devDependencies. So, even if we manualy install peerDependencies.
is not installed as it is missing from both
peerDependencies. If we serve such package from a
cdn service, they tend to fail.
Since the package is missing from both the places. So,
cdn's fail to reolve to fetch any
meta like package version etc. And few cdn's resolve to the latest version of the package in that case.
Package exporting cjs under main !
The most common approach that package authors take is, exporting
main field in package.json. And
This is a pattern that is encouraged and followed mainly on the bundlers userland.
But in node environment you need to export using only
main field. Since,
module attribute is not supported or a standard in node.
Instead we have a another field called,
type : module which is a node standard to define if the code we are exporting is a es mdoule or cjs.
esm modules we the file extension is
.mjs and not
.js. But the bundlers userland will accept even if we use the
.js extension for modules.
To overcome this issue,
jspm does a ahead of time static analysis. This makes then understand the package structure and the files and exports.
But during this step, they mainly rely on
exports field which is a
node standard. And which is again not used by most of the pacakge authors, since bundlers are not using them at the moment.
In such instances, the static analysis fails and
cdn's faile to serve the packages when request a specific module.
Change in sub-dependencies !!
If you are using a sub-package called
a as your main package and
a is at version number
b is initally at
5.0.6 and you built your project around it and things are fine. But then
b had a new
5.0.7 which have a breaking change.
"This happens when semver is not properly followed. Because breaking changes shouldn't go under patch or minor released"
Currently the cdn's are not pinning down this so, in future you might pull
5.0.7 and the package will stop working.
JSPM is solving this issue in their future release of
jspm-cli version 3, which exports a
import-map of every dependency that is used.
For more details please visit the discussion in here
Loading multiple versions of the same library
It's difficult to pin down the version of library that you are using withou scopes in importmaps.
For example if you are using
react and your
dependencies are also using react of various versions. Then react is
loaded multiple times since you are serving unbundled and imports resolve w.r.t to your packages
Loading multiple versions of react sometimes breaks your project. Here is a jsfiddle example
If you can run the example and inspect the console, you can see react is loaded with
cdn's load the latest version by default. Here is a
@email@example.com. You can see the
peerDeependency of react is
16.8.0. But when loaded from cdn,
16.13.1. Cdn link --> https://firstname.lastname@example.orgSqPebG9J1riQcD3Q8/dist=es2020/antd.js
And, from the jsfiddle example we can understand that, there is a unspoken
Right now, we have
skypack and each have their own approaches in solving these issues. So, when you project is loading from multiple vendors, make sure you are
not running into problems like these. Or they have the potential to break the application.
Then why we don't run into issues like this with our projects when build locally ?
Yes, it's completly logically to think in that way. Then we need to look at how the bundlers work in more details below.
The main source of truth for bundlers is
node_modules. So, in most of the cases they don't care about where
the dependency is mentioned (deps, peerDeps, devDeps). As we run
which installs both
They just check the package and load it from
node_modules at the time of bundling.
Here is a issues https://github.com/saleehk/lowdb-google-cloud-storage-adapter/issues/1 which expains it more clearly.
If you check the PR, i was mentioning to move
Because, if we look at the code
The package is being used, but when checked in
pacakge.json it is present under devDependencies https://github.com/saleehk/lowdb-google-cloud-storage-adapter/blob/master/package.json#L17
So, if any
x package from your project uses
babel-polyfill then during
npm install this is resolved.
And the bundlers don't throw any error, since all they care if
babel-polyfill is in
node_modules.But when you
using it in project where no other package loads
babel-polyfill, this does breaks your build. As,
babel-polyfill is not resolved during installation.
The Future !!
I persoanlly like approach that
deno has taken. Betting on
esmodules is the future i am looking forward.
- Loading code from any source and not relying on
node_modulesor some CDN service to do the heavy lifting of transpilation for us.
- Or waiting for a package author to provide a format that we want to load.
- Loading only a specific module from a specific pacakge using
exportmaps, gives us tree-shaking out of the box. Yes, the bundlers which we use gives us the
tree-shakingsupport but it has it's limitations.
And i am excited about the
jspm version 3 release.
If anyone is looking for a package, which follows all these standards and can serve as a good learning point. Please check preact
To have a quick solution for solving the above issues, the best way is have a hybrid version of cdn.
- Transpile the pacakges to
- Bundle it with
peerDependencies. So, we are not dynamically relying on anything or any change in sub-packages etc.
Initial iteration of the hybrid version of cdn-packer teleport-registry-packer