Reducing build time by 50% for a Typescript Monorepo

+

Monorepo's

The state of Javascript developer tooling is exciting and moving at much faster pace than ever. JavaScript being a high level and dynamic language, we end up coming across many runtime errors than a traditional compiler based languages (i.e rust, go, c++). It has it's ownpros and cons at the same time.

One major advantage is, learning curve is little easier as the feedback loop of seeing our projects running is quick. But the same will bite us back in large scale projects.

But with typescript, linters,and prettier and many other build time and development time targeting tools. These JS tooling has made our lifes as developers much more easier and at the same time reducing the run-time errors that we often run into. But, these tools being again written in JS, starts to show the short comings when the applications starts to grow bigger.

Ok, Now enoughranting about the slowness of build tools, but what actually got me thinking into all this ?
Why should i bother, moving my build pipelines which is absolutely stable at the moment ?

There is a high chance of you coming across esbuild I could direct you to some of the blogposts that talk about esbuild.

Monorepo build times

Now, let's talk about monorepos. I wrote a article before on setting up a monorepo using typescipt and lerna. You can find it on medium. And we adopted the same in our monorepo of code generators. As, the project started to grow, we started running into delayed build times.. The build times are taking a massive hit. Our monorepo, consists of 60+ packages to support all frameworks that we target at the end. And so, we are looking at 10~13minutes of build time at minimum. Here is a screenshot from our CI.

You can see, on an average the build time is minimum 10mins. And this started to show effect in local too, even when we had a custom-watcher. That takes care of watching the package and rebuilding them, which you can find here. But still the initial startup and build time at end is consuming a considerable time.

We don't use Webpack or rollup build tools. We just need to transpile the code to es6 and publish to npm. Any bundling with dependencies, transpiling will handled by the users consuming the library. That's the best way to go, or else we might end up bundling two times, transpiling towo times. First when we do it before publishing to npm and once again when the users using in their build systems.

Command that we use for building all the packages. Now, let's look into reducing these build times.
So, let's address the two main missing points if we totally move towards usingesbuild. At the moment of writing, esbuild doesn't support generating types and static-type checking. Which is fairly reasonable because, that's whattsc compiler is for. It means, esbuild can read a typescript code-base. But it just strips away typescript related meta.

So, we will be missing on,types generated by the project andtype-checking. Type-checking is one of the main up-side that we discussed at the start of this post, which helps in minimising the runtime errros.So, we needesbuild'stranspile's performance andtypescript's typesandstatic-type-checking.

Custom Build Scripts

I started to put together some custom scripts to achieve this.
A custom build script that walks through all the packages at depth as 1. And builds the package using esbuild's node-api. And the package.json of each package.

A Custom watcher to look for changes in packages, and rebuilding them.

and now configuringnpm scripts to run typescript compiler to generate types and a static type checking.

And here is the final result.

- Packages transpiled using esbuild. And types are generated in parallel using lerna.
- All the packages are transpiled toes6 and withcjsandesmformats. Transpiling till es6 is enough since, if the end users want to support a much lower syntax. The user's build systems can do that again.
Ok, let's talk about the results. It's clear that we are having an average of 5~6minutes in our build time now.

PS: This may not be the best of the solutions in configuring a mono-repo with custom scripts. But hey, it works for us and saves half a time spent on transpiling the code-base.