First Gulp... ummm Maybe?

Jeff

• 6 min read
First Gulp... ummm Maybe?

TL;DR

My first foray into toolchains, using Typescript in the Node-verse - to build a teeny web-page - with JS, CSS and HTML. This is all about the Toolchain - how to set it up, use it and develop with it. I don't cover packaging, source-control or deployment - because I haven't got there yet!

Excuses

So this is just my personal journey - and it's what I discovered in two four days - but maybe I missed a super easy alternate? I kind of hope so...

Endgame

My goal is to learn to write, update and deploy web-applications using current (and I know that will change!) best-practice (is there such a thing?) in Web technologies.

I know from experience that the latest and greatest can easily turn into yester-years dusty bygones. So I aim to use tools that have some maturity. Hopefully they've been through at least one, and maybe more, major iterations (aka re-writes) - because you only know what you know once you know!

At the same time, I am looking for technology and tools that have an active, positive and excited community - things that are currently being made more shiny because people love them in action. I want some 'smarts' in that community - I prefer conscious design, over mindless evolution - and believe that it shows in structure and response to change.

Be warned, that I an INFJ - and if that's not bad enough, I am so much more a what-works, rather than a what's-right, person. I also learn by doing - but do believe in a reasonable tilt at research before jumping in.

My Endgame here is to use TypeScript and ES2015 all the way through my new stack, with a flexible responsive toolchain that emits tight deployable components. I want good integration into source-control and a release mechanism based on reconstructable source check-points.

Starting point

My last stint at JS coding was in the early 2000's - with target deployment being in IE6 (yes - 6 yikes!), so my JS is limited constrained. I've coded in a bunch of other environments, and even had (gasp) formal education in Structured Programming with Gane and Sarson. I know 3NF, ER, UML and even remember the OLE vs. OOP debates.

So... minimal JS, and none of the current goodness - and wow - there is a lot of JavaScript goodness today in what used to be a true PITA.

Tech Starting Point

I've just installed Visual Studio Code on my Windows laptop. I have a headless Linux server somewhere in the garage, running Debian Buster - on which I've installed the current Node-verse:

jeff@fogh:~$ cat /etc/debian_version
10.5
jeff@fogh:~$ sudo nginx -v
nginx version: nginx/1.14.2
jeff@fogh:~$ node --version
v12.18.3
jeff@fogh:~$ tsc --version
Version 4.0.2
jeff@fogh:~$ npm --version
6.14.8

The server has Samba installed, so I can edit files directly on the file-system - and I've created a link at ~/src/myproj/$ ln -s dist/ /var/www/test-local/dist/ so in my browser I can access http://fogh/dist/ and with autoindex on; in my Nginx site file, I can see an empty directory index.

Maybe that sounds complex? My view is that testing it all via an actual Nginx web-server will tease out issues (like ES5 v ES2015) as soon as possible, as the testing is happening in the deployment environment.

Aside: The main downside so far is that the local Windows installation of Visual Code Studio does dynamic TS syntax checking - and this seems sometimes to be different to the tsc transpiling that is happening on the server.

The Journey

Took a whole two four days! It was harder than I expected - with several restarts after I decided to switch from Webpack to Gulp - and then later from browserify to rollup... and ultimately threw that out too, for microbundle

Webpack

My friend Google provided some morning reading, and led me to decide that Webpack would be a good start.

I followed this tutorial https://webpack.js.org/guides/getting-started/ - tweaked to ignore lodash as I haven't decided to use that. Easy.

So now I added TypeScript to the mix from this tutorial - https://webpack.js.org/guides/typescript/ - again, pretty good. Adding CSS to the mix was also easy, https://webpack.js.org/guides/asset-management/ - and also a couple of .svg files to act as img-src - all of which worked fab.

So I carried on down this path - and was pretty pleased. I then noticed that my app.bundle.js was getting pretty bit - from about 8K of source (including assets) I was coming in at a bundle of about 30K - hmmm, time to minify?

...and that's where the first rabbit-hole appeared. I couldn't seem to find a good guide on minifying the transpiled JS and the CSS - which was being injected as in-line style in <head>. I found a few - but things kept breaking or not working as advertised. A lot of old web guff proved more confusing than useful. Webpack started to feel too controlling - what I wanted to do seemed a simple common requirement. Aside: Looking back, I think that I should revisit this - Webpack is very poplar, and comprehensive. It's definitely worth another crack...

So what are the cool kids using then, if not Webpack?

Grunt or Gulp

Dr Google to the rescue - and in summary, it seemed that Grunt is config-drive whereas Gulp is procedural (as in you encode your desired workflow) - encoded in JS (oh, and supports TS out of the box!).

Maybe as a reaction to my frustration with the controlling nature of Webpack, I let my wild-child free - and decided to take the Gulp... oh, and I love Streams too...

One Big Gulp

So I started with this https://www.typescriptlang.org/docs/handbook/gulp.html - and it all went - okish. I did get there in the end, but it felt harder than it should be.

Transpilation of gulpfile.ts

I decided to use a TypeScript gulpfile.ts - because, why not - all my project code is in TS. This was where most of my first set of issues came from - and perhaps because my understanding of Gulp is rather limited.

I had issues in gulpfile.ts with import vs require and exports.default vs export - it was all confusing until I read about the evolution of modules. I decided to use ES2015. Even after I had upgraded my gulpfile.ts to ES2015, I continued to have issues. In the end, the fix was a change in tsconfig.json - the Gulp transpile options can be set in a ts-node leaf - which is distinct from the main tsc compiler options.

{
  "ts-node": {
    "compilerOptions": {
      "module": "commonjs"
    }
  }
}

tsconfi.json snippet

I was also getting transpile errors because of the gulp function-callback arg - so had to add an interface to fix that. I also wanted to centralise config.

// TS goodness for gulp callback arg 
interface Callback { ( e? : Error ): void; }

// Gulp 
import { parallel, series, src, dest, watch as watchify } from 'gulp'

// Configuration
const $config = require('./package.json')
const $path = {
  main:   $config.main,
  dist:   "dist/",
  build:  "tmp-build/",
  bundle: "tmp-bundle/",
  ts:     [ "src/ts/*.ts"],
  js:     [ "src/js/*.js", ],
  html:   [ "src/html/*.html"],
  css:    [ "src/css/*.css"],
}

gulpfile.ts snippet - initialisation

Workflow - Browserify

I initially used Browserify and Uglify and Tsify as per the tutorials. They all seemed to play nicely - but a whiff of Browserify-does-everything seems to linger in the air. I'm kind of used to chaining a collection of best-in-class tools. I didn't find an easy way to get my CSS minified.

Workflow - CBB - clean, build, bundle.

So - I decided to try this workflow:

  1. clean - remove temporary build artefacts from the file-system
  2. test-unit - TBC
  3. build - lint checking, transpilation, minification and all other asset transforms
  4. bundle - collection and packaging for distribution - including merging JS into a single .js
  5. test-e2e - TBC
  6. ship - TBC... via with source-control

Implemented using this file-system

src - all input files I create/update (not 3rd-party)
  ts   - Typescript files
  css  - CSS files (SCSS later?)
  html - html files (will change with templating...)

tmp-build
  - munged / transpiled / tweaked files
  - hmmm - where have my streams gone?

tmp-bundle
  - all files that are ready for bundling
  - CSS, JS, SVG, JPG
  - etc?

dist
  - all assets to be deployed
  - including source-maps 
  - bundle.zip for draft-release on Github?

file-system workflow

Workflow CBB - clean

Got gulp clean working - using del - not everything has to be a gulp-thingy - remove all files in tmp-build, tmp-bundle and dist 

// cleanup filesystem pipeline
//    src -> build -> bundle -> dist
//    src          -> bundle -> dist
import { default as del  } from 'del' 
export function clean ( cb: Callback ) : void {
  del([
    paths.dist + '/*',
    paths.build + '/*',
    paths.bundle + '/*',
  ])
  cb()
}

gulpfile.ts snippet - clean

Workflow CBB - the rest

Sorry, ran out of steam.

Conclusion

Its just too much klutzy effort - in a couple of years we just won't bother? Except maybe legacy projects that have invested too heavily and are nervous of changing.

Follow-up

  • Revisit Webpack - there must be a way to minify the JS and CSS... and to bring the size of the bundle down.
  • Give Grunt a go - some smart folk are using it.
  • Is there a better work-flow that doesn't rely on a collection of several tmp-folders?

Credits