September 17, 2016

Creating a data-driven page loader in JavaScript

Ramblings regarding a JavaScript project I created.

A programmer worth their salt will always be striving to better their skills and increase their knowledge. They realize just how little they know and how inadequate they feel and work to improve their abilities. This can be done in a variety of ways: attending a programming conference, participating in a 24-hour event, learning a new language or the latest JavaScript framework, to name a few examples.

Some months ago I was asked what new programming thing I had done lately. I replied with a simple “nothing.” See, for some months now, my programming goals have not been to merely do something new for the sake of newness. Instead, I have been actively pursuing developing my existing skills and refining my “cached” knowledge. As I once heard someone say:

“If you aim for breadth, you will not gain depth. If you aim for depth, you will consequently gain breadth.”

Thus in my endeavors, you could say I am going for depth, taking what I already know and seeing how far I can push it.

One way I have been doing that is by paying more attention to my Twitter feed. I have followed some HTML and JavaScript news feed accounts for a while but never really paid attention to them. Now I have been reading a good chunk of the articles they post and learning a great deal in areas I only briefly knew. 10 things to need to know about CSS has probably been the most interesting article (#1 surprised me).

However, nothing ever beats just writing code and experimenting. While my laptop was out of commission for two months, I had a good bit of time to simply think of interesting things to try or imagine how something might work.

One idea I had (actually a dream from one night) was a lightweight, data-driven page loader written completely in client-side JavaScript. Basically, think of an Apache or NGINX server (maybe AngularJS? I do not know.) but on the client. My idea was to simply write the individual pages, store them in a folder (such as `_pages`), then through an simple function call, state the page filename, load it, then inject it into the main page.

If you are a web programmer, you are likely already thinking how you might do that, and I will explain more about that, but first, a quick bit of back story (you can skip this section if you want).

As I previously wrote, I have been managing a church’s WordPress site. It is full of technical debt caused by mismanagement and lack of understanding of how to manage a site. When I gained access, I discover a very large mess full of copious amounts of junk JavaScript (the YouTube embed player being a huge offender). For the record, after a while I managed to tame most of the monster.

While cleaning the site, I started looking for a new theme to use (because that is part of the maintenance plan). The pastor wanted a one-page site, which I happened to be thinking about, so I went on a hunt for a good one-page theme (Thanks to previous experience, I have no desire to write a WordPress theme unless I absolutely must). In the search, I found most all of the one-pager themes either:

  1. Use widgets acting as “pages” to construct the sections (bad for maintainability and ease of revision)
  2. Use a widget to embed a proper page into the section (my preferred method).

Sadly, very few themes used method #2 and those who did were complicated to set up or not really what I was looking for (BTW, have you seen how many themes restrict a custom logo to paid versions?!?!).

Between minimizing site resources and finding a one-page theme built on individual pages, I began to wonder if there was a way to create an entire website built on the concept of pages but only needed to download all the resources once.

Thus the idea was born.

After writing down my ideas on paper, I began implementing the system, using a few ES6 features to learn how they work. It ended up working something like this:

// This is actually passed as a function parameter,
// but defined as such here for illustrative purposes
let pageName = "index";

window.fetch(`_pages/${pageName}.html`).then(r => {
  return r.text();
}).then(body => {
  document.querySelector(".page-content-  wrapper").insertAdjacentHTML("beforeend", body);
}).catch(err => {

The whole system boils down to these few lines. Everything else I have written, such as most advanced error handling, duplicate page load prevention, and a quick-and-dirty cache based on sessionStorage, are all extras.

Demo using original demo site layout
Revised demo site with brief URL scheme visualization (`.com#page`)

At the time of writing, including the Google Fonts load, initial page load for the site is 26.92 KB, with my own JavaScript accounting 10.29 KB, or roughly 39.14% of the total (additionally, nothing is minified or gzipped). Any additional data size is dependent on the page content. Even then, may not be very much because of the sessionStorage cache and the Fetch API’s own caching.

I really like how the core of this concept came out. It is small, fast, uses “proper” pages, and runs entirely on the client. I can easily see myself using it to make some interesting, multi-page sites without actually navigating to other pages or building up one-page sites. Obviously the concept comes with a cost: a11y is practically non-existent, I accidentally broke the browser’s back button (for now), and people without JavaScript enabled cannot use it (something I can likely work around with a server-side pseudo-implementation which may even improve a11y). Despite the trade-offs, I may end up using it on my personal website.

In developing this, I was able to expand my existing knowledge of JavaScript and think in a different manner than I often do. I found out that, as simple as such a concept sounds, it is actually very complicated. I learned that I need to learn how to use Promises more. :P I discovered it is possible to wrap a non-asynchronous task in a Promise. I learned a lot in this project, and I think I did because I did not start out with a plan to go so wide and encounter all that I did. I knew this direction would be deep and I would learn a lot, but I did not realize I would end up covering so many topics(and more, as I am still building it!).

In this project, depth really did produce breadth. With confidence, I can say that building out my little dream code was fun and a great experience. :) If you would to check it out yourself, I have a demo available on my website. Do note it may be broken at any given time due to extended development in the little bit of free time I have, and it is totally broken in browsers that lack Promise and Fetch support. I have not yet uploaded the code anywhere, but I intend to when it is more stable.

Buy Me a Coffee at