Sections
TL; DR;
This post explains how to add the needed parts to a web application in order to make it a PWA. The post is mostly focused in the service worker implementation, which is a key part of the convertion process.
Intro
In this post I'll describe how to convert an existing web application into a PWA.
Did I say "convert to PWA" instead of "add PWA capabilities"? Yes, because although you can add the missing parts to an existing app, the end result is a new type of application that is web in its core, but has some native-like abilities.
What are PWAs
For those of you that already know what a PWA is, feel free to skip to the next section.
Before telling you the "what", let me explain you the "why" with some wish-like questions:
Have you ever wanted to have your web app lauched from the home screen of any device, instead of making the user open the browser and enter a tedious-to-type url?
Have you ever wanted to view your app like the native ones, i.e. without a browser's address bar, and looking just like any other installed app?
Have you ever wanted to check something really quickly in your app, just to notice you don't have a network connection?
Those are capabilities of PWAs: can be installed, can run in standalone mode (no browser window) and can work even when a network is not available.
So now I'll bring you a more in-depth explanation of how PWAs handle all those situations.
PWA means "Progressive web application". The "progressive" part aludes to the evolution of web applications with capabilities that were first implemented
by native apps, like offline browsing (with the help of caching resources while online), an installation experience, a home screen icon and so on.
A PWA web application is not a native application, but it disguises as one in the device where it's installed on.
In their core, PWAs are no more than web applications that implement 4 things:
A manifest file
A set of icons for the home screen
A secure url (https://)
A service worker
The manifest file
The manifest file contains information about the application, like language, icons, background colors, way of displaying it, etc.
It's common to find it in source code repositories with a name like manifest.json, although it could be named whatever you want.
The browser recognizes the manifest file because in your main page it is referenced using a the following tag:
<link rel="manifest" href="[name here]" />Here is a pretty simple example of how a manifest file looks like:
{
"lang": "en-us",
"name": "My Awesome Pwa",
"short_name": "MyPwa",
"description": "My Awesome PWA",
"start_url": "/",
"background_color": "#121415",
"theme_color": "#121415",
"orientation": "any",
"display": "standalone",
"icons": [
{
"src": "/images/logo.png",
"sizes": "512x512"
},
{
"src": "/images/logo192.png",
"sizes": "192x192"
}
]
}A minimal but functional manifest file should provide lang, name or short_name, the start_url, display and a set of icons.
Set of icons
The icons are used in the home screen of the device that installs the PWA (more on the installation later).
There are several recommendations that vary depending on which platform the PWA will be installed on, but common icon sizes are:
44x44 - app icon.
50x50 - store logo.
71x71 - small tile.
150x150 - medium tile.
192 x 192 - used by Chromium
350x150 - wide tile.
310x310 - large tile.
512 x 512 - used by Chomium
620x300 - splash screen.
Tip!: I found an interesting site that mentions the required icon sizes based on the Operating System (not the browser), and also covers how to create a baseline set of icons without loosing your patience on the way.
You can find the article here: https://love2dev.com/blog/easily-create-a-baseline-set-of-progressive-web-app-icons
For more information about the structure and contents of the manifest file, please check https://developer.mozilla.org/en-US/docs/Web/Manifest
Secure url
All PWAs require to be reachable through a secure connection, that is, one that uses the Https protocol, otherwise no device installation will happen.
Nowadays, having a web app served through a secure connection is a must, and there are plenty of ways to do so without a hassle.
For self-hosting, the service I use to get on running asap securely is letsencrypt but several SaaS and hosting platforms offer SSL out of the box.
Note: the underlying security protocol used nowadays is TLS. SSL is and older protocol that comes from the days when it was the only encryption standard used by http, so the name "SSL" stuck in people's head.
Service worker
The Service Worker (SW for short) is the most complex part of implementing a PWA, because many web developers aren't used to work with service workers
or the technologies they are related to like the Cache API.
A SW is just a javascript file that runs on its own thread, separate from the main app.
It cannot access the DOM as the app does, but that acts as a proxy to it by handling cached resources or intercepting network requests and acting upon them.
Service workers thus improve the web app's performance and experience.
Service workers go through a set of events in order to be activated. The most important events are: install, activate and fetch.
Install happens the very first time the browser detects a service worker is available, so it downloads the code and dispatches this "install" event.
The install handler is where all the other needed resources get downloaded and installed in the cache.
Activate is the second event called, which happens right after install, either when the app is used for the first time or when a new version of the service worker is installed.
The last important event to be handled is fetch, where all the heavy lifting is done in order to allow the app to work offline.
Below is a sample service worker so you can grasp a better picture of the mentioned events and the SW event's life cycle:
const RESOURCE_VERSION = "1";
const CACHE_NAME = `myPwaApp-v${RESOURCE_VERSION}`;
const RESOURCES = [
`/css/bootstrap.min.css`,
`/css/bootstrap-dark.min.css`,
`/css/icons.min.css`,
`/css/site-dark.min.css`,
`/css/site-light.min.css`,
`/js/account-activate.js`,
`/js/account-forgot-password.js`,
`/js/account-login.js`,
`/js/admin-companies.js`,
`/js/cldr.js`,
`/js/colors.js`,
`/js/company-areas.js`,
`/js/company-eventlog.js`,
`/js/CountryCode.js`,
`/js/datatables.min.js`,
`/js/functions.min.js`,
`/js/globalize.js`,
`/js/index.js`,
`/js/notifications.js`,
`/js/pagingNavigator.js`,
`/js/pdfviewer.es.json`,
`/js/push.min.js`,
`/js/serviceWorker.min.js`,
`/js/signalr.min.js`,
`/js/site.min.js`,
`/js/sweetalert2.min.js`,
`/js/toastr.min.js`,
`/js/typeahead.bundle.min.js`,
`/fonts/fontA-Regular.ttf`,
`/fonts/fontB-VariableFont_wght.ttf`,
`/fonts/fontC-Light.otf`,
`/fonts/fontD.ttf`,
];
// Add a cache-busting query string to the pre-cached resources.
// This is to avoid loading these resources from the disk cache.
const RESOURCES_WITH_VERSIONS = RESOURCES.map(path => {
return `${path}?v=${RESOURCE_VERSION}`;
});
// On install, fill the cache with all the needed resources we know we need.
// Install happens when the app is used for the first time, or when a
// new version of the SW is detected by the browser.
// In the latter case, the old SW is kept around until the new one is
// activated by a new client.
self.addEventListener('install', event => {
self.skipWaiting();
event.waitUntil((async () => {
const cache = await caches.open(CACHE_NAME);
cache.addAll(RESOURCES_WITH_VERSIONS);
})());
});
// Activate happens after install, either when the app is used for the
// first time, or when a new version of the SW was installed.
// We use the activate event to delete old caches and avoid running out of space.
self.addEventListener("activate", event => {
event.waitUntil((async () => {
const names = await caches.keys();
await Promise.all(names.map(name => {
if (name !== CACHE_NAME) {
return caches.delete(name);
}
}));
await clients.claim();
})());
});
self.addEventListener("message", async (event) => {
console.log(`Message for sw received: ${event.data}`);
if (event.data.type === "clear-cache") {
await clearCache(CACHE_NAME);
await cacheStaticResources();
}
});
// Main fetch handler.
// A cache-first strategy is used, with a fallback to the network.
// The static resources fetched here will not have the cache-busting query
// string. So we need to add it to match the cache.
self.addEventListener('fetch', event => {
const url = new URL(event.request.url);
//return; // cache KILL SWITCH. Uncomment to bypass the cache-check feature.
var networkFirst = false; // set false to use cache-first strategy, true for network-first
// Don't care about other-origin URLs.
if (url.origin !== location.origin)
return;
// Don't care about anything else than GET.
if (event.request.method !== 'GET')
return;
// we'll only cache resources under certain folders
if (!url.pathname.startsWith("/js") &&
!url.pathname.startsWith("/css") &&
!url.pathname.startsWith("/fonts") &&
!url.pathname.startsWith("/images"))
return;
console.log(`sw: checking request in cache: ${event.request.url}`);
event.respondWith((async () => {
const cache = await caches.open(CACHE_NAME);
const appendChar = event.request.url.indexOf("?") === -1 ? "?" : "&";
const url = new URL(event.request.url);
const removeSlash = ((url.pathname === "/" || url.pathname === "") && event.request.url.endsWith("/"));
const curedUrl = removeSlash ? event.request.url.substr(0, event.request.url.length - 1) : event.request.url;
const versionedUrl = `${curedUrl}${appendChar}v=${RESOURCE_VERSION}`;
console.log(`sw: looking for ${versionedUrl} in cache.`);
var cachedResponse;
if (!networkFirst)
cachedResponse = await cache.match(versionedUrl);
if (cachedResponse) {
console.log(`sw: returning cached response: original=${curedUrl}, cached=${versionedUrl}`);
return cachedResponse;
} else {
console.log(`sw: fetching response from backend: original=${curedUrl}, fetched=${versionedUrl}`);
var fetchResponse;
try {
var cred = event.request.credentials;
if (!cred)
cred = "same-origin";
fetchResponse = await fetch(versionedUrl, { redirect: 'manual', credentials: 'same-origin' });
if (fetchResponse.type !== "opaqueredirect" && !fetchResponse.redirected && fetchResponse.status >= 200 && fetchResponse.status < 300) {
cache.put(versionedUrl, fetchResponse.clone());
}
return fetchResponse;
} catch (e) {
// catch usually means network error
cachedResponse = await cache.match(versionedUrl);
if (cachedResponse) {
console.log(`sw (OFFLINE): returning cached response: original=${curedUrl}, cached=${versionedUrl}`);
return cachedResponse;
}
throw e; // not in cache, and we're offline
}
}
})());
});
async function cacheStaticResources() {
const cache = await caches.open(CACHE_NAME);
await cache.addAll(RESOURCES_WITH_VERSIONS);
}
async function clearCache(cacheName) {
const names = await caches.keys();
await Promise.all(names.map(name => {
if (name === cacheName) {
caches.delete(name);
}
}));
}
If you want to start with very basic stuff, a service worker that only caches static resources (as in the sample above) is enough.
It won't handle all scenarios of offline browsing, but it'll allow you to start with a basic PWA and keep adding "progressively" (no pun intended) the missing parts to the code later.
With all these things in order, when you browse to your app you would see a visual cue (an icon usually) in the agent indicating that the app can be 'installed'.
Here's a real example of that notification:
A nice thing of PWAs is that whenever you update the code in the hosting platform, the app in client devices gets updated automatically.
As a difference with native apps that are downloaded and updated via their app stores, PWAs download the service worker always when a different version is detected by the agent, and if you handled the cache API the right way, the updated resources will also be downloaded behind the scenes without the user noticing it.
Why a PWA anyway?
Nowadays, several well known apps have a PWA version running along their native one.
Take for example Youtube (as seen in the previous screenshot), Spotify, Outlook, WhatsApp, etc.
Having a PWA version available for consumers has several advantages:
Users that don't have administrative permissions over the device can still "install" PWAs. Since the application is in fact a disguised web application, thus handled by a browser engine, its sandboxed from the enviroment and there's no real installation performed in the underlaying operating system beyond adding an Icon to the home screen.
Single source code base: the source code is composed of the well-known html, js and css type of files. They run on iOS, Android, Windows, Linux, etc., and having a single source code base is a big maintenance/deployment advantage and a great cost-saving feature.
Web standards are evolving fast, so the gap between native and PWA is getting narrower more often. With every new web standard that's being adopted by major browsers, there are less things left for the native-only domain.
They can be discovered through web search engines.
But... (there's always a 'but')
Before going full into PWAs, you may also check on your specific needs to:
Deeply integrate with hardware technologies like sensors, bluetooth, nfc, etc.
Expose your app in a store, i.e. for branding/marketing purposes
Make high use of CPU, i.e. for multitasking or compute-heavy processes
Harness any api that's still not available in web but's used on mobile
If you checked positive in any of those points, your best bet is to go native, otherwise PWAs are a great alternative.
Considerations when converting to PWA
Although adding the necessary parts to an existing app to make it a PWA is not that difficult, there are some issues that must be considered beforehand.
Some of those issues -specially in regards of the service worker- can be enumerated as follows:
Caching resources
The Cache API works by storing key-value pairs in the cache that consist of the resource urls and the received responses. This is all done while the user is online. See the emphasis on the word "responses".
You don't use the cache API to store raw contents like the payload of an image or a stylesheet, but instead you cache the response object returned by a fetch request.
Response objects aren't made only of the resource content's payload (raw bytes), but they also contain http headers and status information, like the http 200 status for OK, 301/307/308 for redirects, and so on.
Note: I won't cover the whole Cache API in this post, but I can recomend you a very helpful Youtube series where all the quirks of service workers and the Cache API are explained. The series are called "Understanding service workers" and you may find them here: https://www.youtube.com/watch?v=ZY0WTY-g_js&list=PLyuRouwmQCjl4iJgjH3i61tkqauM-NTGj
Why is important to know the status code of a response?
GET requests are supposed to be indempotent, meaning that no matter if you request a resource once o 100 times, the response should always be the same and no state change should happen server-side.
In real life things are somewhat different, though...
If you cache request responses of static resources like images or style sheets, which usually don't depend on any other conditions (like being authenticated), chances are you won't have any issues storing their responses and consuming them later.
But take into the picture the case where a request to the root path ("/") of an app is triggered in order to get the home page, and the app uses authentication. The response will vary depending on whether the user is authenticated or not.
A response for an authenticated user would return the contents of the home page along with an HTTP status code 200.
On the other hand, an unauthenticated user request would return a redirect to the login page (HTTP status codes between 301 and 308).
Moral: Redirect responses should not be cached.
Redirect responses should not be cached, but should be handled anyway, for instance by leting them flow undisturbed and performa a redirect in the agent.
If you try to cache responses without knowing whether they are a redirect, you may end up associating the "/" route to a login page instead of a home page.
A not carefully designed cache strategy could lead to unwanted results.
Handling redirects properly
A crucial thing the service worker has to do before caching a respone is to check the http status of a (GET) request, and see if its OK (2xx) or a redirection (3xx), along with some additional checks to the response properties.
In the following code snippet, the section in blue shows those additional checks:
var fetchResponse;
try {
var cred = event.request.credentials;
if (!cred)
cred = "same-origin";
fetchResponse = await fetch(versionedUrl, {
redirect: 'manual',
credentials: 'same-origin'
});
if (fetchResponse.type !== "opaqueredirect" &&
!fetchResponse.redirected &&
fetchResponse.status >= 200 &&
fetchResponse.status < 300) {
cache.put(versionedUrl, fetchResponse.clone());
}
return fetchResponse;
} catch (e) {
// catch usually means network error
cachedResponse = await cache.match(versionedUrl);
if (cachedResponse) {
console.log(`(OFFLINE): returning cached response: original=${curedUrl}, cached=${versionedUrl}`);
return cachedResponse;
}
throw e; // not in cache, and we're offline
}The "type" property of the response should not be "opaqueredirect". This value means that a fetch call was made with the "redirect: manual" option set (either by code or by the browser itself), and as such, the originator is supposed to handle the redirection on his own.
Then is the "redirected" property, which should be false to allow the response to be cached, otherwise we're dealing with a redirection.
The final check is done with the status code. It should be some code between 200 and 299 to be valid for caching. This check is also important, because opaqueredirects return no information whatsoever (neither in headers nor in the body) and have a status of 0.
Also, if there was a problem fetching the resource (i.e. it was not found), the status code would be >= 400, or >=500 if there was a server error.
Passing all these checks means we can store the response in the cache to be consumed later.
Static resource patching
Service workers in PWAs can utilize a technique called static resource patching to efficiently update static resources such as HTML, CSS, JavaScript, and other files. This approach allows PWAs to deliver faster updates and improve the overall user experience.
When a new version of the service worker is detected by the browser, it is automatically downloaded and updated. This component then will go through its life cycle: install -> activate.
During install, it's common for SWs to download the necessary files and store them in cache with the Cache api.
Static resource patching involves using service workers to intercept network requests for static resources made by the PWA. When an updated version of a static resource is available on the server, the service worker intercepts the request and compares the version of the resource stored on the client device with the version on the server.
If the versions differ, the service worker retrieves the updated resource from the server and replaces the old version stored on the client. This process is often performed in the background without interrupting the user's interaction with the PWA. Once the new resource is installed, subsequent requests for that resource are served directly from the client cache, resulting in faster and more efficient delivery.
But resource patching could be more challenging than expected:
Versioning and cache management: static resource patching requires careful management of versioning to ensure that clients receive and cache the correct updated resources. It can be challenging to handle versioning and cache invalidation effectively, especially in scenarios where multiple clients or different versions of the PWA are in use. Inadequate versioning or cache management can result in outdated resources being served or caching inconsistencies.
Granularity of updates: static resource patching primarily focuses on updating individual static resources rather than the entire application. While this approach improves efficiency by minimizing data transfer, it can be challenging to ensure the consistency and compatibility of different resource versions. If an update involves interdependencies between multiple resources, ensuring that they are all compatible and synchronized can be complex.
Limited update scope: static resource patching is primarily suitable for updating static resources like HTML, CSS, and JavaScript files. It may not be as effective for updating dynamic content or server-generated data, which often require different strategies such as background sync or real-time data fetching to keep them up to date. As a result, static resource patching may not cover all aspects of an application's update requirements.
Increased development complexity: implementing static resource patching and managing versioning and cache invalidation can add complexity to the development and maintenance of a PWA. It may require careful planning, testing, and monitoring to ensure that updates are applied correctly across different client devices and versions of the PWA.
Limited control over client devices: when applying updates through static resource patching, there is limited control over when and how clients receive and apply those updates. Clients may have varying levels of compatibility, caching behavior, or network conditions, which can affect the consistency and timeliness of updates across different devices.
Caching requests other than GETs
GET request themselves can be divided in two categories depending on the backend resource:
1- The ones that fetch static resources, like images, stylesheets and so on, and
2- The ones that load a dynamically-generated resources from the server.
The formers can be easily cached, but one must take into account whether the url has query string parameters.
When query strings are added, the requested resource is treated as a separate entity as the url without the query string, so it's not the same resource "/images/logo.png" as "/images/logo.png?size=M".
In fact, a single query string parameter may change the whole response depending on the application, and it could generete several types of responses which all should be stored in the cache.
Keep adding variations to this parameter's value, or adding more query string parameters, and you'll may hit a space limit using the browser's Cache API.
The latter type of GET requests is even more complex, and in fact, from a caching-wise perspective, they should be treated as POST requests.
This kind of requests are frequently found in SSR frameworks like php or asp.net where the retrieved contents may vary depending on other factors, like the user's logged-in status, stored preferences, and app-specific logic.
Care must be taken when caching this sort of requests.
POST requests
Dealing with POST requests and the Cache API together isn't usually a good idea, and to my mind is something one barely wants to do.
Because of the complexities explained with GET requests involving query string parameters or dynamic content, POST request can vary the response by including a body that affects the app's internal state.
Even when there is no body in the request, the dynamic nature a POST implies that some kind of state change is made in the backend side, and as such, caching those responses can become quite difficult, sometimes impossible, due to the exponential variations or complexity involved.
Most of the times, service workers should let POST request pass through and reach their final destination undisturbed, and avoid caching any responses whatsoever.
Handling an offline state
When there's no network connection, fetch calls will fail, and the SW has an opportunity to handle those situations to present a graceful experience to the user.
If we rely only on the Cache API to enhance the app responsiveness, and a certain response is not available (cached), the app's offline experience might feel limited, but different approaches can be implemented in such cases.
Not interacting with the Cache API when using POST (or PATCH, PUT and DELETE) requests doesn't mean we should hinder the user's offline experience, and here's where another set of APIs come in hand: the Background sync API, Periodic background sync API and Background fetch API.
Background Sync API
This API lets you defer user requests when the app is offline to a later time when the connection is reestablished.
It works by requesting a Sync event (either in the UI or in the SW), and reacting to it when the app is online again, sending the request to the intended endpoint.
There are several ways to store the request for later use, depending on its complexity, going from LocalStorage to IndexedDB.
Periodic Background Sync API
This API is used to get new content in the background so it is available when the user opens the app.
To be used, a permission request must be accepted by the user, and after that, a periodic sync event must be registered, very much like as with the Background Sync API.
Background Fetch API
This API allows an app to delegate the download of large files to the browser engine, thus liberating itself and the SW from doind that task.
Since this engine know how to handle (fragile) network connections, it's a nice way to offload the processing of large streams (like movies, songs, and so on) and have them ready when the download completes.
Wrapping up...
Converting an existing web application into a PWA shouldn't be difficult, it just takes some time and commitment to put the needed parts in order.
There are things that are set in the application itself, like the manifest file, the set of icons for the home screen and the service worker; and others that are set in the hosting environment such as the certificate to allow for a secure connection.
From the things that are set in the application, the service worker could be considered the trickiest part, however a simple worker that caches the startup static resources could do the job to get running ASAP, and more complex logic could be added afterward, in subsequent versions.
The important things to remember is why would you like a PWA at all:
To access it from a home screen?
To provide an offline experience?
To look like a native app but avoid coding in native platforms?
They could be all, or just one of those motivations, enough reason to embrace the PWA approach.
Comments