We engineers tell ourselves and others that we like solving difficult problems. This statement, even though it’s true, is also overly broad. What problems are considered difficult? And which are worth solving in their own right, and not just because they’re difficult?

Until someone solves the difficult problem of human mortality, we’re all stuck with the need to aggressively budget our time, lest we run the risk of squandering what little time we have on things that don’t matter. Selectivity of this kind is probably an evolutionary thing: a large predator can’t survive by hunting prey that’s too small — individually and collectively — to sustain its organism. It’s no wonder, then, that we humans tie so much of our self-esteem to engaging in activities that we consider impactful, important, or simply fun.

This brings me to web development: once the Wild West of the engineering world, the place where web designer hustlers struck web gold on behalf of early web pioneers, this gold mine has turned into a salt mine. What was in the late 90s and early naughts considered acceptable sacrifice for the sake of striking gold as a kind of great societal leap forward, has turned into a desperate struggle to survive endless hours of coughing up salt dust and rock shards just to make another picture sharing or “Uber for $foo” or dull business CRUD web app.

The point I’m working up to is this: developing serious commercial applications atop the currently available “web platform” (however you like to understand the term) is not a difficult challenge worth solving in its own right. Web development is merely a form of mediocrity arbitrage.

Let’s define “arbitrage” before proceeding further. Among other — smarter — things, Wikipedia states:

In economics and finance, arbitrage is the practice of taking advantage of a price difference between two or more markets […] For instance, an arbitrage is present when there is the opportunity to instantaneously buy low and sell high.

We can thus flesh out mediocrity arbitrage as follows:

In this world, the web developer is both king and slave. He commands high prices for his services, yet he’s unable to ever make it out of the salt mine.

Why the bleak view, you ask? Let’s take a brief tour through the web world. Many of you will undoubtedly be familiar with the pains of making anything of consequence happen in the browser. Sure, it’s easy to get started and knock something out quickly using commonly available frameworks. Unfortunately, this only lures even experienced practitioners of the software craft into a false sense of security. Tools built to enable a quick sprint don’t often fare well during the marathon that responsible, long-term engineering ultimately is.

First, there’s the runtime. Imagine a balkanized landscape, where even the most well-intentioned attempt at standardization is met with interminable waits while multiple vendors — open source and proprietary — take their sweet time to implement the fledgling standard. New web technologies take forever to achieve any significant market penetration. This isn’t market penetration: it’s a long, painful, bloody burrowing through a rock that just does not want to yield.

Secondly, JavaScript is still king of browser programming. Conceived over the course of ten days in 1995, JavaScript is an understandably poor choice for building, well, anything! There are no types, scoping is a crazy mess, and Twitter is full of running jokes about various inane things that JavaScript does. Being fast and ubiquitous is not sufficient to enable programming in the large. Serious commercial projects require more oomph, and JavaScript simply can’t deliver it. Sure, there are various improvements in the pipeline, but let’s see how long it takes to drop the polyfills that we have to keep hammering into place until the horde of browser vendors brings their products up to date with newer standards.

Next, the styling mechanism is notoriously difficult to use properly. Vendor differences aside, CSS has scaled poorly with the various uses we’ve thrown at it. I will, however, retract this statement if one intrepid volunteer from the audience will endeavor to explain in simple terms the difference between absolute and relative positioning. Go ahead, give it a try.

Frameworks in web land deserve a beating of their own. Their proliferation has become the butt of many jokes, but I’m personally not laughing. How is it that the collective brainpower thrown at the problem of making web development bearable has failed thus far to produce a solution that can last longer than a year or two before being supplanted by some new hotness of the week? Even juggernauts like Angular aren’t immune to this never ending churn: Angular 2 is on the horizon, yet it’s so vastly different from Angular 1 that many teams have no plans to migrate to 2 at all due to the high cost of what will undoubtedly become a full application rewrite. The tradition of leaving previously built applications eating the dust of JS framework developers moving at breakneck speed will, without doubt, continue unabated.

At this point I’d like to take a short break from the doom & gloom in order to look at some positive developments. A major direction of research has been the relegation of JavaScript and CSS to the status of compilation targets for other, higher-level languages. Many of us have come to love CoffeeScript, TypeScript, ClojureScript, PureScript, Scala.js and many others I must be forgetting. CSS has found some new life as the underpinning for Sass and LESS, which both do a pretty good job at making stylesheet composition a somewhat less painful exercise that it would be without such tools.

Still, not all is peachy here: writing a commercial-grade project in any of the languages targeting JavaScript as a compilation target is far from a trivial matter. The problem of gluing plain JavaScript libraries to newfangled projects built in TypeScript, for example, requires the tedious creation and maintenance of interface files. Determining compilation order and enabling distribution of compiled artifacts are also problems which are far from solved. Painful experience taught me to tread carefully here: demarcation zones between not fully matured programming languages can be dangerous territory. Many an early adopter has perished there.

Lastly, I’m going to harp on for a bit longer about developer tooling. Recently I was showing my wife the ropes of full-stack-ish development. After a whirlwind tour of what a simple but sufficiently modern web app looks like, she asked what I initially considered a silly question:

Why did you have to set up two separate package managers (NPM and Bower) before writing even a line of code?

This got me thinking about the developer tooling landscape in web land. Turns out it’s a city-killing garbage fire! Here xkcd #927 seems particularly applicable. I went ahead and googled for “javascript package manager”, and oh boy! I found nine of ’em:

I truly am in awe of the tenacity with which the JS community appears to produce new package managers. Perhaps the creation of a brand new package manager is some sort of developer’s rite of passage. Many of these seem derivative of each other. Functionality is duplicated. Confusion ensues as a natural consequence of so many different options thrown at the unsuspecting novice. This situation is replicated in many shapes and sizes in other functional areas; just look at Grunt vs Gulp, or Browserify vs webpack, or Uglify vs Closure, or … The list goes on. You’re lucky to find yourself struggling to pick between a measly two reasonably well-maintained and full-featured options. What happens when there’s a bunch, but none fit the bill? You end up rolling your own and … Wait, I think we’ve come to my main thesis at last!

I think that we’re attacking the wrong problems in web and/or JavaScript land. The cards are stacked against us; the game is rigged; the bookie will collect his vig no matter what; the house always wins, and we’re mere hamsters running against an invisible wheel that imprisons us all.

We can’t win against an opponent as formidable and as well-entrenched as institutionalized mediocrity, which is what web technology is, through and through. Looking over my audience I suspect — no, I know! — that each and every one of you realizes, deep down inside, that we simply can not win here. Instead, many highly intelligent people have become content with engaging in a game of mediocrity arbitrage.

Mediocrity arbitrage is happening because we must engage in it. Companies pay us to develop web apps because they must deliver their services over the web. Consumers use web apps because they must do so, or else how will they find a way to bridge the gap between themselves and an increasingly diverse base of service providers, all over a distributed electronic medium that mere mortals will never understand?

Mediocrity arbitrage is our way of extracting a respectable living by exploiting a fundamental inefficiency that exists in the way applications are built. I think there’s absolutely nothing respectable in trading off a status quo steeped in mediocrity. I think it’s outright shameful that we, the engineering community, have not been able to come up with something better after more than 20 years. What a damn shameful waste!

Where do we go from here? We’ve seen the native-apps push make a convincing case, yet the deluge of web development work and newly built applications has shown no signs of abating. There’s the new invisible app concept, which seems to have a lot of promise, but could still go either way. The path forward isn’t quite clear, else it wouldn’t be my “thought leadership” that will light the way.

I want to leave the audience with one parting thought: take a long, hard look over your place of employment, be it a scrappy startup or investment bank, a hospital or non-profit, a government facility or hedge fund, an entertainment giant or freelance development shop. Think back to a time when you and your comrades had wasted countless hours bashing heads against some aspect of a web app that does not work, can not work, but yet must work. It must work because your organization’s mission depends on it. Do you think it’s fair to stake so much on technology that is seemingly tailor-made to perpetuate more misery and mediocrity? Are you sure that we — the engineering culture — couldn’t do better? Imagine the difference a better technical base would have made for your business.