Any creative field undergoes periods of divergence and convergence, and technology is no exception. Right now, we’re in the late swing of a divergent phase, with a plethora of new languages, frameworks, paradigms and methodologies so vast that it’s impossible, at this point, even to know a respectable fraction of them. There are “back-end programmers” who know nothing of Javascript, and “front-end guys” who’ve never used a profiler. That will end, as people grow tired of having to learn a new technology stack with every job change, and also as the dimensionality of the bilateral matching problem between engineers and employers becomes intolerable. As painful as convergence can be, such a time will come out of necessity.
For a single company, convergence is often enforced internally, with “approved tools” lists and style guides. Often the communication that drives these decisions takes a familiar form: flame wars.
These partisan fights are common in technology. There’s one set of programmers who believe Java is the only language that has a right to exist and that, if you’re not using an IDE, you’re using “stone tools”. There’s another contingent that thinks everything should be written in Ruby, because that’s what Rails is in and why do we need another goddamn language? There’s a third that’s fixated on C++ on the basis that, if you can’t manage memory, how can you make sure an application performs? For now, I don’t care who’s right in this– really, that depends on the problem, because there clearly isn’t “one language to rule them all”– and language or style comparisons are not what I care to discuss today. I think we can all agree these flame wars are stupid and counterproductive. Do tabs/spaces debates really require long email chains? No, they don’t.
This, if anything, is the thing I dislike second-most about programmers. The worst thing about programmers is that most of them (although this is a largely a product of a corporate environment that rewards mediocrity) lack taste and curiosity, and therefore never become competent. The second-worst is that, among those who have taste, most of the heated debates come down to superficial nonsense. Yes, code quality matters. It’s critically important; it can make or wreck a company. Cosmetic differences, on the other hand, are just not that important or interesting. Tooling choices are important, but rarely as critical as opinionated engineers make them out to be, in the sense that they’re rarely existential matters for the company. Closed allocation is cultural death, and unethical management will ruin your startup, but the difference between Python and Scala is unlikely to be lethal. Can you write performant systems using mostly Python? Of course. Can high-quality, attractive code be written using an inelegant language like C++? Yes, it can. I have strong preferences regarding languages, but I’ve learned over the years not to take them too seriously. More dangerous than picking a suboptimal language is not getting any work done because of partisan bickering.
Language and tool wars are an outgrowth of a careerist idiocy that one would expect software engineers to be above. Engineers engage in this hyperbolic mudslinging because their career needs, in a typical corporate power struggle, often necessitate some control over the technical environment– and that’s often a zero-sum squabble. The software industry isn’t a real meritocracy, but it insists on looking like one, so the object of the game is to convince the team to use the tools with which one is most familiar, in order to become (because of tool fit, not intrinsic superiority) the most productive. This is the problem with technology in the late-autumn phase of a divergent spell: an engineer’s job performance is going to be more of a function of his match with the work environment and tools than of his overall competence, so there’s an enormous incentive to change the game if one can. There’s more individual gain in changing existing technical choices than grinding away at regular work. Most software engineers have learned this, and I can’t count on one hand the number of times I’ve seen a company go through a scorched-earth code rewrite because some douchebag managerial favorite only knew one language, and (of course) that language became the only possible option for the firm. This is terrible for the company that has to reinvent all of its technical assets, but beneficial to the favorite who naturally becomes the master of the new technical universe.
In a world of tight deadlines, short employment stints, and extremely detailed job requirements, this is only getting worse. It’s no longer acceptable to spend months in a “ramp up” period on a new job. It won’t get a person fired, but you’re unlikely to get the best projects if you don’t blow someone away in your first 3 months. So you need to find or carve out a niche where you can use tools you already know, in order to get some macroscopic achievement (a “launch”) as soon as possible. This forces companies into one of two undesirable alternatives. The first (sprawling divergence) is to let everyone use the tools they like, and the predictable result is “siloization” as engineers stick to the tools they already know how to use, rather than take the time to understand what other people are doing. Taking these systems into production and supporting them becomes a nightmare, because they often have 7 different NoSQL databases to support what’s essentially a CRUD app. The other (enforced convergence) is to require certain uniformity in development. The company becomes an “X Shop” for some specific language (or database, or methodology) X. That’s when the flame wars begin, because everyone is going to have an enormous stake in the choice of X. Now, it’s not enough for X to be the best tool for your project; you have to shove X down other peoples’ throats in the company, or you won’t be able to use it at all. This also falls down as necessity requires exceptions to the uniformity requirements, and the decision of who can get an exception becomes immensely political.
In truth, we as programmers are, in many companies, behaving like executives, spending more time arguing about how to do work than doing the actual work.
This, I think, is the fall-down calamity of the software industry. It’s why 50 percent of people who are professional programmers won’t be, seven years from now. Software engineering is all about improvement– building things that didn’t exist before, automating tedious processes– and it’s only natural that we want to improve ourselves. In order to get better, one needs a reliable stream of increasingly high-quality projects. What makes great software engineers extremely rare (and they are) isn’t a lack of talent alone; the limiting factor is the paucity of quality work. At a certain point, one reaches a level where it’s difficult to get quality work, even for a qualified (or overqualified) engineer. You end up spending more time convincing managers to give you good projects (playing politics, acquiring control of the division of labor) than you actually get to spend on the work. People get enervated and drop out, or they become managers and lose touch with the day-to-day process of writing code. Tooling wars are one component of this nasty slog that programmers have to enter in order to have a shot at the best work.
What’s the solution? I mentioned the alternative to uniformity, which is the “everyone uses whatever they want” divergent approach that often generates an intolerable support burden. It can work, but it will fail in the typical hierarchical corporation. When you have this proliferation of creativity and ideas, you need two things. First, there has to be a pruning process, because some of what is produced won’t be very good. Managers never get this right. Their decisions on what modules to keep (and, thereby, force engineers to maintain and use) and which ones to burn will be based on political factors rather than code or product quality. Second, it requires extremely strong cross-hierarchical communication, because it requires that people begin to make technical choices based on the long-term benefit of the group rather than short-term “productivity” in an attempt to acquire status. People who are building technical assets will need to actually give a damn about making their clients succeed, as opposed to the selfish, parochial goal of pleasing managers. You won’t get this kind of communication and pay-it-forward attitude– not as prevailing behaviors, rather than occasional serendipities endowed by the comfortable– in a closed-allocation company.
The open-allocation sociology is one in which there is cross-hierarchical collaboration because the conceptual hierarchy necessitated by the problem doesn’t congeal into a rigid hierarchy of people. Because there’s so much internal mobility, other teams are potential future teammates, and people will generally treat them well. What this means is that, while there will be divergence in the name of exploration and creative license, people will also take care of the convergence tasks, such as integrating their work with the rest of the company and teaching other teams how to use the assets they generate.
For a contrast, the closed-allocation sociology is one in which people strive, in the short term, for the image of superior productivity, in order to advance rapidly into a role where they control the division of labor instead of being controlled by it. This encourages people to diverge carelessly, for the twin purposes of (a) appearing highly productive and useful, and (b) creating personal security by generating technical assets that, although management can be convinced of their importance through political means, become so opaque to use as to leave other teams beholden to their creator. Of course, this productivity is illusory, the reality being that costs are externalized to the rest of the company. The heavy-handed, managerial antidote is to mandate convergence, but that inducing battles regarding which technologies and approaches live and which die. The result are hyperbolic, counterproductive and fear-driven arguments that devolve into the technical flame wars we all know and loathe.
As long as we have hierarchical corporations and closed-allocation regimes where one must either control the division of labor or be controlled by it, we will have endless flame wars over even the smallest of technical choices. The stakes are too high for them not to exist.
