We've Been Doing This The Wrong Way

(The following is an unedited, stream-ofconsciousness diary entry. The purpose is to explore new ideas that may or may not pan out. Enter at your own risk)

My journey writing the Info-Ops books has been quite interesting. For many years I learned to code, lead, and architect teams and systems. I enjoyed this. It was something I could pick up (mostly) easily and it fit into my natural personality of enjoying activities and hobbies that have lots of technical details. (It's one of the reasons I love being a pilot so much)

But once I started working with teams-of-teams, process, and changing large groups of people, things did not work the way I expected. Between 2005 and 2015 I found myself increasingly giving good, practical advice only to see it "bounce off" the folks I was trying to help.

It lead to an eventual crise du coeur, or crisis of the heart. I found myself repeating the same things over and over again and getting the same substandard results. Was I doing it wrong? I watched other folks in my industry, and hell if they didn't seem to be in the same spot. It's led friends of mine to pursue all sorts of seemingly tangential skills, like psychology, playing with legos, juggling, magic, and so forth. At first I thought many of these things to just be marketing gimmicks, a way to get noticed by the industry. But as I watched and participated, I came to realize that these were honest efforts to get more traction.

That revelation was disturbing on a deep level.

Pulling at the First Thread

Two things changed this for me.

First, I decided on a clear mission and supporting habits instead of a series of small goals separated by sometimes herculean efforts. Frankly, I got worse at learning new stuff, the things I was learning didn't seem to be helping folks much anyway, and what is the point of a series of small steps that lead to a destination that looks like the same place you're in, anyway? Not only did I get tired of watching my friends repeating the same things over and over again, snatching at this or that new shiny in an effort to stay relevant, I got tired of seeing it in myself as well.

I decided my mission: to help technology developers lead happier and more productive lives. In retrospect, I think this decision was the most important and impactful one of my career.

Second, I picked one thing and tried to take it apart from top-to-bottom in order to explain it and help folks optimize. I didn't want slogans or aphorisms. I didn't want anything that looked like selling or marketing. Instead I just wanted to understand one thing at a level of depth that I could bring new things to the table. Hopefully some of these things might be useful.

I looked around using my experience and the experience of others. Looking across hundreds of companies and thousands of projects, the common failure point between people and tech seemed to be backlogs. If you taught technical work like coding and testing, once you started improving teams you ran up against backlogs that were task systems, WBS, or untestable. It was a piece of crap that most teams did not feel empowered enough to fix. Likewise, if you started with programs, orgs, and process, as things percolated down you ended up in the same spot: backlogs that didn't work. For folks in upper management, backlogs became overly-technical and unwieldy quickly, leading to less actual management and more shuffling reports around. You'd ask simple questions, get insane answers, and when you probed you ran into huge designs and backlogs that were logical, needed, and prevented any progress in solving the problem. Worse, these things got worse year-by-year.

Everybody, all of the tech folks, had already done design. Frankly, design is derivative anyway: if you don't know what you want, any kind of design works great. If you know exactly what you want, you'll also know exactly what qualities the design must have. The common problem looked like it was conversations involving "What do you want us to create?"

Watching the Entire Sweater Unravel

So I grabbed that one thread, backlogs, and started pulling. Yup, it was a topic that had been done many, many times before, but no matter how much was done, it all seemed like adding a bunch of bullshit on top of a conversation around "what is it you want?" instead of how that actual conversation happens.

We tend to layer complicated, technical, and perhaps interesting things on top of other things we don't understand and then declare them "solved". I didn't want to play that game no more.

So I started pulling on that backlogs thread, moving into analysis in general, then design. Pretty soon, instead of a sweater I was left with a big pile of threads on the floor. They didn't look like they'd be good for anything. I could hear a disparaging in my mind and in the tone of voice of others. Gee, Daniel, now you've got a lot of nothing. Hell, anybody could create or demonstrate a pile of threads, ie, at some point you can generalize anything into a bunch of poorly defined concepts. Our job is to do something useful, not take everything apart into small; pieces of wonky bullish.

I get it.

But I wanted a completely new outfit, not simply a hat or a sash to put around the sweater we already had. That didn't work for me, and looking at others in the industry, it didn't seem to be working much for them, either. (Aside from making a lot of money, of course! Remember: missions tend to clarify your life. Goals tend to complicate it.)

Learning to Sew

Once I broke it all down, I started putting it all back together. I ended up with a bunch of new stuff I hadn't seen before. Probably most of it was weird, ugly, or impractical. That's fine, I've never done this before. But some of it was not.

Even more surprisingly, these interesting and useful new concepts fit into their own system of creating technology, and this system matched up with many of the patterns I had seen in good coders and companies. The principles were being validated by folks who never knew them.

I started from the ground-up, and I ended up with a new system of creating technology for folks, one that matched up with what great teams and companies were already doing.

Ok, so perhaps I'm creating just more of the same overly-prescriptive wonky bullshit that all of the other process guys have created before me, but fuck, it was new bullshit. I'd much rather go down a new road than the same old other ones. Plenty of other folks have that covered.

Flipping it Around

There's a scene in the movie Planes, Trains, and Automobiles where our heroes accidentally get on a freeway going the wrong way for the lane they're in, ie, heading directly into oncoming traffic. People try to tell them: you're going the wrong way! That's bullshit, they reply, how do you know where I'm going?

Not only is this a joke, there's actually something quite profound here: acknowledging that something is broken works at a much higher level of abstraction that putting the broken thing together in the first place. Generalized negatives have magic powers that exacting positives don't.

Failure avoidance is easier to understand, teach, and implement than telling people exactly what to do. It didn't matter where our heroes were going. It mattered that they were doing it in a potentially fatal manner.

This clicked with what I was observing in the larger development community as "experts" like me tried to help devs improve. Everybody wanted to program their way out of failure. They wanted principles, tactics, rules, evaluations, certifications, and more. Tell me what to do. And don't be vague. Tell me exactly!

As experts, we realized that the more exact we were the less it looked like it applied to every situation, but at the same time we didn't want to become so generalized that we were useless. This has led to a lot of effort over the decades of trying to make an abstract thing much more specific. It's also how, over and over again, simple concepts about making good tech turned into process nightmares.

Turns out, it wasn't the process. It wasn't one thing being better or worse than another thing. Our basic concept of how to proceed forward was flawed. We were going the wrong way.

Twenty-First Century Technology Process

When you start building these new outfits from all of those threads lying around, you end up creating things you didn't plan. This is what is happening to me as I strive to finish up the series. I started with friendly, generic fun history and philosophy. I set up a base and used that base to move into practical how-to-code better territory. Not only was this more practical for readers, it started matching up with what others were doing. Most of these folks were extremely successful.

Now, however, as I move from creating-tech-better to creating-better, these concepts have painted me into various corners. To continue our overloaded and somewhat tortured metaphor, I've finished sewing up everything but the hat, and I only have red thread left. Guess what? We're getting a red hat, whether we wanted one or not.

One of the most surprising places I ended up that I didn't expect was the concept of "Negative Stack; Positives Don't". It has implications far beyond backlogs or coding. Additionally, it seems to match up with what practitioners in other fields are exploring, like the mathematicians and physicists working on Constructor Theory.

It is never going to work to describe how to create technology (program computers). We're always either too generic or extremely precise about things that are not applicable or not important. Instead of finding success, we need to talk much more about avoiding failure.

In every other area of life, success means learning how to put together and use extremely small pieces using precise and prescriptive language into large things that are useful. In the area of philosophy-to-science, which includes technology creation, it does not. In fact, it's an antipattern. Here we have to use extremely large, vague, and proscriptive language in order to mark out enough negative space so that a productive area of inquiry can then begin.

It doesn't work here because all of those small pieces and detailed instructions are built on previous efforts and describe things we may no longer need. Worse, all of these pieces and instructions are cumulative, getting more and more unmanageable no matter how good they were to start with.

Good intentions don't count, and nobody should care about past successes, only future ones. Our intuition fails us. If it feels good, don't do it. (Sorry, couldn't resist that pun)

Markham's Guaranteed, Surefire Road To Success

Don't do stupid shit.

Specifically, have an objectively-measurable list of the top failure modes and actively manage to avoid them.

I propose that each year the technology industry vote on and refine a top ten list of "ways we keep screwing up creating tech". I further propose that your organization or industry do the same thing, since there are failure modes common and specific to each problem domain and situational context.

Will this give us a successful way to create technology at all levels? Yes.

What's that process going to look like? That's unknowable.

How do you know this is the right thing to do? You don't. That's the beauty of it. You can never be right, but you're always wrong to some degree. Minimize that.

Won't this just create some other stupid system of testing or certification? Yes/No. It will create a constantly-shifting yearly list that you strive to complete. That's like certification. But it's never the same and everybody will strive in their own way. That's like natural learning.

How do you know that we'll pick the right things to avoid? I don't. In fact, just like learning ATDD or TDD/TCR, I expect the initial efforts to be quite bad. But just like ATDD and TDD/TCR, we'll get better over time, and as things screw-up, like they always do, we'll adapt to evolve. That's a damned sight better than anything we currently have.

Out.