Problems that get us stuck often create a feeling of overwhelm. It feels like there is too much to manage, and we see all the ways the problem has different obstacles and sub-problems. A practical example comes from learning. A skill is often nested between other knowledge and sub-skills, like getting your web application to talk to a database. On top of that, it could be that you are operating in a new programming language, or any other context that is another variable you don’t feel fully confident in.
Herbert Simon spent his career on how people solve problems with imperfect information, and one idea runs through most of it: the problem space. Picture the whole territory you have to cross to get from where you are to a solution. Some of that territory is mapped. A jigsaw puzzle or an exam question is what Simon called well-structured, meaning the start, the finish, and the legal moves are all defined in advance, so the only thing left to spend is the energy of moving. Many real problems are ill-structured. The territory is large and unmapped, and deciding where to enter it and which direction counts as forward is not the preparation for the work. It is the work.
I think the reason unmapped terrain is so expensive is that every alternative you have not ruled out is an alternative you are still holding. The database question stays open while you are also holding the language question, which stays open while you are holding the design question. Even when none of them is especially hard on its own, holding all of them at once is what produces the feeling of too much to manage. And it gets worse when you move. Every direction you take opens problems that were not visible from where you started.
So the move is to cut the space down to a size you can actually explore. Not small, since it will keep growing as you walk through it, but small enough to enter. But that is only half of the idea. You cut every time you pick a language, choose a library, or accept a deadline. The part that decides the outcome is not how much you cut away. It is which direction you cut in, because that determines which solutions remain visible afterwards and which ones disappear from view. Most of the time you make that choice without noticing you made it.
It is useful to keep in mind that even though you cannot know in advance everything a constraint will remove, you can at least ask what it is selecting for.
I have tried to apply this in deepening my knowledge in programming. I dislike frameworks and tools that abstract away what I’m learning, because in the end I would just learn the conventions of said framework. So the self-imposed constraint is “standard library first”, and I handle the new issues that arise out of the principle as they come.
What I think that constraint actually selects for is worth being exact about, because it is not “better code”. It produces understanding at the level of mechanism rather than convention, which means more of it transfers to the next language and the next problem. And it produces slowness. Not as a side effect but as part of the same thing, since the mechanism is only visible while you are the one doing the work. What it does not produce is shipped software, at least not quickly.
A framework selects the mirror image: speed, coverage of edge cases I cannot yet see, and a working system whose insides I do not have to look at. For experienced people, the “black belt” programmers, that is the right trade almost every time. Neither constraint is better. They are aimed at different results, and I am aimed at the first one for now.
Which leaves the question of when to stop. If a constraint is aimed at a result, then it expires when the result does, and the result I am after will not stay the same. At some point I will want to ship more than I want to understand, and that day is not going to announce itself. So the constraint stays for now, but I will continue to pay attention to what I start missing, because what I start missing is the first sign that I am aimed somewhere new.
I ran into Simon’s problem space in David Epstein’s new book Inside the Box, which is about constraints in different forms and the benefits for people working with them. Which is what sparked this post.