I am thinking about information theory. Premise 1: the quantity of information is growing. Premise 2: uniqueness and dissimilarity from surroundings makes any individual piece of information easier to find. Premise 3: much effort goes in to classifying information so that it can be grouped with other similar things.
So: If our efforts at information classification are successful then the increasing quantity of information will necessarily grow the classes which will decrease the uniqueness of any one piece of information in the class rendering it harder to find.
This yields an interesting corollary and a recursive response with diminishing returns.
The corollary is this: while it is harder to find any given piece of information, it is easier to find a piece of information that is reasonably similar to the desired item. This is usually acceptable for “research” and discovery oriented searching. It is absolutely infuriating when that one thing you need is the only one that will do and you are thwarted in your location attempts by a deluge of similar but useless items.
The recursive response with diminishing returns is this: within any class attempts are often made to sub-classify to ever increasing levels of “granularity” or descriptiveness. Yet the intent of classification is inherently negative; it is abstracting rather than adding detail. It is the zooming out on the map where small features are paid in exchange for the bigger, aggregate, picture. So when you seek to add detail with an abstracting practice your ends end up competing with your means. Furthermore, if you take this practice to its extreme, where you have the most description possible on a piece of information, you end up replicating the information entirely.
This ups your chances of finding it in your class (because now you have 2 identical items in the same class) but you have not gained any efficiency or descriptive benefits. Rather, you have introduced a whole new host of problems and challenges. Like, which one is the “real” one? How to keep them synchronised? And, most importantly, when you have a class with a single member, you have not gained anything for all your classification (abstraction, aggregation & grouping) efforts so why bother in the first place?
Therefore, the extent to which we gain benefits from the classes and groups and not the individual items should drive our classification and similar efforts (tagging, metadata, URI, linking, graphing, parsing etc).
Where unique items are needed I am still truly perplexed on whether or not there is a better way than a brute force, full-index-and-scan-for-matches approach. This is both inefficient and slow.
Even semantic technologies (of which I am strong proponent) are, at their core, intent on the aggregative approach. It is just that instead of aggregating documents and pages they aggregate words. At the end of the perfect semantic process you are left with an amalgam of concepts correctly inferred from words. The intersection or graph of those concepts presumably points to a piece of information. But this is simply a more granular classification process.
Inferential processes move from specific to generality. Deductive processes move from generality to specific. So as information quantities increase and we rely on inferential processes to describe the groups of information we inevitably move deeper into an informational Mandelbrot set where we expose a never-decreasing organisation of concepts that are close but not quite what we’re really after.
So the next time you lose your car keys and you realise that it’s little comfort that you see your house keys right there, it’s probably best to just wait and stumble across them later. When you rely on serendipity, you’re often pleasantly surprised. Maybe that’s why so much stuff out there isn’t good, but rather good enough.
Somehow though, I think it ought to be better.
