Intellectually Tractable Code

The task of writing software consists in large part of minimizing the intellectual burden necessary to completely understand a problem and its solution. In fact, I would argue that our intuition about how "clean" a piece of software is is, in essence, a measure of how closely our understanding of a solution matches the "essential" complexity of an ideal solution. A "clean" module will have a low intellectual burden for complete understanding.

As an example, consider two alternative library implementations of a singly-linked-list data structure:

• In implementation A, list operations include semaphore locks, to ensure that the list is always in the appropriate state.
  
• In implementation B, there are no locks - it is assumed that the caller will always take care to ensure that list manipulation / use is appropriately protected for concurrent access.

It is my contention that implementation B is cleaner, for two major reasons:

1. There is no one design decision for the locking semantics that will be universally applicable. For example, there may be state-variable relationships to be maintained that cross the locking boundary for that entry on the list (like "links 1 and 2 must be added and removed from two different lists at the same time"); the type of locking is set in stone at compile-time, though it may be better to be different for different scenarios (e.g. lock using semaphores for this list; lock using critical sections for this other list; no locks necessary at all for this third list). This means that the developer must _still_ reason about locking in order to use this module effectively - only now it isn't as obvious that she needs to do so, or (to another developer) it isn't as obvious if she _has_ reasoned about locking to protect the relationships between her state variables. Run-time use is less intellectually tractable.
   
   
2. It adds unnecessary dependencies for the linked-list module to work. Linked-list maintenance is well-suited to use in a single-threaded environment, but embedding locks within the linked-list maintenance code prevents deploying the module to environments that lack appropriate locking primitives. Unnecessary dependencies can also lead to layering problems - as when the linked-list maintenance depends upon the scheduler's locking primitives, what happens if the scheduler depends upon the linked-list module? What happens if the scheduler depends upon the linked-list module during initialization, prior to the locking primitives becoming available for use? Instantiation is less intellectually tractable.

There exist general guidelines for making code intellectually tractable, which I will try to expand upon in later posts.

Mastery Against Generality (or, "Keep It Concrete, Stupid")

LtU pointed me to Design Principles Behind Smalltalk. I'll quote the first design principle named, because it illustrates what I think is a mistake in reasoning common to those of a mathematical bent:

Personal Mastery: If a system is to serve the creative spirit, it must be entirely comprehensible to a single individual.

The point here is that the human potential manifests itself in individuals. To realize this potential, we must provide a medium that can be mastered by a single individual. Any barrier that exists between the user and some part of the system will eventually be a barrier to creative expression. Any part of the system that cannot be changed or that is not sufficiently general is a likely source of impediment. If one part of the system works differently from all the rest, that part will require additional effort to control. Such an added burden may detract from the final result and will inhibit future endeavors in that area. We can thus infer a general principle of design:

Good Design: A system should be built with a minimum set of unchangeable parts; those parts should be as general as possible; and all parts of the system should be held in a uniform framework.

I strongly sympathize with the point of view outlined here. If one can master simple, general principles, then that reduces the burden for understanding some set of more specific ideas, and can potentially greatly increase the number and scope of the ideas one can understand and use at any given time -- it can improve one's intellect.

That said, it is considerably more difficult to impart understanding of general ideas than of specific ones. If this isn't immediately obvious to you, consider the order in which you learned some mathematical concepts. Take the following problems:

1. If joey has three apples, and gives two away, how many does he now have?
   
2. Solve for x: x = 3 - 2.
   
3. Prove that the addition operation under the set of integers modulo some constant forms a group.
   
4. Give an example of a non-Abelian group.
   
5. What the hell is a left-adjoint functor?

I think you can be expected to gain mastery of each of these problems in the same order in which the problems are listed. Each problem is more abstract than the previous, and each successive problem is, in a sense, simpler and more general than the previous. But each successive problem is also, to my mind, more difficult than the previous: We needed to understand the more specific ideas before we could be expected to generalize. The mechanism for "good design" quoted above can probably be considered to be in some tension with the stated goal of maximizing "personal mastery" of the system.

The implications of this tension are, I think, important. In particular, under the assumption that a more abstract understanding of a problem domain can make the problem more tractable, it's usually in any given author's interest to move "up" the "abstraction ladder", in order to better solve a problem herself. To the extent that this means she happens upon a good solution faster than others, this is all to the good. But to the extent that this means she happens upon much different solutions than others would, then however much more "elegant" her own solution is will be weighed against the cost in comprehension for others she works with.

What's Missing

I believe, but will not attempt to demonstrate here, that the widely used applications on the Internet are great for people looking to encounter information about subjects that they already know they are interested in. Google will generally find us the Wikipedia article for just about any search. This formulation hints at what I think is a big shortcoming in these tools: getting information to a decent depth on any topic is much easier than determining the breadth of topics in which we would want to be interested.

I mean:

• Pandora Radio will be able to tell you that, if you like Beirut, you will probably like Andrew Bird and Devotchka. If you like Andrew Bird, it will tell you that you'd probably like Beirut and Devotchka, and if you like Devotchka it will tell you that you'd probably like Beirut and Andrew Bird. But you won't easily find out, starting from any of these bands, if you like Faye Wong. How can we find out?
  
• There was a lot of discussion on a number of political blogs recently about Epistemic Closure, a term used in this context to describe a conversation that occurs only between individuals who already agree on the topic discussed, and that has the effect of cementing the participants' opinions. This is certainly facilitated by the Internet.
  
• It's been argued that America is self-segregating into demographically and ideologically similar clusters (i.e. geographically constrained units in which within-region variation is much lower than between-region variation).
  

A theme of this blog is the danger associated with too much social learning (too much "received wisdom"), and too little individual learning (including techniques like deriving results from first principles, or from experimentation). Obviously some balance needs to be maintained, and I believe this balance will be different for different subjects, and for different people. One thing I have tried (and am trying) to contribute is a discussion about what this balance should be, under what circumstances. A mathematician probably believes that she needs to understand all the ideas she uses for her own work thoroughly. In my first undergraduate statistics courses, we were taught how to use particular statistical methods, but were taught almost nothing about how or why they work. The mathematician favored individual learning, the statistician (at least at that low level) favored social learning.

In this post, though, I'd like to pose the question: what is the best way to leverage the breadth of communities on the Internet, so that we can discover things in domains in which we haven't shown much interest before? How can we easily discover things that will challenge our beliefs, instead of reinforcing them? If our beliefs are never really challenged, we'll never really know who we are as individuals... Some things are happening in this area already. I have some ideas of my own. I'll try to expand on this later.

Population Variance

Trying to put down why I think other animals are not as cultural as humans, I think it's important to go off on a little tangent about the importance of variation in traits within populations... It'll take me a little while to get to stuff I consider less appreciated (and therefore more interesting), so please bear with me.

Darwin formulated his theory of evolution while ignorant of genetics. Rather than the atomic inheritance model developed by Mendel and others, Darwin believed in blending inheritance - offspring would tend to be about halfway between each of their parents in any trait. I know R. A. Fisher pointed out that it is very difficult to reconcile blending inheritance with evolution (though I don't believe he was the first to make the point), because blending inheritance naturally reduces the variance of a trait in a population from generation to generation. In fact, I believe variance is supposed to be halved in each generation, when each child is thought to be the mathematical average of the parents. The existence of wide variation in nature, when coupled with blending inheritance, implied an impossible amount of adaptively more-or-less neutral mutation.

Hardy and Weinberg showed that this is not the case for Mendelian inheritance - for each generation reproducing by mixture of atomic genes, in a large enough population, with no selective advantage between genetic variants, and with a couple of other constraints, there will be no change in population variance in genes, or in the traits expressed by these genes. This equilibrium is actually a pretty strong force in a large number of species - in other words, the current gene pool has a lot of inertia, and evolutionary forces will generally act very slowly.

Let's take an example: Peter and Rosemary Grant were able to observe evolution occurring in Darwin's finches through two severe ecological changes - in one case, a drought, and in the other, a flood - and watched the distribution of beak sizes in the population change in response to both events - in the one case, the beaks became shorter and wider to crack open the tougher seeds during the drought, and then longer and narrower after the flood to more efficiently access the softer, smaller seeds that then became abundant. To give a sense of what may be happening evolutionarily, I'll construct a deliberately simplistic genetic model of beak shape: say there are four genes, each with two alleles, that determine the beak shape of a finch. For each gene, one of the alleles will cause the beak to be more stout, and the other will cause the beak to be more lean. We'll call the "stout" allele 0, and the "lean" allele 1, and overall beak shape is determined by how many "stout" and (by definition) "lean" alleles the organism possesses. Any individual in this population will then have K stout alleles, and 4-K lean alleles.

I've set this up to deliberately produce a binomial distribution of beak shapes (though there are complications even here - I note these complications in order to ignore them). In our model, we'll say that after the drought individuals with 3 stout alleles and 1 lean allele will have the optimal beak shape; after the flood, individuals with 3 lean alleles and 1 stout allele will have optimal beak morphology. In neither case should any allele be completely eliminated from the population, which means that there will always be the potential to adapt to the flood after a drought, and to the drought after the flood.

This leads directly to what I think is the core of why culture is not nearly so intensely used in non-human animals as it is in humans: cultural evolution includes a lot of blending of received information, blending inheritance reduces population variance, and reduced variance lowers the ability to respond to ecological change. I may unpack this more later...

Culture and the Central Limit Theorem

James Surowiecki famously applied the Central Limit Theorem of statistics to market behavior in his book The Wisdom of Crowds. I have not read the book, however the wikipedia page seems coherent enough (though I disagree with some of it), and anyone with a background in statistics is probably already familiar with the idea. I do, however, want to point out an irony in his approach already apparent from the title of the book: it is difficult for a "wise" crowd to access the wisdom it generates. I think that unpacking this further may begin to account for why there are no other animals as extensively cultural as humans (but does not explain why we are so cultural).

I don't want to get much into the discussion of the statistics, which are probably valid. Rather, I want to talk about the assumptions required for these statistics to work. The major assumption in the CLT is that the random variables in one's sample are independent and identically distributed. In the sorts of scenarios Surowiecki (apparently) describes (like guessing the number of beans in a jar of jelly beans), these assumptions more-or-less holds until the point when the average is taken. I'll try to make my point more clear by a couple of scenarios:

In both scenarios, we'll place a jar of, say, dollar coins in front of a crowd, and the jar gets awarded to the person who guesses closest to the number of coins in the jar. In the first scenario, everyone makes a private guess about this number; in the second, we'll follow a "Price-is-Right" model, and ask participants in order (and never asking more than once) what they think the number is, while allowing later participants to eavesdrop on earlier. As a further assumption, lets say that everyone in the crowd understands and can use the central limit theorem. There are two questions that immediately arise:

1. What is the best strategy for an individual to choose?
   
2. In which of the two scenarios does the average guess of the crowd come closest to the actual number of coins in the jar?
   

In scenario 1, the answer to the first question is obvious: the respondent should make the guess that matches his or her internal idea about the number of coins in the jar. The responses will be independent, and probably have pretty similar distribution centered around the actual contents of the jar, so the CLT ought to be able to characterize the answer to the second question quite well.

In scenario 2, the answer is much less obvious. Each person's answer will depend on both what that person believes the true answer to be, and on the circumstances in which he or she answers the question; If I know I'm the last to answer the question, but my internal idea about the count is widely larger from what all others have answered so far, the best guess to make is *not* what I believe the number is, but rather the guess which seems likely to be closer to the true value than the others, rather than that which will be closest. An epsilon greater than the largest value picked by previous guessers. The upshot is that the best strategy is no longer independent of the actions of other participants, and the CLT can not be applied.

This is likely to be true of any game with eavesdropping. Early guesses may be relatively independent, but later participants who know this will also know that they can make a better guess by taking the mean of those early, independent guesses. The obvious danger is that maybe you've sampled the guess of someone else playing the exact same strategy as you have, without knowing it - now you've increased the number of samples, and decreased the variance within the population of the parameter you've attempted to guess. After several iterations of this strategy, the sample variance is now smaller than the population variance, and the variance of the sampling distribution of the mean rapidly approaches 0. It appears, statistically, like there is a very narrow confidence interval for the population parameter, but it's an illusion - there's no more information added to the system when later samples are describable as functions of earlier samples. The result roughly approximates mobbing behavior, like we see in the market.

Later I'll try to argue why I think this makes culture a generally losing proposition across species...

I'd just like to appropriate this...

I noticed the Atlantic had a new blog by Jeffrey Goldberg, who wrote one of the best articles I've recently read on Israel. I don't know how well the blog will turn out, but I wanted to align myself with this sentiment from his opening post:

This is almost certainly a mistake.

Friends tell me that I will take naturally to blogging because I am in possession of many poorly considered opinions about issues I understand only marginally. I am dubious, however.

Yup.

Evolution of institutions

One of the themes from the post on religion below is that institutions can profitably be regarded as evolved entities in their own right. This echoes group selection theory, but I don't really intend it that way. An evolutionary theory to account for modern human behavior needs to include the concept that individual humans are (or can be) members of several communities at once, which may have conflicting hierarchies. It is not clear to me that the usual formulations of group selection consider humans in this way. Besides which, the success of an institution is not necessarily linked to its ability to create copies of itself - often, stability over a long time period is intuitively a better measure of an institution's success. The generally declining marginal contribution of additional group members to group success (which is an assumption frequently made in economics to describe why firms eventually stop growing or shrinking) suggests that group selection theory must ultimately favor the creation of new groups. This is not necessarily true of institutions.

Institutions, to me, can be described more or less as basins of attraction. I say this with some reservation, because I'm not well enough versed in the mathematics to be precise. I understand a basin of attraction as a concept in dynamical systems, used to describe more-or-less regular trajectories through the space under consideration. For example, the trajectory of astronomical bodies (planets, stars, etc.) are well described by differential equations - astral body acceleration at any instant depends on the body's position and mass, and the position and mass of any other astral bodies nearby. Without getting too deep into it, this dependence of acceleration on the positions of several mobile bodies makes modeling the motion of the planets actually quite a bit more complex than one might naïvely expect (see the N-body problem for details). The orbits of the planets within the solar system are regular to the extent that they are caught within the solar system's basin of attraction. (I'm happy to be corrected on this.) Incidentally, "adaptive peaks" on Sewall Wright's "fitness landscape" are themselves essentially basins of attraction... As are other evolutionary scenarios, such as rock-paper-scissors games described by Maynard-Smith (in which an individual is committed to a strategy at birth - in this case, there is no ESS. nevertheless, the path through time is quite regular: scissors is replaced by rock is replaced by paper is replaced by scissors. I've wondered if this scenario perhaps accounts for nuptial-gift patterns in insects, which I may expand on later).

This sounds excessively complicated, it probably is excessively complicated. But that doesn't mean that this sort of thinking isn't useful. Ecology in general is considered by those who study it as a very complicated dynamic system. Adaptation is considered to follow a trajectory on a fitness landscape defined by the entities in the ecosphere, which is a very complicated beast. Studies of biological adaptation, however, greatly simplify this picture by considering a denuded subset of the ecosphere, and model but a few variables of interest.

What I am trying to get at can be expressed much more simply: a successful institution will select for (that is, recruit and/or develop) the individual or individuals best suited to carry out a role in that institution. The method of selection is a property of the institution, and of the available "talent pool", and will also reflect interactions with other institutions in the cultural web. Understanding the evolution of institutions is a question of how these institutions' mechanisms for selecting or cultivating new individuals in different roles change over time, and understanding the adaptations of institutions will mean describing the circumstances under which different forms of selection or cultivation tend to be more or less successful.

It's probably hubris, but I suspect that research following such a program would do better describing behavior on a global scale than any approach which treats institutions as mere epiphenomena.

I hope to unpack some of this discussion later.