The human-induced great filter. Nature’s great civilizational test. In an earlier post, I mentioned stochastic gradient descent being analogous to natural selection optimizing for fitness across different species in a given environment. Humans, as a collective, can be thought of as agents, operating in this highly complex non-linearly dynamic environment, primarily trying to optimize over a range of terminal objectives. Mesa-objectives can be conceived, in this context, as partially detrimental, to the extent that they hinder the terminal objectives, whether through apparent pseudo-alignment or terminal override via deceptive alignment.

I must establish some parallels. When one speaks of Mesa optimizers and their Mesa objectives, as well as base optimizers and their base objectives, one could draw parallels between the mechanism of operation between base optimizers and stochastic gradient descent. manifested as natural selection, as well as the base objective function and relative fitness in an evolutionary context. Looking further, one could also find even more parallels between Mesa optimizers and human sociocultural values, and mesa objective functions and preservation and propagation of said sociocultural values goading to inner alignment predilections(temperament, predisposition to violence, etc).

Screen Shot 2022-08-08 at 10 12 11 AM

From the advent of society circa 10000BCE, humans, across large sets of people and cultures have had their fair share of war and religiously and culturally induced conflicts, primarily fueled by a “fight for civilization” as opposing sides were generally motivated by the preservation and propagation of their socio-cultural values on to the other side. These conflicts usually always led to the loss of life in large quantities. The motivation for such violent conquests perpetrated by humans was rooted primarily in differing cultural and religious norms - or differing mesa-optimizers, as well as phenotypic differences which are even more apparent.

Sociocultural norms and values, in this context, can be conceived as a sort of aberration, an unaccounted-for parameter in nature’s optimization algorithm. Cultural norms and values emerged in tandem with civilization. The earliest civilizations we have a record of - the ancient Mesopotamians, had their own set of gods and goddesses, which one could call deities. They derived all their societal norms from the framework laid down by such a proto-religion. The same can be said for a wide range of functional societies that arose after the Mesopotamians. So, it seems as though a precursor for any such advanced society, entails initially operating on a quasi-religious framework - deriving societal norms from said quasi-religious framework conjured up by progenitors and passed down across generations.

We already established earlier that most frameworks that societies are predicated on presuppose a somewhat cohesive culture, as culture is the meta-optimizer; the superset for which all other subsets, such as religion and norms fall under.

One could see how war and other such destructive products of human collective inner alignment, manifest themselves as convergent instrumental goals of collective human intelligence, ultimately as an attempt to attain and maintain the mesa objective - cultural preservation.

Think of population A and population B defined as distinct sets. Both sets A and B are subsets of a set Z that contains all humans. The superset is broken down into subsets on the basis of socio-cultural norms, with each subset operating on widely differing sociocultural norms. Of course, the ultimate goal of the collective human optimizer is to propagate the human race as far and wide as possible, with the terminal objective being fitness maximization across individuals and populations. The problem arises from the fact that as sub-collective agents, each population is endowed with a Mesa optimizer, an inner alignment mechanism that allows for instrumental goals being a necessary precursor to Mesa objective maximization. Given that both populations are culturally distinct, as a consequence of the convergent instrumental goal for mesa-objective preservation, both populations, even though part of the same superset of humans, would still go to war with each other, ultimately leading to loss of lives on both sides, and a subversion of the terminal objective via the instrumental objective of “going to war” or “winning a war” as a method of mesa objective preservation.

The problem of inner alignment of mesa-objectives is one that should be alarming generally. Looking at humans, we see how our mesa-objectives might actually be the end of us. Wars, genocides, poverty, environmental degradation, etc are all seemingly emergent consequences of instrumental convergence goading to mesa-objective preservation.

I would like to write a bit more on this topic, but I’m quite busy most days so I can’t really find the time.