Conversation with Ralf-Peter Behrendt

Hi,

Very late to this - found whilst googling thinking about times past. Ralf was a friend of mine, I knew him from his time in Sheffield.

His death was very much a shock, though I have a suspicion of what may have happened. He was a very intelligent man, great fun to talk to, but also very troubled.

I miss him still.

Adrian Hill

I met Ralf in person. I spent a lot of time with Ralf. We watched Mighty Bush. We would sit on his carpet and watch it. We loved it, we were like little kids.

A patient reminded me of Ralf today. Episodic memory. Ralf will always be the most intelligent person I have ever met. I heard him though. Loud and clear.

I miss him so much now.

Hey Ralf,

I've been working on my latest revisions of the paper, in many places

starting from scratch. Most of the arguments are the same, I'm just

unpacking them much more thoroughly.

There is one detail that I'm starting to see differently, and I'd love

to hear your take on it. It's about the same referee question I sent

you recently, about how does the hippocampal simulation become

experience? I have, heretofore, been agnostic about that question,

and just left the simulation within the hippocampus.

However, that answer seems theoretically troubling to me. I had built

the hippocampus argument on the model of a 'pyramid of reports', i.e.

each module sends its report to the next module, which does its

computation, and then sends an output report on to the module after

that. In other words, information can only be known in the brain, if

it is shared. In this model, there must be a transmission. The

encode of memory cannot just exist, 'unseen', or it could not be

subjective experience.

Of course subjective experience is not 'seen', but I do think that the

metaphor of the hippocampus as the TV news function of the brain is

useful anyway. My current explanation in the paper, the one that

makes the most sense to me at this point, is that subjective

experience is not the memory encode itself, rather that it is the

process of that encode being 'broadcast' back out to the rest of the

brain. Again, the rest of the brain is not 'watching' the news.

Rather, the brain is activated by the memory broadcast, in the same

way that the visual cortex doesn't watch the outside world; rather, it

is activated by it. Same thing here: as the broadcast signal

activates the neocortex (presumably at layer 6), as a top-down signal

to help bias the next moment's predictions, that activation creates

experience throughout the brain. That experience has already been

unified and simulated in the hippocampus, but it is now made 'real' by

being shared with the rest of the brain.

One piece of evidence that I find compelling in this direction: the

fact that patients with hemispatial neglect exhibit the same neglect

when remembering scenes that they had first committed to memory,

pre-morbidity. If they imagine themselves at one side of St. Mark's

square, they can only describe items on their right, but if they

imagine themselves at the other end, looking back, they describe the

opposite half. What this suggests to me is: the encode is intact, CA3

is intact, but the recalled memory is only 'known' to the rest of the

brain, if those parts of the brain can be re-activated by the

broadcast of the recall. Hence, subjective experience of the recall

fails, because of neocortical damage. Which suggests: subjective

experience 'happens' at the neocortex, even though it's formed in the

hippocampus.

Now, that's not the only possible answer, I'm sure, but right now,

it's the one that makes the most sense to me. As you know, I've been

bugged with some variant of this question for many months now, and

this is the first formulation that really feels elegant and

explanatory. This doesn't change the overall theory, as I see it, but

it does make specific predictions that I was not previously ready to

commit to.

I've often seen you state that consciousness is the patterns in CA3,

but I have a hard time with that theoretically (especially since the

downstream CA1 seems to be where events are placed into chronological

order, and embedded into theta cycles). But it's also hard to imagine

that the entire brain 'believes' that the new memory encode is

reality, unless the entire brain (or most of it) receives that

broadcast transmission first. And it's hard to explain a rich

experience that only exists in the hippocampus, as if the hippocampus

could somehow experience its own creation. I think of CA3 as the

auto-associator and decoder of previous memory encodes, and CA1 as the encoder of the new memory (as prepared for it by CA3, DG and the EC, and prior to them, PA and PHC, and prior to them, everywhere else in

the brain). The parts of the brain which are capable of experience

seem like they should be the very sense cortices that process initial

stimuli. This closes the loop, and it explains why subjective

experience is happening in the background, even during 'selfless' flow

(i.e. neocortical) states.

I'd love to hear your thoughts.

best,

matt faw

Hi Ralf,

I think I see what you mean. Even though 'subjective experience' is

important from a human-scale perspective, it is merely a epiphenomenon

of the patterns that underly it, because it is the patterns that

actually run the show. The patterns aren't asking themselves: "what

color was that?" The patterns don't ask anything, because they are

just patterns. They reflect natural processes, just at an extremely

high level of complexity. Those patterns happen to be members of the

'ecosystem' or 'weather' of the mind; they're part of a more complex

set of patterns, but there is no identity in there, no will, no actor,

no perceiver.

-----------

BTW, when I sent you the reviewer's criticism, I should have included

the following paragraph, because it's more to the point, as far as the

actual locus/mechanism of subjective experience (SE). It's a question

that I'm sure is answerable, but I have no way of being able to make

anything more than a wild guess. I'd love to hear your thoughts on

it.

"Why is the HF so important for SE?

The authors have not actually answered a key question - why does the

HF have this special property of SE, and not other regions? What they

have done is picked out some functional properties of the HF that map

quite well onto important features of SE, but this is merely

correlational. This does not, therefore, answer the fundamental

question of how the (according to the authors) unconscious processing

occurring across the rest of the brain is somehow transformed into SE

within the HF. I do not understand the rationale for this. One could

accept the authors arguments about the functional role of the HF in

binding together many different processing streams, and creating a

"movie" simulation etc., but could then generate many different

theories about the actual neural correlate of SE on the basis of this,

other than their own argument that it is in the HF. For instance, it

could be that the HF processing is also unconscious, and that the

"movie" created within the HF is then passed to other areas (such as

the posterior cingulate cortex or medial prefrontal cortex, or a

distributed network of high-level regions) which use this information

to generate SE. Or one could argue that SE is an emergent property of

a complex, dynamic distributed system, such that the HF contributes to

SE only to the extent that it contributes unique aspects of

processing. By this account (which I personally find the most

plausible), the HF is essential for certain kinds of very important

SE, such as SE of an extended, spatially coherent world as opposed to

disjointed visual objects, and such as episodic memory and

imagination, but it is not the route of all SE. Extensive

clarification is needed."

Thanks, Ralf, for your quick reply. I am very grateful for your

ongoing friendship and conversation, so please, no need to apologize.

I completely agree with what you wrote. Clearly, there is no homunculus.

On this draft, I'm trying to simplify and spell out absolutely

everything, leave nothing unsaid. Most of the times our reviewers

said some answer was missing, I had actually covered it, but

apparently unsuccessfully. So I'm trying to break everything down

into small little steps, like a Joy of Cooking recipe, so there's no

gap through which understanding can fall.

On this particular issue, I can see a reasonable path of explanation,

but actually finding the right words to convey that explanation is a

whole other issue. It's like the HM thing; it doesn't seem that

difficult or controversial to me, but it's damnably hard to get across

(even to my own dad). Oh well, it's a good challenge.

best,

matt

Hi Ralf,

I hope you're well.

Sad to say, my paper just got rejected from a third journal.

Thankfully, the responses are much kinder this time than the first

two, due I think, to all the revisions I've made. I'm now trying to

figure out how best to revise and address these new criticisms.

Almost all the reviewers' comments have been addressed within the

paper already, so it's clear most of them are about me spelling things

out more clearly. However, there are a couple of questions they have

that are still stumpers.

In particular is an issue we've talked about before: yes, the

experience 'movie' is bound together in the hippocampus, but that

still leaves open the question of whether that movie is then

'experienced' by other brain parts. I've mentioned the metaphor of

subjective experience as the news of the brain, that each module sends

its report to the EC to be bound in the HF, and then broadcast back to

the rest of the brain as multimodal memory, a news report of the big

picture from a moment before. This is the solution that makes the

most sense to me, but it's out there on the periphery of the my

understanding, and I can't assert anything with confidence.

Following is the reviewer's comment. I'd love to hear what you think.

best,

matt faw

"Where is subjective experience actually located?

I am left feeling extremely unclear as to exactly what this theory is

suggesting with regard to the neural locus of SE. Most of the time the

authors appear to be suggesting that the HF is entirely responsible

for generating SE, but at other times they appear to flatly contradict

this view, and suggest that the HF actually contains indices which do

not themselves generate SE, but are used to activate the various

cortical representations from which SE arises e.g. p.9-10

"Technically, a memory's perceptual content is not embedded within the

encode, but is instead indexed, like hyperlinks, to sensory cortices

which were originally responsible for sending their report to the HF

for memory...the HF, which then follows the encode's indexing

instructions, to enlist the requisite sensory cortices in supplying

the data necessary to re-construct the episode. The more vivid the

memory, the more the sensory cortices are involved in its

reconstruction.". These are very different theories, which produce

very different predictions, and the current manuscript does not make

it clear as to which the authors are actually suggesting."

Hi Bob, yes I agree.

A word on "Pain doesn’t exist context free." I think there's

something to be learned by the recent experiments where the subjects

would rather shock themselves than be left alone with their thoughts.

I think that the body sends pain signals to the brain all the time, as

information as to where to stretch, where to be careful about, etc.

Pain demands to be listened to, but it is also part of the brain's

overall balancing scheme that there are inhibitors to block pain, for

a certain time period, so the organism can escape from harrowing

circumstances, and wait to feel pain when it is back in safe

territory. I'm speculating that most of the discomfort that was felt

by the subjects was the pain that they usually had distractions to

cover. No distractions means having to face what's already there,

which of course just makes the thoughts that must worse.

However, this ability to inhibit pain can be over-used, and I think

people, be default, do so, we tune out from bodily pain. I think this

is taught by parents to their children, so that kids learn fear of

injury beyond what is necessary, and they generalize it to pain. Pain

is bad. It is something to be avoided, to be "killed'.

But pain is just a sense, like any other. And in extreme cases,

people have inhibited other senses, including vision. So it is no

surprise that we do what we can to minimize the pain in our

simulations. Once the simulator habit includes "exclude pain" then it

creates a reality simulation that intentionally restricts a sizable

portion of its own input data. Only when the delta signal is over

some threshold do we even notice pain in the body. So old injuries

can hide for years, not because the signal is not being generated, but

because they are too familiar to consider. All of a sudden they

'flare up', but if we had been paying attention to pain all along,

surely they would have made themselves known. We have the same

attitude in geopolitics; the general public only responds when we're

freaked out by changes. Up until then, we're numb. But if we were

paying attention all along, we'd use an ounce of cure, and not have to

wait for the freak-out.

The parable of the frog in hot water is everywhere, but nowhere do I

see it more displayed, than in the way people retreat from pain, as

they age. People's bodies, as they get older, turn into caricatures

of their peculiarities when they were younger. If my back is a little

bit hunched in my 30s, you sure know I'm going to be defined by my

hunch in my 70s. But what is it that keeps people forced in these

postures? I propose that it is only pain that stops them, and if they

learned to listen, and to stretch, and to really take care of the

parts that hurt, then we'd all age a lot more gracefully. That's my

experience, anyway.

So in that sense, I think that pain is constant. The body is always

offering up pain as feedback, to help the brain make decisions. I

experience the immediate moment and respond to it with input from all

my stimulations, including pain. But since I (the planning self of

the default network) have excluded pain from my experience, I do not

believe it to be one of my motivators, and I create stories that make

other salient characters (like other people) to become the sources of

whatever pain and discomfort I feel. Pain is out-sourced, made alien,

and seen as an invader, whenever it flares up.

matt faw

Hello gentlemen,

I don't know if you've yet seen this TED talk, but I highly recommend it. I think he is very much talking about the division between the neocortical expression of self vs. the hippocampal interpretation of that. This is related to the Martin Conway material I've been pushing on both of you, about how a self is created in memory, that is parallel but separate from the self which reflects neocortical activity. The first-person experience of self is always hippocampal, because that's where my experience becomes available to my default system. Whereas the 3rd person experience of me is essentially of my neocortical self, what I do in the moment (but of course they experience a neocortical self that is informed by the hippocampal story).

http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory?language=en

I hope you enjoy it!

matt faw

And here is a short article that I think is very

worth reading, on mise en place, or how our perception of our environment changes how we think and act. For example, a playful fantastical environment, in which some things seem to contradict our notions of what is possible, is actually the ideal learning environment, precisely because it liberates children from the tyranny of the known.

This may seem somewhat tangential to the question of consciousness, but I think place only happens at the consciousness level; this is where everything comes together on the "spatial scaffolding" to use Lynn Nadel's term. The gestalt is the subjective mise en place.

matt

Interesting Ralf.

And, I have to say, this is where I think story fits in again. The "way we ought to be" is exactly story, whereas the "way we are tempted to be" is really who we are, it's our biology.

As I see it, the dissonance is really between who I am, vs. the story I have of myself.

Following up on my post yesterday: just found this review article on motivated forgetting. The gist is that the right dlPFC and vlPFC both activate to inhibit the memory, the entorhinal cortex also activates, as if this is where the lateral PFC is acting upon, and the hippocampus de-activates. It's as if the lateral PFC is "choking off" the data stream at the EC, to reduce likelihood of hippocampal encoding.

matt faw

Hi Bob,

Maybe it's a semantic issue, but it seems to me that confabulation as a phenomenon is much wider than confabulation as a diagnosis.

Yes, confabulation as I formally know it, is something to fill in the blanks for patients with frontal lobe amnesia.

But the very fact that these patients regularly do this behavior says a lot, I think, about the human need to make sense of things, put things in some kind of order (that reflects their beliefs).

Why is it that we know everything at age 18 (especially if we've just found religion or nationalism) but in our 40s, we know less and less? It is because the 18 year old has confabulated reality, has created a level of certainty that the more experienced version of him cannot rationally support. I see this with my (somewhat troubled) nephew, who is trying on different absolutist victim, martyr and hero stories, each which satisfies some longing in him, at various times.

Even you saying, that we "perceive what we believe" seems to be describing a form of confabulation, a constructed reality that specifically avoids causing us anxiety by not challenging our previous views.

matt faw

Hi Ralf,

I would agree that confabulation often is self-serving, but my speculation is that confabulation is equally distributed between what we wish and what we fear. If my girlfriend starts with "I have something serious to talk about", I can hear my little theory-generator sparking all kinds of worst-case-scenario stories to anticipate her subject.

Another example: there are some internet memes running around Facebook, etc. that show "baby yoga" and "Roma acrobats training babies" which show adults doing all kinds of absurd and dangerous-looking things with babies, in the name of making them limber. The videos themselves never show kids actually getting hurt, or even acting scared, but the comments (especially from parents) are like they are watching the work of the devil. Every parent comments with extreme revulsion, and utmost certainty that they have seen child abuse and damage on the video, even though it is not to be found. That is because they have confabulated that damage, in order to justify their own fears. They "know" that the yoga is bad, precisely because their inner voice yells out its freak-out so loudly.

I think that homophobia has been justified similarly; since there is some part of me which reacts with revulsion, therefore I confabulate a moral dimension to explain my disgust.

matt faw

Hi Bob and everyone,

A few thoughts:

1. I've recently gone through my interview with Steven Macknick. He focuses on the art of magic, to help explain the neuroscience of attention. He uses the word "confabulation" to explain our visual perception of the world, that what we perceive is dramatically different than the noisy incomplete data that hits the retina. This is pre-language story-telling, inner movie-making, in which we direct stimulus into a film that conforms to our expectations. One could also call this "pattern completion", but I think confabulation is a usefully alarming term for it, because subjectively we only see our perception as "what's real" instead of realizing how much of it we're making up or filling-in.

2. I've always been fascinated by the "new recruit" phenomenon of religions, in which the person who just converted does indeed seem to experience a short-term dramatic shift in the sense of self, one's 'mission, and how one fits into the world. Especially considering how little change happens from self-declared resolutions, like those at New Year's, it is pretty amazing to see how much change can happen (at least for a while) in the lives of the recently converted. Unless the religious shift accompanies a new-found sobriety or whatever, it appears as if the new belief system is responsible for most of the changes. But I don't think it's the religion, because it seems to happen to new recruits to (almost?) all religions. Instead, it seems to be a shift in story. In fact, it may be a lot like what happens when we travel out of the country: our right hemispheres seem to be called into action to explain all of the new stimuli, and we get out of our left brained stereotypical processes.

Now #1 is not necessarily about character, but it is about the drama. Because the movie confabulation may just be: e.g. I find the world ugly (or beautiful). If I am leaning towards seeing ugliness, then I will emphasize that aspect of reality, in my simulation. And I will be offended, just by looking at the world. That is drama, regardless of whether other people are involved.

#2 often involves a shift in #1: world perception, plus in other-people perception, event perception, etc. Many events become translated by a new paradigm. The victim may become the hero and the former perpetrator may decide that he was really a victim.

What I think is unlikely to change in #2 is the overall importance of self, within the model. Even if the previous view was: I'm an asshole, and I make people miserable, then I was still somehow an important asshole. If the new persona is now important, even if in the need to save others, to recruit them to the "right and just cause", then the original persona was probably equally important, just in a different paradigm.

Here's confabulation upon a different axis:

3. Another interview I conducted in Tucson this year was with Jeffery Martin, who studies self-designated "enlightened" people. The diminution of the sense of self is one of the hallmarks of this population, across all kinds of religious and spiritual backgrounds. So in their case, the story of self seems to become a lot less important, at least according to self-reports. However, their backgrounds do tend to explain how they experience "enlightenment" with Christians and Buddhists each recognizing elements of their own belief system within the process. So even though self (which I see as a constellation of habits, a sub-set of which are beliefs) is explicitly less important, the belief system actually becomes more solidified. They begin to see their own beliefs (and even their random thoughts) as perfect, as somehow divinely inspired. So in many ways their delusion is the greatest, because no one can ever convince them to consider a different point of view. Plus, when different "enlightened" people get together, especially those from different backgrounds, they do not recognize each other as such, and assume the others are deluded or fakers. They each know what it's like to be enlightened (at least from the inside), so obviously this other person is not it. It's also important to know that the self-proclaimed "enlightened" people universally feel radically different, after their enlightenment experience, but most of the people in their lives do not notice a difference in their behavior.

Just some thoughts.

matt faw

Bob and Ralf, I just came across this excellent article about the "hero narrative" and confabulation of the self-image. Very interesting, and very fitting for our discussion.

http://aeon.co/magazine/psychology/how-a-hero-narrative-can-transform-the-self/

Dear Ralf,

I'm curious if you know Martin Conway's work. Other than your papers, his were the closest in parallel to my thesis. I relied on Thomas Metzinger's model of self and experience as representations, but Conway's writing covers a lot of the same ground, and is specifically about the relationship between self and memory. He also wrote a lot about memory and narrative, the idea that I was trying to get across in our conversation with Bob. I only discovered his writing at the very end of my research phase, so he didn't influence the paper as much, but I think he will be very helpful for the documentary.

Conway, M. A., & Pleydell-Pearce, C. W. (2000). The construction of autobiographical memories in the self-memory system. Psychological review, 107(2), 261. http://homepage.psy.utexas.edu/HomePage/Class/Psy394U/Bower/11%20Soc%20Cog%20Personality/XX%20Constr%20Life%20Stories/Autobio%20Mem-Conway.pdf Conway, M.A. (2005). Memory and the self. Journal of Memory and Language, 53(4), 594-628. http://homepages.abdn.ac.uk/k.allan/pages/dept/webfiles/4thyear/conway%202005%20jml.pdf

Hi Bob, thanks for your reply.

I think that the brain works on multiple levels, simultaneously. One level is defined by society and language, and includes the "I" as a unified agent, a character, as you might say, working within a confabulated milieu, that is constructed to interface, as well as possible, with the consensus reality around us. On the other level, we're talking about brain modules, algorithms, modulatory neurochemicals, and neuronal democracy, i.e. interpretation of stimulus by population vote.

Personally, I think that the latter level is the more accurate one, and can help illustrate causes of pathology, etc. But clearly, the other level of processing includes all that we consider to be "conscious". On the brain level, self is defined by habit, but on the conscious level, self is defined by belief (belief being a semantic, non-procedural form of habit). These two "selves" may be radically different from each other.

Interestingly, I just got finished reading a blog talking about this, about how confabulation and delusion may manifest themselves very differently through language than through behavior.

http://imperfectcognitions.blogspot.co.uk/2014/06/are-delusions-acceptances.html?m=1

The blog's example is that sufferer's of Cotard's syndrome believe themselves to be dead, and will tell you so. However, they may continue to behave much as they always have, so you'd never know that belief exists.

Likewise, my understanding of patients with anosognosia for hemiplegia: they will not act as if they are fully functional, unless you ask them to move. When you ask them to move, they engage the social self, the self of belief, which views reality from the point of view of the hippocampus. In the hippocampus, the body is a figment, a concept, and so it can move easily, just as it would in a dream. However, in the neocortex, the body response has been severed by the lesion, and so the organic self, i.e. the body/brain, does not try to defy its own paralysis.

In fact, delusion seems to be precisely about the schism between the self of belief and the organic self. The further afield the belief sense roams from the actual responses and needs of the body/brain, the more deluded it is. The simulated self, although it is generally adaptive, can become cocooned by the rigidity of belief, until it is no longer able to function in the realm of the material or social worlds.

Or that self can limp along, as does your character "Eddy", interfacing as best as possible, but unable to witness the inner biochemical milieu which constrains the confabulated one. From within consciousness, which is the confabulated now, only the confabulated self exists, but all of that, of course, is represented on a neurological level by massively distributed and parallel processes.

Personally, I think neither view of self and reality is sufficient to explain pathology, nor even normal function. We have to be able to describe brain events from a neurological POV, but describe human events from a social-/self-model POV. Both make us who we are and influence our behavior, so both need to be considered.

best,

matt faw

Dear Ralf and Bob,

Very happy to hear both of your responses. I'm not surprised that we all have different views on this; precisely why it's nice to hash out these ideas.

By coincidence, I was just yesterday listening to a science podcast in which they were talking about the importance of labels (i.e. language tags for what we experience). One striking example they gave: Russians have separate words for "blue" and "light blue" and thereby, they are able to distinguish shades of the two with much greater accuracy than people who speak languages that only have the one word for "blue". In other words, the ability to perceive is shaped by language.

In communication, the dualism of words forces fake dualism upon the world. We treat "smart" and "dumb" or "good" and "bad" as polar opposites, instead of relative ranges of multiple overlapping spectra. This is probably most pernicious when it comes to race-related words like "black" and "white".

I think language plays an important role in what Bob calls the "characterological drama". If we internalize labels like "rebel", "good girl", "lazy", racist, sexist or homophobic epithets, etc., then our drama can be shaped by our understanding of what those words mean.

But the territory that I think language plays the most important role is in memory. Martin Conway wrote a lot about this, about how memories get boiled down from being the full multimodal movie to becoming something more like a narrative: "my summer in France", or "my hellish commute to work today". The remaining episodic movie clips, audio and stills become illustration for the story. Nadel and Moscovitch's multiple trace theory suggests something along these lines, too: the hippocampus is needed to replay the episodic movie, but the narrative has been recorded in the neocortex, as an organizing principle for the memory.

In my life, I recognize the time lying in bed before falling asleep, as one time in which my brain is hungry to make sense of memories from the day. It plays back events, seeking meaning in them, and that meaning is encoded as language. I.e. "he did this to me" or "I really messed up" or "she was just being mean". These thoughts become encoded with the memory, and inform future recall of it. If I self-justify (e.g. "I was being 'righteous' not 'angry'"), then I will re-classify the emotional context of the memory, affecting not only what I remember, but how and when I remember it.

Michael Gazzaniga, in his 2011 "Who's In Charge?" and elsewhere, talks about "the interpreter", a left-hemisphere language module that constantly feeds a narration of events to help the organism make sense of experience, as it happens. Of course, since I believe that "experience" is actually a brand new episodic memory, then I think the "interpreter" is precisely helping to consolidate memories. After all, language makes an excellent mnemonic, and is much more compact and efficient (albeit less accurate) than a movie. It is much more efficient to say "my boss was a total jerk to me today" and have a few small hyperlinked memory movies attached to that story, rather than to have a full movie of the whole workday.

By the way, Bob, when I'm talking about a 'representational workspace', I am referring to the hippocampus. Specifically, I'm talking about the work by Demis Hassabis, Eleanor Maguire, Dan Schacter, et al, who have been imaging the hippocampus, in relation to the default network, in the staging of mental rehearsal and other interior scenarios, including social theorizing. In each of various simulations, this network is representing other people, objects, one's own body in physical rehearsal, etc., in order to pre-visualize how things will play out.

We can see how language plays a role in this inner representation when we read a menu, and invoke sights, sounds and smells in our head. Or when a friend tells us a story, and we involuntarily see some of the details in our own mind, even though we didn't experience them directly. So how different is it, when I tell myself a story, and re-arrange my own memory details, in order to support that story?

best,

matt faw

Ralf, following is a hypothesis that's been buzzing around my head for awhile. Bob and Dad have already read the following, from my dad and my paper, about language thought and the hippocampus. Still, I think there's plenty here to unwrap and discuss.

best to you all,

matt faw

Language Thought:

In a brain in which data moves very quickly between modules, it is a mystery that we are also reliant upon inter-brain data transmission which is represented in a form as slow and imprecise as language. Surely other forms of intra-modular communication must be more efficient than language thought.

HST: We think it likely that language thought is one form of representing pre-memory cognitive processes, for use in the simulation. One aspect of this is specifically to simplify memory processes, by giving the episode a narration [e.g. ‘he did this to me’ (Gazzaniga, 2011)]. This allows the memory to be lean in terms of content, and to be easily understandable upon recall. It’s also a potential source of error, as language implies a level of certainty that is often not warranted in a world of complex causality and incomplete evidence.

Language thought is probably also a tool of intrinsic simulation, like weighing the pros and cons of a decision. Various behavioral maps [e.g. in the ventromedial prefrontal cortex (Behrendt, 2013b)] suggest different choices or outcomes, which are then sequentially formed into language using the left hemisphere’s language encoder, and then passed into the HF to buffer in memory. This allows multiple options to be weighed simultaneously. This is what we know as ‘conscious deliberation’, but not because there is a consciousness doing the deliberation. Rather, pre-memory structures use the HF as a problem-solving workspace in which to manipulate representations, and the HF activity consequently gives rise to subjective experience / memory.

Thank you, Bob. I wonder if the end of your e-mail points toward a bridge between your book and a hippocampal model of subjective experience.

In particular, in my paper we claim that the neocortex has its own set of representations that are specific to how it processes immediate interaction with the world. However, none of those representations are available to experience (i.e. memory). What is available is the post-facto reports of neocortex activity, sent to the hippocampus for simulation into the experience, for possible storage.

This implies (at least) two levels of mapping. External reality (especially viz. vision) cannot really be mapped until birth, because of the lack of reality to interact with (mostly proprioceptive, motor, and some audio mapping during gestation). So upon birth there begins to be a neocortical mapping of external stimuli and their meanings, that helps the baby recognize objects and people.

The hippocampus, however, takes some time to come online, and so its mapping comes later. In the MTL, most visual objects are mapped to the perirhinal area (via concept cells), which supplies a mature hippocampus meaningful objects to be added to the spatial simulation (which has been mapped through the parahippocampal cortex). These two mappings come together in the entorhinal cortex to form a scene with objects, and are brought into the hippocampus for final simulation that includes self- and other-mapping as well as location and objects. Each step in the MTL is about activating mapping to bring meaningfulness into the scene, including reports from the amygdala, vmPFC, hypothalamus etc. Ralf's excellent "Appetitive Behaviors" paper from late last year dealt with a good deal of that.

The neocortical mapping drives immediate interaction with the world, and so this is the personality that other people see, and may be very different than the personality that is represented back to the person, from within the hippocampus. In fact, there can be a great divorce between the actions and feelings in immediate behavior vs. the introspected actions and feelings that one creates in memory. For example, someone can appear and behave as if they are full of anger, without being aware (from within experience/memory) that they are feeling that anger at all.

The belief we have about ourselves may be formed from experience, but of course, it also shapes experience of ourselves. Not only in the immediate moment, but also in how we remember our past. Repression of old memories, like the times when I acted like a jerk, is the norm, I think. We are very good at maintaining the memories that support our self-image, and equally good at forgetting those memories which don't.

IMO, the hippocampal simulation is a good deal of where the play of reality takes place. It is at least where the play is remembered, which shapes future drama. It is also where the mapped 'other' is brought into the simulation, where attempted mind-reading is added to experience. You may remember my hypothesis about Capgras Syndrome, in which the mapping for "mom" breaks down, and therefore she is simulated as a stranger-who-just-happens-to-look-like-mom, an impostor.

In my paper, I dealt only superficially with the causal nature of "habits of simulation", tendencies to construct the world in a way that satisfies some self-image or biases about reality. I dealt with it superficially because it felt like a whole other can of worms, but I think that it's some of the most important take-away from this theory. We need to know that we are creating and biasing reality moment-by-moment, because otherwise reality continues to seem self-evident. And we need to realize how much we confabulate our experience of self, because otherwise we won't wake up to our responsibility to behave well in society.

best,

matt faw

I was reading Bob's blog just now, about nature and nurture, and that brought me back to the question of mapping.

The way I look at it is: when the blastula in the womb begins to subdivide, there is no differentiation between cells, not to mention structure of the body. But gestation is a process not only of division but of cells identifying themselves, strictly by the nature of their neighbors. What neighborhood you are in decides whether you become a heart cell or a neuron. This is autopoetic mapping, organization from within.

As the body and brain continue to mature in the womb, "nature" is still connecting wires, but it's starting to involve "nurture" (i.e. experience). As the fetus kicks, it is testing muscular pathways, proprioceptive response, etc. This is a Hebbian mapping of the body upon itself, through internal experimentation.

The first 6 months outside the womb reflect the body still getting to know itself, learning its muscular feedback, starting with mouth and fingers and eyes. This is when vision begins to get mapped (albeit onto a cortex predisposed toward human sight). What are these blobs? What do they signify? What are the rules by which they operate? If the infant is deprived of some aspects of vision (like color or vertical lines), then its brain doesn't learn how to process them.

And then, as you point out, the emotional mapping starts (probably in the womb, if there is too high a level of cortisol, etc., but also) postpartum, in engagement (or lack thereof) with the parents, siblings, etc. Emotional mapping is not only about temperament, but about survival: strategies of navigating this particular family and environment. If mom freaks out whenever baby cries, baby may learn alternate ways of expressing herself.

I think that's part of why those first few years are so powerful in the shaping of personality, because the major emotional strategies and habits have been deeply mapped onto the organism, and they shape the lens by which the person interprets and interacts with the world, thereafter.

Hi Bob,

Thanks for your divergence!

Ralf could probably address this better than I, but I see various forms of mapping at play here. One is via concept cells, primarily in the perirhinal area, but also in the EC and hippocampus. These are indexing place holders which bind various object features into one concept, whether various viewed angles of the same object, or various ways of considering Jennifer Aniston, for example: her image, voice, name as spoken, name as written.

Another form of mapping is the way that memories index to each other; the previous experience informs how I simulate the current one. That mapping actually happens within the hippocampus, attractor dynamics in field CA3 eliciting related memories. Again, Ralf can tell this a lot better.

There are maps of actual geographic locations that help us navigate 3D environments. These are probably accessed through the parahippocampal place area, before being fed through the EC grid cells to the hippocampal place cells for binding with object information. The PHC place area translates the parietal lobe's egocentric "where" dorsal visual stream and translates it to an allocentric hippocampal representation, and as you mentioned, Bob, the retrosplenial cortex seems to translate the allocentric hippocampal output back into an egocentric view for outputting back to the parietal lobe.

Ralf wrote quite a bit about behavioral maps in the ventral-medial PFC, in which the current "situation" (i.e. hippocampal output) elicits motivational arousal and behavior. I think of this kind of mapping as complex algorithmic if/then mapping. The amygdala also affects immediate behavior, through concept cell mapping of apparent threats.

I have also heard speculation that the vmPFC contains Theory of Mind mapping of other people, although I'm not sure Ralf would agree with that. Somewhere in the default network, at least, there seems to be personality mappings that allow us to make predictions about other people, especially people we know well.

The body is mapped in the parietal lobe (somatosensory), the motor cortex (as procedural memory), the cerebellum, and (in my paper, at least) also in the hippocampus, as a concept. The neocortical maps all seem to interact together to guide intentional movement, and the cerebellum probably compensates for that movement, adjusting the rest of the body to keep balance and coordination. In my paper, the body is represented in memory predictively as a concept, which is why so little of the vast amounts of information that the brain receives about the body actually makes it into experience. As Michael Graziano said it (in my interview with him in April): the body seems subjectively to move by magic, because most of the interaction of muscle and sinew is left out of the simulation.

And then self is mapped through all of these, as a chain of memories, as concept cells, as body schema, as behavioral repertoire. Self is not easily reducible to any brain part, process or concept, since it really reflects processing of the whole brain and body.

H.M. certainly would respond like you said: he'd forget that he had met any of the researchers, but would be inclined to move towards the ones who were nice to him and away from those who were not. This shows clearly that episodic memory is not the only kind of memory we have and use (although without episodic memory, new semantic and procedural memories take a lot longer to become knowledge/habit). H.M. had a good deal of self-mapping, and would recognize his wife. But there was also a good deal of what we normally think of as self, that was probably missing with H.M. The Damasio quote I used about Daniel was the most clear-cut description I've heard of what's missing. But I've also heard from Lynn Nadel and others that the AOCBHD patients are missing important elements of spatiotemporal processing and of self-awareness, introspection and personal narrative.

best,

matt faw

Dear Ralf, in particular,

Bob and my dad have both seen this newer section from our paper, but I don't think I had such an extensive H.M. rebuttal in the draft that Ralf has seen.

Whenever dad and I explained our theory to people in the field (e.g. during our poster presentation in Tucson this April), the first question was always about H.M.

Following is my response about H.M. and similar patients. I'm curious to hear what you think, whether my response is sufficient, and whether are other arguments that should be brought into this question.

best,

matt faw

What does this say about H.M. (and other patients like him)? Henry Molaison, commonly known as H.M., is the most famous patient with adult-onset complete bilateral hippocampal damage (AOCBHD). Most of his MTL was removed surgically, to save him from the overwhelming epilepsy that was focused there (Scoville & Milner, 1957). Patients like H.M. are unable to form detailed or coherent intrinsic simulations, like mental navigation, future projection, imagination and social rehearsal (Spreng, Mar & Kim, 2009). These patients are afflicted with anterograde amnesia, the inability to form new episodic memories, and retrograde amnesia, the inability to replay previous episodic memories (Kartsounis, Rudge, & Stevens, 1995).

In the model we are proposing, such patients do not have SE, which may sound implausible, given SE’s usual definition. But this theory’s definition of (what those of us with working hippocampi call) ‘subjective experience’ is a brand new episodic memory, an after-the-fact representation of the perceptual, cognitive, affective and motor processes that came before it. That is what H.M. was missing.

We with working hippocampi have the same approximate pre-memory representations and processes as H.M. None of us, including H.M., act directly in response to the hippocampal SE simulation; we act according to representations in the neocortex and other pre-memory structures (see figure 1, below). As we can see in the case of ‘Driving Mind’ (section 2.1), the sensory and motor cortices, guided by the task-positive network, can drive the car, at the same time that the MTL is being used by the default network to construct intrinsic simulations (e.g. daydreams). This shows that SE, although it usually feeds back to the rest of the brain one unified story of “what just happened”, is not necessary for complex interaction with the world. Only upon introspection or recall [both which are forms of memory search (Edelman, 1989)], do we act specifically according to our hippocampal simulation (see section 2.13 on Anosognosia).

What is different for those of us with working hippocampi, is that episodic memory is being generated (almost) all the time, and so we believe that episodic representation to be the experience, the ‘I’, the actual self-in-the-world. H.M. could interact with the world as we can, but was never generating the movie of memory, and so did not suffer from the illusion that the rest of us do. Those of us with intact MTLs are fooled by memory in a way that H.M. could not be (see section 2.10 on Confabulation).

Antonio Damasio, based on his work with AOCBHD patient Daniel, said that Daniel does not have “an elaborate sense of self … at a point in individual historical time, richly aware of the lived past and of the anticipated future” (1999, p.16). What is missing in Daniel is the sense of self and context that is provided by memory.

Unfortunately, it is impossible to guess what H.M.’s non-memory experience is like, since our own normal pre-memory experience is not available to our subjective view. Nor can we ask patients with AOCBHD whether their perceptions have changed, post-morbidity because, missing their MTLSN, they can no longer evoke the vivid memories of previous experience to compare to their current way of seeing the world.

But we predict there are probably subtle testable differences, [e.g. egocentric vs. allocentric views of the world, or differences in timing or ability for certain associational, metaphorical, spatial and representational tasks], which may help distinguish our SE from AOCBHD patients’ non-memory experience.

Dear Ralf, Bob and Dad,

As I mentioned to each of you, I was hoping to get a conversation started that included all of us, and which hashes out some of the more ambiguous areas of our respective theories. I know that Bob has read Ralf's paper, and I have sent Bob's blog and a link to his book to Ralf, so y'all know some about each other. My dad is very busy preparing to move, but I'm hoping he can at least enjoy the conversation and join in here and there.

The first big topic was suggested by Robert Reid who found our poster presentation at Tucson compelling, and who has since read our and Ralf's papers. He asks: "Have you thought of your own confirming brain experiment – or is one even possible? (Is self-reporting on subjectivity reliable?)"

Well, I don't have a confirming brain experiment, although I had a couple ideas for possible directions.

One is a take-off on Libet's timing experiments; if our subjective experience of will follows the neocortical activation of action, then a patient with adult-onset complete bilateral hippocampal damage (AOCBHD) may not have the same delay. However, they also may not be able to follow the instructions, either, because of memory loss, so that might be a confounding issue. More likely would be Libet's other timing experiments, in which he stimulated the somatosensory cortex, and witnessed a half-second delay in the patient's report of the sensation. I think that delay is likely to vanish, in patients with AOCBHD.

Another possible experiment could utilize the 'boundary extension' function of the hippocampus, as detected by Eleanor Maguire's lab. This is the function that allows a greater scope of the scene to be 'perceived' than what the eyes' immediate saccades can allow. This gives vision its sense of stability and breadth. Patients with AOCBHD can reproduce pictures more accurately than those of us with working hippocampi, because our brains automatically add more boundary than exists in the original. I'm not sure exactly what the experiment would be, but that seems like an area that is ripe for testing.

It might also be possible to use trans-cranial stimulation to inhibit the entorhinal cortex (hub to the hippocampus), and see what happens to subjective experience. Or perhaps Itzhak Fried's UCLA lab (in which epileptics are wired up with deep-brain single-cell electrodes, often in their hippocampus) can try certain timing experiments (his timing experiments thus far have corresponded very closely with Libet's findings). Or he may even be able to stimulate the hippocampus, and observe results.

Anyone have thoughts as to what would be the smoking gun experiment?

Thanks, and I look forward to hearing y'all's ideas!

best,

Matt Faw

Hey Ralf,

I don't recall where I heard this metaphor, but I think it's a great one for the documentary:

The hippocampus is like the TV station of the brain. It receives reports from embedded reporters throughout the whole brain, and it turns them into the "news", a broadcasted master report which lets all the various modules know what's going on. Like all broadcast news, it's a vastly simplified view of reality, but because it is the report which is most widely shared, it's also what is believed.

I'm curious what you think.

matt faw

Hi Ralf, When I was in Tucson recently for the Consciousness conference, I found out that hippocampus specialist Lynn Nadel was teaching at U of AZ, so I contacted him for an interview. By coincidence, he had somehow found a copy of some draft of my paper online, and read it before the interview. Not surprisingly, the idea of putting experience in the hippocampus was novel to him, and he had the same thalamocortical assumptions as everyone else. But during the interview, we ended up basically talking about the theory, and he ended up agreeing with all the technical points. One such point was my metaphor of there being a "pyramid of reports" from the senses, through the sensory cortices to the MTL, with the hippocampus on the top of the pyramid, receiving reports from all over the cortex. In fact, Nadel had just published a paper with his vision specialist wife, Mary Peterson, in which they described the visual scene only coming together at the hippocampus. http://media.wix.com/ugd/b6bc32_851529a9dd4c4c31ad13d3fdc1cc3aa8.pdf Of course, the hippocampus is only on the top of the perceptual pyramid, but is at different places in regards to associational and simulational work. I'm just now editing the interview with Demis Hassabis, which is all about simulation in the hippocampus. Speaking of interviews, I just contacted Martin Conway, whose work I cited a good deal in my paper. He's in London, so I have to find an opportunity to fly over and interview you both. Any thoughts as to what time of year would be good to come over? Maybe there is a good consciousness or memory conference in the UK that I could also attend? How far are you from London? best, matt faw

Dear Ralf,

Following is a hypothesis that's been buzzing around my head for awhile. Wondering what you think about this:

Language Thought: In a brain in which data moves very quickly between modules, it is a mystery that we are also reliant upon inter-brain data transmission which is represented in a form as slow and imprecise as language. Surely other forms of intra-modular communication must be more efficient than language thought. HST: We think it likely that language thought is one form of representing pre-memory cognitive processes, for use in the simulation. One aspect of this is specifically to simplify memory processes, by giving the episode a narration [e.g. ‘he did this to me’ (Gazzaniga, 2011)]. This allows the memory to be lean in terms of content, and to be easily understandable upon recall. It’s also a potential source of error, as language implies a level of certainty that is often not warranted in a world of complex causality and incomplete evidence. Language thought is probably also a tool of intrinsic simulation, like weighing the pros and cons of a decision. Various behavioral maps [e.g. in the ventromedial prefrontal cortex (Behrendt, 2013b)] suggest different choices or outcomes, which are then sequentially formed into language using the left hemisphere’s language encoder, and then passed into the HF to buffer in memory. This allows multiple options to be weighed simultaneously. This is what we know as ‘conscious deliberation’, but not because there is a consciousness doing the deliberation. Rather, pre-memory structures use the HF as a problem-solving workspace in which to manipulate representations, and the HF activity consequently gives rise to subjective experience / memory.

I think this very question (the anticipatory image) is what was tested with DB in the Klein, Loftus and Kilhstrom paper in 2002:

http://guilfordjournals.com/doi/abs/10.1521/soco.20.5.353.21125 Plus all of the Maguire and Hassabis, Dan Schacter, etc. experiments in the same vein. The problem with the simple argument is that everyone I've had this discussion with is so sure that consciousness is necessary for current interaction with the world, that they won't even consider otherwise. Even my use of the term "subjective experience" still ignites the same argument, but with less certainty than when I use "consciousness". I think that part of that problem is that "conscious" also can mean "responsive" (as in, not in a coma), and since H.M. was responsive, he was assumed to also be "conscious". Of course, plants and protozoans are also responsive, but presumably have no experience, which is really the distinction we're making. I think that Gerald Edelman would've arrived (or inspired someone to arrive) at the hippocampus, a long time ago, if not for H.M., DB, KC, etc. And it is damnably hard to imagine what their experience must be like, lacking memory.

Hi Ralf, good to hear from you.

You wrote: "from what you wrote i got the impression that part of you does believe after all that conscious experience of the world is somehow a function of the neocortex; you seem to not completely agree with my suggestion that conscious experience of the world around us is indistinguishable from episodic memory; memory is not just the past it is also the present; memory captures the present and can be recalled as the past"

What I believe is as follows: in the present moment, the functioning of the sensory, motor and task-positive cortices is not directly dependent on the hippocampal simulation, or else HM would have not been able to function. The new moment memory is (in those of us with working hippocampi) being generated in the background of the current task, but it is only relevant in the way it top-down informs the various neuronal populations of 'what just happened'. During deep involvement in a world-oriented task, neocortical perceptual processing mostly ignores the hippocampal simulation, because it would be too slow to be the main input.

All of the actual task-positive work is being done via 'unconscious' (i.e. pre-memory) processes working on bottom-up data flow from the senses (as informed by top-down guidance from the MTL simulation as well as all the various higher processing neocortical areas). In fact, all processing in the brain is essentially unconscious, because 'conscious' only means "included in the memory representation". It is never the process which is conscious, only the after-the-fact representation of or report from, the process.

Since the memory is all that is left of the moment, we believe the memory to be true, and so any mis-representations from the task get enshrined in our view of reality.

In much of life, however (especially social life), we are not just deeply focused on a task, but are continually shifting back and forth from engaging with our TP network, and engaging with intrinsic simulations in the MTL. For example, "how do I look right now?", "is she getting bored?", or "what was that look on his face just now?" are all forms of hippocampal simulations, of the self, of her, or recall of the immediate past. All introspection is retrospection, and this is when we are most involved in the hippocampal simulation, when it forms the most immediate sense of reality and of self.

This is born out by an experiment that I've seen described, but have not yet found online. The researcher (I think a Dr. Johnson) first asked the subjects to hit a button as soon as a cue appeared. They were able to respond in a little over 100ms. The researcher then asked the subjects to delay ~100ms, after seeing the cue, before hitting the button. However, their response times in the second experiment jumped to ~800ms. My interpretation of that is that in the first situation, the subjects were responding directly to neocortical representations of the cue, and thus, were able to respond quickly. They were responding in the way that H.M. can respond. However, given the second condition, the subjects had to introspect upon when they noticed the cue, in order to add a delay. But because they were introspecting a stimulus that could not be predicted, they were not 'conscious' of the cue until ~500ms, which is the time necessary for the predictive memory simulation to be retroactively edited, to add in the cue. When the subject references the neocortical representation of the cue, the response is much faster than when he references the hippocampal representation.

So yes, I agree, our introspective view of reality is always of the hippocampal simulation, even in the present moment. This world that I'm experiencing is the hippocampal simulation, which is built out of reports from the neocortex, but which does not take place within it. This sense of self is entirely hippocampal. All experience itself is memory, the brain reflecting back to itself the reports from other modules, those reports which are bound together in the MTL.

But I think a good deal of the time, our processes are relatively independent of that new memory, and are based upon pre-memory representations. These are all considered to be "unconscious" but they are not "dumb" or "automatic" (or no more so than any brain process). These processes are fast, and are equipped to deal with partial information, fragmented bottom-up representations, and are not reliant on the memory simulation to sense, decide and act.

These neocortical processes are NEVER directly available within conscious experience (i.e. memory). All that is available is a report of those processes, in the form of a thought, image, intuition, fringe sense, etc., sent from the processing module to the MTL, specifically for the purpose of being bound into the full 'consciousness' simulation.

As I wrote in the paper: there is no such thing as a 'conscious process'. Just like dreams of me-performing-processes seems real, so too does the memory of me-performing processes. But in each case, the process happens unconsciously, and is only represented after the fact by the memory simulation. And/or, during intrinsic hippocampal simulations, the unconscious brain uses the hippocampus as a workspace, and manipulates representations within it. But the sense that 'I' am doing the manipulation is just an illusion, as 'I' am just another cobbled-together representation.

Hi Ralf,

Thanks for the distinctions viz. task-positive and -negative modes.

I'm glad to hear your mom is well. Both my parents came to Tucson, so it was a good chance to catch up, and have them meet my girlfriend. My folks live on the east coast, 3000 miles from me, so I don't see them as often as I'd like.

When at Tucson, my dad and I presented the theory as a poster, and received some interest. I hope you don't mind I'm sending some people your paper, as well as ours, to give them a more robust treatment.

While presenting the poster, the most common first response was about H.M. and other patients like him. My argument is that it seems implausible that H.M. not have 'subjective experience' only given the old definition of the term, but if we just see subjective experience as the memory that follows the neocortex's interaction with the world, and forget the notion that it is the "inner light" or intelligence of the brain, then it no longer need seem impossible. I use "driving mind" as an example of how the neocortex can have complex interaction with the outside world, while the hippocampus is being recruited in a daydream, recall or social rehearsal.

However, I know this issue is sticky for a lot of people (and was part one referees' objection), so I've been trying to find some literature, somewhere, in which some researcher said: "something's missing" (other than memory). Whether it's some spatial misunderstanding, or inability to distinguish temporality of sequence, some kind of fragmentation of the scene, inability to introspect, something. All I've really found is that these patients don't confabulate (which I think fits the theory) Damasio's statement that his patient didn't have "extended consciousness" and a couple of vague references to difficulty with spatial perception. Do you have any insight into this? Any good research you've found, which you think helps solidify the case?

Thanks and take care!

matt faw

p.s. I did find the autopsy report on patient D.B. who had lesions exclusively in field CA1, bilaterally. He suffered no retrograde amnesia at all, just complete anterograde.

http://www.jneurosci.org/content/6/10/2950.full.pdf

Hey Ralf,

You're right, I didn't mean "thinking style" per se, as much as distinct process.

For example, maybe task-positive is more about "what's now?", whereas task-negative is more about "what's next?" TP about immediate action, TN about planning the next move. Or TP about reacting to the world, whereas TN defines a social self, the organism representing itself, interacting with others. vmPFC about planning action, dlPFC about putting the plan into action (or inhibiting it).

I know these are overly broad generalizations, but it's helpful for me to get an overall grasp on how things fit together.

I hope you had a good time visiting Germany (assuming you're back in the UK now). I hope your mother is well.

best,

matt faw

Hi Ralf, I hope you and your family are well!

I've just finished up on an intense week in Tucson, interviewing Daniel Dennett and Bernard Baars, among others. Truthfully, the most interesting interviews of the week included Michael Graziano, who is preaching a radically parallel theory, except he's focusing on the TPJ, instead of the hippocampus. Also, through my interview with Alison Gopnik, and talking about the hippocampus, she recommended I talk to U of AZ professor Lynn Nadel, who of course has been publishing hippocampus research since the 70s. That interview was pure gold, as he confirmed many of the steps along the way to seeing the hippocampus as the center of subjective experience!

Oh yes, and I interviewed Thomas Metzinger as well, a week after Tucson, since he happened to be in California. As you saw, I referenced him in the paper, more than anyone else but you, because he was talking about experience and self as representation, rather than process.

And now, I'm back to the paper, and trying to resolve whatever was missing in the previous drafts, and seeking a great deal more research, and finally ending up with this surprising discovery today:

I did not realize that the task-positive network did not include the primary perceptual and motor cortices, so when I read a paper today (co-authored by Raichle and Fox among others) that said they were not part of the TP network, I was flummoxed.

But, on retrospect, it makes sense. "Driving mind" makes more sense if the perceptual and motor cortices keep on chugging along, even when a robust daydream is brewing in the default and simulation networks. It makes sense that these networks should be divorced from the cognitive ones (positive/negative). Pre-motor is considered task-positive, but motor is not, which divorces the decision from the doing.

This all suggests to me that the positive/negative anti-correlation is really about thinking styles. DLPFC is more about what's now? whereas VMPFC is more about what's next? DL is more about the immediate moment, responding to environment, whereas VM seems to be more about the future, using the Hippocampus to figure out a plan for the near or far future.

I'm curious to hear what you think about all of the above.

sincerely,

matt faw

Hey Ralf,

I hope you're well.

I just wanted to share an excellent article: "The Construction of Autobiographical Memories in the Self-Memory System" by Conway and Pleydell-Pearce (2000). I haven't found anything else quite like it. http://homepage.psy.utexas.edu/HomePage/Class/Psy394U/Bower/11%20Soc%20Cog%20Personality/XX%20Constr%20Life%20Stories/Autobio%20Mem-Conway.pdf It describes why we remember what we remember, why we forget, how the self fits into all of it, etc. I'm sure it would be very helpful for any work on the Self.

best,

matt faw

Hi Ralf! Great to hear from you!

I'm doing well; focusing on prepping for the Tucson consciousness conference (which starts in 10 days). I have a nice collection of interviews lined up, including Dennett and Bernie Baars. I'm also prepping my first conference poster, to explain the hippocampal simulation theory.

Sad to say, my paper was rejected by the Journal of Consciousness Studies. Evidently, as a novice writer, I fell into pitfalls I didn't know existed. We got attacked on one side for our anatomical explanation not being comprehensive enough, and from the other side for our philosophical argument. Perhaps I should just focus on the philosophical part, and say: if you want an anatomical argument, go read Behrendt's paper! 🤣

I probably won't be able to work on the paper until after Tucson, but I'm committed to making it better. I'd rather be working on the movie, but I think the future of the doc is intimately connected to the paper, so I'm not giving up on it. Hopefully with the poster, I can engage in discussion with the critics, rather than just get their comments, and I can have a better idea of what needs fixing and addressing.

I hope you are doing well and enjoying spring!

best,

matt faw

Dear Ralf,

Thank you for for your swift response on repression. What you wrote about the superego makes me think about a phenomenon I've noticed in myself, and have heard described elsewhere. It's the sense of a watching presence, that I used to feel all the time, and brought shame to me, even when I was alone. I knew perfectly well that no one was there, and theorized that it was an internalization of my parents' presence, that they had trained it into me, so I would behave as if they were watching, even when they weren't.

In fact, I realized at some point, that when I was experiencing strangers as being judgmental towards me, that I was really projecting this watching presence out to them, making them into the cause of my shame, instead of realizing it was a self-reflective loop. I have since wondered if that loop function is part of the default-network-driven simulation, a simulation of other people that arose from fear of social approbation.

I can see why you say that the sense of self is intimately tied to this sense of being observed. I've often wondered if the reason why we spend so much time in default mode, is because we're so often projecting out to some 3rd party point of view, and trying to witness ourselves.

In ecstatic dance, this is the element I try to surrender: any attempt at viewing myself from outside. "Dance as if no one is watching". Including: as if I am not watching. Dance without a sense of story, right or wrong, shame or glory. It's an ongoing practice.

best,

matt faw

Dear Ralf,

Yes, I agree. The subjective self is an illusion, a construction for memory. But what about the capital-S Self: the body and its brain? Cannot the body+brain be fooled by the simulation? If a vision module receives hippocampal output, which helps it predict the next moment more efficiently, cannot we say that that module "believes" the report that it receives from the hippocampus, and can be fooled by it? If I meet someone new who reminds me of someone else I don't like, I end up simulating them with distaste built in. Am I not fooled into a false reality of that other person? If I have a strong self-image, then I craft my memories to cater to that image. Am I not fooling myself with my simulations? Isn't the sufferer of Capgras syndrome fooled by the fact that his mother is simulated without strong emotional content? Isn't the clueless drunk guy at the bar fooling himself when he believes that: "She was so into me!"? Aren't Fox News viewers fooling themselves when they intentionally avoid letting contradictory information into their memories? Aren't we all fooling ourselves when we treat pain as something to be ignored or tuned out from? I don't see any contradiction in saying that the subjective self, the self of belief and memory, is a fiction, but that the organism itself can be fooled by that fiction. That seems to describe the exact world we live in. It doesn't even make sense to me to say that the "self is just an illusion" without there being some entity that is fooled by that illusion. Still scratching my head, matt faw

Thanks Ralf, for your e-mail.

I wonder if you can help me bridge the gap in my understanding. I have told you many reasons why I think the simulation, by defining reality to the rest of the brain, has powerful causal effects on human behavior. Absolutely agreed that the simulation is ultimately nothing but patterns in the hippocampus. But the whole explanatory power in this model, as I wrote in the paper, and throughout this conversation, is that people are fooled into thinking that the simulation is reality.

I understand your assertion, but not your reasoning behind it. You've heard all my reasons; can you please explain yours, so I can understand better?

Thanks!

matt faw

Dear Ralf,

Thank you for your primer on attactor state dynamics. This helps me a great deal in understanding the auto-associative processes of CA3.

Forgive me, however, if I still have resistance to seeing the simulation as a "one-way process".

What I hear you describing is the function of the "hardware" of the system. What I propose is that the simulation itself can act as software, that it can be a causal agent, even though it is only a story, a series of patterns.

In society, we can witness many many stories that work as causal agents, even though they're essentially just ideas. Patriotism, marriage, religion, justice, art, values, money, rules, societal norms, etc., are all stories that have powerful causal effect on behavior.

Likewise, even though the self and experience are only stories, are only patterns of neural encoding, they still shape behavior by defining reality. The anosognosic hemiplegic behaves according to the simulated memory of self-movement. The dissociative identity patient behaves according to whatever memory stream is activated. The gay man who is closeted from himself behaves accordingly to his simulation of himself as straight, rather than how the rest of his brain would have him behave. I think that memory would serve no purpose, if it did not represent actual things in the world, including the self. Memory would only be like the accretion of rings in a tree, a record of what happened that is meaningless to the tree itself. The CA3 attractor patterns must encode within them facets of personal history, or we could not recall those facets, could not use them in our future decision-making. The brain is the hardware, and does all the work. But I think the simulation ends up becoming software, an ersatz reality, including a self. This, despite the fact that it is only a story, a chain of memories that is entirely dependent on the attractor dynamics that underlie the hardware process. It doesn't make decisions, but it continually informs them.

Thank you again for your continued conversation and for hearing my thoughts on the matter.

With much respect,

matt faw

Hi Ralf, thanks for your reply.

I think I'm finally understanding the difference in how we're talking about this.

When I wrote that the rest of the brain "believes the simulation", I did not mean this literally, like the way some people "believe Fox News". Because there are no homunculi in the brain, and there is no screen.

Because I am not a neuroscientist, and do not have access to the all the subtle mechanics of the brain, I use the colloquial language to describe the brain processes. But it is clearly the processes that run the ship.

On my computer, I interface with the GUI, which is the people-friendly appearance of the programming. That is the part that I can understand and manipulate, but clearly it is not what gets things done; it is the underlying code (and the physical processes of the computer) which make everything possible.

Likewise, "consciousness" is the memory-friendly appearance. Within it, memories interact with each other, in this experience-like way. However, every memory itself, and all of its interactions, are clearly just processes of the underlying structures.

But I'm also wary of portraying the hippocampus as a "holodeck", a place of entertainment that has no bearing on the actual function of the ship. Nor as the "captain's log", just a record of what happened. Instead, I see it as an associative engine, a predictive engine, a workspace in which other brain modules are able to create more complex views of reality, to behave beyond simple conditioned algorithms. To me, the hippocampus seems like it's intimately involved in ongoing processing of the now, and seems to powerfully inform behavior and beliefs. And that's what I hear from your "appetitive behaviors" paper, as well.

In order to influence the other modules of the brain, and to elicit data from them, it must share its output somehow ( albeit clearly not as a TV show for an audience). That mechanism is what I'm curious about.

best,

matt faw

Dear Ralf,

Good to hear from you. Thank you for your reply.

Forgive me if I'm still not getting it. Let me at least say how things appear from my (admittedly naive) position.

I think of belief as straddling the line between memory and habit. Every episodic memory encode can be thought of as a belief, that "this is what happened". And people will react as if you contradicted their belief, when you contradict their memory (at least the emotional ones).

The episodic memory informs the semantic memory, and then the belief becomes "unconscious" because it no longer needs the hippocampus. It could also be said to start being habit at this point, because the association is "automatic".

Habit I think of as an algorithm: if A + B + C stimuli cross some threshold activation, then X outcome, whether X is a thought, an emotional activation and/or a behavior. If I understand your "appetitive behaviors" paper, the vmPFC "behavioral map" works as such an algorithm. Of course, there are competing algorithms with mutually inhibitive outputs, so the formulas get complicated.

In the case of semantic ('unconscious') belief, the mechanisms may be largely concept cells, mapping associations. So that whenever I see a picture of "Halle Berry" or "Bill Clinton", the appropriate concept cells are activated, which co-activate semantic, emotional, and perhaps episodic associations.

Now story-based beliefs are another thing, and I think they tend to fall more in the territory of 'conscious' associations. For example, ask a religious bigot their feelings on homosexuality, and they will bring up a story-based answer about some verse from the scriptures. However, their physiological response to the idea of homosexuality has almost nothing to do with the story of scripture; scripture is a self-story told to explain to themselves why they have such intense feelings against the idea. The actual feelings probably are conditioned by years of being raised in a society in which homosexuality is treated as a perversion, in which the person has witnessed or experienced horrific teasing and bullying. There may even be some Freudian self-loathing going on, fear that some part of the unconscious self is gay, which makes the response even more violent. But none of these really have anything to do with the stated belief, the quoted scripture.

That said, these story-based beliefs, even if they are postdictive rationales given to explain self-phenomena, are still forms of habit. The algorithm is: if X (someone questions my feelings about homosexuality), then Y (the scriptural story comes up to explain it).

The most deeply held (and least conscious) belief, I think, is that the simulation is real. This is the real me, this is my body, these events are what happened, this construction is the real reality. This belief is at the heart of the pathologies of subjectivity. Weaken this belief, as Buddhist meditators supposedly do, and the simulation becomes less dogmatic, less concrete. Reality is understood as a construct, and beliefs/habits are revealed as conditioning. This allows for slower, deeper responses, in which more task-positive stimuli is allowed in, to inform the simulation, and a wider range of algorithms are allowed to participate in the decision-making. This is the opposite of "having one's buttons pushed".

As far as the hippocampus is concerned, I understand that the CA3 field is auto-associative, which I understand to mean that no other brain module is needed, to elicit related memories.

However, CA3, as I understand it, then feeds its associative data to CA1, which has already received the input from the EC on a faster route. CA1 then generates a simulation that is informed by 1. the task-positive network's flow from the EC, 2. its own predictive functions, and 3. the associations from the CA3 field.

Then the feed from the CA1 simulation informs not only the vmPFC and multimodal areas like the IPL, but also returns re-entrantly to all of the major neocortical modules, as "top-down" information, to help speed up predictive processing. For example, "object identification" gives way to the less-processing-heavy "change detection"

In your "appetitive behaviors" paper you talk about the output from the hippocampus then being fed to the vmPFC, where the "situation" is the conditioning stimulus. It seems to me that the "situation" and the "simulation" are the same thing, so in this way, the simulation directly informs behavior and emotion.

Plus, all of the intrinsic simulations (prospection, mental rehearsal, theory of mind, etc.) are constructed in the hippocampus, and help inform possible future actions.

It is in all these loops in the brain in which it seems as if the simulation becomes known elsewhere. The hippocampus is informed, and then it informs. No neuron is sitting in a theater, watching the show, but many different modules receive at least part of the report from the hippocampus, and react accordingly.

This is my understanding, and it says very clearly to me that the simulation is not epiphenomenal. It is not itself a cognitive structure, but it IS a packet of cognitive information. The hippocampus, as I understand it, is an associative engine that connects many different modules, helps line them all up temporally, brings their content into alignment, and then prepares for the possible long-term storage of the associations, primarily during sleep.

The very fact that our bodies have to reduce muscle tone during REM sleep, so we don't act out our dreams, says to me that the hippocampal simulation must have direct influence on our behavior.

So that's my understanding of the relationship between association, simulation, memory, habit, belief and behavior. I see a ton of evidence that the simulation influences other parts of our brain, including behavior. Again, maybe I'm missing something, but I don't know what that something is.

Thanks again for your continued conversation!

best,

matt faw

Wow, Ralf, that's an amazing statement. I'm still trying to wrap my brain around it.

Certainly pathologies like schizophrenia or anosognosia show that the simulation can become severely detached from consensus reality, can be like a dream.

The one thing that I can't get past, though, is belief. The simulation shapes belief so powerfully, and subjective reality is so compelling, that I have a hard time thinking about that as epiphenomenal.

Maybe some of that subjectivity is in the pre-memory processes, but much of it is not.

For example, people neglect their bodies, because they are not well-represented in the simulation. But I'm confident that the body is very well represented in the brain. We have a sitting society, precisely because we tune out of our bodies, and we imagine ourselves as being something other than the body. We tune out from pain and discomfort, even though they are the body's own signals, designed precisely to get us to pay attention. These are serious consequences of the simulation.

Plus, we know that the brain can manage multiple interpretations of stimuli at once, can handle extremely fuzzy processing and multiple perspectives. But most of that is gone by the time we get to the hippocampus, which basically represents one perspective for memory. And that is what is believed.

All of the pathologies of subjectivity that I wrote about in our paper are consequences of believing the simulation. Racism, homophobia, sexism, etc. are all consequences of believing the simulation. People often live scripted lives, lives that are more informed by story, than by experience, because the simulation is so well trained in childhood.

Belief in the self has powerful ramifications. Almost everyone believes that the aesthetic qualities of the world are actually out in the world, rather than in the simulation. I'd venture to say that most people think their opinions on the world are basically the "right" ones, despite the sheer statistical unlikelihood of that. And people get very angry and defensive about their ideas, tastes, and values, even though these are non-corporeal things, that cannot be threatened. We believe in entities like "reputation" or "relationship" as if they were concrete, singular things, as opposed to just descriptors for our projections and feelings. We even believe that our simulations of other people are real, despite how suspect that is.

Plus, we can look at H.M. and other complete bilateral hippocampal patients, and see that although they can still function, they have stunted lives, lives with no imagination or possibility. They are no longer doers, and can only be patients, recipients of others' welfare.

It's very hard to see all that as epiphenomenal. To me, this says that our brains are fooled, whenever we are awake, by the simulation, and by the long chain of memories.

Maybe I'm misunderstanding you, but I have a hard time seeing the simulation as being completely "orthogonal" to brain function, since it shapes our behavior so powerfully.

respectfully,

matt faw

Hey Ralf,

If you've got a minute, I've got a burning question.

I've just started to learn more about the temporoparietal junction's role in the simulations. I've just recently read that it's active during memory recall and theory of mind, among other things. Plus, lesions to the right TPJ can create neglect, not only of the left half of the body, but also of the left visual and spatial hemifield. The TPJ is a multimodal area, and is associated with switching between default and task-positive modes, and for orienting attention in space,

This constellation of evidence is very suggestive, and I can't help but look for connections to the simulation theory. I know that anatomically, it is highly connected the vmPFC, which I find interesting.

The hypothesis that immediately occurs to me is that the TPJ might present the simulation to the sensory cortices. We have a metaphor of memory being a player-piano roll, that plays back on the hippocampus, which then recruits the neocortex perceptual, motor and somatosensory cortices to "play back the tune". So one possibility is that the vmPFC uses the TPJ to access those various task-positive cortices, for playing back memory and other simulations.

This would explain hemispatial neglect, since the simulation would invoke both left and right visual hemispheres, and only receive back images and spatial information from the left, thereby constructing a limited world. Also, in anosognosia, when the MTL body schema predicts a movement, it would be the interaction with the somatosensory and motor cortices that report prediction violations. However, if the pathway from the simulation to those cortices is cut, then the question of "was my prediction right?" never gets through, and therefore the prediction is allowed to stand, and the simulation reflects that the paralytic's prediction of movement, which creates a memory of having moved.

Of course, since these pathologies arise only from lesions to the right TPJ and greater parietal cortex, and not the left, it may suggest that the bilateral simulation is fed to the right TPJ, but for some reason, only the unilateral simulation is sent to the left hemisphere.

So, those are the ideas that my TPJ research is prompting in me. I wonder if your research has found an important connection between the simulator system and the TPJ?

matt faw

Dear Ralf,

Thank you very much for reading the paper, and for the feedback. I really appreciate you taking the time.

Now I have to wrestle with how best to implement and integrate your notes. I'm also forwarding your comments to dad, see what inspiration he can get from them.

Thanks also for sharing your previous work on idealism. The epistemological implications of simulation has been at the heart of my own 'spiritual' path for years, learning to surrender certainty and stories, in order to experience beyond my habits. Trying to surrender my sense of self for a new model of my brain as a community, and learning to trust the brain parts that I can't directly observe. That's why, when I first heard that there was an experience simulator in the brain (in that conversation with my dad last year), that it was a short leap of logic to the hippocampus as home of consciousness, and from there, seeing subjective experience (and self) as memory.

When I started conducting interviews, I expected everyone to talk about experience as a construction, but I got surprisingly little of that, mostly from the specialist on psychedelics. I think Dennett's Cartesian Theater argument is so compelling that it has put many off the scent, which is why I had to address that argument. Thomas Metzinger, who I quoted at the top of the paper, is one of the few I've seen who is championing the idea of simulation of experience and self. Others hint at it, but I think the illusion of consciousness as command and control unit is so powerful, that it is hard to go deep down that rabbit hole. That's why "driving mind" and Libet's experiments are at the top of my list of evidence, because both specifically illustrate that the sense of an agentive 'I' is a feature of memory, and is not needed for choice.

A thought about the self: what I think of as a personality, at least from the 3rd person perspective, is a constellation of habits, including habits of simulation. From your "situational behavior" paper, it's interesting that the vmPFC stores and elicits behavioral maps, which I interpret as habit. Of course, there is habit throughout the brain, Hebbian, concept cells, conditioning, semantics, etc. But behavior maps, in particular, seem to be the defining features of a personality.

I'm curious if you know how the default network and the hippocampus relate to Michael Gazzaniga's idea of a left-hemisphere narrator module. That seems to me to be an important part of the story (not for this paper, but for the documentary), how language can serve as a low-bandwidth mnemonic, simplifying memories into stories. That simplification, and the need to tell a story that fits within the context of one's self-model, is I think, the source of a good deal of error. I had included his theory in previous versions of the paper, but left it out this time, in order to be leaner and more concise. I'm hoping to interview him, as well, for the doc.

Thanks again for your time and energy, and for this continuing conversation!

Much respect,

matt faw

Thank you so much, Ralf! I really appreciate it.

You'll see that this is version 38. We actually submitted a different version of the paper earlier, last August, but were rejected. Not because of the theory, but because I had never written a paper before, and my prose was reviewed as being too "unserious".

I completely re-wrote my sections of the paper for this new version, and I think it's now pretty clear, and well supported. I hope you agree. It helps a lot that I now have your paper to cite, for some of the more speculative aspects of the theory. The first two sections, which are both philosophical arguments, are mine, and the last section, which is an argument from neuroanatomy, is mostly written by my dad. I don't think dad has yet read your paper on consciousness, unfortunately, so this draft does not reflect your influence there. Your paper is much more comprehensive and thoroughly researched, so I'm happy to have your feedback there, and throughout.

The file is in .pdf format. Please let me know if you have any problems accessing it. Feel free to highlight, cross-out, add, etc.

Thank you again, so much, for being willing to look at our paper. I look forward to hearing your response.

sincerely,

matt faw

Dear Ralf,

Thank you for your kind wishes about my sister. She is indeed, recovering well.

Yeah, dad has been focusing more and more on the neural correlates of emotions, over the last several years, and has been presenting talks on the insula, etc. at various conferences. When I sent him the notes I generated, while reading your paper, his response was: "Wow! I love your description of Behrendt's paper. I must now read it. Every bit of it deals with my book. I am dealing with each of those areas you mentioned. Behrendt puts it together in a way that I must incorporate." He's recently been really focused on my sister, of course, but he's a voracious researcher, and I'm sure he'll devour your paper hungrily.

Ralf, I have a favor to ask of you, beyond being interviewed for the documentary. As I'm sure you can tell, I've been thinking about the hippocampus' role in consciousness for some time now. I had reached an impasse in the documentary, where science was not connecting the dots, and I knew that simulation had something to do with it. Then, in a conversation with my dad last February, I mentioned how similar I thought that recall and imagination were, and dad told me about the recent work tying the hippocampus to these various simulation tasks, and the light bulb went off. By the end of that day, I was convinced that subjective experience was a simulation, generated in the hippocampus, for the sake of memory.

When I started looking for researchers connecting consciousness and the hippocampus, I did not find you immediately, and so it seemed like no one was yet saying what seemed to me to be a powerful explanatory model, the only model I've heard that really made sense to me. I thought I saw an elegance and clarity in the hypothesis, which explained all these major mysteries of subjectivity. But, I had no one to interview about it, so instead I interviewed other specialists on the hippocampus, like Demis Hassabis and Itzhak Fried, and worked on a paper to float the idea to the scientific world, see if anyone would respond.

Now, of course, that I've found you, I am grateful to have a chance to finally interview someone who has already approached the same hypothesis from an neuroanatomical bent. And I am very much looking forward to the chance to interview you.

However, something is still missing. I cannot release a documentary claiming what science is saying about consciousness, unless the broader community has first tested that theory. So I still want to submit my paper to a journal, and present a poster at the Tucson TSC conference, etc., start a P.R. blitz with this theme, in order to try to encourage others to test this theory. That way, hopefully, by the time the film comes out, this theory really will be reflective of what science says. (and if it's not, and I'm wrong, then I want science to show that, so I can change my film).

I was trying to finalize our paper (dad wrote the neuroanatomy section), and was looking for final citations, when I found your 2013 paper on consciousness and the hippocampus. Dad and I both feel that our paper is about ready to submit, but I sure would feel better, having an educated 3rd party read over it first. I don't know of anyone else on the planet who even knows the theory, and therefore no one else is better suited to give constructive feedback about it. I'm sure you are a very busy person, but if you think you might have time to look over our latest version of the paper (which of course, you are now heavily cited in), I would be extremely grateful.

Respectfully yours,

Matt Faw

p.s. Below is the copy of my notes of your paper, that I sent my dad. He and I refer to the hippocampal formation as HF.

Hey pops, I doubt you've had the chance to read Behrendt's "appetitive behaviors" paper yet. I'm making my way through it now, and I wanted to collect some notes and thoughts as I do so: - One of the major themes, thus far, is in explaining behavioral conditioning in rats, via interaction between the hippocampus, vmPFC, and amygdala. Since behaviorism experiments tend to adjust one variable at a time, Behrendt I think is trying to explain how conditioning happens in the real world, in which the "situation" is the conditioned stimulus. The situation being a complex multimodal multidimensional stimulus, a combination of allocentric location map, place cell firings within that map, head direction or view direction cells, motor response, sounds, smells, vibrations and lighting, etc. Behrendt says the hippocampal output = the situation = the stimulus that is conditioned to. That conditioning happens in replay, after the salient event. After the rat has run the track to get its reward, the place cells that led it up to the reward now fire off backwards, to connect the memory of the reward to the steps that led up to it. This replay seems to be managed by the vmPFC, and while this new memory is still labile, the vmPFC brings in meaningfulness information from the basolateral amygdala, which gets encoded with the new memory for context. This maps "how did it make me feel" onto the memory encode, and strengthens the probability of that memory being recalled and being put into use later. Before running on a familiar track, the rat's place cells also pre-play the necessary behaviors to get to the reward, as mental rehearsal / prediction. Goal-oriented behavior is seen as depending on a loop between HF and mPFC, which keeps the expected outcome and its behavioral map in working memory. The medial PFC is explained, among other things, as the store house for mature associative representations. The index of memory eventually gets consolidated / copied to the mPFC, so it is no longer dependent on the HF for storage. Also, the mPFC receives output from the HF, and builds maps and indexes of various kinds, which then become short-hand models or assumptions about allocentric space, behavior sequences, meaningfulness of situations, and in humans, at least, maps of other people. More data that supports the view of HF as predictive agent. The rat's dorsal HF increases firing in anticipation of a reward. The closer the rat gets to the reward site, the more activity. Also, if new information arrives that fits within a pre-existing schema (i.e. if it matches expectations), then the HF activity is decreased, compared to when expectations are violated. The implication seems to be that the vmPFC can integrate this new semantic information into its mapping system, without as much need to re-simulate what just happened, with the new data included. It is when expectations are violated that the HF becomes more active, re-simulating its prediction to include the new data.

The orbitofrontal cortex may hold the mapping of reward (medial OFC) and punishment (lateral OFC) with objects. These two sections of the OFC seem to be mutually inhibitive. Sharp wave-ripple complexes are part of the HFs replay and consolidation process during times of rest, slow-wave sleep, and while consuming. They are associated with time-compressed but forward in sequence replay. The mPFC strong connection to motor and sensory areas may reflect its replay not only in the allocentric place map, but also replay of contextual stimulus and behavioral motor firing. Replay in the HF and mPFC is mirrored by replay in the ventral striatum, which Behrendt thinks is replaying the meaningfulness of the memory, its reward or punishment value. Motivation is provided by the nucleus accumbens core (reward) and shell (motivation toward reward). The prelimbic cortex (PLC) of the mPFC projects to the core, and the infralimbic cortex (ILC) to the shell. The core and PLC (which holds the behavioral maps) orient toward previously learned rewards, whereas the shell and ILC are necessary to orient toward open foraging without expectation of reward. PLC motivates precise behavior sequences, whereas ILC motivates more global behavior, like fight or flight. Stress increases ILC activation. PLC activation inhibits the ILC. Stress encourages the locus coeruleus to release noradrenaline into the PLC, effectively inhibiting it, and thereby removing the PLC's inhibition of the ILC. The ventral HF, when activated by stressors (via the locus coeruleus or basolateral amygdala), regulates the hypothalamic-pituitary-adrenal axis. It also increases, via a multisynapic pathway, activation of VTA dopamine output to the nucleus accumbens shell, increasing motivation to act. I get somewhat confused, when Behrendt is talking about dopaminergic pathways of motivation, but this much I think I follow: mPFC can activate the ventral tegmental area into delivering dopamine to the nucleus accumbens and into the mPFC, in order to keep motivation oriented toward the goal. Novelty signals or violations of predictions increases CA1 to VTA activation, increasing dopamine flow. And the central nucleus of the amygdala can also increase VTA activation. Dopamine increase in a rat's "preferred place" may mirror the increase in human social bonding and mating to "preferred people". The subgenual (subcallosal) anterior cingulate (area 25) is the primate version of the rodent's infralimbic cortex. It and the wider vmPFC are involved in initiating global fight or flight modes, as well as orienting in response to painful stimuli, including social discomfort. The subgenual anterior cingulate activates during anxiety and is chronically over-active in depressed patients. The anterior agranular insula receives motivationally salient information about interoceptive body states, including pain, thirst, hunger, cold, anger, fear etc. It provides bias and constraint to the vmPFC, in the picking of appropriate behavioral sequences. The insula effects the HF simulation via input through the lateral EC. In rats, instinctive behavior (like eating or drinking) is activated by connections from the ILC to the medial hypothalamic zone, and general appetitive arousal is invoked by ILC to lateral hypothalamus connections. The infralimbic cortex also projects to the autonomic and visceromotor centers in the hypothalamus and brainstem. In particular, it projects to the dorsolateral hypothalamic nucleus, a central part of the "visceromotor pattern generator network". The ILC also regulates the hypothalamus' "neuroendocrine motor zone". Motivational body states are part of the current hippocampal situation, and can motivate task focus. Planning is done in the default network, by invoking previous memories in the HF, and re-constructing them into a rehearsal. The plan, a newly minted behavior/context map, is followed by the task positive network, until violations of expectations force the switch back to the default mode, where the vmPFC uses the HF to re-simulate that part of the plan. The human dorsal anterior cingulate cortex helps provide attention toward ongoing task-relevant stimuli. That section (plus the wider dorsal mPFC) is also sensitive to violations of expectations. The rostral anterior cingulate, in conversation with the HF, learns about aversive stimulus to avoid, in the pursuit of the task, and then helps guide that avoidance behavior, along the route to the goal.

Dear Ralf,

My apologies for taking so long to get back to you. My sister had surgery last week (it went well), and I went to the east coast to support her.

I just got back and read your "situationally appropriate behavior" paper. It's very exciting, because it explains behavior and conditioning. I also sent the paper on to my dad (my science advisor on the documentary), along with your question about publishers. He and my mom are still taking care of my sister, so I don't think he's had any chance to get to it yet. But he's also working on a book, about the neural correlates of emotions, and I know he'll find your paper very interesting. Hopefully he'll also have some insight into publishers.

As a layman, I did find some parts of your paper hard to follow, mainly because most of my research, thus far, has been on the hippocampus and related MTL structures. So there were a lot of structures and concepts that were new to me. I wrote down notes for myself, and sent them off to my dad (copied below). If you see that I was way off on my understanding of the paper, I'd appreciate the correction.

I also did a rash of quick research on pathologies of the vmPFC, and found some interesting data that seems to support the idea of that structure as the driver of emotionally significant simulations, of both self and other. In the gambling task scenario, vmPFC patients have a hard time imagining their own future emotional response to losing or gaining money (even though the actual response is not later inhibited). In the Ultimatum game, vmPFC patients are mis-led by their immediate emotional response of fairness-violation to turn down relatively large percentages of offered money, perhaps because they are not able to create a prediction of how losing the money will make them feel later. ToM emotional tasks are inhibited (in damage to the right vmPFC), like being able to perceive irony or faux pas. And these patients have trouble seeing the moral violation in the person who intends ill, but does not succeed (like the person who intends to put poison in someone's coffee, but accidentally puts in sugar, instead), as if they are unable to take on the emotional perspective of the perpetrator. And even the emotional outbursts of frustration that seem to plague vmPFC patients may have to do less with emotional volatility (since they have overall flattened affect) and more to do with inability to predict how inappropriate their emotional response will be, how others will perceive them.

I was wondering how the vmPFC relates to emotional memory recall, and your paper answers that pretty well. And your paper is clear that it is at the heart of situational conditioning and regulating behavioral repertoire. That's a lot of function.

Thank you again for sending the paper, and for this continuing dialogue. I hope you are well!

best,

matt faw

Excellent! Thank you, Ralf, for explicating about emotions and the simulation; good stuff! And yes, I'd love to read the .pdf that you wrote about this.

What you wrote reminded me of the interview that Jesse Prinz gave me, talking about how we project our aesthetics and values on the world. How we tend to mistake things for being inherently beautiful or repulsive or whatever, rather than understand that we are representing them with that added value.

It's like Capgras syndrome, how the absence of the emotional marker in the simulation of mom means that I must radically re-explain reality, in order to explain how I feel. That is powerful stuff.

I've been thinking about the amygdala and insula, not just as emotional centers, but also as "meaningfulness" centers. They seem to do a good deal of the "painting" of the simulation, giving it context and helping to evoke previous similar memories.

I'm also curious, and wonder if you know about this: lateralization of function in the hippocampi. I've read occasional accounts which suggest that there is some task-specialization between the two. But I have to wonder: do they each simulate? Is it simultaneous? Are they simulating different versions of the same reality, say the left one for objects in the scene and the right one for meaningfulness or relationship? Interesting also that the two hippocampi have their own commissure, and do not rely on the corpus callosum.

I am very interested in the subject of the self, and am looking forward to hearing what you have to say about it. I've been thinking a lot about the fMRI studies that show the default network (and particularly the vmPFC) in conversation with the hippocampus, in daydreaming, projection,, theory of mind and other simulations. I think this suggests that the hippocampus is the "mind's eye" to the default network, the workspace within which "I" am able to manipulate representations. This, plus the fact that the vmPFC seems to store models of other people, makes it seem like a likely candidate for much of the sense of self.

In Buddhism, they invite the practitioner to look inside for the self; after unwrapping that concept for awhile, the students usually arrive at "the observer", the one who is perceiving consciousness. Which of course sounds a lot like Dennett's dreaded homunculus (except the observer does not have the ability to control the body). But I think it's a fair question: if the vmPFC is the manipulator of the hippocampus, could it also not be considered the observer of it? Isn't that what feedback is? This is one of my major questions about subjective experience: is it the simulation, being created within the hippocampus? Or is it the output of the hippocampus, being witnessed by another brain structure, like the vmPFC?

I'd love to hear your thoughts on all this. Thanks again, so much, for engaging in this conversation with me!

best,

matt faw

Dear Ralf,

Thanks so much for your reply. I'm afraid I can't speak as to whether Dr. Koch has seen your paper; I was just reading an older article he had written, in which he momentarily entertained the idea of the hippocampus as the center of consciousness, and then used H.M. to dismiss that possibility. However, Antonio Damasio, who worked with complete bilateral damage hippocampal patient Daniel, makes it clear in "The Feeling of What Happens" that Daniel does not have the kind of phenomenal "consciousness" that someone with access to episodic memories would have. So I see H.M. and Daniel as having Actual Experience and Interaction with the world, but never having a subjective experience representation of that, for memory.

I also think that Daniel Dennett's "Cartesian Theater" argument has scared a lot of people off of the idea of subjective experience as a multimodal movie. But his argument really shows the absurdity of an inner me, and doesn't get rid of the evidence for experience-as-construction. If the "me" is part of the construct, then there is no infinite regress of homunculi.

I absolutely agree with you that episodic memory and the self still do play important causal roles in the life of the mind, as H.M. and Daniel demonstrate.

After all, memory defines reality, quite literally so with veterans who experience flashbacks. Sufferers of dissociative identity disorder have their memories segmented off from each other. Since the 'I' is generated within memory, each memory stream has its own 'I'.

As you point out, schizophrenics receive confused and redundant data for creating their simulations, and the result is a very messy reality. In fact, I think schizophrenics may point to why most brain data is left out of the simulation: because to have our memories cluttered with multiple interpretations and background processing would create a reality that was nearly impossible to navigate.

Hemispatial neglect shows that when incomplete data arrives at the hippocampus, the simulator does its best to make a reality out of it, anyway. Likewise, a patient with Capgras syndrome builds his memory simulation with certain emotional data missing, so when mom is simulated, she looks and sounds like mom, but does not feel like her. It is actually more appealing to come up with the preposterous notion that mom is an impostor, then it is to doubt one's own reality simulation. That's powerful.

Confabulation is another interesting case. Patients with frontal lobe amnesia (as I'm sure you've experienced) can create a memory-like simulation out of whatever data is available, but patients with complete bilateral hippocampal damage do not confabulate. They cannot construct memories, and so they cannot mis-construct them, either.

And on and on. I see tons of evidence pointing in the direction of your theory. The fact that the visual cortices do not construct an entire scene, but rely on the parahippocampal gyrus to construct it, and the hippocampus to bind the objects, scene and self-information. The fact that prospection, imagination, mental navigation, episodic recall and even theory of mind all display similar neural patterns of the default network activating the hippocampus, implying that the hippocampus is the "mind's eye" for the default network. In particular, this line of research seems to implicate the vmPFC in some very powerful self-processing, in connection with the hippocampus as its "workspace", to use Baar's term.

Really amazing stuff, and I'm so happy to be making this documentary now, to be able to feature this cutting-edge science and theory. Of course, I know that most of the scientific community has not yet grasped what you are proposing, but I am confident that that will happen soon.

with much respect,

matt faw