A programmer who is obsessed with giving experimenters
a better environment for developing biologically-guided
neural network designs. Author of
an introductory book on the subject titled:
"Netlab Loligo: New Approaches to Neural Network
Simulation". BOOK REVIEWERS ARE NEEDED!
Can you help?
Scientists at UC Berkeley have taken brain scans of subjects in an fMRI machine while they watched a movie clip. They then reconstructed the movie the subjects were watching using only the brain scan data, and a database of 18 million seconds of random video gleaned from the web.
First, they used fMRI imaging to measure brain activity in visual cortex as a person looked at several hours of movies. They then used those data to develop computational models that could predict the pattern of brain activity that would be elicited by any arbitrary movies (i.e., movies that were not in the initial set). Next, they used fMRI to measure brain activity elicited by a second set of movies that were also distinct from the first set. Finally, they used the computational models to process the elicited brain activity, and reconstruct the movies in the second set.
The amount of new understanding this could allow us to gather about mind-brain correlates and first person knowledge should be considerable. If this lives up to the hype, a lot of new research ideas should come out of it. Keeping fingers crossed here.
In the above clip - the movie that each subject viewed while in the fMRI is shown in the upper left position. Reconstructions for three subjects are shown in the three rows at bottom. All these reconstructions were obtained using only each subject's brain activity and a library of 18 million seconds of random YouTube video that did not include the movies used as stimuli. The reconstruction at far left is the Average High Posterior (AHP). The reconstruction in the second column is the Maximum a Posteriori (MAP). The other columns represent less likely reconstructions. The AHP is obtained by simply averaging over the 100 most likely movies in the reconstruction library. These reconstructions show that the process is very consistent, though the quality of the reconstructions does depend somewhat on the quality of brain activity data recorded from each subject. [source: Gallant Lab (see resources below)]
Stanford University School of Medicine has developed a relatively simple new imaging technique that provides a very exact way to capture the synapses of a connectome with pinpoint 3D positional accuracy, and considerable contextual resolution.
Stanford has performed a study (see below), which was admittedly done primarily just to showcase the new technique. That said, the study managed to produce a very impressive new find.
“
In the course of the study, whose primary purpose was to showcase the new technique’s application to neuroscience, Smith and his colleagues discovered some novel, fine distinctions within a class of synapses previously assumed to be identical.
”
Volunteers watched three films of everyday types of activity, and then were asked to recall one of them while in an fMRI machine. A computer learning algorithm was trained to recognize the fMRI data produced, and was able to discern which of the three films the subjects were recalling.
This is fairly astounding, in and of itself. Over-reaching media-speak on the subject notwithstanding, we can now, very literally, read minds. It should be cautioned though that this is sharply abstracted, and only possible at a very rudimentary level for the time being.
The study also observed that the brain-area producing the most easily describable activity patterns, when recalling episodic information, was the hippocampus. This moves us further away from the historic, simplistic, view of the hippocampus as merely a place where long-term declarative memories are maintained. Emerging (for some time now, actually) from studies like this one, is a much more nuanced understanding of a detailed interplay between both existing memory of events and activities, and the mechanisms involved in the extraction and acquisition of new or more pronounced memories from the current situation.
The improvements and innovations are coming fast and furious now. It is a great time to be into this stuff.
Traditional fMRI (it is strange to type that) detected blood flow. Hemoglobin in blood contains iron, which mediates changes in magnetic fields. These are, in turn, detected and converted to three-dimensional image data by the MRI. That same iron-containing molecule carries oxygen to the cells of the body. When neurons become more active they require more oxygen-carrying blood. Arteries in the vicinity of the active neuron cells respond by dilating in order to increase the supply. It is this extra blood that is detected by fMRI (traditional fMRI, that is).
This has been a boon to understanding the topographical correlates of thought and brain function. That's the upside. The downside is that it produces very course correlates. That is, it only measures increased blood flow in the vicinity of neural activity. It doesn't pin-point the actual activity topographically.
It is also contextually course information. In other words, it displays all activity, and can't discern between, say long-, and short-term PTP, or any of the staggering number of other protein interactions that are involved in different types of mental activity. These different types of activity are often as important as locational activity.
Finally, it has a course temporal aperture as well. Because it measures blood-flow, it tends to see the activity many milliseconds, or even seconds after the activity has started
Some of the timing and delay deficiencies have been overcome by coupling it with EEG scans as well. EEG scans have almost no positional information, just giving general areas of the brain where electrical activity is sourced, but it does give immediate feedback, which can then be narrowed by the fMRI imaging that comes in some time later.
Now, the researchers at MIT have begun to work on new ways for the fMRI to image actual protein/neurotransmitter mediated activity in the brain. Instead of simply measuring the amount of activity through increased hemoglobin in the area, these will image on the actual molecules involved in brain functions. They are accomplishing this by coming up with contrast agents (things that make an MRI image brighter or darker), which bind directly to the various molecular sites and chemicals in the brain used in brain and neuron function.
The upshot? Faster, more tightly synchronized time windows, more fine-grained spacial resolutions and magnification scales, and a whole new dimension of functionality. The functionality is based on being able to contrast specific molecular mechanisms having to do with specific types of brain activity.
The following video from the MIT press release further explains the project...
I'm not sure there's anything new here, other than demonstrating that something which has been confirmed by a variety of other means, does what you'd expect it to do in an fMRI machine. Pavlov referred to this phenomenon as the investigatory reflex.
This study shows that the reflex is represented in brain-activity as well as outward behavior. This gets us a little closer to the origin point, taking us a little farther down the path. In essence it shows us one more "observable" appearance of the investigatory reflex, which Pavlov had documented in the early nineteen-hundreds. That further validates and clarifies our understanding.
Notwithstanding the gee-whiz appeal of the fMRI, the real search—for me at least—is for the underlying mechanisms and brain characteristics that are responsible for the investigatory phenomenon. This particular study seems to have been restricted to only the appearance of the investigatory reflex in V1 (BA17).
This doesn't detract from the study's value, however. It isn't really fair for me, or anyone, to expect a custom fit to our questions in an "off-the-rack" world. This study should prove important, because it may help to show that the investigatory reflex has brain-global origins. That is, it may show that the investigatory reflex is a natural byproduct of individual neurons acting together, regardless of what brain-region they occupy. If not caused directly by local, cell-centric activities, the investigatory reflex may, at least, be facilitated by intrinsic cell behavior in some way. We are learning, for example, that the hippocampus seems to be tied into this reflexive phenomenon in a fairly substantial way.
I have some questions for the tailor: If the novel stimulus had been auditory, that probably would not have lit up V1 at all, but this is just a hunch on my part. It would have been interesting to find out—one way or the other—if V1 would have been lit up with novel auditory stimulus. This could lead to even more interesting questions about cross-modal stimuli. For example, what would happen at V1 if the novel auditory was sourced from specific locations around the organism? One interesting experiment along these lines might be to repeatedly play a given sound always from the same "location", and then suddenly (the novel part), make it emit from a different location.
Such questions simply aren't addressed in this study, which restricts itself, not only to just area V1, but also to just the effects of novel visual stimuli at V1. Also, there is no attempt made in the study to isolate the investigatory reflex at V1 from other areas that may have facilitated it. That is, the study doesn't really attempt to drill down and expose the brain structures and mechanisms actually responsible for the reflex. It is primarily about gathering more data-points, and there is nothing wrong with that.
Sources & Resources
The Press Release: "The human brain processes predictable sensory input in a particularly efficient manner"
From the more aptly named study: "Stimulus predictability reduces responses in primary visual cortex. - (abstract) - Full Text($)
A recent University of Minnesota study helps to clarify the role of the hippocampus in the formation and ongoing operations of long term memory. Specifically, with regard to the replay role that it has been know to participate in.
The hippocampus has long been known to aid in the playback of recent memories and has been thought to be a mechanism whereby recent, short-term memories could be repeated to aid in their transfer into long-term memory mechanisms within the brain.
The study demonstrated that, when faced with a novel challenge, the animals don't merely play back their most recent memories in their hippocampus activity patterns. Instead, it was found that they are most likely to play back experiences they had encountered least. Dr. Redish and his team also discovered that the animals often played back sequences that were never before experienced.
IBM has found that picking up a single carbon-oxide (CO) molecule onto the tip of their AFM (Atomic Force Microscope) allows them to get a much sharper "point" with which to obtain much sharper images of atomic scale entities. This image is simply amazing...
Just like in the textbooks. This certainly clarifies the level of abstraction used within those molecular diagrams (not much abstraction at all).
Stand Out Publishing
fMRI, that is).
