Loligo Blog

Netlab's Compatibility Mode

2012-01-19T20:17:00Z

Spent some time today doing minor edits to glossary entries. Of all the small edits, the most significant change made was to add the following section to the entry for weights.

“

. . . . . . .
Netlab's Compatibility Mode

ANN models that use floating point signed-value weights in the conventional fashion are math-centric. That is, they typically are concerned only with the signed numeric weight-value, rather than with the connection-strength represented by its absolute value. In this case, for example, increasing the weight value will make it more positive, regardless of whether it is representing an excitatory or inhibitory connection.

Netlab's default behavior is to operate directly on connection-strength representations, regardless of how they are implemented internally. Netlab neurons facilitate the conventional practice, however, by allowing it to be specified in the learning method for each weight-layer.

The table below shows how Netlab facilitates compatibility with existing practices. The table documents how the translation is carried out between the traditional math-centric convention, and Netlab's connection-strength-centric convention.

Connection-Type->
v--Operation Excitatory Inhibitory

Increase Increase
Connection Strength Decrease
Connection Strength

Decrease Decrease
Connection Strength Increase
Connection Strength

Translations performed when conventional adjustment
practice is specified for a connection.

”

One possible analogy for the conventional, value-based, adjustment practice is that of adjusting for a specific water temperature from a faucet. If the water is too cold, for example, adjusting the weight value is comparable to simultaneously increasing the hot and reducing the cold (hot being the negative inhibitory weights, and cold being the positive excitatory weights in this analogy). Conversely, if the water is too hot, it is adjusted by simultaneously decreasing the hot, and increasing the cold.

Netlab fully supports the practice of working directly with the numeric value of a signed weight, but it also supports its own alternative strategy of adjusting connection strength representations. This strategy seems to be more representative of what has been learned about the cell, and molecular biology of neurons. The faucet analogy used above to describe the value-based adjustment is not sufficient to describe this strategy[1].

^[top]

Sources and Resources

Related glossary entries:

===========
Notes:

[1] - This is not to say the connection-strength adjustment strategy can't be related with an analogy, just that I have been too lazy, or too unfocused to come up with one that feels satisfyingly apt.

AI - Time for a New Name?

2012-01-05T16:33:00Z

Linguists have recently discovered [1] that almost all words are metaphorical at their base, and some people (e.g., me) posit that they all are. Though speculative, it is at least conceivable that even the sub-language signaling in the brain, which eventually leads to language, is also metaphorical. Consider that the bell becomes a metaphor for food in the mind of Pavlov's dog.

Language is also able to relate ambiguity about the concepts it conveys. The word “life,” for example, can mean life-biology, or life-consciousness. Up until now, it has been perfectly acceptable to use these two meanings interchangeably. There simply has never been an instance of consciousness that existed outside of a biological body — at least none that we could directly experience with our physical senses.

Things may be changing now. . .

Continue reading "AI - Time for a New Name?"

Happy New Year !

2011-12-30T19:54:00Z

For many, this is a time of year to reflect.
It is often a mixture of the sadness of letting go of times passed,
and the excitement and hopefulness of embracing times to come.

http://www.youtube.com/watch?v=W4r74IIKbh8

AULD LANG SYNE

Should auld acquaintance be forgot,
And days of auld lang syne?
Should auld acquaintance be forgot,
And days of auld lang syne.

For auld lang syne, my dear,
For auld lang syne.
We'll tak a cup o' kindness yet,
And days of auld lang syne.

. . .

Happy New Year.

===========

For more information, see.

Unto Us A Child Is Born!

2011-12-16T19:12:00Z

Merry Christmas, Happy Hanukkah, and happy new year. May your days be filled with happiness, love, and joy this Christmas season, and may your new year be a blessing to you and others.

http://www.youtube.com/watch?v=DKk9rv2hUfA

-djr

Multitemporal Synapses and Our Perception of a Present Moment

2011-11-20T03:44:00Z

Overview

“Certainly, one of the most relevant and obvious characteristics of a present moment is that it goes away, and that characteristic must be represented internally.”

Stated plainly[1], the principle behind multitemporal synapses is that we maintain the blunt “residue” of past lessons in long-term connections, while everything else is learned in the instant. In other words, we re-learn the detailed parts of our responses as we are confronted with each new current situation.[2]

. . . . . . .
Contents

^[top]

An Interesting Visualization

An earlier blog entry makes various attempts—using statically presented explanations—to have readers visualize the concept. For the most part, those attempts seem to miss the mark.

The following video, however, was produced by people who probably have never heard of multitemporal synapses. Their amazing experiment inadvertently does a much better job of relating the concept of multitemporal learning than I ever could with static presentations.

http://www.youtube.com/watch?v=wM6lGNhPujE

Long face?: What you are viewing in this video may be your immediate responses—driven by long-term connections—before your short-term connection-components have had a chance to form/learn finer “present moment” responses.

Continue reading "Multitemporal Synapses and Our Perception of a Present Moment"

Site Glossary - A Better Way to Relate Ockham's Razor?

2011-11-14T22:06:49Z

The Glossary entry for William of Ockham here at the site has a new section titled “In Other Words?”. This new section attempts to provide a nutshell explanation of William's original advice more accurately than the nutshell statement commonly used today. The advice in question is commonly referred to as Ockham's Razor. Here's the suggested new nutshell definition from the glossary entry.

"Always express things using the most general representation possible for the context in which the representation is being used."

The glossary entry goes on to clarify that this is just an attempted improvement over the current vague fashion statement, and it welcomes other suggestions.

-djr

Learning is Adaptation is Learning - Using Batesian Mimicry As an Explanatory Device

2011-11-03T20:11:00Z

The book on the Netlab project often returns to the notion that learning is merely a form of adaptation and that, conversely, adaptation is merely a form of long-term learning. This, in turn, all fits under the umbrella notion that memory is behavior.

The idea that learning is adaptation is learning is forwarded as a possibility, mainly as a better means of discussing the concepts. This (in my opinion) provides a clearer and more converged understanding of how memory works in biological organisms. This could be very wrong, of course, so it's important to describe it properly. That way it, and not a straw man, can be critiqued. This article represents one such attempt to properly describe it. . .

Batesian Mimicry

Batesian mimicry is when a non-noxious/poisonous plant or animal projects the appearance of a poisonous plant or animal, allowing it to avoid being eaten by predators.

Those predators, goes the logic, which have partaken of the poisonous organism and survived, would have become very sick, and would have learned to avoid ingesting anything that appears to be that organism in the future. This will include those organisms who are not poisonous, but merely look, or act, like the poisonous organism.

Continue reading "Learning is Adaptation is Learning - Using Batesian Mimicry As an Explanatory Device"

Goodbye, World - Dennis Ritchie Creator of C Has Died

2011-10-13T00:43:00Z

Dennis Ritchie, the creator of the C programming language, died on Saturday after battling a long illness. The C programming language, arguably, changed the world. It can be found at the heart of most modern computer applications, operating systems, and successor programming languages.

   Dennis Ritchie
   Creator of the C programming language

   9 September 1941 — 8 October 2011



There's an obituary, and a very well researched history, at the Guardian

From his book, "The C Programming Language"

main()
{
        printf("hello, world\n");
}

Neural Networks Backgrounder: Ce n'est pas une l'histoire

2011-10-03T15:24:00Z

Overview

This article provides a layman's-level discussion of neural network technology within the framework of a sketchy historical sequence. Neural networks are described while presenting an overview of just one of the many routs taken by the field in the last half-century or so.

It is not for those interested in a full history of neural networks (i.e., connectionism). It is just a quick backgrounder, which should suffice to give readers a little bit of perspective into how we got from "there" to "here." The actual history of this field is storied, and sometimes even checkered and controversial. I highly recommend to anybody who is interested, that you get a good book or two on the subject.

This entry will also serve as a place to accumulate links to resources and information on the subject of neural networks and their history at this layman's level.

. . . . . . .
Contents

Continue reading "Neural Networks Backgrounder: Ce n'est pas une l'histoire"

UC Berkeley - Scientists use brain imaging to reveal the movies in our mind

2011-09-24T19:41:21Z

Scientists at UC Berkeley have taken brain scans of subjects in an fMRI machine while they watched a movie clip. They then reconstructed the movie the subjects were watching using only the brain scan data, and a database of 18 million seconds of random video gleaned from the web.

First, they used fMRI imaging to measure brain activity in visual cortex as a person looked at several hours of movies. They then used those data to develop computational models that could predict the pattern of brain activity that would be elicited by any arbitrary movies (i.e., movies that were not in the initial set). Next, they used fMRI to measure brain activity elicited by a second set of movies that were also distinct from the first set. Finally, they used the computational models to process the elicited brain activity, and reconstruct the movies in the second set.

The amount of new understanding this could allow us to gather about mind-brain correlates and first person knowledge should be considerable. If this lives up to the hype, a lot of new research ideas should come out of it. Keeping fingers crossed here.

In the above clip - the movie that each subject viewed while in the fMRI is shown in the upper left position. Reconstructions for three subjects are shown in the three rows at bottom. All these reconstructions were obtained using only each subject's brain activity and a library of 18 million seconds of random YouTube video that did not include the movies used as stimuli. The reconstruction at far left is the Average High Posterior (AHP). The reconstruction in the second column is the Maximum a Posteriori (MAP). The other columns represent less likely reconstructions. The AHP is obtained by simply averaging over the 100 most likely movies in the reconstruction library. These reconstructions show that the process is very consistent, though the quality of the reconstructions does depend somewhat on the quality of brain activity data recorded from each subject. [source: Gallant Lab (see resources below)]

Continue reading "UC Berkeley - Scientists use brain imaging to reveal the movies in our mind"

Pub Notes - Fifth Printing - Corrections. . .

2011-09-23T18:40:00Z

It's called the fifth printing, but that is a bit of a misnomer. These days the actual physical printings are done in small runs in multiples of six. It is really more like the fifth “edit.”

In practice, corrections have been made to each new edit while its previous edit was running. After five edits, there have been hundreds of corrections. The issues addressed have included a lot of out-of-place commas, over-used hyphens, typos, and more grammar errors than I'd like to admit.

How do you know you've got the fifth (or better) printing?

Easy. Look at the bar code on the back cover. If the price code (the smaller bar-code, to the right of the ISBN number) says “90000” (no price specified), you have an older copy. If it reads “54795” (USD $47.95) you have the fifth or better edit.

Continue reading "Pub Notes - Fifth Printing - Corrections. . ."

True Random Number Generator Using Only Logic Gates?

2011-09-07T22:42:00Z

In the book, Netlab Loligo, repeated calls are made for true random number generators (TRNGs) to be included in all CPUs, or at least in those that are intended for use in neural network applications. Naturally, I was very excited to see a headline about Intel having developed one with general purpose use in mind.

Intel's Low-Power “True” Random Number Generator

IEEE has an article about a new “true” random number generator from Intel that has been 10 years in development. Its primary advantage is that, while it is a true RNG, it operates entirely in digital mode using digital devices to obtain randomness from hardware. The slow, energy hogging, analog technology normally needed to glean randomness from Quantum phenomena has been eliminated. It has a few quirks, such as the need to force the outputs of its two mutex inverters high, and the seemingly unavoidable need to compensate using averaging techniques. I expand just a little on these quirks below.

In the spirit of not critiquing something without also offering, at least, a sincere attempt at a solution, I've forwarded a quick (if dirty) attempt at an “all logic gates” DTRNG (Digital True Random Number Generator) below. Only the equations were scratched out at the IEEE blog, I've since produced a circuit diagram graphic, which is included here as well.

Continue reading "True Random Number Generator Using Only Logic Gates?"

Robot Arm Inventor Dies

2011-08-11T16:59:00Z

George Charles Devol
Inventor of Robot Arm

20 February 1912 — 11 August 2011

The self taught inventor of the robotic arm, which has become an icon of factory automation from Detroit to Asia, has died.

Story: Pioneer of the long arm of factory floor technology

Four New Species of Zombifying Ant Fungus Found

2011-05-18T02:43:52Z

From the press release: “Once infected by spores, the worker ants ... leave the nest, find a small shrub and start climbing. The fungi directs all ants to the same kind of leaf: about 25 centimeters [(9.8 inches)] above the ground and at a precise angle to the sun (though the favored angle varies between fungi). How the fungi do this is a mystery.”

[. . .article]

The paper:

Hidden Diversity Behind the Zombie-Ant Fungus Ophiocordyceps unilateralis: Four New Species Described from Carpenter Ants in Minas Gerais, Brazil

Background & Further Reading.

A few interesting links to related stuff. Includes some strange and interesting background about ants.

Youtube short video about Cordyceps fungi
Ant Colony Optimization Site - Behavior of real ants
AntWeb
AntBase (database of ants)
Ant Genera Distribution Maps
Bug Guide
Honey Ants
Ant Death Spiral
- Ant Spiral Video

Book Excerpt Pages Now Include the Preface

2011-05-10T00:52:50Z

Hi,

The Preface of the book has now been added to the Excerpts Pages that are available here at the site. -djr

Connection-Type-> v--Operation	Excitatory	Inhibitory
Increase	Increase Connection Strength Decrease Connection Strength
Decrease	Decrease Connection Strength	Increase Connection Strength