Site Journal

=================
2016-04-23: Updated glossary entry for Memristor

Added resources, including where you can purchase discrete memristor components.





=================
2016-04-22: Updated glossary entry for: haptic

Added resources




=================
2016-04-11: Updated the glossary entry for panpsychism

Expanded discussion and clarification of the various connotations, and added resources




=================
2016-04-08: Added to glossary: Plagiarism

The glossary now has fields and template codes allowing for the addition of a description, and a list of keywords for each entry.

Of course, it is still necessary to go back and add them to the terms that are there.

Today I've added descriptions and keyword lists for about 80 entries. This is pretty close to my limit for one sitting.

When combined with earlier work, I'm happy to report that four letters have been filled so far. This leaves 23 letters still to do. . .

Aha! Did you notice that this adds up to 27 letters? Good eye! The glossary has a separate "letter'" for terms that start with numbers or symbols.

Some recent glossary updates have included:

• Connection Strength [Refreshed]
  Banging away some more on the "basics" drum. Attempting to address some common confusion surrounding weight-values vs. the connection-strengths they represent.

• Catastrophic Forgetting [New]
  A definition of a fairly common term from neural network literature, which labels one of the major problems that have been encountered. The entry also includes a discussion of how the problem is fully resolved by Multitemporal Synapses.

• Interference [New]
  Definition and some resources regarding current understanding and ideas on interference in neuronal memory acquisition.

• The Stability Plasticity Problem [New]
  Definition of another term from neural network literature, which is used as a more general label for the problem that is responsible for catastrophic forgetting. This entry also explains how the problem it labels is resolved by Multitemporal Synapses.

• Memory [Refreshed]
  Added some pics to the section on non-neuronal biological learning, along with a terse discussion of things like herding, schooling, and flocking behaviors. These are used to demonstrate how learning can occur in biological systems via an extra-neuronal mechanism.

• Multitemporal Synapses [Refreshed]
  Better explanations and editing plus a diagram (5.1) from the book (originally from the patent application).

• Kinetic Depth Effect [Refreshed]
  Added content, plus reorganized.

• Multilayer Perceptron (MLP)[New]
  Quick add to support other entries.

The site was switched to a new hosting service at the end of February. The blog and glossary were the pieces I was most anxious about, but they seem to have handled the move just fine.

So far, this host seems to be providing much faster responses. It should also provide better up-time.

Responses have gone from often taking 40-70 seconds, down to less than ten seconds. In fact, I haven't counted a single response greater than 12 yet.

The previous provider would regularly (about once a month) make changes that completely hid most, or all, of the site's content from the search-engines and in-links. Those down-times would typically last from two to six days. Many down-times, including the last one, only ended when I wrote some defensive code to work around their new server-settings.

Hoping this provider will do better in that department as well.

So far, I'm happy with it.

Spent some time today doing minor edits to glossary entries. Of all the small edits, the most significant change made was to add the following section to the entry for weights.


“

 

. . . . . . .
Netlab's Compatibility Mode

ANN models that use floating point signed-value weights in the conventional fashion are math-centric. That is, they typically are concerned only with the signed numeric weight-value, rather than with the connection-strength represented by its absolute value. In this case, for example, increasing the weight value will make it more positive, regardless of whether it is representing an excitatory or inhibitory connection.

Netlab's default behavior is to operate directly on connection-strength representations, regardless of how they are implemented internally. Netlab neurons facilitate the conventional practice, however, by allowing it to be specified in the learning method for each weight-layer.

The table below shows how Netlab facilitates compatibility with existing practices. The table documents how the translation is carried out between the traditional math-centric convention, and Netlab's connection-strength-centric convention.

Connection-Type-> v--Operation	Excitatory	Inhibitory
Increase	Increase Connection Strength	Decrease Connection Strength
Decrease	Decrease Connection Strength	Increase Connection Strength

Translations performed when conventional adjustment
practice is specified for a connection.

 

”


One possible analogy for the conventional, value-based, adjustment practice is that of adjusting for a specific water temperature from a faucet. If the water is too cold, for example, adjusting the weight value is comparable to simultaneously increasing the hot and reducing the cold (hot being the negative inhibitory weights, and cold being the positive excitatory weights in this analogy). Conversely, if the water is too hot, it is adjusted by simultaneously decreasing the hot, and increasing the cold.

In this way, Netlab is able to fully support the practice of working directly with the numeric value of a signed weight, but it also supports its own alternative strategy of adjusting connection strength representations. This strategy seems to be more representative of what has been learned about the cell, and molecular biology of neurons. The faucet analogy used above to describe the value-based adjustment is not sufficient to describe this strategy[1].

• Related glossary entries:
  
  • Weights
  • Connection Strengths
  • Synapses
  • ANN (Neural Network)
  • Learning
  • Memory
  • [Book] The Neuron: Cell and Molecular Biology

===========
Notes:

[1] - This is not to say the connection-strength adjustment strategy can't be related with an analogy, just that I have been too lazy, or too unfocused to come up with one that feels satisfyingly apt.

The Glossary entry for William of Ockham here at the site has a new section titled “In Other Words?”. This new section attempts to provide a nutshell explanation of William's original advice more accurately than the nutshell statement commonly used today. The advice in question is commonly referred to as Ockham's Razor. Here's the suggested new nutshell definition from the glossary entry.


"Always express things using the most general representation possible for the context in which the representation is being used."


The glossary entry goes on to clarify that this is just an attempted improvement over the current vague fashion statement, and it welcomes other suggestions.


 -djr

Hi, The Preface of the book has now been added to the Excerpts Pages that are available here at the site. -djr