Franchisehunter368

CPSC 352 -- Artificial Intelligence
Notes: Machine Learning: Neural Networks

This indicator is common when the main router is not communicating back to the netTALK DUO with an IP Address. Check the Ethernet cable to see if it is properly connected between the netTALK DUO and the router. At certain times the main Ethernet cable may go bad. You may consider changing out the Ethernet at any point during troubleshooting. Your VoIP Settings allow you to use your netTALK Residential VoIP Phone Service with various types of SIP phones, software programs, or adapters. Below is a list of recommended devices. Below is a list of recommended devices. Specialties: Our flagship product the TK6000 connects you to the largest digital phone network in the world, the Internet, where you can make a lifetime of unlimited phone calls within the USA and Canada. Enjoy free TK6000 to TK6000 calling. DUO auto syncs with our network: The first time the DUO is connected it may take a minute to sync/register. After the DUO registers, it will ring your phone once. You should now hear a dial tone and be able to place and receive calls on the DUO. If you experience any issues during this process, please submit a trouble ticket at faq.nettalk.com. DUO auto syncs with our network: The first time the DUO is connected it may take a minute to sync/register. After the DUO registers, it will ring your phone once. You should now hear a dial tone and be able to place and receive calls on the DUO. If you experience any issues during this process, please submit a trouble ticket at faq.nettalk.com.

Introduction

In this lecture we consider the basics of machine learning in neural networks.

An Artificial Neuron

Connectionist Learning

Hebbian Learning (1949):

Repeated stimulation between two or more neurons strengthens the connection weights among those neurons. One problemwith this model is it had no way to model inhibition between neurons.

Perceptron Learning (1958):

A perceptron is a single-layer network that calculates alinear combination of its inputs and outputs a 1 if the result is greaterthan some threshold and a -1 if it is not:

Supervised Perceptron Learning

• c : learning-rate parameter
• d: desired output value
• signal() is the perceptron's actual output value, which is always +1 or -1
• cases
  1. d - signal = 0 > do nothing
  2. d - signal = +2 > increment w_i by 2cx_i
  3. d - signal = -2> decrement w_i by 2cx_i

By repeatedly adjusting weights in this fashion for an entire set oftraining data, the perceptron will minimize the average errorover the entire set.

Minsky and Papert (1969) showed that if there is a set of weights that give the correct output for an entire training set, a perceptron will learn it.

Example: Perceptrons can learn models for the followingprimitive boolean functions: AND, OR, NOT, NAND, NOR. Here's anexample for AND:

Limitations of Perceptrons

Minsky and Papert (1969) showed that perceptrons could not model the exclusive-orfunction, because its outputs are not linearly separable. Two classes ofoutputs are linearly separable if and only if you can draw a straight line intwo dimensions that separates one classification from another.

The Delta Rule (Rumelhart, 1986)

The perceptron activation function is a hard-limiting threshold function.A more general neural network uses a continuous activation function. One popularfunction is the sigmoidal (s-shaped) function, such as the logistic function:

f(net) = 1/(1 + e^-L*net)
where L is lambda, aparameter for 'squashing' the function and net is the output orsum of the weights.

The delta rule is a learning rule for a network with acontinuous (and therefore differentiable) activation function. It attemptsto minimize the cumulative error over a data set as a function ofthe weights in the network:

Delta(w_ji) = c(d_i - O_i)f'(net_i)x_j
where c is the learning rate, d_i and O_i are thedesired and actual outputs for the ith node, and f'(net) is the derivative of the activationfunction for the ith node, and x_j is the jth input to the ith node.

Key Point: The delta rule is tries to minimize the slope of the cumulativeerror in a particular region of the network's output function. This makes is susceptibleto local minima.

Back propagation Learning for Multilayer Networks

Back propagation starts at the output layer and propagates the error backwardsthrough the network. The learning rule is often called the generalized delta rule.

Back propagation activation function is the logistic function:

f(net) = 1/(1 + e^-L*net)

The logistic function is useful for assigning error to the hidden layers in a multi-layernetwork because:

• It is continuous and has a derivative everywhere.
• It is sigmoidal.
• The derivative is greatest where the function is steepest. This assigns the most error to nodes whose activation is least certain.

The formulas for computing the adjustments of the kth weight of the ith node:

Delta(w_ik) = -c(d_i - O_i) * O_i(1 - O_i)x_ikfor nodes on the output layerDelta(w_ik) = -c * O_i(1 - O_i)Sum(-delta_j * w_ij)x_ikfor nodes on the hidden layers.

NETtalk System (Sejnowski and Rosenberg, 1987)

Nettalk is a neural network, developed in 1987, that learns to pronounceEnglish text. It learns to associate phonemes with string of text.

Properties of NETtalk

• Learned to pronounce English text.
• Inputs: String of text, e.g. 'I say hello to you' (7 letter window)
• Input Unit: 29 units, one for each letter and 3 for punctuation and spaces
• Outputs: Phonemes (26 different ones)
• Hidden Elements: 80 (These units learn the pronounciation rules)
• Connections: 18,629
• Learning rule: back propagation
• Interesting Properties
  • Performance improves with training but at a slower rate.
  • Graceful degradation
  • Relearning was highly efficient

NETtalk Comparison with ID3 (Shavlik, 1991)

• Both ID3 and NETtalk were able to pronounce 60% after 500 training examples
• ID3 required 1 pass through the training data
• NETtalk was allowed 100 passes through the 500 training data

Using Encog Java Neural Network Framework

Homework Exercise: Using the links below, download theEncog Framework intoa directory on your Linux account. Then perform the exercises.

Downloads Download and unzip each of the following Encogpackages from the Encog Download Site:

• encog-workbench-3.0.1-release.zip
• encog-examples-3.0.1-release.zip
• encog-core-3.0.1-release.zip

Exercises

1. Take a look atthe GettingStarted Documentation.
2. Command Line Exercise: Do the EncogJava XORHelloWorldexample. Try working through the ANT version. On my system, this is the Javacommand you need to run from within the .../encog-examples-3.0.1/lib:
3. GUI Exercise: Dothe WorkbenchClassification Example.

NETtalk is an artificial neural network. It is the result of research carried out in the mid-1980s by Terrence Sejnowski and Charles Rosenberg. The intent behind NETtalk was to construct simplified models that might shed light on the complexity of learning human level cognitive tasks, and their implementation as a connectionist model that could also learn to perform a comparable task.

NETtalk is a program that learns to pronounce written English text by being shown text as input and matching phonetic transcriptions for comparison.^[1]

Achievements and limitations[edit]

NETtalk was created to explore the mechanisms of learning to correctly pronounce English text. The authors note that learning to read involves a complex mechanism involving many parts of the human brain. NETtalk does not specifically model the image processing stages and letter recognition of the visual cortex. Rather, it assumes that the letters have been pre-classified and recognized, and these letter sequences comprising words are then shown to the neural network during training and during performance testing. It is NETtalk's task to learn proper associations between the correct pronunciation with a given sequence of letters based on the context in which the letters appear. In other words, NETtalk learns to use the letters around the currently pronounced phoneme that provide cues as to its intended phonemic mapping.

References[edit]

1. ^Thierry Dutoit (30 November 2001). An Introduction to Text-to-Speech Synthesis. Springer Science & Business Media. pp. 123–. ISBN978-1-4020-0369-1.

External links[edit]

Nettalk Network Wireless