Neural Network Structures - IEEE

1 3 Neural Network StructuresThis chapter describes various types of Neural Network Structures that are usefulfor RF and microwave applications. The most commonly used Neural networkconfigurations, known as multilayer perceptrons (MLP), are described first,together with the concept of basic backpropagation training, and the universalapproximation theorem. Other Structures discussed in this chapter includeradial basis function (RBF) Network , wavelet Neural Network , and self-organizingmaps (SOM). Brief reviews of arbitrary Structures for ANNs and recurrentneural networks are also IntroductionA Neural Network has at least two physical components, namely, the processingelements and the connections between them.

2 The processing elements arecalled neurons, and the connections between the neurons are known as link has a weight parameter associated with it. Each neuron receivesstimulus from the neighboring neurons connected to it, processes the informa-tion, and produces an output. Neurons that receive stimuli from outside thenetwork ( , not from neurons of the Network ) are called input whose outputs are used externally are called output neurons. Neuronsthat receive stimuli from other neurons and whose output is a stimulus forother neurons in the Neural Network are known as hidden neurons.

3 There aredifferent ways in which information can be processed by a neuron, and differentways of connecting the neurons to one another. Different Neural networkstructures can be constructed by using different processing elements and bythe specific manner in which they are Networks for RF and microwave DesignA variety of Neural Network Structures have been developed for signalprocessing, pattern recognition, control, and so on. In this chapter, we describeseveral Neural Network Structures that are commonly used for microwave model-ing and design [1, 2].

4 The Neural Network Structures covered in this chapterinclude multilayer perceptrons (MLP), radial basis function networks (RBF),wavelet Neural networks, arbitrary Structures , self-organizing maps (SOM), andrecurrent NotationLetnandmrepresent the number of input and output neurons of the neuralnetwork. Letxbe ann-vector containing the external inputs (stimuli) to theneural Network ,ybe anm-vector containing the outputs from the outputneurons, andwbe a vector containing all the weight parameters representingthe connections in the Neural Network . The functiony=y(x,w) mathemati-cally represents a Neural Network .

5 The definition ofwand the manner in whichyis computed fromxandw, determine the structure of the Neural illustrate the notation, we consider the Neural Network model of anFET shown in Figure The inputs and outputs of the FET Neural modelare given by,x=[LWaNdVGSVDS freq]T( )y=[MS11PS11MS12PS12MS21PS21MS22PS22]T( )wherefreqis frequency, andMSijandPSijrepresent the magnitude andphase of the S-parameterSij. The input vectorxcontains physical/process/Figure physics-based Network Structuresbias parameters of the FET. The original physics-based FET problem can beexpressed asy=f(x)( )The Neural Network model for the problem isy=y(x,w)( ) of the Neural Network Modeling ApproachIn the FET example above, the Neural Network will represent the FET behavioronly after learning the originalx yrelationship through a process calledtraining.

6 Samples of (x,y) data, calledtraining data, should first be generatedfrom original device physics simulators or from device measurements. Trainingis done to determine Neural Network internal weightswsuch that the neuralmodel output best matches the training data. A trained Neural Network modelcan then be used during microwave design providing answers to the task itlearned. In the FET example, the trained model can be used to provideS-parameters from device physical/geometrical and bias values during further highlight features of the Neural Network modeling approach,we contrast it with two broad types of conventional microwave modelingapproaches.

7 The first type is the detailed modeling approach such as EM-based models for passive components and physics-based models for activecomponents. The overall model, ideally, is defined by well-established theoryand no experimental data is needed for model determination. However, suchdetailed models are usually computationally expensive. The second type ofconventional modeling uses empirical or equivalent circuit-based models forpassive and active components. The models are typically developed using amixture of simplified component theory, heuristic interpretation and represen-tations, and fitting of experimental data.

8 The evaluation of these models isusually much faster than that of the detailed models. However, the empiricaland equivalent circuit models are often developed under certain assumptionsin theory, range of parameters, or type of components. The models have limitedaccuracy especially when used beyond original assumptions. The Neural networkapproach is a new type of modeling approach where the model can be developedby learning from accurate data of the original component. After training, theneural Network becomes a fast and accurate model of the original problem itlearned.

9 A summary of these aspects is given in Table in-depth description of Neural Network training, its applications inmodeling passive and active components and in circuit optimization will be64 Neural Networks for RF and microwave DesignTable Comparison of Modeling Approaches for RF/ microwave ApplicationsEmpirical andBasis forEquivalent CircuitPure Neural NetworkComparisonEM/Physics ModelsModelsModelsSpeedSlowFastFastAccur acyHighLimitedCould be close toEM/physics modelsNumber of training0A fewSufficient trainingdatadata is required, whichcould be large forhigh-dimensionalproblemsCircuit/EM theoryMaxwell.

10 OrPartially involvedNot involvedof the problemsemiconductorequationsdescribed in subsequent chapters. In the present chapter, we describe structuresof Neural networks, that is, the various ways of realizingy=y(x,w). Thestructural issues have an impact on model accuracy and cost of model Multilayer Perceptrons (MLP)Multilayer perceptrons (MLP) are the most popular type of Neural networksin use today. They belong to a general class of Structures called feedforwardneural networks, a basic type of Neural Network capable of approximatinggeneric classes of functions, including continuous and integrable functions [3].