ARTIFICIAL NEURAL NETWORKS - IASRI

1 ARTIFICIAL NEURAL NETWORKS GIRISH KUMAR JHA Indian Agricultural Research Institute PUSA, New Delhi-110 012 1. Introduction ARTIFICIAL NEURAL NETWORKS (ANNs) are non-linear mapping structures based on the function of the human brain. They are powerful tools for modelling, especially when the underlying data relationship is unknown. ANNs can identify and learn correlated patterns between input data sets and corresponding target values. After training, ANNs can be used to predict the outcome of new independent input data. ANNs imitate the learning process of the human brain and can process problems involving non-linear and complex data even if the data are imprecise and noisy. Thus they are ideally suited for the modeling of agricultural data which are known to be complex and often non-linear.

2 ANNs has great capacity in predictive modeling , all the characters describing the unknown situation can be presented to the trained ANNs, and then prediction of agricultural systems is guaranteed. An ANN is a computational structure that is inspired by observed process in natural NETWORKS of biological neurons in the brain. It consists of simple computational units called neurons, which are highly interconnected. ANNs have become the focus of much attention, largely because of their wide range of applicability and the ease with which they can treat complicated problems. ANNs are parallel computational models comprised of densely interconnected adaptive processing units. These NETWORKS are fine-grained parallel implementations of nonlinear static or dynamic systems.

3 A very important feature of these NETWORKS is their adaptive nature, where learning by example replaces programming in solving problems. This feature makes such computational models very appealing in application domains where one has little or incomplete understanding of the problem to be solved but where training data is readily available. ANNs are now being increasingly recognized in the area of classification and prediction, where regression model and other related statistical techniques have traditionally been employed. The most widely used learning algorithm in an ANN is the Backpropagation algorithm. There are various types of ANNs like Multilayered Perceptron, Radial Basis Function and Kohonen NETWORKS .

4 These NETWORKS are NEURAL in the sense that they may have been inspired by neuroscience but not necessarily because they are faithful models of biological NEURAL or cognitive phenomena. In fact majority of the network are more closely related to traditional mathematical and/or statistical models such as non-parametric pattern classifiers, clustering algorithms, nonlinear filters, and statistical regression models than they are to neurobiology models. ANNs have been used for a wide variety of applications where statistical methods are traditionally employed. The problems which were normally solved through classical statistical methods, such as discriminant analysis, logistic regression, Bayes analysis, multiple regression, and ARIMA time-series models are being tackled by ANNs.

5 It is, therefore, time to recognize ANN as a powerful tool for data analysis. ARTIFICIAL NEURAL NETWORKS 2 2. Review Mc-Culloch Pitts (1943) proposed a model of computing elements called Mc-Culloch Pitts neurons, which performs weighted sum of the inputs to these elements followed by a threshold logic operation. Combinations of these computing elements were used to realize several logical computations. The main drawback of this model of computation is that the weights are fixed and hence the models could not learn from examples which is the main characteristic of the ANN technique which later evolved. Hebb (1949), proposed a learning scheme for adjusting a connection weight based on pre and post synaptic values of the variables.

6 Hebb s law became a fundamental learning rule in neuron network literature. Rosenblatt (1958), proposed the perceptron models, which has weight adjustable by the perceptron learning law. Widrows and Hoff (1960) and his group proposed an ADALINE (Adaptive Linear Element) model for computing elements and LMS (Least Mean Square) learning algorithm to adjust the weights of an ADALINE model. Hopfield (1982), gave energy analysis of feed back NEURAL NETWORKS . The analysis has shown the existence of stable equilibrium states in a feed back network , provided the network has symmetrical weights. Rumelhart et al. (1986), showed that it is possible to adjust the weight of a multilayer feed forward NEURAL network in a systematic way to learn the implicit mapping in a set of input output patterns pairs.

7 The learning law is called generalized delta rule or error back propagation. An excellent overview of various aspects of ANN is provided by Cheng and Titterington (1994) and Warner and Misra (1996). Kaastra and Boyd (1996) developed NEURAL network model for forecasting financial and economic time series. Dewolf et al. (1997, 2000) demonstrated the applicability of NEURAL network technology for plant diseases forecasting Zhang et al. (1998) provided the general summary of the work in ANN forecasting, providing the guidelines for NEURAL network modeling, general paradigm of the ANNs especially those used for forecasting, modeling issue of ANNs in forecasting and relative performance of ANN over traditional statistical methods.

8 Sanzogni et al. (2001) developed the models for predicting milk production from farm inputs using standard feed forward ANN. Gaudart et al. (2004) compared the performance of MLP and that of linear regression for epidemiological data with regard to quality of prediction and robustness to deviation from underlying assumptions of normality, homoscedasticity and independence of errors. More general books on NEURAL NETWORKS and related topics contain separate chapters/sections on NEURAL NETWORKS , to cite a few, HHaassssoouunn ((11999955)),, PPaatttteerrssoonn ((11999966)),, Schalkoff (1997), Yegnanarayana (1999), AAnnddeerrssoonn ((22000033)),, Software on NEURAL NETWORKS has also been made and are as follows: Commercial Software:- Statistica NEURAL network , TNs2 Server,DataEngine, Know Man Basic Suite, Partek, Saxon, ECANSE - Environment for Computer Aided NEURAL Software Engineering, Neuroshell, Neurogen, Matlab: NEURAL network Toolbar, Tarjan, FCM(Fuzzy Control manager).

9 Freeware Software:- NetII, Spider Nets NEURAL network Library, NeuDC, Binary Hopfeild Net with free Java source, NEURAL shell, PlaNet, Valentino Computational Neuroscience Work bench, NEURAL Simulation language version-NSL, Brain NEURAL network Simulator. In India, few studies have been conducted using ANN models. Kumar et. al. (2002) studied utility of ARTIFICIAL NEURAL NETWORKS (ANNs) for estimation of daily grass reference ARTIFICIAL NEURAL NETWORKS 3 crop evapotranspiration and compared the performance of ANNs with the conventional method used to estimate evapotranspiration. Pal et al. (2002) developed MLP based forecasting model for maximum and minimum temperatures for ground level at Dum Dum station, Kolkata on the basis of daily data on several variables, such as mean sea level pressure, vapour pressure, relative humidity, rainfall, and radiation for the period 1989-95.

10 Madhav (2003) forecasted wheat productivity of Junagadh (Gujarat) using data upon weather parameters based on ANN model. 3. Development of an ANN Model Development of ANN model is discussed here briefly. ANNs are constructed with layers of units, and thus are termed multilayer ANNs. A layer of units in such an ANN is composed of units that perform similar tasks. First layer of a multilayer ANN consists of input units. These units are known as independent variables in statistical literature. Last layer contains output units. In statistical nomenclature, these units are known as dependent or response variables. All other units in the model are called hidden units and constitute hidden layers. There are two functions governing the behaviour of a unit in a particular layer, which normally are the same for all units within the whole ANN, the input function, and the output/activation function.