Book review: ANNs applied in vivo to industrial and environmental systems
By Shalin Hai-Jew, Kansas State University
A Comprehensive Guide to Neural Network Modeling
Steffen Skaar
Nova Science Publishers, Inc.
2020
160 pp.
Steffen Skaar’s edited collection A Comprehensive Guide to Neural Network Modeling (2020) presents four chapters around various practical applications of artificial neural networks to represent various aspects of the world for awareness, decision-making, research, and other applications. Modeling is representational and strives to offer insights with fidelity to the world, to inform people’s consciousness and actions.
Artificial Neural Networks, Generally
In a simple sense, artificial neural networks (ANNs) are about inputs and outputs. Particular variables thought to inform a particular outcome in a process or context are input into this artificial intelligence system (which can be set up in various different ways), and various informational outputs may be garnered. ANNs themselves are comprised of various nonlinear regression and discriminative statistical methods in order to “learn” from available data. This approach was conceptualized in 1943 by Warren McCulloch and Walter Pitts (https://en.wikipedia.org/wiki/Artificial_neural_network), as a biomimetic emulation of biological neurons linked by synaptic connections. Where the human brain has 1011 number of neurons and 104 connections per neuron (Tušek, Valinger, Benković, Kljusurić, & Jurina, 2020, p. 58), albeit with neurons dying off over time without new ones being created, ANNs are smaller in terms of neurons and are targeted to particular issues in particular spaces, based on available and relevant data.
This computational analytics method is used for “nonlinear and non-parametric modeling” and works stably with sufficient data even if there is noise in it (Skaar, 2020, p. viii). ANN algorithms are also used for “recognition, detection, classification as well as for the search of patterns, predictions of on-line parameters, image processing and optimization” (Skaar, 2020, p. ix).
The basic visual representation is of incoming information from the external environment, layers of internal or hidden neurons, and output neurons. The I/O dynamic is shown left-to-right.
The “hidden” layers are so-referred to because the various specific mathematical functions may not be specifically defined in every case, and the 321sprocesses in these layers are sometimes referred to as “grey-box” (vs. total “black box”) processes, with some parts partially known. [One authoring team in this collection explains: ANNs are sometimes considered black boxes since “there is no possibility of obtaining the exact mathematical model which correlates the input and output variables in a clear way” (Kovačević, Banjac, Podunavac-Kuzmanović, & Jevrić, 2020, p. 92).] The ANN regression model can generalize relationships between dependent and independent variables based on the available data. It systematizes the analysis of input variables to target outcomes in supervised learning (with labeled training data), unsupervised learning (with unlabeled training data), and reinforcement learning. Data are often normalized before input into the ANN. The data may be non-parametric, and the relationships identified are nonlinear.
Figure 1: A General Feed-Forward Artificial Neural Network (ANN)
The four chapters show the application of ANNs in food drying, food engineering, chemometrics for novel chemicals, and river water quality. These works combine both the practical and the theoretical, and they do demonstrate well some of the affordances and constraints of ANNs.
ANN in Drying Food Products
Rachel Guiné, Iman Golpour, Maria João Barroca, and Mohammad Kaveh’s “Application of Artificial Neural Networks (ANNS) Modelling in Drying Technology of Food Products: A Comprehensive Survey” (Ch. 1) portray food drying in the present day as involving complex physical, chemical, and biological processes. On the physics alone: Drying is an “extremely complex thermal process during which unsteady heat and moisture transfer happen simultaneously” (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 8).
In various contexts, based on the fruits, vegetables, meats, cereals, seeds, and other contents, there are particular processes to preserve the nutrients, flavor, color, texture, and other aspects, while saving on energy and engaging at industrial scale. Think processed foods. Think convenience foods. Think packages and powders. Those who engage in food drying need to understand how this process changes the food product’s “structural properties (e.g., shrinkage, porosity, volume, density, pore size distribution and surface area) and also impacts on its physicochemical properties (e.g., texture, colour, nutritive value and appearance) of most food products” (Guiné, Golpour, Barroca, & Kaveh, 2020, pp. 3 - 4).
In ancient times, people used sunlight and air to dry foods. Now, such work is achieved by “convective stove, tray drying, spray drying, drum drying, lyophilization, osmotic dehydration, extrusion, fluidization, and the use of microwaves and radio frequency” individually or in combination (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 4). Some machines require direct contact for heat transfer and others indirect contact. The (art and) science of drying food products also depends in part on the sequencing of various processes and fine-grained adjustments.
Drying modes vary based on “modes of operation, pressure, heat input type, number of stages or state of the material in the dryer” (Guiné, Golpour, Barroca, & Kaveh, 2020, pp. 3 - 4). Much of industrial food drying is modeled mathematically; however ANNs have also become more important in this space, which requires minute and detailed awareness. The co-authors write about ANNs, citing the perceptron model as the basic one:
ANNs are a computational model consisting of many nonlinear and parameterized analog signal-processing units (neurons) connected by links (synapses) of variable numerical weights between the neurons. The neuron network and its synapses establish the model that could be executed artificially via software programs and specific hardware (Guiné, 2019, as cited in Guiné, Golpour, Barroca, & Kaveh, 2020, p. 10)
The neurons are conceptualized as interacting with each other in concert. In this process, there are differing inputs to the respective artificial neurons. The inputs are weighted and processed in the respective neurons in a number of layers. The findings are summed up and “fed into the transfer function for generating an important result as the output” (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 10). Some ANNs have feedback functions that change how the respective neurons and layers reprocess data, towards more accuracy. Determining the activation thresholds of computational neurons for their output is “analogous to synapses, soma and axon” (as cited in Aghbashlo et al., 2018, as cited in Guiné, Golpour, Barroca, & Kaveh, 2020, p. 12).
A neutral network is trained by using data exemplars (known as the “training group”).
The network can be trained by supplying it with a group of instances (the training group) of true neutral network manner: {p1, t1}, {p2, t2}…{pn, tn} where pi is an input parameter to the network, ti is the parameter of the respective target output, and n is the specific number of the training pairs.Actually, in this method, including a teacher, a lot of pairs of input/output training patterns can supply a learning data set whereas after using the inputs, the network outputs can be compared by using the targets and then the error signal is utilized to adapt the weights so that the weights and biases of the network are modified by this training algorithm for reducing the error between the outputs and the targets of the networks. (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 14)
Where the data are labeled, that is generally supervised machine learning. Where the data are not labeled, that is generally unsupervised machine learning and requires the ANN to find self-organization patterns. [Here, there are only inputs fed into the ANN, and then the ANN has to find its own clustering patterns and applying classifications.] The neurons operate in parallel.
ANN performance is assessed based on how well it handles data in three types. The general data proportions are 70% of randomly selected empirical data for training data, 15% for validation data, and 15% for test data. The modeling performance is based on various variables: goodness of fit to the data, the determination (or regression) coefficient (R2), the mean square error, the mean absolute error, the F-test, the variation coefficient, the p-value, and others. The optimal one is the one with the lowest error of prediction (and optimally efficiency). There are data processing approaches that strive to arrive at a true measure of model accuracy, including cross-validation.
There are different types of ANNs. Feed-forward Neural Networks (FNNs) include Single Layer Perceptron (SLP), Multi-Layer Perceptron (MLP), Radial Basis Function Nets,” and others (Guiné, Golpour, Barroca, & Kaveh, 2020, pp. 14 - 15). These ANNs are without loops or cycles (so they are acyclic), which means the signals only feed forward through the layers to an outcome. Recurrent Feedback-Back Neural Networks (RFNNs) include the following types: “fully recurrent (Hopfield network and Boltzmann machine), simple recurrent, echo state, long short-term memory, bidirectional, hierarchical, and stochastic neural networks” (Yi & Tan, 2004, as cited in Guiné, Golpour, Barroca, & Kaveh, 2020, p. 18). Recurrent neural networks enable “feedback connections among neurons so that each neuron is connected to others” and even to itself (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 17). ANNs may include various features like fuzzy logic and so-called grey box technologies.
The co-authors explain ANN use in the food drying space: “Genetic algorithm, Fuzzy Logic and reinforcement Learning can help to forecast, model and control the food drying process beyond the range of the used data…” (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 21) These approaches amplify the signals in the available data and perhaps enable inferences beyond the known data. In various contexts, then, if particular inputs to a process are seen to affect outcomes positively or negatively, then adjustments may be made.
This chapter offers a powerful summary of some of the prior research on ANN applications to the drying of fruit (granulated grapes, terebinth, cantaloupes, strawberries, white mulberry, kiwi, quince, oranges), vegetables (peppers, mushrooms, squash, turnips, zucchini, onions, tomatoes), nuts (hazelnuts, pistachio, almonds), legumes (green chickpeas, beans), leaves (tea), and other consumables. The works describe different types of dryers and sequences: “convective, microwave vacuum, infrared convection with vacuum pretreatment, ultrasound pre-treatment with Clevenger, fluidized-bed drier, microwave-hot air” in the reviewed research (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 31). Depending on the context, there are some known optimized models.
There is also a sequential schematic of the pathway to set up deep neural network models in this space:
Development of a physics based model for drying, experimental validation of model, parametric sensitivity analysis for input parameters, identifying sensitive, i.e. characteristic input parameters, generation of various combinations of input parameters, simulation on physics based model for various food samples, experimental validation of model, extraction of temperature and moisture variation for various food samples, neural network platform, (and) trained neural network setup. (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 40)
ANN has also been harnessed to program factory machine controllers to modify settings on industrial scale dryers. Full industrial applications have not yet been fully realized. The researchers here propose the building out of the capability of ANN algorithms in this applied space. They write:
In the future, it is envisaged that great possibilities in development of hardware and software will be created by the ANN approach to discover real-time applications, but there are some hardware restrictions as the main problems in this type of the applications by ANNs that can be solved in the future.The expanded ANN models for various types of food dryers have to be incorporated into user-sociable software. This causes the designers of dryers and the managers of corresponding plants who are unfamiliar with AI techniques to easily use this software entering requested conditions via the customers and also receiving a rapid and acceptable controlling system along with a reliable confidence. (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 41)
Here, the data may come from a variety of sensors to inform real-time controls. The input data may come from the following: “electronic tongue and nose, bioelectronics tongue, imaging, spectral, acoustical, spectroscopic procedure, and electrical techniques for pattern tracing and optimization and control of processes” (Guiné, Golpour, Barroca, & Kaveh, 2020, p. 42).
In Food Engineering
Ana Jurinjak Tušek, Davor Valinger, Maja Benković, Jasenka Gajdoš Kljusurić, and Tamara Jurina’s “Application of Artificial Neural Networks in the Food Engineering” (Ch. 2) focuses on some of the studies in this space. One summarized work focuses on medicinal plants and “total dissolved solids (TDS), extraction yield (EY), total polyphenolic content (PC) and antioxidant activity (AA)” of the aqueous extracts of the plants (Jurina et al, 2018, as cited in Tušek, Valinger, Benković, Kljusurić, & Jurina, 2020, p. 69).
ANNs in Chemometrics
ANNs are seen to add value to various human endeavors. Strahinja Kovačević, Milica Karadžić Banjac, Sanja Podunavac-Kuzmanović, and Lidija Jevrić’s “Artificial Neural Networks as a Chemometric Tool in Analysis of Biologically Active Compounds” (Ch. 3) focuses on the application of ANNs to chemometric methods to anticipate / predict the features of new biologically active compounds. Biologically active compounds (including “steroids and steroidal derivatives, bezimidazoles, benzoxazoles, s-triazines, terpenes, alcaloids and many other” may be tested for various biological effects expressed as “anticancer, antiinflamatory, antioxidant, antifungal, antibacterial, antiviral” and other activities (Kovačević, Banjac, Podunavac-Kuzmanović, & Jevrić, 2020, p. 84). Some synthesized steroidal derivatives that have “antiproliferative activity towards various cancer cell lines” (p. 84). The designs of such biologically active compounds stand to benefit from predictions of “their physicochemical and biological properties” (p. 84). Such initial information from ANNs may be suggestive of which synthetic compounds to pursue. The co-researchers explain:
An ANN regression model does not provide a specific mathematical function, but it (can) generalize the relationship between one or more dependent and independent variables. Since the biological response of a compound is a consequence of many factors (chemical nature, molecular structure, reactivity, lipophilicity, bulkiness, shape, etc.), multivariate modeling is inevitable in its prediction, particularly for in vivo and in vitro correlations. (Kovačević, Banjac, Podunavac-Kuzmanović, & Jevrić, 2020, p. 85)
In terms of data processing, the authors point to a need to data scale or normalize often using a minimum-maximum and Z-score method (based on mean and standard deviation), often with outlier data points removed so as not to skew the numbers (Kovačević, Banjac, Podunavac-Kuzmanović, & Jevrić, 2020, p. 93). They explain the criticality of selecting the likely relevant variables that should be “mathematically related with an output variable, but also must be able to explain the predicted phenomenon” (p. 94). In a summary table, in this space, there are particular ANNs and techniques applied to particular compound types, based on a review of the literature, including QSRR and QSAR modeling, and a more recent method integrating Bayesian analysis as the Bayesian Regularization feed-forward artificial neural network (LASSO-BR-ANN) (pp. 96-110).
ANNs for Monitoring River Water Quality
Eda Göz, Erdal Karadurmuş, and Mehmet Yüceer’s “River Water Quality Modelling Using Artificial Intelligence Techniques” (Ch. 4) focuses on probe- and sensor-based river water quality data collected at a measurement station on the Yeşilirmak River in Amasya, Turkey, to benefit river basin management planning. The data include “luminescent dissolved oxygen (LDO), pH, conductivity, nitrate nitrogen (NO3-N), ammonium nitrogen (NH4-N), total organic carbon (TOC), chloride, orthophosphate, temperature, turbidity, suspended solid and flow rate” (p. 124) in brief five-minute intervals. From the known information, they use ANN to extrapolate “parameters that are expensive to measure” from those lower-cost data ones (p. 124). The new information may better inform decision making. Classical models have been poorer at emulating natural systems, which “tend to be complex and nonlinear for deterministic modeling methods. AI techniques provide a fast and flexible means of creating models for estimation of river water quality” (Göz, Karadurmuş, & Yüceer, 2020, p. 123).
In their review of the literature, ANNs are used for “classification, pattern recognition, image indexing and retrieval, forecasting problems, robustness and fault tolerance problems” (Göz, Karadurmuş, & Yüceer, 2020, p. 129). In their detailed summaries of research, they point at different ANN models optimized for different types of prediction and work. The data and the tuned ANN program are considered “an ‘expert’ in the category of information to be analyzed, and it can be used for answering ‘what if’ questions” (Zakaria et al. 2014, as cited in Göz, Karadurmuş, & Yüceer, 2020, p. 129).
This chapter points at different types of algorithms: Support Vector Machine (SVM), Least Squares Support Vector Machine (LS-SVM), An Extreme Learning Machine (ELM), A Kernel Extreme Learning Machine (KELM), and others. They show how various equations are used in ANNs.
This is an engaging work, with evocative photos of the various stations and diagrams showing the setup for the study at Ankara University. They explain the following:
The water sample was taken from the river into the tank inside the station…Conductivity and temperature values were measured in the tank with a probe. After this, the samples were sent to other analyzers where the pollution parameters were measured. All data were collected in a database inside station and transferred to the monitoring Center in Ankara via a General Packet Radio Service (GPRS) transmission channel. The data were continuously monitored and displayed in different formats at this Central Office. Real-time data were measured at five-minute time intervals for the parameters of luminescent dissolved oxygen (LDO), pH, conductivity, nitrate nitrogen (NO3-N), ammonium nitrogen (NH4-N), TOC, o-phosphate, chloride, temperature, turbidity, suspended solid and flow rate (Göz, Karadurmuş, & Yüceer, 2020, p. 142)
For their work, this team used MATLAB and Neural Network Toolbox. As is fairly typical, the various ANN algorithms performed with high accuracy, and the variance in performance was low. The team provided comparisons among the various approaches. In terms of correlation and closeness of performance, the accuracy was between 0.9048 (ELM) and 0.9751 (LS-SVM) for TOC modeling (Göz, Karadurmuş, & Yüceer, 2020, p. 146).
Conclusion
Steffen Skaar’s A Comprehensive Guide to Neural Network Modeling (2020) offers some fresh ideas to the general public about how artificial neural networks (ANNs) can and are being applied in a variety of contexts. If there is one quibble, the title smacks of over-reach in claiming comprehensiveness, given the four chapters from several disciplines. Well beyond, ANNs have been applied in various practical fields, such as finance, medicine, geology, physics, and other areas (Tušek, Valinger, Benković, Kljusurić, & Jurina, 2020, p. 58). This data analytics approach seems very cyborg-ian, combining human insights and machine capabilities.
About the Author
Shalin Hai-Jew works as an instructional designer / researcher at Kansas State University. Her email is shalin@ksu.edu.
| Previous page on path | Cover, page 14 of 21 | Next page on path |
Discussion of "Book review: ANNs applied in vivo to industrial and environmental systems"
Add your voice to this discussion.
Checking your signed in status ...