رسائل ماجيستير
English
2014
New Algorithms for Deep Learning Machines
Lahouari Ghouti, عصام الحاج لعراجي
كلية علوم وهندسة الحاسب الآلي-جامعة الملك فهد للبترول والمعادن · السعودية
The objective of the thesis is two-fold. First, it provides an outline of the fundamental concepts revolving around artificial neural networks (ANNs) and extreme learning machines (ELMs). Second, built on these concepts, it presents three new deep learning algorithms. The first algorithm uses the least squares technique to learn spatio-temporal patterns. It addresses problems including the prediction of stock market values, wind speed and chaotic patterns. The algorithm uses information provided by previous time-steps to improve its decision function. Two variants are proposed for of this algorithm: 1) recurrent-hidden ELM and; 2) recurrent-output ELM. Computer simulations showed that, on average, the algorithm outperforms traditional ELMs on six different datasets. Called ELM-based autoencoder (ELM-AE), the second algorithm is a feature extractor based on the least squares technique too. To extract structural hidden features from the inputdata, the least squares algorithm is preferred over the backpropagation algorithm for its learning speed. Further, the second algorithm can be used to train larger neural networks consisting of two or more hidden layers. Using the optical character recognition (OCR) problem, simulation results revealed that the extracted digit strokes using the ELM-AE and sparse autoencoders are similar. However, it is worth noting that the ELM-AE algorithm has a faster learning curve. Moreover, large networks, trained by the ELM-AE algorithm, perform as well as deep belief networks trained with the backpropagation algorithm using the same number of hidden layers. Like its predecessors, the third algorithm, called the ELM-based convolutional autoencoders (ELM-CA), uses the least squares algorithm to train a convolutional layer to reconstruct the input images. Such design allows constructing large convolutional neural networks (CNNs) that train very quickly as opposed to those based on backpropagation. In fact, the latter networks can take weeks to train. Reported results showed that CNNs trained using the ELM-based CA algorithm achieve competitive results that greatly outperform support vector machines (SVM) and standard ELM algorithms. The contributions, made in this thesis, aim to start a new trend of deep learning algorithms that are quick to train and efficient as well.