Features of the implementation of streaming recurrent neural networks on graphical processing units
The relevance of the problem of optimizing data stream processing with recurrent neural networks (RNNs) is driven by the growing volumes of multivariate time series in complex dynamic systems, where traditional implementations fail to provide the required real-time prediction speed and the ability for continuous learning without system interruption. The aim of this work is to expand the capabilities of processing multivariate time series streams with recurrent neural networks (RNNs) with controlled elements through their efficient implementation on graphics processing units (GPUs). The paper proposes an algorithm for such an implementation, taking into account the specific features of these networks. As the research material, a constructed dataset containing 1,000 elements of multivariate time series was used, on which models of all considered architectures were trained; the test procedure adopted is one prediction cycle, including training on the entire sample and computing a forecast with a horizon of 72 time steps. Experiments comparing the data processing cycle time in RNNs for CPU- and GPU-based implementations were conducted. For each experiment, a neural network with identical parameters was implemented on the aforementioned architectures. A series of experiments with RNNs of different sizes was carried out to evaluate the scalability of the proposed GPU architecture. The experimental results show that for small-scale RNNs (approximately 650 neurons in each layer), the performance gain of the proposed GPU architecture is about 10 times compared to the CPU implementation, whereas, with increasing network size, the acceleration grows nonlinearly and reaches 90 times for large-scale configurations. Thus, implementing such RNNs on a GPU platform significantly expands the applicability of these networks for solving time series forecasting problems with continuous learning.
Authors: V. M. Taits
Direction: Informatics, Computer Technologies And Control
Keywords: recurrent neural network, graphics processors, implementation algorithms, performance
View full article