Correcting Predictions from Oceanic Maritime Numerical Models via Residual Learning Tingwei Wang1 , Song Gao2 , Jiangling Xu2 , Yaru Li2 , Peng Li1 , and Peng Ren1 1. College of Information and Control Engineering, 2. North China Sea Marine Forecasting Center of China University of Petroleum (East China). State Oceanic Administration, China.
[email protected] [email protected] Abstract—Accurately predicting significant wave height is essential for reducing the damages caused by storm surge. There are two main widely used predicting methods, i.e. predicting by numerical models, and correcting predictions using the statistical property of historical observations and predictions. However, the prediction accuracy of these methods rarely satisfy the requirements of the operational tasks. In this paper, we propose a novel prediction correcting framework using a residual singlehidden layer feedforward neural network, which is able to obtain efficient and effective corrections of numerical models. The neural network utilizes random input weights to predict the residual between numerical predictions and historical observations. The proposed framework is conducted in two steps: first, it explores the pattern of the residuals generated from numerical models compared with the observations. Second, it predicts the residuals using the trained model to improve the prediction accuracy of significant wave height. Experimental results reveal that the proposed framework predicts the significant wave height more accurately and efficiently compared with the maritime numerical model.
I. I NTRODUCTION Storm surge is one of the most dangerous catastrophes from hurricanes and typhoons. It results in billions of dollars in damage to the coastal regions and is one of the primary threats for loss of life [1]. In 2008, the storm surge from hurricane Ike killed more than 100 people, and it caused 20 billion dollars in property damage directly and indirectly [2]. Some researches [3][4][5] reveal that the storm surge damage will increase among the world due to the climate change and sea level rise. To reduce the severe damage of storm surge, it is essential to predict the significant wave height of storm surge. But the prediction accuracy from conventional methods hardly satisfies requirements in disaster prevention. One straightforward strategy for significant wave height prediction is manual prediction. Forecasters predict the significant wave height based on historical observations and other supporting materials. However, the performance heavily relies on the forecaster’s professional knowledge and experience. Another notable method is numerical ocean wave model. Booji et al. [6] proposed a third-generation numerical wave model based on Eulerian formulation of the discrete spectral balance of action density. Wahle et al. [7] adopted the atmospherewave regional coupled model to improve the prediction accuracy of wave height and surface wind in dynamically complicated coastal ocean areas. Kumar et al. [8] described how to use a sequential neural network to learn the mapping between
978-1-5386-1654-3/18/$31.00 ©2018 IEEE
historical observations and forecasts directly. Kang et al. [9] structured the complex nature of the significant wave height by introducing autoregressive integrated moving average model. Because of the complicated physical mechanism of ocean wave, the prediction of significant wave height using numerical models and from observations directly is not effective. In the literature, many prediction correcting methods have been developed to improve the accuracy of significant wave height prediction. These methods mainly correct the predicted consequences of significant wave height from numerical models. Parker et al. [10] proposed effective bias correction methods for numerical wave model. Janssen et al. [11] adopted a neutral regression based triple collocation method to obtain relative calibration of the prediction of significant wave height. Deshmukh et al. [12] developed wavelet neural networks to improve numerical ocean wave forecast. These methods commonly exploited statistical discipline of historical observations or numerical predictions of ocean wave. However, they are not effective because of adopting predictions or observations of wave height only. One potential solution is to develop a model that considers the error between predictions from numerical models and historical observations. In this paper, we develop a residual learning method to improve the correcting accuracy of wave height by exploring the mechanism behind residuals between numerical wave height predictions and historical observations. There are several reasons for developing a residual learning based correcting model. First of all, because of the complicated character of ocean wave patterns, it is difficult for a numerical model to predict the significant wave height directly. Furthermore, conventional correcting methods are based on the statistical discipline of historical observations, or they just learn an endto-end mapping from historical observation to errors without the supporting of physical models, which limit its ability to correct predictions. In addition, the numerical models are established based on the physical patterns of ocean wave with limited priors and mathematical relaxations. As a result, it is essential to develop a correcting model utilizing the residuals between numerical wave height predictions and observations. The main contributions of this paper are summarized as follows: • We develop a residual learning method, which learns the pattern of residuals between numerical predictions and observations of ocean wave to correct numerical models.
The residual learning method combines the advantages of numerical models and statistical property of observations to correct the prediction of significant wave height. • We adopt residual single-hidden layer feedforward neural network (RSLFN) to learn the pattern of residuals between wave height predictions and observations, and the model is faster and general than others. The RSLFN is also formulated as ELM [13] and FNNRWs [14]. Furthermore, experimental evaluations validate that our framework achieves better performance compared with the wave height numerical model in the situation of different time range correcting.
B. Single-Hidden Layer Feedforward Neural Networks for Residual Pattern Learning
II. C ORRECTING N UMERICAL P REDICTIONS VIA R ESIDUAL L EARNING In this section, we commence by introducing the principle of residual learning for prediction correction, as well as the training data organization strategy. Then, we introduce the proposed RSLFN for residual learning. Next, we describe how to train the proposed neural network to explore the pattern of residuals between numerical predictions and historical observations. Finally, we describe how to correct the prediction of significant wave height from numerical models using the trained model.
ai and bi are input weights and bias between input layer and hidden layer, and they are specified randomly. βi is the learned weight connecting the ith hidden node to the output node. G(ai , bi , X) is the output of the ith hidden node with respect to the input. Huang et al. [13] and Cao et al [14] have proved that single-hidden layer feedforward neural networks with a wide type of random computational hidden nodes have the universal approximation capability. For a given set of training examples n m {(xi , εi )}N i=1 ⊂ R × R . Supposing that the output matrix H of hidden layer and output weights β are formulated as follows:
A. Principle of Residual Learning and Training Data Organization In our work, we proposed a residual learning framework for significant wave height prediction correction. As illustrated in Fig. 1, the historical observations Ho and numerical predictions Hn are utilized to extract the residual attributes X. In this framework, the residual attributes are treated as the training data of the residual learning model, which is initialized by random input weights a and b as illustrated in Fig. 1. Specifically, for each time t, the observation Hot and the numerical prediction Hnt are utilized to compute the residual attribute εt = Hot − Hnt . Thus, we organize the training data as (1): ⎡ ⎤ ⎡ ⎤ x1
εt−1
⎢ x2 ⎥ ⎢ εt−2 ⎥ ⎢ X=⎢ ⎣ ... ⎦ = ⎣ ... xN
εt−2 εt−3 .. .
εt−N −1 εt−N −2
. . . εt−m . . . εt−1−m ⎥ ⎥ .. .. ⎦ . . . . . εt−N −m
(1)
where t is the time of numerical predictions to be corrected, and N is the length of training data. We use previous m residuals x = [εt−1 , εt−2 , . . . , εt−m ] to predict the residual εt of time t. Then, the predicted residuals is added to the numerical prediction to improve the accuracy of numerical models. In order to train the single-hidden layer neural network, the corresponding targets E are required: ⎡ ⎤ εt ⎢ εt−1 ⎥ ⎢ ⎥ E=⎢ (2) ⎥ .. ⎣ ⎦ . εt−M +m
In this part, we adopt RSLFN with random input weights for residual pattern learning. The learned patterns are then used for correcting the significant wave height predictions from numerical models. Fig. 1 illustrates the structure of the proposed framework for numerical correction. The output weights of RSLFN need not be turned, and the overall description of the proposed model is given by fL (X) =
L
βi G(ai , bi , X)
(3)
i=1
H(a1 , · · · , aL , b1 , · · · , bL , x1 , · · · , xN ) = ⎤ G(a1 , b1 , x1 ) · · · G(aL , bL , x1 ) ⎥ ⎢ .. .. .. ⎦ ⎣ . . . ⎡
G(a1 , b1 , xN )
· · · G(aL , bL , xN ) ⎡ T⎤ β1 ⎢ T⎥ ⎢β2 ⎥ ⎢ ⎥ ⎥ β=⎢ ⎢ . ⎥ ⎢ .. ⎥ ⎣ ⎦ βLT
(4)
N ×L
(5)
L×m
Here, H is so called the hidden-layer output matrix of the network, whereby the ith column is the output vector of the ith hidden node with respect to inputs x1 , · · · , xN , and the jth row is the output vector of the hidden layer with respect to input xj . β and E are corresponding matrices of the output weights and targets, respectively. L is the number of hidden neurons. The description of the proposed model (3) is rewritten as (6) Hβ = E (6) After the hidden nodes are randomly generated and the training data are given, the hidden-layer output matrix H is known and need not be turned iteratively. The solution to the system (5) is given explicitly as follows: βˆ = H† E †
(7)
where H is the Moore-Penrode generalized inverse of the hidden layer output matrix H.
The RSLFN with random input weights is trained by specifying input weights randomly and then the output weights are calculated using (7). By training the proposed model, the pattern hides the residual attributes between wave height numerical predictions and observed data are learned. The trained model is then used to correct the coming numerical predictions based on the historical data. +
+
56/)1
D
ȕ
«
«
激
compared to the corresponding numerical predictions. We also validate the efficiency of the proposed model, and the results are presented in Tab. I. As is illustrated in Fig. 2, the performance of the proposed residual learning method is better than the predictions from wave height numerical model. Furthermore, benefiting from the scheme of RSLFN with random input weights, whose parameters need not be turned by iterative strategy, the efficiency of training and testing is remarkable than the conventional methods. Experimental results reveal that the proposed residual learning method achieves a good performance both in effectiveness and efficiency.
«
«
«
«
TABLE I T HE TRAINING AND TESTING TIME ( S ) FOR PROPOSED METHOD OVER DIFFERENT CORRECTING TIME RANGE .
E +LVWRULFDO 1XPHULFDO 2EVHUYDWLRQV 3UHGLFWLRQV
,QSXW /D\HU
+LGGHQ /D\HU
RXWSXW /D\HU
3 hr
6 hr
12 hr
24 hr
48 hr
72 hr
Training
0.0156
0.0313
0.0469
0.2969
0.5781
0.8562
Testing
0.0082
0.0156
0.0938
0.2813
0.7031
0.9948
Fig. 1. The proposed Residual Single-hidden Layer Feedforward Neural Network.
IV. C ONCLUSIONS C. Correcting Numerical Predictions Using Single-Hidden Layer Feedforward Neural Networks We train the RSLFN with random input weights in Section II-B, and then correct the numerical predictions using the trained model. We first extract the residual attribute and organize the residuals in the form of samples in (1) as X∗ . Then the output of the residual learning model can be calculated using (8) fL (X) =
L
ˆi G(ai , bi , X∗ ), β
(8)
i=1
where the input weights ai and bi are given randomly, and ˆ is learned in Section II-B. After the coroutput weights β recting residuals is generated from proposed model, it will be added to the numerical prediction to improve the accuracy of numerical model. If the predictions needed to be corrected are more than one point, the predicted residuals will be used as input attribute to predict next residuals until finishing the correction. III. E XPERIMENTAL VALIDATION To validate the effectiveness of the proposed correcting scheme, we test our strategy on the predictions of significant wave height from numerical models and corresponding observed data. The observed data and numerical predictions are collected from 1 January, 2016 to 31 December, 2016 of a buoy in the Bohai Sea, China, and they are updated every one hour. We use the residual learning method to correct the predictions over a range of three hours, six hours, twelve hours, twenty-four hours, forty-eight hours and seventy-two hours. The performances under different time range of the proposed residual learning method are presented in Fig. 2, which are
In contrast to the existing correcting methods, which are based on the statistical property of historical observations, or just learn an end-to-end mapping between observations and errors. The proposed RSLFN learns the pattern of the residuals between observations and numerical predictions, which makes full use of the statistical property of observations with supporting of the numerical model to predict the significant wave height. Furthermore, the RSLFN with random input weights need not be turned using iterative strategy in the training procedure, the efficiency of the proposed model is notable than other neural network based correction methods which need to be turned iteratively. The proposed residual learning method involved two steps, i.e. residual pattern learning and inference. In our future work, we will go deep in to investigate the impact of the ocean surface wind, atmospheric pressure on the ocean wave height. ACKNOWLEDGMENT This work was supported in part by the National Natural Science Foundation of China under Project 61671481, in part by the Qingdao Applied Fundamental Research under Project 16-5-1-11-jch, and in part by the Fundamental Research Funds for the Central Universities under Project 18CX05014A. R EFERENCES [1] R. Morss, K. Fossell, D. Ahijevych, C. Davis, and C. Snyder, “Storm surge predictability,” in AGU Fall Meeting Abstracts, 2016. [2] R. E. Morss and M. H. Hayden, “Storm surge and certain death: Interviews with texas coastal residents following hurricane ike,” Weather, Climate, and Society, vol. 2, no. 3, pp. 174–189, 2010. [3] M. F. Karim and N. Mimura, “Impacts of climate change and sea-level rise on cyclonic storm surge floods in bangladesh,” Global Environmental Change-human and Policy Dimensions, vol. 18, no. 3, pp. 490–500, 2008.
[4] S. Hallegatte, N. Ranger, O. Mestre, P. Dumas, J. Corfeemorlot, C. Herweijer, and R. M. Wood, “Assessing climate change impacts, sea level rise and storm surge risk in port cities: a case study on copenhagen,” Climatic Change, vol. 104, no. 1, pp. 113–137, 2011. [5] S. Hoshino, M. Esteban, T. Mikami, H. Takagi, and T. Shibayama, “Estimation of increase in storm surge damage due to climate change and sea level rise in the greater tokyo area,” Natural Hazards, vol. 80, no. 1, pp. 539–565, 2016. [6] N. Booij, R. C. Ris, and L. H. Holthuijsen, “A third-generation wave model for coastal regions 1. model description and validation,” Journal of Geophysical Research, vol. 104, pp. 7649–7666, 1999. [7] K. Wahle, J. Staneva, W. Koch, L. Fenogliomarc, H. T. M. Hohagemann, and E. V. Stanev, “An atmospherecwave regional coupled model: improving predictions of wave heights and surface winds in the southern north sea,” Ocean Science, vol. 13, no. 2, pp. 289–301, 2016. [8] N. K. Kumar, R. Savitha, and A. A. Mamun, “Regional ocean wave height prediction using sequential learning neural networks,” Ocean Engineering, vol. 129, pp. 605–612, 2017.
Fig. 2. range.
[9] B. H. Kang, T. H. Kim, and G. Y. Kong, “A novel method for longterm time series analysis of significant wave height,” in Techno-Ocean (Techno-Ocean), 2016, pp. 478–484. [10] K. Parker and D. Hill, “Evaluation of bias correction methods for wave modeling output,” Ocean Modelling, vol. 110, pp. 52–65, 2017. [11] P. A. E. M. Janssen, S. Abdalla, H. Hersbach, and J. Bidlot, “Error estimation of buoy, satellite, and model wave height data,” Journal of Atmospheric and Oceanic Technology, vol. 24, no. 9, pp. 1665–1677, 2007. [12] A. N. Deshmukh, M. Deo, P. K. Bhaskaran, T. B. Nair, and K. Sandhya, “Neural-network-based data assimilation to improve numerical ocean wave forecast,” IEEE Journal of Oceanic Engineering, vol. 41, no. 4, pp. 944–953, 2016. [13] G. Huang, Q. Zhu, and C. Siew, “Extreme learning machine: Theory and applications,” Neurocomputing, vol. 70, no. 1, pp. 489–501, 2006. [14] F. Cao, D. Wang, H. Zhu, and Y. Wang, “An iterative learning algorithm for feedforward neural networks with random weights,” Information Sciences, vol. 328, pp. 546–557, 2016.
The comparison of performance between Residual Single-hidden Layer Feedforward Neural Network and Numerical Predictions over different time
