Roll force prediction using hybrid genetic algorithm with semi-supervised support vector regression

Abstract

Roll force prediction plays a significant role in rolling schedule and optimization. For a specific steel grade, the roll force can be determined by the aid of several factors: rolling speed, initial thickness, ratio of thickness reduction, the starting temperature of the strip, and the friction coefficient in the contact region. Roll force prediction mathematical models are sometimes rare and inaccurate. This paper presents a new approach to predict roll separating force using semi-supervised support vector regression (SSSVR). The parameters affecting the sensitivity of the SSSVR were optimized using the genetic algorithm to maximize the r-squared accuracy score. The intelligent system is evaluated using two quality metrics: the root mean square error (RMSE) and the mean absolute error calculated between the measured force from the industrial rolling field and the predicted force using the proposed system from one side, and the measured force and the calculated force from another side. Obtained results show the improvement while using the intelligent predictive system. The reduction in RMSE was achieved by the proposed system by 66.9% and 32.1% for oval and round shape passes, respectively in comparison to the conventional calculation method.

Introduction

The era of industry 4.0 shows the rise of the scientific field (Artificial intelligence) as a promising technology widely applied in different industrial applications [1] (e.g. efficient energy management systems (EEMS) [2], microgrid system [3]) Modeling and soft computing are highly interested in predicting the roll force and torque rolling. Roll force prediction plays a significant role in rolling schedule and optimization. Several methods have been devoted to predicting roll forces and torque through modeling and soft computing. Some of these methods were based on traditional models, such as the numerical finite element method, and others were based on the optimized heuristic method.

The prosperous application of semi-supervised support vector machines has been used in many applications. Therefore, this paper had paid attention to the semi-supervised support vector machines to predict roll separating force.

The main contribution in this paper is:

• 1. The development of a machine learning based algorithm that can predict accurately the roll force with few experiments in laboratory

• 2. The new algorithm outperforms the conventional regression models and also the mathematical model used for calculation of roll force

• 3. Saving the cost and the time deployed in laboratory and having results very close to the actual readings can certainly improve the industrial process

The main novelty of the paper is the prediction of roll separating force without deploying big efforts to carry results from the experiments in the laboratory. The semi-supervised support vector regression is a semi-supervised learning technique that requires very few input data records with labels. Most of the data records can be without labels to train the network. Input data labeled can be taken from one or two passes and the unlabeled data can complete the input from five or six passes. Once the model is trained, the output force of the following passes is predicted easily.

The organization of the rest of the paper is as follows: “Literature survey” section presents a brief on previous work related to the paper achievement “Methodology” section introduces the methodology. “Experimental results and discussion” section presents the experimental results and discusses them. “Conclusions and future work” section gives conclusions on work done in the paper and suggests future works.

Literature survey

Aghasafari et al. [4] introduced a model based on an inverse analysis technique to obtain the flow curve of materials in a hot rolling finishing mill. The model has relied on the minimization function of the differences between the experimental and computed values. The model can simultaneously determine more accurate flow stress and enhance the estimation of the interface friction factors.

Yang et al. [5] applied an artificial neural network model trained by data conducted from developed a finite element model. The finite element model has been developed and validated to predict the stock temperature, strain, strain rate, and stress profiles during the rolling process. Then, the finite element was used to build the training data for the development of the neural network (NN) models. Mahdi and Hosein [6] developed a neural network model to a hot strip mill to enhance the model's prediction for rolling force and rolling torque as a function of different process factors. The neural network was trained and validated using many three-dimensional finite element simulations carried out for various sets of process parameters. Fei et al. [7] proposed a concept of enabling the computation of the material model factors via a direct way from the rolling process on an industrial scale using an artificial neural network.

The main difficulty in the usage of the finite element model to produce the data needed to train the artificial neural network is the large amount of data needed as input for the construction of the mesh and those data are mainly acquired from exhaustive experiments in the labs, also the output may vary with large differences according to this input data.

Liu et al. [8] proposed a roll force prediction method based on genetic algorithm (GA), particle swarm optimization algorithm (PSO), and multiple hidden layer extreme learning machine (MELM). In the method proposed, GA is used for the search of the optimal number of hidden layers and nodes, and PSO is used to determine the best input weights and biases. One disadvantage to this model is that it depends on completely supervised learning and labeling the input data for training needs always big efforts, cost and time in labs’ experiments. Hwang et al. [9] suggested a hybrid model based on mathematical and artificial neural network for prediction of rolling force and temperature in hot rolling processes. They trained their new model by collecting previous history data (e.g. 6–12 months ago). A drawback to this method is the need to wait for months to collect data which is not always realistic for industry.

In [10], an ensemble system was used to merge a number of machine learning techniques and average them to produce one final predictive model where the k-cross validation was carried out to validate the results and enhance the model. The R-squared value of the machine learning model was over 0.98, while for calculation using Sims theory it was 0.922. This research work lacks the existence of the different shapes of roll force operations (e.g. round, oval, etc.)

In [11], a new online model using the gradient boosting decision tree (GBDT) method was proposed to predict rolling force. The new online model improved the time performance however the accuracy of prediction remains poor in comparison to the actual readings of roll force in labs.

Li et al. [12] proposed a novel mechanical properties interval prediction model based on the sparrow search algorithm as an optimization for the neural network learning model. Cases examined in the research were very few.

Our main objective in this research is to develop a model independent from the finite element model and its disadvantages by introducing a method based on semi-supervised learning instead of completely supervised learning. In this method, very few experiments are needed to label a part of the data and not all the data. We also prove that the new method gives better accuracy in predicting roll force.

Semi-supervised learning is a technique widely used in machine learning that merges a small number of labeled data and a large number of unlabeled data to form the training dataset. Semi-supervised learning is between the unsupervised learning and the supervised learning. The heuristic technique of self-labeling is considered as the oldest model of the semi-supervised learning [13]^.

The examples of applications started in the 1960s [14]. Vladimir Vapnik 1970s introduced a transductive learning framework [15]. Nearly correct learning model for semi-supervised learning of a Gaussian mixture was introduced by Ratsaby and Venkatesh in 1995 [16]. Several classical methods are used in semi-supervised learning, such as the self-training method [17, 18], expectation-maximization method, multi-view method [19], graph-based method. Dópido et al. [17] developed a new approach for semi-supervised learning that adapts available active learning methods to a self-learning framework for applying hyperspectral image classification. Yuanqing Li et al. [18] made the analysis of the convergence of a semi-supervised support vector machine algorithm for the classification in small training dataset.

Adankon and Cheriet [19] suggested a genetic algorithm to improve the objective function of the semi-supervised learning support vector machine. This algorithm was applied to applications of brain-computer interface. Shipeng et al. [20] proposed an improved method for co-training, which is a novel co-training kernel for Gaussian process classifiers. Ahmet et al. [21] employed a transductive label propagation method based on manifold assumption to predict the entire dataset generates pseudo-labels for the unlabeled data and train a deep neural network.

Methodology

Genetic algorithms (GAs)

GAs are stochastic search techniques that can search spaces using ideas from the natural evolutionary principle [22, 23]. The idea of this method was first proposed by Holland [24]. A genetic algorithm works with a population of chromosomes, each identifying a possible solution to the problem tackled. Each chromosome is assigned a fitness value given the fitness function. Chromosomes with high fit are given more chances to reproduce, and the offspring take features from their parents. The GA is a simple tool for searching the global solution to an optimization problem. It is widely used for large-scale and complex nonlinear optimization problems [25].

The procedure of a GA can be summarized as follows:

• 1. Initialize a randomly generated population.

• 2. Compute the fitness of each chromosome in the population.

• 3. Do the offspring by genetic operators: selection, crossover, and mutation.

• 4. Check the stopping criteria. If the stopping criteria are met, the genetic algorithm should be stopped. Otherwise, repeat steps 2–4 using the generated offspring.

There are two types of coding methods for GAs. They are real and binary-coded GAs. Here, we adopt the binary-coded GA

Semi supervised-support vector regression

Support Vector Machines (SVM) is a robust machine learning technique introduced for Supervised Learning [26] and classification problems. Suppose a training set $L_{d} = \left\{ {\left( {x_{1} ,y_{1} } \right), \ldots ,\left( {x_{n} ,y_{n} } \right)} \right\}$ with $x_{i} \in R^{d}$, $i = 1, \ldots ,n$ and $y_{i} \in \{ 1, - 1\}^{n}$, the problem of Support Vector Regression (SVR) is defined by Bennett and Demiriz in [27]as follows:

$$\mathop {\min }\limits_{w,b,\varepsilon } \left( {\frac{1}{2}w^{2} + C\mathop \sum \limits_{i = 1}^{n} \xi_{i} } \right)$$

(1)

s.t. $y_{i} \left( {wx_{i} + b} \right) + \xi_{i} \ge 1,\xi_{i} \ge 0,$ where $C > 0$ is the error parameter and $\xi_{i} \ge 0$ is an added variable to every point, in a manner that $\xi_{i} \ge 1$ if the point is classified wrong, $b$ is a scalar and $w$ is an n-vector such that:

$$y_{1} \left[ {w.x_{i} - b} \right] \ge 1,i = 1, \ldots ,l$$

(2)

where $l$ is the number of labeled samples.

For the semi-supervised approach, suppose a set of $l$ labeled samples $L_{d} = \left\{ {\left( {x_{1} ,y_{1} } \right), \ldots ,\left( {x_{l} ,y_{l} } \right)} \right\}$ and a set of $u$ unlabeled samples $U_{D} = \left\{ {x_{l + 1} , \ldots ,x_{l + u} } \right\}$ where $x_{i} \in R^{n} , i = 1, \ldots ,l + u$ and $y_{i} \in R$, the problem of SS-SVR is as follows [28]:

$$\mathop {\min }\limits_{w,b,\xi ,\eta ,z} C\left( {\mathop \sum \limits_{i = 1}^{l} \eta_{i} + \mathop \sum \limits_{j = l + !}^{l + u} {\text{min}}\left( {\xi_{j} ,z_{j} } \right)} \right) + w$$

(3)

s.t. $y_{i} \left( {wx_{i} + b} \right) + \eta_{i} \ge 1, \eta_{i} \ge 0, i = 1, \ldots .,l$

$$wx_{i} - b + \xi_{j} \ge 1,\xi_{j} \ge 0,j = l + 1, \ldots .,l + u$$

$$- \left( {wx_{i} - b} \right) + z_{j} \ge 1,z_{j} \ge 0$$

where $C > 0$ is an error parameter, $\eta_{i}$ is a slack term added for each point such that if the point is misclassified $\eta_{i} \ge 1$.

Figure 1 shows the semi-supervised regression general framework

Fig. 1

The SSR general framework

BGA-based optimization of the SS-SVR model

The SS-SVR generalization estimation accuracy depends on the correct setting of the hyperparameter (C). However, no general guidelines are available to select this parameter. Therefore, we adopted a binary genetic algorithm (BGA) to search for the optimal parameters of SS-SVR to improve the prediction. In the proposed BGA–SSSVR model, the parameter C is encoded with binary data; the values of the SS-SVR parameter are optimized using the BGA evolutionary process to estimate the wanted parameter to construct an optimized SVR model.

Figure 2 shows the framework for optimizing the SS-SVR parameters with a binary genetic algorithm, summarized as follows.

Fig. 2

The BGA-SSSVR model

Step 1—Binary Coding of C to generate the chromosome randomly. Here, the range of C is defined as [100, 2000], the value of $\xi_{{}}$ is set to 1, and the value of η is set to 0.001.

The population size is 100.

Step 2—Fitness definition of the training data set as maximum determination coefficient R-squared

Step 3—Running the Genetic algorithm operations as a standard roulette wheel operates to select excellent chromosomes to reproduce. Single-point crossover is used between two chromosomes. The probability of generating new chromosomes is 0.9. Then, the mutation operation probability is set to 0.05.

Step 4—If the new population does not meet the stopping criterion, steps 3–4 are repeated until C satisfies the maximum determination coefficient R-squared. The best C would be given based on the optimum fitness function value.

$${\text{R}}^{{2}} = {1} - \left( {{\text{sum}}\;{\text{of}}\;{\text{the}}\;{\text{residuals}}\;{\text{squared}}/{\text{sum}}\;{\text{of}}\;{\text{squares}}} \right)$$

(3)

where the sum of squares is the sum of the difference between the data and the mean all squared and the residuals are the absolute subtraction of the actual value from the predicted value

In our work, the BGA-SSSVR was run five runs to get the mean value of the output C and the mean values of the predicted roll forces. We used the standard parameter settings for the GA. The genetic algorithm is a stochastic algorithm, i.e., Each run gives different results and we can calculate the mean of the results in different runs.

The design of roll force prediction model based on BGA-SSSVR

For any Machine Learning (ML) model, the input parameters' design is critical for the accuracy of its output results. To get an accurately predicted roll force, the input parameters to the BGA-SSSVR model are set to the parameters that determine the roll force for a particular steel grade which are:

• 1. Initial thickness (mm)

• 2. Reduction in thickness (%)

• 3. Rolling Speed (rpm)

• 4. Initial temperature (°C)

• 5. Friction Coefficient $\mu$

While the output is the predicted roll force (N)

Figure 3 shows the roll force prediction model based on BGA-SSSVR

Fig. 3

The design of the Roll Force Prediction model

Experimental results and discussion

Data set

In [29], the authors validated a model for predicting roll force during rebar steel processing. The readings from the rolling field industry were used as input for the model to check the accuracy of the model calculations. The steel section varies from pass to pass in the rolling process, i.e., round-oval-round. Measured flow stresses were bigger than those calculated by the model in range of 30–40 MPa. The difference between the measured roll forces values and calculated values is noticeably high.

In this paper, we use data of the first six passes to train our machine learning proposed model and the data of the following passes from the 7th pass to the 21st pass for the testing phase.

Tables 1 and 3 show the input parameters to the BGA-SSSVR for the Oval and Round shape respectively.

Table 1 The input parameters to the BGA-SSSVR for the Oval shape

In machine learning, we stop to acquire data for training when we want to avoid overfitting. One way to avoid overfitting is to apply the cross-validation method, in which the data being used for training the model is divided into folds and the model is done for every fold. Then, the overall error is averaged. However, our data acquired from experiments are always too few to cause the overfitting problem

Quality metrics

As quality metrics, we use two quality metrics: the mean absolute error (MAE) and the root mean square error (RMSE)

The mean absolute error is defined as follows:

$${\text{MAE}} = \frac{{\mathop \sum \nolimits_{i = 1}^{N} \left| {{\text{AF}} - {\text{PF}}} \right|}}{N}$$

(4)

And the root-mean-square error is defined as follows:

$${\text{RMSE}} = \sqrt {\frac{{\mathop \sum \nolimits_{i = 1}^{N} \left( {{\text{AF}} - {\text{PF}}} \right)^{2} }}{N}}$$

(5)

where AF is the actual force, PF is the predicted force, and $N$ is the number of observed passes.

Comparison criteria

We propose the following set of criteria to be used to evaluate the performance of machine learning to predict the roll force in a fourth industry generation:

• Two shapes will be studied: oval and round

• Three values will be compared for each sample: actual measured—calculated through the mathematical model—predicted using the proposed ML model

• The standard description of management data

• Three quality measures: RMSE—MAE-MAPE

Experimental setup

All implementations performed in this paper were using Python 3.8. The size of the training set was 105 records; only 15 records were labeled data. The size of the testing set was eight records for oval shape and seven records for round shape. The key parameters for the genetic algorithms are: bounds of penalty parameter C = [100, 2000], number of bits = 20, number of iterations = 100, number of population = 100, rate of crossover = 0.9, rate of mutation = 0.05. Tables 1 and 3 show the input parameters to the BGA-SSSVR for the oval and round shapes, respectively

Results and discussion

As mentioned in “The design of roll force prediction model based on BGA-SSSVR” section, the proposed BGA-SSSVR model has five input parameters: initial thickness, the percentage reduction in thickness, rolling speed, initial temperature, and friction coefficient $\mu$. It gives one output which is the predicted force. The experiment applies an oval shape manufacturing process and compares the three forces. Table 2 shows the obtained predicted force (PF) compared to the actual (MF) and the mathematically calculated ones in eight different observed passes. From this table, it could be shown that the PF approaches the MF in passes 2 to 8.

Table 2 Comparison between the measured (actual), calculated, predicted roll force in Newton for the Oval Shape

The same conclusion could be deduced from Fig. 4. The fitness function for the oval shape R²=0.9192 and the best value of C was found to be 1738.58. The mean absolute error (MAE) is 289,246.61 between the measured force and the calculated force, while its value equals 95,684.98 between the measured force and the predicted force. This means that the percentage of reduction reaches 66.9%. The root mean square error (RMSE) is 317,262.98 between the measured force and the calculated force, while its value equals 108,881.5 between the measured force and the predicted force, leading to a reduction of 65.7%. Also, in case of oval shape, the mean absolute percentage error (MAPE) is 49.6045 between the measured force and the calculated force, while its value equals 20.27056 between the measured force and the predicted force leading to a reduction in error of 59.14%. The reduction in RMSE and MAPE is above the 50% which means that the new method succeeded in reducing error resulted from prediction of roll force using the mathematical model by a great proportion in case of oval shape (Table 3).

Fig. 4

Comparison between the measured (actual), calculated, predicted roll force in Newton for the Oval Shape

Table 3 The input parameters to the BGA-SSSVR for the Round shape

Moving to the round shape manufacturing process, Table 4 shows the obtained predicted force (PF) compared to the actual (MF) and the mathematically calculated ones in seven different observed passes. From this table, it could be shown that the PF approaches the MF in all passes, as well. The same conclusion could be deduced from Fig. 5.

Table 4 Comparison between the measured (actual), calculated, predicted roll force in Newton for the Round Shape

Fig. 5

Comparison between the measured (actual), calculated, predicted roll force in Newton for the round Shape

From Fig. 5, it could be observed that both the calculated and the predicted values do not show the same behavior moving from one pass to the other, i.e., sometimes are lower than the measured value (pass 3 to 7) and in other case are higher than the measured values. However, we can see that the predicted values are always very close to the measured (actual) values. This enforces the need to predict using machine learning methods rather than calculate the force by the model given in [29].

The fitness function for the round shape R²=0.7937, and the best value of C was 888.37. The mean absolute error (MAE) is 147,594.12 between the measured force and the calculated force, while its value equals 100,223.64 between the measured force and the predicted force. This means a percentage reduction of 32.1%. The root mean square error (RMSE) is 167,557.94 between the measured force and the calculated force, while its value equals 109711.2 between the measured force and the predicted force, giving a percentage reduction of 34.5%. Also, in case of round shape, the mean absolute percentage error (MAPE) is 32.76025 between the measured force and the calculated force, while its value equals 26.18651 between the measured force and the predicted force leading to a reduction in error of 20.07%. The reduction in RMSE and MAPE is above the 20% which means that the new method succeeded in reducing error resulted from prediction of roll force using the mathematical model by a good proportion in case of round shape.

To better evaluate the machine learning model for prediction of roll force, the k-fold cross validation evaluation method was used in both cases oval shape and round shape. For the oval shape, the 3-fold cross validation produces a root mean square error (RMSE) of value 360.66 while when k equals 5, the RMSE was decreased to 342.86. For the round shape, the 3-fold cross validation gives an RMSE equals 375.71, while the 5-fold cross validation gives a reduction in value of RMSE 373.73. The results of the usage of the k-fold cross validation as an evaluation method shows that the assignment of k to be 5 is better than 3 and that the error is found to be less in the case of oval shape than that of round shape. In general, the new machine learning model suggested in this paper to predict roll force succeeded in improving the prediction’s accuracy using small efforts in data acquisition.

In order to prove the efficiency of the new BGA-SSSVR in predicting the roll force, a comparison between the new approach and four conventional regression models (Linear Regression, Random Forest Regression, Support Vector Regression and Decision Tree regression) was established in addition to the comparison made between the BGA-SSSVR and the calculation model presented in [21]. Tables 5 and 6 show these comparisons for the oval and the round shapes, respectively.

Table 5 Comparison between BGA-SSSVR and other regression models for the oval shape

Table 6 Comparison between BGA-SSSVR and other regression models for the round shape

Conclusions and future work

Over the last decade, machine learning (ML) and especially deep learning (DL) have started to empower applications within the industrial field. Many industrial areas indicate ML is becoming one of the main components to upgrade traditional manufacturing to the Industry 4.0 level. This paper applies ML in the roll force prediction industrial process. Here, the mathematical models suffered from significant errors compared to actual measured data. A new approach to predict roll separating force using SSSVR was introduced, and GA was applied to optimize the parameters affecting the sensitivity of the SSSVR. Maximizing the R² accuracy score has been considered as a fitness function for the GA. The proposed model was evaluated using the RMSE, the MAPE and the MAE, calculated between the actual measured force, the predicted force, and the calculated forces. Obtained results show the reduction in RMSE by 66.9% and 32.1% for oval and round shape passes, respectively. Moreover, the MAE has been reduced by 65.7% and 34.5% in shapes oval and round, respectively. Also, the results show the reduction MAPE by 59.14% and 20.07% for oval and round shape passes, respectively. The K-fold cross validation used for the evaluation of the new model had shown its efficiency in both shapes.

Experimental and simulation results showed that the predicted force values surpassed the mathematically calculated values when compared to the actually measured ones.

In sum, this method can be applied to different materials, form factors and similar processes as long as we train the model by data given from passes belonging to the corresponding process or material. By entering an initial input file about the first given passes to train the model, the new method will be able to predict the other passes accurately.

As future work, we suggest to introduce the notion of fuzzy logic in the machine learning model in order to acquire data inputs in the form of range of values which better describes data and hence yields to more accurate results