An advanced hybrid meta-heuristic algorithm for solving small- and large-scale engineering design optimization problems

An advanced hybrid algorithm (haDEPSO) is proposed in this paper for small- and large-scale engineering design optimization problems. Suggested advanced, differential evolution (aDE) and particle swarm optimization (aPSO) integrated with proposed haDEPSO. In aDE a novel, mutation, crossover and selection strategy is introduced, to avoid premature convergence. And aPSO consists of novel gradually varying parameters, to escape stagnation. So, convergence characteristic of aDE and aPSO provides different approximation to the solution space. Thus, haDEPSO achieve better solutions due to integrating merits of aDE and aPSO. Also in haDEPSO individual population is merged with other in a pre-defined manner, to balance between global and local search capability. The performance of proposed haDEPSO and its component aDE and aPSO are validated on 23 unconstrained benchmark functions, then solved five small (structural engineering) and one large (economic load dispatch)-scale engineering design optimization problems. Outcome analyses confirm superiority of proposed algorithms over many state-of-the-art algorithms.

upper limits for j th decision vector. L and K : total number of inequality and equality constraints. Many conventional optimization algorithms like Newton or quasi-Newton have been developed to solve engineering design optimization problems. However, they have certain inherent drawbacks like high computational complexity, local optimal stagnation and derivation of the search space [1]. Also, it is difficult to find the optimal solution in the solving process. Presently, to overcome the drawbacks of conventional optimization methods, a bunch of optimization methods known as meta-heuristics algorithms (MAs) has been introduced to solve complex engineering design optimization problems. According to the mechanical differences, the MAs can be categorized into four groups as follows: swarm intelligence algorithms (SIAs)-inspired from behavior of social insects or animals, evolutionary algorithms (EAs)-inspired from biology, physics-based algorithms (PBAs)-inspired by the rules governing a natural phenomenon and human behavior-based algorithms (HBAs)-inspired from the human being. Some instances of these algorithms are listed as follows.
Among many MAs, DE and PSO have been widely used in continuous/discrete, constrained as well as constrained/unconstrained optimization problems. DE has remarkable performance and become a powerful optimizer in the field of real-world problems. However, it has few issues such as convergence rate and local exploitation ability. In order to overcome its shortcomings, lots of DE has been designed in the literature like JADE [17], SADE [18], CDE [19], modified DE [20] and DSS-MDE [21]. Also, PSO has attracted attention to solve many complex optimization problems due to its efficient search ability and simplicity. However, main drawback of PSO is that it may easily get stuck at a local optimal solution region. Therefore, accelerating convergence speed and avoiding local optimal solutions are two critical issues in PSO. To overcome such issues many modified PSO are proposed in the literature such as HEPSO [22], RPSOLF [23], CPSO [24], IPSO [25], QPSO [26], PSO-OPS [27], MTVPSO [28], IPSO [29], MPSO-TVAC [30], IPSO-TVAC [31], θ-PSO [32] and MPSO [33]. Furthermore, hybrid strategy is one of the main research directions to improve the performance of single algorithm. Therefore, to enhance the performance of DE and PSO, lots of their hybrids are presented in the literature like FAPSO [34], PSOSCALF [35], CSDE [36], PSOSCANMS [37], DEPSO [38] and DPD [39]. Although a large number of MAs are introduced in the literature, they could not able to solve variety of problems [40]. In other words, a method may have the acceptable results for some problems, but not for others. Thus, there is a need to introduce some effective algorithms to solve a wider range of optimization problems. Also, hybrid techniques are now more favored over their individual effort. Hence, it is the motivation of this study to present novel variants of DE and PSO with their hybridization.
Moreover, after an extensive literature review on different variants of DE and PSO with their hybridization, following points are analyzed and motivated from them.
(i) In DE mutation and crossover strategy with their associate control parameters utilized to produce global best solution and beneficial for improving convergence behavior. Therefore, appropriate strategies and their associated parameter values of DE are considered a vital research study. (ii) The performance of PSO greatly depends on its parameters like acceleration coefficients (guide particles to the optimum) and inertia weight (balancing diversity). Hence, many researchers have tried to modify control parameter of PSO to achieve better accuracy and higher speed. (iii) Hybrid algorithms have aroused interest of researchers due to its effectiveness for complex optimization problems. Since DE and PSO have complementary properties, therefore their hybrids has gained prominence recently. To best of our knowledge, finding ways to combine DE and PSO is still an open problem.
Motivated by above observations and literature survey, following major contributions have been outlined for solving small-and large-scale engineering design optimization problems. Small-scale engineering design optimization problems include welded beam design (WBD), three-bar truss design (TRD), pressure vessel design (PVD), speed reducer design (SRD) and tension/compression spring design (T/CSD), whereas in largescale engineering design optimization problem namely economic load dispatch (ELD) with or without valve-point effects considering 3-, 6-, 15-, 40-and 140-unit test system.
(i) Developed an advanced differential evolution (aDE) where combination of novel strategies with their associated parameter values are familiarized. (ii) Suggested an advanced particle swarm optimization (aPSO) which consists of novel gradually varying (decreasing and/or increasing) parameters. (iii) Designed an advanced hybrid algorithm by hybridizing advanced DE and PSO (haDEPSO: hybridization of aDE and aPSO).

Methods
In this section, following proposed methodology has been described in detail: (i) advanced differential evolution (aDE), (ii) advanced particle swarm optimization (aPSO) and (iii) hybrid haDEPSO.

Advanced differential evolution (aDE)
In suggested advanced DE (aDE), modified mutation strategy and crossover rate as well as changed selection scheme are introduced as follows.

Mutation:
where x t i,j : target vector, v t i,j : mutant vector, rand(0, 1) : uniformly spread random number between 0 and 1, best j : best vector and τ : convergence factor (elects searching scale of all vectors). The dynamic adjustments of convergence factor ( τ ) are given as follows. (i) If τ = 1, then a vector will be randomly generated in the range [ x t i,j , best j ]. This can improve convergence rate of DE, but it may take risk of increasing probability of encountering local optima and (ii) if τ = µ(1 − t/t max ) + 1 , where t and t max : current and total iteration, µ : positive constant (determining the maximal searching scale of all vectors). In 1 st iteration, τ ≈ µ + 1 (as t = 1 is much smaller than t max , then term t/t max can be ignored). In max iteration, τ = 1 (as (1 − t/t max ) = 0 ). Therefore, τ decreases linearly from µ + 1 to 1 during the whole optimization process. This can improve the convergence as well as avoid local optima.
Since τ is composed of a series of large values and enlarges the exploring scale of all vectors earlier, whereas, τ composed a series of large values earlier and small values later which ensure global and local search capacity respectively. Also, it guaranteed to exploring the search space for all vectors of the proposed mutation strategy. These ensure global and local search capacity as well as exploring search space of all vectors of the proposed mutation strategy.

Crossover:
In order to keep the global searching ability and improve convergence speed C r is set as e (t−t max ) t max . It guarantees of individual diversity in early stage which improves global search ability. Further reduce degree of difference among individuals which accelerate convergence rate in later stage.
Selection: It emphasizes on the random nature of aDE which is formulated as follows.
where f (·): fitness function values and p : random value in (0, 1]. In this selection each pioneer vector gets chance to survive and share its observed information with other vectors in the next steps. It implies searching capabilities are more enriched and advantageous for stabilizing essential exploration and exploitation trends to aDE. The pseudocode of the proposed aDE is presented below.

Advanced particle swarm optimization (aPSO)
Preferably, PSO needs strong exploration ability and exploitation capability at early and later phase of the evolution, respectively. In velocity update equation of PSO, inertia weight ( w ) and acceleration coefficient (c 1 and c 2 ) are important factors to satisfy the above requirement with following concept. Considering all concerns like advantages, disadvantages and parameter influences of PSO, an advanced PSO (aPSO) is introduced in this study. It relies on novel gradually varying (decreasing and/or increasing) parameters ( w, c 1 and c 2 ) stated as follows (5)  where w i and w f : initial and final values of w ; c 1i and c 1f : initial and final values of c 1 ; c 2i and c 2f : initial and final values of c 2 ; t and t max : iteration index and maximum number of iteration. Hence, velocity and position of the i th particle are updated by following equations in the proposed aPSO. The pseudocode of proposed aPSO is presented below.

Hybrid advanced DEPSO (haDEPSO)
An advanced hybrid algorithm (haDEPSO) is proposed to further improve solution quality. In haDEPSO, entire population is sorted according to fitness function value and divided into two sub-populations, i.e.,pop 1 (best half ) and pop 2 (rest half ), since pop 1 and pop 2 contain best and rest half of the main population which implies good global and local search capability, respectively. In order to maintain local and global search capability, applying the proposed aDE (due to its good local search ability) and aPSO (because of its virtuous global search capability) on the respective sub-population ( pop 1 and pop 2 ). Evaluating both sub-populations and then better solution obtained in pop 1 (by using aDE) and pop 2 (by using aPSO) are named as best and gbest separately. If best is less than gbest, then pop 2 is merged with pop 1 thereafter merged population evaluated by aDE (as it mitigate the potential stagnation). Otherwise, pop 1 is merged with pop 2 afterward merged population evaluated by aPSO (as it established to guide better movements). Basically, haDEPSO is based on relating superior capability of suggested aDE and aPSO. The flowchart of haDEPSO is demonstrated in Fig. 1 and the pseudocode described below.

Implementation of haDEPSO for optimization
The stepwise implementation of the proposed haDEPSO for Sphere function (continuous, convex and unimodal) is demonstrated in this section. The initial parameters for the considered function are given as follows.
Stepwise execution of the proposed haDEPSO for Sphere function for iteration t = 1.
Step I: Initialization.
Arranging the population according to the fitness function values and dividing into two groups as pop 1 and pop 2 .   Step III: Applying. aDE in pop 1 and aPSO in pop 2 .    Best function value obtained by aDE is 1.0608 named as best on pop 1 .

Advanced particle swarm optimization (aPSO) for pop 2
of aPSO are given as follows.
Best function value obtained by aPSO is 12.6788 named as gbest on pop 2 .
Step IV: Condition best < gbest then merging pop 2 with pop 1 and applying again aDE else merging pop 1 with pop 2 then applying aPSO (best < gbest condition is applicable for iteration t = 1).

Mutation
Crossover     Step V: Termination criterion Stop if optimal solution obtained otherwise repeat go to step II. Similarly

Results and discussion
The validation of the proposed algorithms with the solution of small-and large-scale engineering design optimization problems is included in this section.

Validation of proposed algorithms
The justification of suggested component aDE and aPSO with the proposed haDEPSO algorithm evaluated on 23 unconstrained benchmark functions (UBFs). The descriptions of UBFs are listed in Table 1.
Simulations were conducted on Intel (R) Core (TM) i5-2350 M CPU @ 2.30 GHz, RAM: 4.00 GB, Operating System: Window 10, C-free Standard 4.0. After an extensive analysis, the parameters used in the proposed algorithms are fine-tuned and recommended as follows.w i (initial inertia weight) = 0.4, c 1i (initial cognitive acceleration coefficient) = 0.5 and c 2i (initial social acceleration coefficient) = 2.5 and w f (final inertia weight) = 0.9, c 1f (final cognitive acceleration coefficient) = 2.5 and c 2f (final social acceleration coefficient) = 0.5. Moreover, to handle constraints constrained optimization problems transfers to unconstrained one by adding penalty term into objective function. The bracket operator penalty [41] is picked in the present study due to its higher efficiency. And after many experiments fine-tuning value of R = 1e 03 is acclaimed to use in proposed algorithms. The overall best values in each table are highlighted with boldface letters of the corresponding algorithms.
The produced result by proposed algorithms on 23 unconstrained benchmark functions (UBFs) is compared with traditional algorithms (HHO [5] and EO [12]), DE variants (JADE [17] and SHADE [18]), PSO variants (HEPSO [22] and RPSOLF [23]) and hybrid variants (FAPSO [34] and PSOSCALF [35]). The parameters of all above compared and proposed algorithms are listed in Table 2. For fair comparison population size, stopping criteria and independent run of proposed algorithms is taken as minimum of corresponding comparative algorithms. The comparative experimental results in terms of mean, std. (standard deviation) and ranking of the objective function values are presented in Table 3 of 30 independent runs.
It should be noted that from Table 3, the mean objective function values of the proposed aDE, aPSO and haDEPSO algorithms are better and/or equal in comparison of          Moreover, all algorithms are individually ranked (as '1' for the best and '2' for subsequent performer and so on) in Table 3 based on mean result values. From this table it is concluded that haDEPSO, aDE and aPSO ranked 1st, 2nd and 4th sequentially. Also, average and overall rank of proposed algorithms Vs others is presented in Table 3. It is clear that (from ranking) performances of proposed algorithms are superior to others. Eventually, proposed aDE, aPSO and haDEPSO produce less std.
(it may 0.00E+00) for most of the cases on UBFs which describe their stability. Furthermore, superiority of proposed algorithms is statistically validated over other algorithms through one-tailed t test (with 98 degree of freedom (df) at 5% significance level) and Wilcoxon signed rank (WSR) test (at 5% significance level). The details of these tests can be found in [42]. The results of t test and WSR test on UBFs are reported in Table 4. From Table 4 it can be seen that proposed algorithms have both 'a (significantly better than other)' and 'a + (highly significance with other)' sign (in case of t test) and perform better or equally (in case WSR test) in most of consequence. Also, the p values as reported in Table 4 of the proposed algorithms are less with others which conclude that simulations are reliable for the majority of runs. The convergence speed of proposed and comparative algorithms is compared over 8 (f 1 , f 5 , f 6 , f 7 , f 8 , f 9 , f 10 and f 11 ) typical 30D UBFs. All plotted convergence graphs (objective function values Vs iterations) are separately presented in Fig. 2a-h. From these figures, it can be concluded that proposed aDE, aPSO and haDEPSO converge much faster than other algorithms in all cases.
Also, an attempt is made to find global optimal solution total of 690 runs (30 runs for each UBFs with 30 population size) and illustrated in Fig. 3. It confers that the proposed algorithms score the highest optimum solutions.
Apart from this, computational time of proposed and compared algorithms on each UBF is computed and presented through box plots in Fig. 4. From this figure, it can be perceived that the proposed algorithms take lesser time to achieve the best value for the entire UBFs.  As a whole, above numerical, statistical and graphical result analysis shows that proposed aDE, aPSO and haDEPSO are performed very competitive and/or equally with other compared algorithms. However, among three proposed algorithms haDEPSO is superior.

Application
In order to further examine proposed algorithms aDE, aPSO and haDEPSO are further applied to solve-following five well-known small-and one large-scale engineering design optimization problem.

Small-scale engineering design optimization problems
Considered small-scale engineering design optimization problems are briefed as follows.

(i) Welded beam design (WBD) problem
Its objective is to find the minimum cost design of a structural welded beam design, subject to constraints g 1 : shear stress (τ ) , g 2 : bending stress in the beam ( σ ), g 3 , g 4 , g 5 : side constraints, g 6 : end deflection of the beam (δ) and g 7 : bucking load on the bar ( P c ) with four design variables h(x 1 ) : thickness of the weld, l(x 2 ) : length of the welded joint, t(x 3 ) : width of the beam and b(x 4 ) : thickness of the beam. The schematic diagram of this problem is presented in Fig. 5, and it can be formulated as follows. where subject to:

(ii) Three-bar truss design (TRD) problem
It is dealt with the design of a three-bar truss structure in which the volume is to be minimized subject to stress constraints. The problem has two decision variables and three constraints. The schematic diagram of this problem is presented in Fig. 6 and it can be formulated as follows.

(iii) Pressure vessel design (PVD) problem
Its objective is to minimize the total cost f (x) , including cost of the material, forming and welding with variables Ts (thickness of the shell), T h (thickness of the head), R (inner radius) and L (length of the cylindrical section of the vessel). Both thickness variables (Ts, Th) must be integer multiple values of 0.0625 inch, which is the available thickness of rolled steel plates. R and L are continuous variables. A cylindrical vessel is capped at both ends by hemispherical heads. The schematic diagram of this problem is presented in Fig. 7 and it can be formulated as follows.

(iv) Speed reducer design (SRD) problem
Its aim is to minimize the weights of the speed reducer subject to constraints on bending stress of the gear teeth, surface stress, transverse deflections of the shafts and stresses in the shafts. The variables x 1 to x 7 represent the face width (b), module of teeth (m), number of teeth in the pinion (z), length of the first shaft between bearings ( l 1 ), length of the second shaft between bearings ( l 2 ) and the diameter of first ( d 1 ) and second shafts ( d 2 ), respectively. This is an example of a mixed integer programming problem. The third variable x 3 (number of teeth) is of integer values while all rest variables are continuous type. The schematic diagram of this problem is presented in Fig. 8 and it can be formulated as follows.
subject to:    (
The convergence graphs of all proposed and best non-proposed algorithm (to avoid complicacy) is plotted for all small-scale engineering design optimization problems and presented in Fig. 10a-e. From these figures, it can be clearly visualized that proposed algorithms converge faster than others. Hence, proposed algorithms are computationally efficient.
In general, from the all above result analysis it can be declared that proposed aDE, aPSO and haDEPSO are performing better and/or equally with others. However, among three proposed algorithms haDEPSO has larger competence.

Large-scale engineering design optimization problem: Economic load dispatch (ELD) problem with or without valve-point effects
Objective function of ELD problem with succeeding constraints can be represented as follows.
where F : total fuel cost, n : number of generating unit, F i (P i ) : operating fuel cost (real power output) and a i , b i &c i : cost coefficient of generating unit i . And P min i : minimum generation limit of unit i. Constraints.
• Generator constraint: P min i ≤ P i ≤ P max i , where P min i and P max i : minimum and maximum power generation by unit i. • Power balance constraint: n i=1 P i = D + P L , with P L = n i=1 n j=1 P i B ij P j + n i=1 P i B oi + B oo , where D : total load demand, P L : total transmission line loss and B ij ,B oi ,B oo : transmission loss coefficient. • Prohibited operating zone constraint: P min . . n i where n i : number of prohibited operating zone and P l i,k and P u i,k : lower and upper limit of k th prohibited zone of generating unit i. • Ramp rate limit constraint: max P min where P t i and P t−1 i current and previous output power and UR i and DR i : up and down ramp limit of generating unit i.
As reported in these tables, the global optimal cost produced by-(i) proposed aDE for TSys-1, TSys-2, TSys-3, TSys-4 and TSys-5 are 8234.0719 ($/h), 15 for all test systems show that the proposed algorithms succeed in finding the best solution in comparison with other algorithms. Furthermore, the mean and maximum fuel cost together with standard deviation and CPU mean time for each test systems are also recorded in the same tables. Also, it can be seen from these tables that proposed algorithms surpassed all other comparative algorithms by providing the best result with regard to minimum, mean and maximum cost. It is noteworthy that the proposed algorithms can still yield better solutions with low standard deviations and acceptable CPU time. It signifies that the proposed algorithm has stronger convergence with higher stability and reliability/robustness compared to other existing algorithms.
The convergence curves of proposed with compared algorithms are plotted in Fig. 11a-e for TSys-1, TSys-2, TSys-3, TSys-4 and TSys-5 of ELD problem in terms of fuel cost and iterations. This figure shows that proposed algorithms aDE, aPSO and haDEPSO have more robust convergence where the results improved as the iterations increased.

Conclusion
In this study, an advanced hybrid algorithm haDEPSO proposed for solving small-and large-scale engineering design optimization problems, where an advanced DE (aDE) and PSO (aPSO) are integrating in suggested hybrid. The brief summary of these proposed algorithms is given as follows.
(i) An advanced hybrid algorithm (haDEPSO) has been established by combining aDE and aPSO. It is based on multi-swarm approach where the population of one is merged with other in a pre-defined manner which yields guaranteed convergence and diversifying the solutions. (ii) To enhance performance and easily adjust the control parameters of DE, an advanced DE (aDE) is developed. The novel mutation strategy, crossover probability and altered selection schemes of aDE will guarantee high and low population diversity at start and end of the algorithm, respectively. (iii) To avoid particles stagnant, an advanced PSO (aPSO) is proposed which consists of novel gradually varying (decreasing and/or increasing) parameters. These control parameters can well-balance the exploration and exploitation capabilities and promotes the particles to search high-quality solution of aPSO.
The effectiveness of the proposed hybrid haDEPSO and its suggested component algorithms aDE and aPSO algorithm are tested on 23 unconstrained benchmark functions, then applied on five well-known small engineering design optimization problems, namely welded beam design (WBD), three-bar truss design (TRD), pressure vessel design (PVD), speed reducer design (SRD) and tension/compression spring design (T/CSD) problem and one large-scale engineering design optimization problem, viz., economic load dispatch (ELD) having five different test systems (3-, 6-, 15-, 40-, 140-unit). The numerical, statistical and graphical analyses of the proposed algorithms are compared against the state-of-the-art algorithms. The comparative results shape that the proposed algorithms become more robust and effective to solve complex engineering design optimization problems. Thus, it is conclusive that the proposed algorithms can be treated as a vital alterative in the field of MAs. Moreover, in the view of feasibilities, superiorities and solution optimality, among all and suggested algorithms haDEPSO outperformed.
In addition, the proposed algorithms have higher time complexity compared to some DE, PSO and hybrid variants. The main reason of time-consuming of proposed algorithms is the matrix operation execution. This operation is repeated per individuals in each iteration and increases the running time of the algorithm to some extent. Moreover, the proposed algorithms may not suitable for all engineering design optimization problems as others. As a part of our future work, some novel parameters will be designed for the proposed aDE, aPSO and haDEPSO in the hope of finding more accurate solutions and reduce the time complexity. Additionally, effectiveness of the proposed algorithms can be tested by some more complicated real-world applications and new MAs will be developed in future. Finally, this paper is expected more attention to the analysis of how to strengthen the robustness of the proposed algorithms for complex optimization problems.