Application of spectral conjugate gradient methods for solving unconstrained optimization problems

Sulaiman Mohammed Ibrahim, Usman Abbas Yakubu, Mustafa Mamat

Abstract


Conjugate gradient (CG) methods are among the most efficient numerical methods for solving unconstrained optimization problems. This is due to their simplicty and  less computational cost in solving large-scale nonlinear problems. In this paper, we proposed some spectral CG methods using the classical CG search direction. The proposed methods are applied to real-life problems in regression analysis. Their convergence proof was establised under exact line search. Numerical results has shown that the proposed methods are efficient and promising.


Keywords


Sufficient descent property, Exact line search, Regression analysis, Spectral CG

Full Text:

PDF

References


Andrei, N. (2008). An unconstrained optimization test functions collection, Adv. Modell. Optim. 10, 147-161.

Barzilai, J. & Borwein, J.M. (1988). Two-point step size gradient methods, IMA J Numer Anal. 8, 141-148.

Birgin, E.G. & Martinez, J. M. (2011). A spectral conjugate gradient method for unconstrained optimization, Appl. Math. Optim. 43 (2), 117-128.

Yakubu, U.A. Mamat, M. Mohamad, A.M. Sukono, A.M. & Rivaie, M. (2018a). Secant free condition of a spectral PRP conjugate gradient method. International Journal of Engineering & Technology, 7 (3.28), 325 – 328.

Yakubu, U.A. Mamat, M. Mohamad, A.M. Sukono, A.M. & Rivaie, M. (2018b). Modification on spectral conjugate gradient method for unconstrained optimization. International Journal of Engineering & Technology, 7 (3.28), 307 – 311.

Yakubu, U.A. Mamat, M. Mohamad, A.M. Rivaie, M. & B.Y. Rabi’u. (2018d). Secant free condition of a spectral WYL and its global convergence properties. Far East Journal of Mathematical science, 12, 1889 – 1902.

Yakubu, U.A. Mamat, M. Mohamad, A.M. Rivaie, M. & J. Sabi’u. (2018e). A recent modification on Dai-Liao conjugate gradient method for solving symmetric nonlinear equations. Far East Journal of Mathematical science, 12, 1961 – 1974.

Zull, N. Rivaie, M. Mamat, M. Salleh, Z. & Amani, Z. (2015). Global convergence of a Spectral conjugate gradient by using strong Wolfe line search, Appl. Math. Sci. 63, 3105-3117.

Hu, C. & Wan, Z. (2013). An Extended Spectral Conjugate Gradient Method for unconstrained optimization problems, British Journal of Mathematics & Computer Science, 3, 86-98.

X. Wu. (2015). A new spectral Polak- Ribière -Polak conjugate gradient method, ScienceAsia. 41, 345-349.

Sulaiman, I. M., Sukono, Sudradjat, S., and Mamat, M. (2019). New class of hybrid conjugate gradient coefficients with guaranteed descent and efficient line search. IOP Conf. Ser.: Mater. Sci. Eng. 621(012021), 1-7.

Mathematical Sciences Issue 65-68, 3307-3319.

W. W. Hager & H. Zhang. (2006). A survey of nonlinear conjugate gradient methods, Pacific Journal of Optimization.. 2 (1), 35-58.

Du, X. & Liu, J. (2011). Global convergence of a spectral HS conjugate gradient method, Procedia Engineering. 15, 1487 – 1492.

Raydan, M. (1997). The Barzilai and J.M. Borwein gradient methods for the large scale unconstrained minimization in extreme problems, SIAM. J. Optim. 7 (1), 26-33.

Batu, T., Dasgupta, S., Kumar, R., & Rubinfeld, R. (2005). The complexity of approximating the entropy. SIAM Journal on Computing, 35(1): 132–150.

Motulsky, H., & Christopoulos, A. (2004). Fitting models to biological data using linear and nonlinear regression. Oxford University Press, New York.

Shirin, S., Mahmudul. H., & Laek, S.A. (2015). Age-Structured Population Projection of Bangladesh by Using a Partial Differential Model with Quadratic Polynomial Curve Fitting. Journal of Applied Sciences, 5: 542-551.

Zoutendijk, G. “Nonlinear programming, computational methods, in J Abadie (ED)”, Integer and Nonlinear Programming, North-Holland, Amsterdam, (1970), 37-86.

Yakubu, U.A. (2019). Enhancing spectral conjugate gradient method for solving unconstrained optimization problems. PhD Thesis. University Sultan Zainal Abidin Kuala Terengganu, Malaysia.

Yakubu, U.A. Mamat, M. Mohamad, A.M. Puspa, L.G. & Rivaie, M. (2018c). Secant free condition of a spectral Hestenes-Stiefel conjugate gradient method and its sufficient descent properties. International Journal of Engineering & Technology, 7 (3.28), 312 – 315.

K.E. Hillstrom. (1977). A simulation test approach to the evaluation of the nonlinear optimization algorithm. Journal ACM Trans. Mathematics Software. 3 (4), 305-315.

Dolan, E. & More, J.J. (2002). Benchmarking optimization software with performance profile, Math. Prog. Vol. 91, 201-213.

Z. Wei, S. Yao, and L. Liu. (2006). The convergence properties of some new conjugate gradient methods, Applied Mathematics and Computation, Vol. 183(2), pp. 1341–1350.

E. Polak & G. Ribiere. (1969). Note sur la convergence de directions conjug´ees, Rev. Francaise Informat Recherche Opertionelle, 3e ann´ee, Vol 16, pp. 35–43.

Sulaiman, I. M. Mamat, M. Abashar, A. Rivaie M. (2015). The global convergence properties of an improved conjugate gradient method Applied Mathematical Science; Vol 9(38), pp. 1857-1868.

Powell, M.J.D. (1977). Restart procedures for the conjugate gradient method, Math. Program. 12, 241-254.

Sulaiman, I. M., Mamat, M., Abashar, A., Zabidin, S. (2015): A Modified Nonlinear Conjugate Gradient Method for Unconstrained Optimization, Applied Mathematical Sciences Vol. 9(54), pp. 2671-2682.

Aini, N. Rivaie, M. Mamat, M. and Sulaiman, I. M. (2019). A Hybrid of Quasi-Newton Method with CG Method for Unconstrained Optimization. J. Phys.: Conf. Ser. 1366(012079), 1-10.




DOI: http://dx.doi.org/10.11121/ijocta.01.2020.00859

Refbacks

  • There are currently no refbacks.


Copyright (c) 2020 Sulaiman Mohammed Ibrahim

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

footer_771

   ithe_170     crossref_284         ind_131_43_x_117_117  Scopus  EBSCO_Host    ULAKBIM     ZBMATH more...