Separable approximations and decomposition methods for the augmented Lagrangian

Rachael Tappenden*, Peter Richtarik, Burak Büke

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

In this paper, we study decomposition methods based on separable approximations for minimizing the augmented Lagrangian. In particular, we study and compare the diagonal quadratic approximation method (DQAM) of Mulvey and Ruszczyński [A diagonal quadratic approximation method for large scale linear programs, Oper. Res. Lett. 12 (1992), pp. 205-215] and the parallel coordinate descent method (PCDM) of Richtárik and Takáč [Parallel coordinate descent methods for big data optimization. Technical report, November 2012. arXiv:1212.0873]. We show that the two methods are equivalent for feasibility problems up to the selection of a step-size parameter. Furthermore, we prove an improved complexity bound for PCDM under strong convexity, and show that this bound is at least 8(L′/L¯)(ω-1)2 times better than the best known bound for DQAM, where ω is the degree of partial separability and L′ and L¯ are the maximum and average of the block Lipschitz constants of the gradient of the quadratic penalty appearing in the augmented Lagrangian.

Original languageEnglish (US)
Pages (from-to)643-668
Number of pages26
JournalOptimization Methods and Software
Volume30
Issue number3
DOIs
StatePublished - May 4 2015

Keywords

  • (block) coordinate descent methods
  • augmented Lagrangian
  • iteration complexity
  • separable approximations

ASJC Scopus subject areas

  • Software
  • Control and Optimization
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Separable approximations and decomposition methods for the augmented Lagrangian'. Together they form a unique fingerprint.

Cite this