Many problems in machine learning today can be cast as minimizing a convex loss function subject to some inequality constraints. As a result, the success of machine learning today depends on convex optimization methods that can scale to sizes reaching that of the World Wide Web. Problems in this class include basis pursuit, compressed sensing, graph reconstruction via precision matrix estimation, matrix completion under rank constraints, etc. One of the most popular optimization methods to use is the Alternating Direction Method of Multipliers. This is extremely well-scalable, but the convergence rate can be erratic. In this talk I will introduce the problem and algorithm with some applications and show how linear algebra can explain the erratic behavior.

Daniel Boley received his Ph.D. degree in Computer Science from Stanford University in 1981. Since then, he has been on the faculty of the Department of Computer Science and Engineering at the University of Minnesota, where he is now a full professor. Dr. Boley is known for his past work on numerical linear algebra methods for control problems, parallel algorithms, iterative methods for matrix eigenproblems, inverse problems in linear algebra, as well as his more recent work on computational methods in statistical machine learning, data mining, and bioinformatics. His current interests include scalable algorithms for convex optimization in machine learning, the analysis of networks and graphs such as those arising from metabolic biochemical networks and networks of wireless devices. He is an associate editor for the SIAM Journal of Matrix Analysis and has chaired several technical symposia at major conferences.