Mastane Achab

Mastane Achab

Nurturing the future

Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

Pages

Posts

Beyond Convexity #2: Barygradient flow

3 minute read

Published:

In my latest paper, I introduced a generalized proximal point algorithm (PPA): given a point $(x,q) \in \mathbb{R}^m \times \mathring \Delta_S$ ($m\ge 1$, $S \ge 2$ and $\Delta_S$ the probability simplex), the next iterate $(x’,q’)$ is given by: \(( \nabla f + \lambda A )(x',q') = \nabla f(x,q) + c \begin{pmatrix} 0_m \\ 1_S \end{pmatrix}\) where \(c = \log(\sum_s e^{\log(q'_s)-\lambda \ell_s(x')}) -\log(\sum_s e^{\log(q_s)}),\) for step-size $\lambda>0$, $f(x,q)=\frac12 ||x||^2 + h(q)$ with $h(q)=\sum_{s=1}^S q_s \log(q_s)$ the negentropy, and \(A(x,q) = \begin{pmatrix} J_\ell(x)^\intercal q \\ -\ell(x) \end{pmatrix}\) where $J_\ell$ denotes the Jacobian matrix of $\ell=(\ell_1,\dots,\ell_S):\mathbb{R}^m \rightarrow \mathbb{R}^S$ with each $\ell_s$ convex ($\forall 1\le s \le S$).

portfolio

publications

Checkered Regression

Published in preprint, 2022

Abstract. This paper introduces the checkered regression model, a nonlinear generalization of logistic regression. More precisely, this new binary classifier relies on the multivariate function $\frac{1}{2}\left( 1 + \tanh(\frac{z_1}{2})\times\dots\times\tanh(\frac{z_m}{2}) \right)$, which coincides with the usual sigmoid function in the univariate case $m=1$. While the decision boundary of logistic regression consists of a single hyperplane, our method is shown to tessellate the feature space by any given number $m\ge 1$ of hyperplanes. In order to fit the model’s parameters to some labeled data, we describe a classic empirical risk minimization framework based on the cross entropy loss. A multiclass version of our approach is also proposed.

Download here

Beyond Log-Concavity: Theory and Algorithm for Sum-Log-Concave Optimization

Published in preprint, 2023

Abstract. This paper extends the classic theory of convex optimization to the minimization of functions that are equal to the negated logarithm of what we term as a “sum-log-concave” function, i.e., a sum of log-concave functions. In particular, we show that such functions are in general not convex but still satisfy generalized convexity inequalities. These inequalities unveil the key importance of a certain vector that we call the “cross-gradient” and that is, in general, distinct from the usual gradient. Thus, we propose the Cross Gradient Descent (XGD) algorithm moving in the opposite direction of the cross-gradient and derive a convergence analysis. As an application of our sum-log-concave framework, we introduce the so-called “checkered regression” method relying on a sum-log-concave function. This classifier extends (multiclass) logistic regression to non-linearly separable problems since it is capable of tessellating the feature space by using any given number of hyperplanes, creating a checkerboard-like pattern of decision regions.

Download here

A Bregman firmly nonexpansive proximal operator for baryconvex optimization

Published in preprint, 2024

Abstract. We present a generalization of the proximal operator defined through a convex combination of convex objectives, where the coefficients are updated in a minimax fashion. We prove that this new operator is Bregman firmly nonexpansive with respect to a Bregman divergence that combines Euclidean and information geometries.

Download here

talks

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.