LDA++
ldaplusplus::optimization::MultinomialLogisticRegression< Scalar > Class Template Reference

#include <MultinomialLogisticRegression.hpp>

## Public Member Functions

MultinomialLogisticRegression (const MatrixX &X, const Eigen::VectorXi &y, VectorX Cy, Scalar L)

MultinomialLogisticRegression (const MatrixX &X, const Eigen::VectorXi &y, Scalar L)

Scalar value (const MatrixX &eta) const

## Detailed Description

### template<typename Scalar> class ldaplusplus::optimization::MultinomialLogisticRegression< Scalar >

MultinomialLogisticRegression is an implementation of the multinomial logistic loss function (without bias unit).

It follows the protocol used by GradientDescent. For the specific function implementations see value() and gradient().

## Constructor & Destructor Documentation

template<typename Scalar >
 ldaplusplus::optimization::MultinomialLogisticRegression< Scalar >::MultinomialLogisticRegression ( const MatrixX & X, const Eigen::VectorXi & y, VectorX Cy, Scalar L )
Parameters
 X The documents defining the minimization problem ( $$X \in \mathbb{R}^{D \times N}$$) y The class indexes for each document ( $$y \in \mathbb{N}^N$$) Cy A different weight for each class in the optimization problem L The L2 regularization penalty for the weights
template<typename Scalar >
 ldaplusplus::optimization::MultinomialLogisticRegression< Scalar >::MultinomialLogisticRegression ( const MatrixX & X, const Eigen::VectorXi & y, Scalar L )
Parameters
 X The documents defining the minimization problem ( $$X \in \mathbb{R}^{D \times N}$$) y The class indexes for each document ( $$y \in \mathbb{N}^N$$) L The L2 regularization penalty for the weights

## Member Function Documentation

template<typename Scalar >
 void ldaplusplus::optimization::MultinomialLogisticRegression< Scalar >::gradient ( const MatrixX & eta, Eigen::Ref< MatrixX > grad ) const

The gradient of the objective function implemented in value().

We use $$I(y) \in \mathbb{R}^Y$$ as the indicator vector of $$y$$ (a vector with all the values 0 except at the yth position).

$\nabla_{\eta} J = -\sum_{n=1}^N C_{y_n} \left( X_n I(y_n)^T - \frac{\sum_{\hat{y}=1}^Y X_n I(\hat{y})^T \exp(\eta_{\hat{y}}^T X_n)} {\sum_{\hat{y}=1}^Y \exp(\eta_{\hat{y}}^T X_n)} \right) + L \eta$

Parameters
 eta The weights of the linear model ( $$\eta \in \mathbb{R}^{D \times Y}$$) grad A matrix of dimensions equal to $$\eta$$ that will hold the result
template<typename Scalar >
 Scalar ldaplusplus::optimization::MultinomialLogisticRegression< Scalar >::value ( const MatrixX & eta ) const

The value of the objective function to be minimized.

$$N$$ is the number of documents (different vectors), $$X_n \in \mathbb{R}^D$$ is the nth document, $$\eta_y \in \mathbb{R}^D$$ is the weights vector for the class $$y$$ defining the hyperplane that separates class $$y$$ from all the other, finally $$y_n$$ is the class of the nth document.

$J = -\sum_{n=1}^N C_{y_n}\left(\eta_{y_n}^T X_n - \log\left( \sum_{\hat{y}=1}^Y \exp\left( \eta_{\hat{y}}^T X_n \right) \right)\right) + \frac{L}{2} \left\| \eta \right\|_F^2$

Parameters
 eta The weights of the linear model ( $$\eta \in \mathbb{R}^{D \times Y}$$)

The documentation for this class was generated from the following files: