Public Member Functions | List of all members
ldaplusplus::em::FastSupervisedMStep< Scalar > Class Template Reference

#include <FastSupervisedMStep.hpp>

Inheritance diagram for ldaplusplus::em::FastSupervisedMStep< Scalar >:
ldaplusplus::em::UnsupervisedMStep< Scalar > ldaplusplus::em::MStepInterface< Scalar > ldaplusplus::events::EventDispatcherComposition ldaplusplus::em::SemiSupervisedMStep< Scalar >

Public Member Functions

 FastSupervisedMStep (size_t m_step_iterations=10, Scalar m_step_tolerance=1e-2, Scalar regularization_penalty=1e-2)
virtual void m_step (std::shared_ptr< parameters::Parameters > parameters) override
virtual void doc_m_step (const std::shared_ptr< corpus::Document > doc, const std::shared_ptr< parameters::Parameters > v_parameters, std::shared_ptr< parameters::Parameters > m_parameters) override
- Public Member Functions inherited from ldaplusplus::events::EventDispatcherComposition
std::shared_ptr< EventDispatcherInterfaceget_event_dispatcher ()
void set_event_dispatcher (std::shared_ptr< EventDispatcherInterface > dispatcher)

Detailed Description

template<typename Scalar>
class ldaplusplus::em::FastSupervisedMStep< Scalar >

Implement the M step for the fsLDA.

Similarly to the UnsupervisedMStep the purpose is to maximize the lower bound of the log likelihood \(\mathcal{L}\). The same notation as in UnsupervisedMStep is used.

\[ \log p(w, y \mid \alpha, \beta, \eta) \geq \mathcal{L}(\gamma, \phi \mid \alpha, \beta, \eta) = \mathbb{E}_q[\log p(\theta \mid \alpha)] + \mathbb{E}_q[\log p(z \mid \theta)] + \mathbb{E}_q[\log p(w \mid z, \beta)] + H(q) + \mathbb{E}_q[\log p(y \mid z, \eta)] \]

We observe that with respect to the parameter \(\beta\) nothing changes thus FastSupervisedMStep extends UnsupervisedMStep to delegate part of the maximization to it. Decoration or another type of composition may be a more appropriate form of code reuse in this case.

To maximize with respect to \(\eta\) we use the following equation which amounts to simple logistic regression. The reasons for this approximation are explained in our ACM MM '16 paper (to be linked when published).

\[ \mathcal{L}_{\eta} = \sum_d^D \eta_{y_d}^T \left(\frac{1}{N} \sum_n^{N_d} \phi_{dn}\right) - \sum_d^D \log \sum_{\hat y}^C \text{exp}\left( \eta_{\hat y}^T \left(\frac{1}{N} \sum_n^{N_d} \phi_{dn}\right) \right) \]

This implementation maximizes the above equation using batch gradient descent with ArmijoLineSearch.

Constructor & Destructor Documentation

template<typename Scalar >
ldaplusplus::em::FastSupervisedMStep< Scalar >::FastSupervisedMStep ( size_t  m_step_iterations = 10,
Scalar  m_step_tolerance = 1e-2,
Scalar  regularization_penalty = 1e-2 
m_step_iterationsThe maximum number of gradient descent iterations
m_step_toleranceThe minimum relative improvement between consecutive gradient descent iterations
regularization_penaltyThe L2 penalty for logistic regression

Member Function Documentation

template<typename Scalar >
void ldaplusplus::em::FastSupervisedMStep< Scalar >::doc_m_step ( const std::shared_ptr< corpus::Document doc,
const std::shared_ptr< parameters::Parameters v_parameters,
std::shared_ptr< parameters::Parameters m_parameters 

Delegate the collection of some sufficient statistics to UnsupervisedMStep and keep in memory \(\mathbb{E}_q[\bar z_d] = \frac{1}{N} \sum_n^{N_d} \phi_{dn}\) for use in m_step().

docA single document
v_parametersThe variational parameters used in m-step in order to maximize model parameters
m_parametersModel parameters, used as output in case of online methods

Reimplemented from ldaplusplus::em::UnsupervisedMStep< Scalar >.

Reimplemented in ldaplusplus::em::SemiSupervisedMStep< Scalar >.

template<typename Scalar >
void ldaplusplus::em::FastSupervisedMStep< Scalar >::m_step ( std::shared_ptr< parameters::Parameters parameters)

Maximize the ELBO w.r.t. to \(\beta\) and \(\eta\).

Delegate the maximization regarding \(\beta\) to UnsupervisedMStep and maximize \(\mathcal{L}_{\eta}\) using gradient descent.

parametersModel parameters (changed by this method)

Reimplemented from ldaplusplus::em::UnsupervisedMStep< Scalar >.

The documentation for this class was generated from the following files: