LDA++
Public Member Functions | List of all members
ldaplusplus::em::CorrespondenceSupervisedEStep< Scalar > Class Template Reference

#include <CorrespondenceSupervisedEStep.hpp>

Inheritance diagram for ldaplusplus::em::CorrespondenceSupervisedEStep< Scalar >:
ldaplusplus::em::AbstractEStep< Scalar > ldaplusplus::em::EStepInterface< Scalar > ldaplusplus::events::EventDispatcherComposition

Public Member Functions

 CorrespondenceSupervisedEStep (size_t e_step_iterations=10, Scalar e_step_tolerance=1e-2, Scalar mu=2., Scalar compute_likelihood=1.0, int random_state=0)
 
std::shared_ptr< parameters::Parametersdoc_e_step (const std::shared_ptr< corpus::Document > doc, const std::shared_ptr< parameters::Parameters > parameters) override
 
- Public Member Functions inherited from ldaplusplus::em::AbstractEStep< Scalar >
 AbstractEStep (int random_state)
 
virtual void e_step () override
 
- Public Member Functions inherited from ldaplusplus::events::EventDispatcherComposition
std::shared_ptr< EventDispatcherInterfaceget_event_dispatcher ()
 
void set_event_dispatcher (std::shared_ptr< EventDispatcherInterface > dispatcher)
 

Additional Inherited Members

- Protected Member Functions inherited from ldaplusplus::em::AbstractEStep< Scalar >
bool converged (const Eigen::Matrix< Scalar, Eigen::Dynamic, 1 > &gamma_old, const Eigen::Matrix< Scalar, Eigen::Dynamic, 1 > &gamma, Scalar tolerance)
 
PRNGget_prng ()
 

Detailed Description

template<typename Scalar>
class ldaplusplus::em::CorrespondenceSupervisedEStep< Scalar >

CorrespondenceSupervisedEStep implements the expectation step of a variant of the correspondence LDA model as it was introduced in [1]. Iinstead of trying to generate labels it tries to generate the class of the document.

The generative process according to this model is summarized below:

  1. Given \(K\) \(V\)-dimensional multinomial distributions as the topics ( \( \beta\)) and \(K\) \(C\)-dimensional multinomial distributions ( \( \eta\)) to sample the class labels from.
  2. For each of the D documents:
    1. Sample from a Dirichlet distribution with Dirichlet prior \( \alpha \) and create the topic distribution for document d, \( \theta_d \sim Dir\left(\alpha\right)\).
    2. For each of the \(N\) words \(w_n\):
      1. Sample a topic \( z_n \sim Mult\left( \theta \right)\)
      2. From that topic sample a word using the \(k\)th \(V\)-dimensional multinomial distribution, namely \(w_n \sim p(w \mid z_n, \beta)\)
    3. From a uniform distribution sample a number \(n\) between \(1\) and \(N\), namely \( \lambda_d \sim Unif\left(1...N\right)\).
    4. Sample a class label for the \(d \) document from a multinomial distribution, namely \( y_d \sim p\left( y \mid \lambda, z, \eta \right)\).

Exact probabilistic inference for this model is intractable, thus we use variational inference methods. The factorized distribution on the latent variables is the following.

\( q\left(\theta, z, \lambda \right) = q\left(\theta \mid \gamma\right)\left( \prod_{n=1}^N q\left(z_n \mid \phi_n \right)\right)q\left(\lambda \mid \tau \right)\)

[1] Blei, D.M. and Jordan, M.I., 2003, July. Modeling annotated data. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval (pp. 127-134). ACM.

Constructor & Destructor Documentation

template<typename Scalar >
ldaplusplus::em::CorrespondenceSupervisedEStep< Scalar >::CorrespondenceSupervisedEStep ( size_t  e_step_iterations = 10,
Scalar  e_step_tolerance = 1e-2,
Scalar  mu = 2.,
Scalar  compute_likelihood = 1.0,
int  random_state = 0 
)
Parameters
e_step_iterationsThe max number of times to alternate between maximizing for \(\gamma\) and for \(\phi\).
e_step_toleranceThe minimum relative change in the variational parameter \(\gamma\).
muThe uniform Dirichlet prior of \(\eta\), practically is a smoothing parameter during the maximization of \(\eta\).
compute_likelihoodThe percentage of documents to compute likelihood for (1.0 means compute for every document)
random_stateAn initial seed value for any random numbers needed

Member Function Documentation

template<typename Scalar >
std::shared_ptr< parameters::Parameters > ldaplusplus::em::CorrespondenceSupervisedEStep< Scalar >::doc_e_step ( const std::shared_ptr< corpus::Document doc,
const std::shared_ptr< parameters::Parameters parameters 
)
overridevirtual

Maximize the ELBO w.r.t. \(\phi\) and \(\gamma\).

The following steps are the mathematics that are implemented where \(\beta\) are the over words topics distributions, \(\alpha\) is the Dirichlet prior, \(\eta\) are the logistic regression parameters, \(\tau\) are the \(N\)-dimensional mutltinomial parameters, \(i\) is the topic subscript, \(n\) is the word subscript, \(\hat{y}\) is the class subscript, \(y\) is the document's class, \(w_n\) is n-th word vocabulary index, and finally \(\Psi(\cdot)\) is the first derivative of the \(\log \Gamma\) function.

  1. Repeat until convergence of \(\gamma\).
  2. Compute \(\phi_{ni} \propto \beta_{iw_n} \exp\left( \Psi(\gamma_i) + \tau_{ni} * \log\left( \eta_{yi} \right) \right)\)
  3. Compute \(\tau_{ni} \propto \exp\left( \sum_{i=1}^K \phi_{ni} * \log\left( \eta_{yi}\right)\right)\)
  4. Compute \(\gamma_i = \alpha_i + \sum_n^N \phi_{ni} \)
Parameters
docA single document.
parametersAn instance of class Parameters, which contains all necessary model parameters for e-step's implementation.
Returns
The variational parameters for the current model, after e-step is completed.

Implements ldaplusplus::em::EStepInterface< Scalar >.


The documentation for this class was generated from the following files: