• Clustering of high-dimensional categorical data.
• LCM: mixture models that assign the set of multivariate categorical observations to a latent class \( z \). Within each \( z \) the observed variables are statistically independent.
• LCM estimates the class probability \( \lambda \) and the probability of observing a particular response for a question conditioned on the latent class.

• \( \alpha,\beta \) are hyper-parameters, govern the sparsity of the model.
• \( \lambda | \alpha \) the size of the classes in simple LCM or non-parametric LCM.
• \( z \) contains the latent class assignment for each individual.
• \( U \) 3-way tensor of size \( J \times K \times R\). Contains the probability for response - content the word - \( r \) from an individual - a document - from class \( k \) for question - the word - \( j \).
• \( X_{j,k} | U_{j,z_i=k} \) specifies that the response of an individual \( i \) that belongs to a class \( k \) is drawn from a Multinomial distribution according to the probability vector \(U_{j,k}\).

Simple LCM

$$ p(\lambda ,z,U,X | \alpha, \beta) = p( \lambda |\alpha) \prod_{i=1}^{I}p(z_i | \lambda) \prod_{j=1}^{J}\prod_{k=1}^{K} p(U_{j,k} | \beta)\prod_{i=1}^{I} \prod_{j=1}^{J} \prod_{k=1}^{K} p(X_{i,j}|U{j,k})^{I(z_i=k)}$$

Nonparametric LCM

$$ p(\lambda ,z,U,X | \alpha, \beta) = \prod_{k=1}^{K_{max}-1}p( \nu_k|\alpha) \prod_{i=1}^{I}p(z_i | \lambda) \prod_{j=1}^{J}\prod_{k=1}^{K} p(U_{j,k} | \beta)\prod_{i=1}^{I} \prod_{j=1}^{J} \prod_{k=1}^{K} p(X_{i,j}|U{j,k})^{I(z_i=k)}$$

C. Ahlmann-Eltze and C. Yau, MixDir: Scalable Bayesian Clustering for High-Dimensional Categorical Data, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), Turin, Italy, 2018, pp. 526-539.

\( ARI \) = Adjusted Rank Index se utiliza para comparar los resultados de clustering. Para cada documento se estima la probabilidad de pertenecer a cada clase (soft clustering). Para obtener \( ARI \) cada documento ha sido asignado a la clase con mayor probabilidad.

El alluvial plot muestra el efecto sobre la agrupación al incrementar la penalización de la creación de nuevas clases - \( \alpha_1 \) en el Proceso de Dirichlet como prior.

Como referencia, utilizamos \( K_{max} = 5 \). Las clases pequeñas se unifican según crece el valor del parámetro.

• Longitud - número de palabras sin stopwords - de los documentos en cada Clase.
• La longitud no se utiliza directamente en el modelo.
• Unsupervised clustering can help uncover interesting underlying structures. One must be careful not to over-interpret the data …