StatLearn 2012 - Workshop on "Challenging problems in Statistical Learning‪"‬ Statlearn2012

Consider the usual regression problem in which we want to study the conditional distribution of a response Y given a set of predictors X. Sufficient dimension reduction (SDR) methods aim at replacing the high-dimensional vector of predictors by a lower-dimensional function R(X) with no loss of information about the dependence of the response variable on the predictors. Almost all SDR methods restrict attention to the class of linear reductions, which can be represented in terms of the projection of X onto a dimension-reduction subspace (DRS). Several methods have been proposed to estimate the basis of the DRS, such as sliced inverse regression (SIR; Li, 1991), principal Hessian directions (PHD; Li, 1992), sliced average variance estimation (SAVE; Cook and Weisberg, 1991), directional regression (DR; Li et al., 2005) and inverse regression estimation (IRE; Cook and Ni, 2005). A novel SDR method, called MSIR, based on finite mixtures of Gaussians has been recently proposed (Scrucca, 2011) as an extension to SIR. The talk will present the MSIR methodology and some recent advances. In particular, a BIC criterion for the selection the dimensionality of DRS will be introduced, and its extension for the purpose of variable selection. Finally, the application of MSIR in classification problems, both supervised and semi-supervised, will be discussed.

Information visualization is a research area that focuses on making structures and content of large and complex data sets visually understandable and interactively analyzable. The goal of information visualization tools and techniques is to increase our ability to gain insight and make decisions for many types of datasets, tasks, and analysis scenarios. With the increase in size and complexity of data sets today, the research area of information visualization increasingly gains in importance and recognition. In this talk I will present principles of data representation and interaction and introduce a number of existing applications, tools, and techniques and show how they can be applied to questions in statistics and statistical learning.

In the early days of kernel machines research, the "kernel trick" was considered a useful way of constructing nonlinear learning algorithms from linear ones, by applying the linear algorithms to feature space mappings of the original data. Recently, it has become clear that a potentially more far reaching use of kernels is as a linear way of dealing with higher order statistics, by mapping probabilities to a suitable reproducing kernel Hilbert space (i.e., the feature space is an RKHS). I will describe how probabilities can be mapped to reproducing kernel Hilbert spaces, and how to compute distances between these mappings. A measure of strength of dependence between two random variables follows naturally from this distance. Applications that make use of kernel probability embeddings include: - Nonparametric two-sample testing and independence testing in complex (high dimensional) domains. As an application, we find whether text in English is translated from the French, as opposed to being random extracts on the same topic. - Bayesian inference, in which the prior and likelihood are represented as feature space mappings, and a posterior feature space mapping is obtained. In this case, Bayesian inference can be undertaken even in the absence of a model, by learning the prior and likelihood mappings from samples.

Functional data are becoming increasingly common in a variety of fields. Many studies underline the importance to consider the representation of data as functions. This has sparked a growing attention in the development of adapted statistical tools that allow to analyze such kind of data : functional data analysis (FDA). The aims of FDA are mainly the same as in the classical statistical analysis, e.g. representing and visualizing the data, studying variability and trends, comparing different data sets, as well as modeling and predicting,... Recent advances in FDA allow to construct different classification methods, based on the comparison between centrality curves or using change points,... We review some procedures that have been used to classify functional data. The main point is to show the good practical behaviors of these procedures on a sample of curves. In addition, theoretical advances on functional estimations related to these classification methods are provided.

A new family of 12 probabilistic models, introduced recently, aims to simultaneously cluster and visualize high-dimensional data. It is based on a mixture model which fits the data into a latent discriminative subspace with an intrinsic dimension bounded by the number of clusters. An estimation procedure, named the Fisher-EM algorithm has also been proposed and turns out to outperform other subspace clustering in most situations. Moreover the convergence properties of the Fisher-EM algorithm are discussed; in particular it is proved that the algorithm is a GEM algorithm and converges under weak conditions in the general case. Finally, a sparse extension of the Fisher-EM algorithm is proposed in order to perform a selection of the original variables which are discriminative.

Cluster analysis is concerned with finding homogeneous groups in a population. Model-based clustering methods provide a framework for developing clustering methods through the use of statistical models. This approach allows for uncertainty to be quantified using probability and for the properties of a clustering method to be understood on the basis of a well defined statistical model. Mixture models provide a basis for many model-based clustering methods. Ranking data arise when judges rank some or all of a set of objects. Examples of ranking data include voting data from elections that use preferential voting systems (eg. PR-STV) and customer preferences for products in marketing applications. A mixture of experts model is a mixture model in which the model parameters are functions of covariates. We explore the use of mixture of experts models in cluster analysis, so that clustering can be better understood. The choice of how and where covariates enter the mixture of experts model has implications for the clustering performance and the interpretation of the results. The use of covariates in clustering is demonstrated on examples from studying voting blocs in elections and examining customer segments marketing.