StatLearn 2013 - Workshop on "Challenging problems in Statistical Learning‪"‬ Statlearn2013

The exponential random graph is arguably the most popular model for the statistical analysis of network data. However despite its widespread use, it is very complicated to handle from a statistical perspective, mainly because the likelihood function is intractable for all but trivially small networks. This talk will outline some recent work in this area to overcome this intractability. In particular, we will outline some approaches to carry out Bayesian parameter estimation and show how this can be extended to estimate Bayes factors between competing models.

The main goal of this work is to tackle the problem of dimension reduction for highdimensional supervised classification. The motivation is to handle gene expression data. The proposed method works in 2 steps. First, one eliminates redundancy using clustering of variables, based on the R-package ClustOfVar. This first step is only based on the exploratory variables (genes). Second, the synthetic variables (summarizing the clusters obtained at the first step) are used to construct a classifier (e.g. logistic regression, LDA, random forests). We stress that the first step reduces the dimension and gives linear combinations of original variables (synthetic variables). This step can be considered as an alternative to PCA. A selection of predictors (synthetic variables) in the second step gives a set of relevant original variables (genes). Numerical performances of the proposed procedure are evaluated on gene expression datasets. We compare our methodology with LASSO and sparse PLS discriminant analysis on these datasets.

When an unbiased estimator of the likelihood is used within an Markov chain Monte Carlo (MCMC) scheme, it is necessary to tradeoff the number of samples used against the computing time. Many samples for the estimator will result in a MCMC scheme which has similar properties to the case where the likelihood is exactly known but will be expensive. Few samples for the construction of the estimator will result in faster estimation but at the expense of slower mixing of the Markov chain.We explore the relationship between the number of samples and the efficiency of the resulting MCMC estimates. Under specific assumptions about the likelihood estimator, we are able to provide guidelines on the number of samples to select for a general Metropolis-Hastings proposal.We provide theory which justifies the use of these assumptions for a large class of models. On a number of examples, we find that the assumptions on the likelihood estimator are accurate. This is joint work with Mike Pitt (University of Warwick) and Robert Kohn (UNSW).

The high dimensional setting is a modern and dynamic research area in Statistics. It covers numerous situations where the number of explanatory variables is much larger than the sample size. This is the case in genomics when one observes (dozens of) thousands genes expression ; typically one has at hand a small sample of high dimensioned vectors derived from a large set of covariates. Such datasets will be abbreviated to HDD-I for High Dimensional Data of type I. A particular setting may correspond to the observation of a collection of curves, surfaces, ... sampled at high frequencies (design points) ; these sets of data are gathered under the terminology functional data (or functional variables) and will be abbreviated to HDD-II (High Dimensional Data of type II). The main feature of HDD-II (and difference with HDD-I) is due to the existence of high colinearities between explanatory variables which reduces the overall dimensionality of the data. Last twenty years have been devoted to develop successful methodologies able to manage such high dimensional data. Essentially sparse linear modelling involving variable selection techniques has been proposed to investigate on HDD-I whereas non selective functional linear approaches have been introduced to handle HDD-II mainly. However, as in the standard multivariate setting, linear assumption may too much restrictive by hiding relevant nonlinear aspects. This is why in the last decade flexible methodologies taking into account nonlinear relationship have been developed to better understand the structure of such high dimensional data. So, the aim of this talk is to present and illustrate on various examples recent approaches connecting nonparametric, selective and functional techniques in order to handle nonlinear relationship in HDD-I or HDD-II settings which allow us to tackle various challenging issues.

The Online Expectation-Maximization (EM) is a generic algorithm that can be used to estimate the parameters of latent data models incrementally from large volumes of data. The general principle of the approach is to use a stochastic approximation scheme, in the domain of sufficient statistics, as a proxy for a limiting, deterministic, population version of the EM recursion. In this talk, I will briefly review the convergence properties of the method and discuss some applications and extensions of the basic approach.

This work is motivated by the challenges of drawing inferences from presence-only data. For example, when trying to determine what habitat sea-turtles "prefer" we only have data on where turtles were observed, not data about where the turtles actually are. Therefore, if we find that our sample contains very few turtles living in regions with tall sea grass, we cannot conclude that these areas are unpopular with the turtles, merely that we are unlikely to observe them there. Similar issues arise in forensic accounting : attempts to determine which companies are apt to misreport their official earnings based on a history of which firms were censured by the SEC are confounded by the fact that we only observe which firms got caught cheating, not which firms cheat (many of whom do not get caught). This sort of confounding is insurmountable from a point-estimation perspective, but the data are not entirely uninformative either. Our present work is devoted to parametrizing observation models in a way that isolates which aspects of the model are informed by the data and which aspects are not. This approach allows us to construct priors which are informative with respect to the unidentified parts of the model without simultaneously (and unintentionally) biasing posterior estimates of the identified parameters ; these priors do not "fight against" the data. In addition, their modularity allows for convenient sensitivity analysis in order to examine the extent to which our ultimate conclusions are driven by prior assumptions as opposed to our data. Joint work with Richard Hahn and Jared Murray.