Header logo is


2010


no image
A Maximum Entropy Approach to Semi-supervised Learning

Erkan, A., Altun, Y.

30th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering (MaxEnt 2010), 30, pages: 80, July 2010 (poster)

Abstract
Maximum entropy (MaxEnt) framework has been studied extensively in supervised learning. Here, the goal is to find a distribution p that maximizes an entropy function while enforcing data constraints so that the expected values of some (pre-defined) features with respect to p match their empirical counterparts approximately. Using different entropy measures, different model spaces for p and different approximation criteria for the data constraints yields a family of discriminative supervised learning methods (e.g., logistic regression, conditional random fields, least squares and boosting). This framework is known as the generalized maximum entropy framework. Semi-supervised learning (SSL) has emerged in the last decade as a promising field that combines unlabeled data along with labeled data so as to increase the accuracy and robustness of inference algorithms. However, most SSL algorithms to date have had trade-offs, e.g., in terms of scalability or applicability to multi-categorical data. We extend the generalized MaxEnt framework to develop a family of novel SSL algorithms. Extensive empirical evaluation on benchmark data sets that are widely used in the literature demonstrates the validity and competitiveness of the proposed algorithms.

ei

PDF PDF [BibTex]

2010


PDF PDF [BibTex]


no image
The effect of positioning aids on PET quantification following MR-based attenuation correction (AC) in PET/MR imaging

Mantlik, F., Hofmann, M., Kupferschläger, J., Werner, M., Pichler, B., Beyer, T.

Journal of Nuclear Medicine, 51(Supplement 2):1418 , June 2010 (poster)

Abstract
Objectives: We study the quantitative effect of not accounting for the attenuation of patient positioning aids in combined PET/MR imaging. Methods: Positioning aids cannot be detected with conventional MR sequences. We mimic this effect using PET/CT data (Biograph HiRez16) with the foams removed from CT images prior to using them for CT-AC. PET/CT data were acquired using standard parameters (phantoms/patients): 120/140 kVp, 30/250 mAs, 5 mm slices, OSEM (4i, 8s, 5 mm filter) following CT-AC. First, a uniform 68Ge-cylinder was positioned centrally in the PET/CT and fixed with a vacuum mattress (10 cm thick). Second, the same cylinder was placed in 3 positioning aids from the PET/MR (BrainPET-3T). Third, 5 head/neck patients who were fixed in a vacuum mattress were selected. In all 3 studies PET recon post CT-AC based on measured CT images was used as the reference (mCT-AC). The PET/MR set-up was mimicked by segmenting the foam inserts from the measured CT images and setting their voxel values to -1000 HU (air). PET images were reconstructed using CT-AC with the segmented CT images (sCT-AC). PET images with mCT- and sCT-AC were compared. Results: sCT-AC underestimated PET voxel values in the phantom by 6.7% on average compared to mCT-AC with the vacuum mattress in place. 5% of the PET voxels were underestimated by >=10%. Not accounting for MR positioning aids during AC led to an underestimation of 2.8% following sCT-AC, with 5% of the PET voxels being underestimated by >=7% wrt mCT-AC. Preliminary evaluation of the patient data indicates a slightly higher bias from not accounting for patient positioning aids (mean: -9.1%, 5% percentile: -11.2%). Conclusions: A considerable and regionally variable underestimation of the PET activity following AC is observed when positioning aids are not accounted for. This bias may become relevant in neurological activation or dementia studies with PET/MR

ei

Web [BibTex]

Web [BibTex]


no image
Justifying Additive Noise Model-Based Causal Discovery via Algorithmic Information Theory

Janzing, D., Steudel, B.

Open Systems and Information Dynamics, 17(2):189-212, June 2010 (article)

Abstract
A recent method for causal discovery is in many cases able to infer whether X causes Y or Y causes X for just two observed variables X and Y. It is based on the observation that there exist (non-Gaussian) joint distributions P(X,Y) for which Y may be written as a function of X up to an additive noise term that is independent of X and no such model exists from Y to X. Whenever this is the case, one prefers the causal model X → Y. Here we justify this method by showing that the causal hypothesis Y → X is unlikely because it requires a specific tuning between P(Y) and P(X|Y) to generate a distribution that admits an additive noise model from X to Y. To quantify the amount of tuning, needed we derive lower bounds on the algorithmic information shared by P(Y) and P(X|Y). This way, our justification is consistent with recent approaches for using algorithmic information theory for causal reasoning. We extend this principle to the case where P(X,Y) almost admits an additive noise model. Our results suggest that the above conclusion is more reliable if the complexity of P(Y) is high.

ei

PDF Web DOI [BibTex]


no image
Multi-task Learning for Zero Training Brain-Computer Interfaces

Alamgir, M., Grosse-Wentrup, M., Altun, Y.

4th International BCI Meeting, June 2010 (poster)

Abstract
Brain-computer interfaces (BCIs) are limited in their applicability in everyday settings by the current necessity to record subject-specific calibration data prior to actual use of the BCI for communication. In this work, we utilize the framework of multitask learning to construct a BCI that can be used without any subject-specific calibration process, i.e., with zero training data. In BCIs based on EEG or MEG, the predictive function of a subject's intention is commonly modeled as a linear combination of some features derived from spatial and spectral recordings. The coefficients of this combination correspond to the importance of the features for predicting the intention of the subject. These coefficients are usually learned separately for each subject due to inter-subject variability. Principle feature characteristics, however, are known to remain invariant across subject. For example, it is well known that in motor imagery paradigms spectral power in the mu- and beta frequency ranges (roughly 8-14 Hz and 20-30 Hz, respectively) over sensorimotor areas provides most information on a subject's intention. Based on this assumption, we define the intention prediction function as a combination of subject-invariant and subject-specific models, and propose a machine learning method that infers these models jointly using data from multiple subjects. This framework leads to an out-of-the-box intention predictor, where the subject-invariant model can be employed immediately for a subject with no prior data. We present a computationally efficient method to further improve this BCI to incorporate subject-specific variations as such data becomes available. To overcome the problem of high dimensional feature spaces in this context, we further present a new method for finding the relevance of different recording channels according to actions performed by subjects. Usually, the BCI feature representation is a concatenation of spectral features extracted from different channels. This representation, however, is redundant, as recording channels at different spatial locations typically measure overlapping sources within the brain due to volume conduction. We address this problem by assuming that the relevance of different spectral bands is invariant across channels, while learning different weights for each recording electrode. This framework allows us to significantly reduce the feature space dimensionality without discarding potentially useful information. Furthermore, the resulting out-of-the-box BCI can be adapted to different experimental setups, for example EEG caps with different numbers of channels, as long as there exists a mapping across channels in different setups. We demonstrate the feasibility of our approach on a set of experimental EEG data recorded during a standard two-class motor imagery paradigm from a total of ten healthy subjects. Specifically, we show that satisfactory classification results can be achieved with zero training data, and that combining prior recordings with subject-specific calibration data substantially outperforms using subject-specific data only.

ei

Web [BibTex]


no image
Causal Influence of Gamma Oscillations on Performance in Brain-Computer Interfaces

Grosse-Wentrup, M., Hill, J., Schölkopf, B.

4th International BCI Meeting0, June 2010 (poster)

Abstract
Background and Objective: While machine learning approaches have led to tremendous advances in brain-computer interfaces (BCIs) in recent years (cf. [1]), there still exists a large variation in performance across subjects. Furthermore, a significant proportion of subjects appears incapable of achieving above chance-level classification accuracy [2], which to date includes all subjects in a completely locked-in state that have been trained in BCI control. Understanding the reasons for this variation in performance arguably constitutes one of the most fundamental open questions in research on BCIs. Methods & Results Using a machine learning approach, we derive a trial-wise measure of how well EEG recordings can be classified as either left- or right-hand motor imagery. Specifically, we train a support vector machine (SVM) on log-bandpower features (7-40 Hz) derived from EEG channels after spatial filtering with a surface Laplacian, and then compute the trial-wise distance of the output of the SVM from the separating hyperplane using a cross-validation procedure. We then correlate this trial-wise performance measure, computed on EEG recordings of ten healthy subjects, with log-bandpower in the gamma frequency range (55-85 Hz), and demonstrate that it is positively correlated with frontal- and occipital gamma-power and negatively correlated with centro-parietal gamma-power. This correlation is shown to be highly significant on the group level as well as in six out of ten subjects on the single-subject level. We then utilize the framework for causal inference developed by Pearl, Spirtes and others [3,4] to present evidence that gamma-power is not only correlated with BCI performance but does indeed exert a causal influence on it. Discussion and Conclusions Our results indicate that successful execution of motor imagery, and hence reliable communication by means of a BCI based on motor imagery, requires a volitional shift of gamma-power from centro-parietal to frontal and occipital regions. As such, our results provide the first non-trivial explanation for the variation in BCI performance across and within subjects. As this topographical alteration in gamma-power is likely to correspond to a specific attentional shift, we propose to provide subjects with feedback on their topographical distribution of gamma-power in order to establish the attentional state required for successful execution of motor imagery.

ei

Web [BibTex]


no image
Solving large-scale nonnegative least-squares

Sra, S.

16th Conference of the International Linear Algebra Society (ILAS 2010), 16, pages: 19, June 2010, based on Joint work with Dongmin Kim and Inderjit Dhillon (poster)

Abstract
We study the fundamental problem of nonnegative least squares. This problem was apparently introduced by Lawson and Hanson [1] under the name NNLS. As is evident from its name, NNLS seeks least-squares solutions that are also nonnegative. Owing to its wide-applicability numerous algorithms have been derived for NNLS, beginning from the active-set approach of Lawson and Hanson [1] leading up to the sophisticated interior-point method of Bellavia et al. [2]. We present a new algorithm for NNLS that combines projected subgradients with the non-monotonic gradient descent idea of Barzilai and Borwein [3]. Our resulting algorithm is called BBSG, and we guarantee its convergence by exploiting properties of NNLS in conjunction with projected subgradients. BBSG is surprisingly simple and scales well to large problems. We substantiate our claims by empirically evaluating BBSG and comparing it with established convex solvers and specialized NNLS algorithms. The numerical results suggest that BBSG is a practical method for solving large-scale NNLS problems.

ei

PDF Web [BibTex]

PDF Web [BibTex]


no image
Dynamic Dissimilarity Measure for Support-Based Clustering

Lee, D., Lee, J.

IEEE Transactions on Knowledge and Data Engineering, 22(6):900-905, June 2010 (article)

Abstract
Clustering methods utilizing support estimates of a data distribution have recently attracted much attention because of their ability to generate cluster boundaries of arbitrary shape and to deal with outliers efficiently. In this paper, we propose a novel dissimilarity measure based on a dynamical system associated with support estimating functions. Theoretical foundations of the proposed measure are developed and applied to construct a clustering method that can effectively partition the whole data space. Simulation results demonstrate that clustering based on the proposed dissimilarity measure is robust to the choice of kernel parameters and able to control the number of clusters efficiently.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Sparse Spectrum Gaussian Process Regression

Lázaro-Gredilla, M., Quiñonero-Candela, J., Rasmussen, CE., Figueiras-Vidal, AR.

Journal of Machine Learning Research, 11, pages: 1865-1881, June 2010 (article)

Abstract
We present a new sparse Gaussian Process (GP) model for regression. The key novel idea is to sparsify the spectral representation of the GP. This leads to a simple, practical algorithm for regression tasks. We compare the achievable trade-offs between predictive accuracy and computational requirements, and show that these are typically superior to existing state-of-the-art sparse approximations. We discuss both the weight space and function space representations, and note that the new construction implies priors over functions which are always stationary, and can approximate any covariance function in this class.

ei

PDF [BibTex]

PDF [BibTex]


no image
Simultaneous PET/MRI for the evaluation of hemato-oncological diseases with lower extremity manifestations

Sauter, A., Horger, M., Boss, A., Kolb, A., Mantlik, F., Kanz, L., Pfannenberg, C., Stegger, L., Claussen, C., Pichler, B.

Journal of Nuclear Medicine, 51(Supplement 2):1001 , June 2010 (poster)

Abstract
Objectives: The study purpose is the evaluation of patients, suffering from hemato-oncological disease with complications at the lower extremities, using simultaneous PET/MRI. Methods: Until now two patients (chronic active graft-versus-host-disease [GvHD], B-non Hodgkin lymphoma [B-NHL]) before and after therapy were examined in a 3-Tesla-BrainPET/MRI hybrid system following F-18-FDG-PET/CT. Simultaneous static PET (1200 sec.) and MRI scans (T1WI, T2WI, post-CA) were acquired. Results: Initial results show the feasibility of using hybrid PET/MRI-technology for musculoskeletal imaging of the lower extremities. Simultaneous PET and MRI could be acquired in diagnostic quality. Before treatment our patient with GvHD had a high fascia and muscle FDG uptake, possibly due to muscle encasement. T2WI and post gadolinium T1WI revealed a fascial thickening and signs of inflammation. After therapy with steroids followed by imatinib the patient’s symptoms improved while, the muscular FDG uptake droped whereas the MRI signal remained unchanged. We assume that fascial elasticity improved during therapy despite persistance of fascial thickening. The examination of the second patient with B-NHL manifestation in the tibia showed a significant signal and uptake decrease in the bone marrow and surrounding lesions in both, MRI and PET after therapy with rituximab. The lack of residual FDG-uptake proved superior to MRI information alone helping for exclusion of vital tumor. Conclusions: Combined PET/MRI is a powerful tool to monitor diseases requiring high soft tissue contrast along with molecular information from the FDG uptake.

ei

Web [BibTex]

Web [BibTex]


no image
Unsupervised Object Discovery: A Comparison

Tuytelaars, T., Lampert, CH., Blaschko, MB., Buntine, W.

International Journal of Computer Vision, 88(2):284-302, June 2010 (article)

Abstract
The goal of this paper is to evaluate and compare models and methods for learning to recognize basic entities in images in an unsupervised setting. In other words, we want to discover the objects present in the images by analyzing unlabeled data and searching for re-occurring patterns. We experiment with various baseline methods, methods based on latent variable models, as well as spectral clustering methods. The results are presented and compared both on subsets of Caltech256 and MSRC2, data sets that are larger and more challenging and that include more object classes than what has previously been reported in the literature. A rigorous framework for evaluating unsupervised object discovery methods is proposed.

ei

PDF DOI [BibTex]

PDF DOI [BibTex]


no image
How to Explain Individual Classification Decisions

Baehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M., Hansen, K., Müller, K.

Journal of Machine Learning Research, 11, pages: 1803-1831, June 2010 (article)

Abstract
After building a classifier with modern tools of machine learning we typically have a black box at hand that is able to predict well for unseen data. Thus, we get an answer to the question what is the most likely label of a given unseen data point. However, most methods will provide no answer why the model predicted a particular label for a single instance and what features were most influential for that particular instance. The only method that is currently able to provide such explanations are decision trees. This paper proposes a procedure which (based on a set of assumptions) allows to explain the decisions of any classification method.

ei

PDF PDF [BibTex]

PDF PDF [BibTex]


no image
Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior

Kim, K., Kwon, Y.

IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(6):1127-1133, June 2010 (article)

Abstract
This paper proposes a framework for single-image super-resolution. The underlying idea is to learn a map from input low-resolution images to target high-resolution images based on example pairs of input and output images. Kernel ridge regression (KRR) is adopted for this purpose. To reduce the time complexity of training and testing for KRR, a sparse solution is found by combining the ideas of kernel matching pursuit and gradient descent. As a regularized solution, KRR leads to a better generalization than simply storing the examples as has been done in existing example-based algorithms and results in much less noisy images. However, this may introduce blurring and ringing artifacts around major edges as sharp changes are penalized severely. A prior model of a generic image class which takes into account the discontinuity property of images is adopted to resolve this problem. Comparison with existing algorithms shows the effectiveness of the proposed method.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Imitation and Reinforcement Learning

Kober, J., Peters, J.

IEEE Robotics and Automation Magazine, 17(2):55-62, June 2010 (article)

Abstract
In this article, we present both novel learning algorithms and experiments using the dynamical system MPs. As such, we describe this MP representation in a way that it is straightforward to reproduce. We review an appropriate imitation learning method, i.e., locally weighted regression, and show how this method can be used both for initializing RL tasks as well as for modifying the start-up phase in a rhythmic task. We also show our current best-suited RL algorithm for this framework, i.e., PoWER. We present two complex motor tasks, i.e., ball-in-a-cup and ball paddling, learned on a real, physical Barrett WAM, using the methods presented in this article. Of particular interest is the ball-paddling application, as it requires a combination of both rhythmic and discrete dynamical systems MPs during the start-up phase to achieve a particular task.

ei

PDF Web DOI [BibTex]

PDF Web DOI [BibTex]


no image
Diffusion Tensor Imaging in a Human PET/MR Hybrid System

Boss, A., Kolb, A., Hofmann, M., Bisdas, S., Nägele, T., Ernemann, U., Stegger, L., Rossi, C., Schlemmer, H., Pfannenberg, C., Reimold, M., Claussen, C., Pichler, B., Klose, U.

Investigative Radiology, 45(5):270-274, May 2010 (article)

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
A Bayesian Framework to Account for Complex Non-Genetic Factors in Gene Expression Levels Greatly Increases Power in eQTL Studies

Stegle, O., Parts, L., Durbin, R., Winn, JM.

PLoS Computational Biology, 6(5):1-11, May 2010 (article)

ei

PDF Web DOI [BibTex]

PDF Web DOI [BibTex]


no image
Estimation of a Structural Vector Autoregression Model Using Non-Gaussianity

Hyvärinen, A., Zhang, K., Shimizu, S., Hoyer, P.

Journal of Machine Learning Research, 11, pages: 1709-1731, May 2010 (article)

Abstract
Analysis of causal effects between continuous-valued variables typically uses either autoregressive models or structural equation models with instantaneous effects. Estimation of Gaussian, linear structural equation models poses serious identifiability problems, which is why it was recently proposed to use non-Gaussian models. Here, we show how to combine the non-Gaussian instantaneous model with autoregressive models. This is effectively what is called a structural vector autoregression (SVAR) model, and thus our work contributes to the long-standing problem of how to estimate SVAR‘s. We show that such a non-Gaussian model is identifiable without prior knowledge of network structure. We propose computationally efficient methods for estimating the model, as well as methods to assess the significance of the causal influences. The model is successfully applied on financial and brain imaging data.

ei

PDF Web [BibTex]

PDF Web [BibTex]


no image
A Robust Bayesian Two-Sample Test for Detecting Intervals of Differential Gene Expression in Microarray Time Series

Stegle, O., Denby, KJ., Cooke, EJ., Wild, DL., Ghahramani, Z., Borgwardt, KM.

Journal of Computational Biology, 17(3):355-367, May 2010 (article)

Abstract
Understanding the regulatory mechanisms that are responsible for an organism‘s response to environmental change is an important issue in molecular biology. A first and important step towards this goal is to detect genes whose expression levels are affected by altered external conditions. A range of methods to test for differential gene expression, both in static as well as in time-course experiments, have been proposed. While these tests answer the question whether a gene is differentially expressed, they do not explicitly address the question when a gene is differentially expressed, although this information may provide insights into the course and causal structure of regulatory programs. In this article, we propose a two-sample test for identifying intervals of differential gene expression in microarray time series. Our approach is based on Gaussian process regression, can deal with arbitrary numbers of replicates, and is robust with respect to outliers. We apply our algorithm to study the response of Arabidopsis thaliana genes to an infection by a fungal pathogen using a microarray time series dataset covering 30,336 gene probes at 24 observed time points. In classification experiments, our test compares favorably with existing methods and provides additional insights into time-dependent differential expression.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Statistical Tests for Detecting Differential RNA-Transcript Expression from Read Counts

Stegle, O., Drewe, P., Bohnert, R., Borgwardt, K., Rätsch, G.

Nature Precedings, 2010, pages: 1-11, May 2010 (article)

Abstract
As a fruit of the current revolution in sequencing technology, transcriptomes can now be analyzed at an unprecedented level of detail. These advances have been exploited for detecting differential expressed genes across biological samples and for quantifying the abundances of various RNA transcripts within one gene. However, explicit strategies for detecting the hidden differential abundances of RNA transcripts in biological samples have not been defined. In this work, we present two novel statistical tests to address this issue: a "gene structure sensitive" Poisson test for detecting differential expression when the transcript structure of the gene is known, and a kernel-based test called Maximum Mean Discrepancy when it is unknown. We analyzed the proposed approaches on simulated read data for two artificial samples as well as on factual reads generated by the Illumina Genome Analyzer for two C. elegans samples. Our analysis shows that the Poisson test identifies genes with differential transcript expression considerably better that previously proposed RNA transcript quantification approaches for this task. The MMD test is able to detect a large fraction (75%) of such differential cases without the knowledge of the annotated transcripts. It is therefore well-suited to analyze RNA-Seq experiments when the genome annotations are incomplete or not available, where other approaches have to fail.

ei

PDF DOI [BibTex]

PDF DOI [BibTex]


no image
Parameter-exploring policy gradients

Sehnke, F., Osendorfer, C., Rückstiess, T., Graves, A., Peters, J., Schmidhuber, J.

Neural Networks, 21(4):551-559, May 2010 (article)

Abstract
We present a model-free reinforcement learning method for partially observable Markov decision problems. Our method estimates a likelihood gradient by sampling directly in parameter space, which leads to lower variance gradient estimates than obtained by regular policy gradient methods. We show that for several complex control tasks, including robust standing with a humanoid robot, this method outperforms well-known algorithms from the fields of standard policy gradients, finite difference methods and population based heuristics. We also show that the improvement is largest when the parameter samples are drawn symmetrically. Lastly we analyse the importance of the individual components of our method by incrementally incorporating them into the other algorithms, and measuring the gain in performance after each step.

ei

PDF PDF DOI [BibTex]

PDF PDF DOI [BibTex]


no image
Temporal Kernel CCA and its Application in Multimodal Neuronal Data Analysis

Biessmann, F., Meinecke, F., Gretton, A., Rauch, A., Rainer, G., Logothetis, N., Müller, K.

Machine Learning, 79(1-2):5-27, May 2010 (article)

Abstract
Data recorded from multiple sources sometimes exhibit non-instantaneous couplings. For simple data sets, cross-correlograms may reveal the coupling dynamics. But when dealing with high-dimensional multivariate data there is no such measure as the cross-correlogram. We propose a simple algorithm based on Kernel Canonical Correlation Analysis (kCCA) that computes a multivariate temporal filter which links one data modality to another one. The filters can be used to compute a multivariate extension of the cross-correlogram, the canonical correlogram, between data sources that have different dimensionalities and temporal resolutions. The canonical correlogram reflects the coupling dynamics between the two sources. The temporal filter reveals which features in the data give rise to these couplings and when they do so. We present results from simulations and neuroscientific experiments showing that tkCCA yields easily interpretable temporal filters and correlograms. In the experiments, we simultaneously performed electrode recordings and functional magnetic resonance imaging (fMRI) in primary visual cortex of the non-human primate. While electrode recordings reflect brain activity directly, fMRI provides only an indirect view of neural activity via the Blood Oxygen Level Dependent (BOLD) response. Thus it is crucial for our understanding and the interpretation of fMRI signals in general to relate them to direct measures of neural activity acquired with electrodes. The results computed by tkCCA confirm recent models of the hemodynamic response to neural activity and allow for a more detailed analysis of neurovascular coupling dynamics.

ei

PDF PDF DOI [BibTex]

PDF PDF DOI [BibTex]


no image
Estimating predictive stimulus features from psychophysical data: The decision image technique applied to human faces

Macke, J., Wichmann, F.

Journal of Vision, 10(5:22):1-24, May 2010 (article)

Abstract
One major challenge in the sensory sciences is to identify the stimulus features on which sensory systems base their computations, and which are predictive of a behavioral decision: they are a prerequisite for computational models of perception. We describe a technique (decision images) for extracting predictive stimulus features using logistic regression. A decision image not only defines a region of interest within a stimulus but is a quantitative template which defines a direction in stimulus space. Decision images thus enable the development of predictive models, as well as the generation of optimized stimuli for subsequent psychophysical investigations. Here we describe our method and apply it to data from a human face classification experiment. We show that decision images are able to predict human responses not only in terms of overall percent correct but also in terms of the probabilities with which individual faces are (mis-) classified by individual observers. We show that the most predictive dimension for gender categorization is neither aligned with the axis defined by the two class-means, nor with the first principal component of all faces-two hypotheses frequently entertained in the literature. Our method can be applied to a wide range of binary classification tasks in vision or other psychophysical contexts.

ei

Web DOI [BibTex]


no image
Animal detection in natural scenes: Critical features revisited

Wichmann, F., Drewes, J., Rosas, P., Gegenfurtner, K.

Journal of Vision, 10(4):1-27, April 2010 (article)

Abstract
S. J. Thorpe, D. Fize, and C. Marlot (1996) showed how rapidly observers can detect animals in images of natural scenes, but it is still unclear which image features support this rapid detection. A. B. Torralba and A. Oliva (2003) suggested that a simple image statistic based on the power spectrum allows the absence or presence of objects in natural scenes to be predicted. We tested whether human observers make use of power spectral differences between image categories when detecting animals in natural scenes. In Experiments 1 and 2 we found performance to be essentially independent of the power spectrum. Computational analysis revealed that the ease of classification correlates with the proposed spectral cue without being caused by it. This result is consistent with the hypothesis that in commercial stock photo databases a majority of animal images are pre-segmented from the background by the photographers and this pre-segmentation causes the power spectral differences between image categories and may, furthermore, help rapid animal detection. Data from a third experiment are consistent with this hypothesis. Together, our results make it exceedingly unlikely that human observers make use of power spectral differences between animal- and no-animal images during rapid animal detection. In addition, our results point to potential confounds in the commercially available “natural image” databases whose statistics may be less natural than commonly presumed.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
A generative model approach for decoding in the visual event-related potential-based brain-computer interface speller

Martens, SMM., Leiva, JM.

Journal of Neural Engineering, 7(2):1-10, April 2010 (article)

Abstract
There is a strong tendency towards discriminative approaches in brain-computer interface (BCI) research. We argue that generative model-based approaches are worth pursuing and propose a simple generative model for the visual ERP-based BCI speller which incorporates prior knowledge about the brain signals. We show that the proposed generative method needs less training data to reach a given letter prediction performance than the state of the art discriminative approaches.

ei

PDF PDF DOI [BibTex]

PDF PDF DOI [BibTex]


no image
Hilbert Space Embeddings and Metrics on Probability Measures

Sriperumbudur, B., Gretton, A., Fukumizu, K., Schölkopf, B., Lanckriet, G.

Journal of Machine Learning Research, 11, pages: 1517-1561, April 2010 (article)

ei

PDF [BibTex]

PDF [BibTex]


no image
Graph Kernels

Vishwanathan, SVN., Schraudolph, NN., Kondor, R., Borgwardt, KM.

Journal of Machine Learning Research, 11, pages: 1201-1242, April 2010 (article)

Abstract
We present a unified framework to study graph kernels, special cases of which include the random walk (G{\"a}rtner et al., 2003; Borgwardt et al., 2005) and marginalized (Kashima et al., 2003, 2004; Mahét al., 2004) graph kernels. Through reduction to a Sylvester equation we improve the time complexity of kernel computation between unlabeled graphs with n vertices from O(n6) to O(n3). We find a spectral decomposition approach even more efficient when computing entire kernel matrices. For labeled graphs we develop conjugate gradient and fixed-point methods that take O(dn3) time per iteration, where d is the size of the label set. By extending the necessary linear algebra to Reproducing Kernel Hilbert Spaces (RKHS) we obtain the same result for d-dimensional edge kernels, and O(n4) in the infinite-dimensional case; on sparse graphs these algorithms only take O(n2) time per iteration in all cases. Experiments on graphs from bioinformatics and other application domains show that these techniques can speed up computation of the kernel by an order of magnitude or more. We also show that certain rational kernels (Cortes et al., 2002, 2003, 2004) when specialized to graphs reduce to our random walk graph kernel. Finally, we relate our framework to R-convolution kernels (Haussler, 1999) and provide a kernel that is close to the optimal assignment kernel of kernel of Fr{\"o}hlich et al. (2006) yet provably positive semi-definite.

ei

PDF Web [BibTex]

PDF Web [BibTex]


no image
Solving large-scale nonnegative least squares using an adaptive non-monotonic method

Sra, S., Kim, D., Dhillon, I.

24th European Conference on Operational Research (EURO 2010), 24, pages: 223, April 2010 (poster)

Abstract
We present an efficient algorithm for large-scale non-negative least-squares (NNLS). We solve NNLS by extending the unconstrained quadratic optimization method of Barzilai and Borwein (BB) to handle nonnegativity constraints. Our approach is simple yet efficient. It differs from other constrained BB variants as: (i) it uses a specific subset of variables for computing BB steps; and (ii) it scales these steps adaptively to ensure convergence. We compare our method with both established convex solvers and specialized NNLS methods, and observe highly competitive empirical performance.

ei

PDF [BibTex]

PDF [BibTex]


no image
Gene function prediction from synthetic lethality networks via ranking on demand

Lippert, C., Ghahramani, Z., Borgwardt, KM.

Bioinformatics, 26(7):912-918, April 2010 (article)

Abstract
Motivation: Synthetic lethal interactions represent pairs of genes whose individual mutations are not lethal, while the double mutation of both genes does incur lethality. Several studies have shown a correlation between functional similarity of genes and their distances in networks based on synthetic lethal interactions. However, there is a lack of algorithms for predicting gene function from synthetic lethality interaction networks. Results: In this article, we present a novel technique called kernelROD for gene function prediction from synthetic lethal interaction networks based on kernel machines. We apply our novel algorithm to Gene Ontology functional annotation prediction in yeast. Our experiments show that our method leads to improved gene function prediction compared with state-of-the-art competitors and that combining genetic and congruence networks leads to a further improvement in prediction accuracy.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Sparse regression via a trust-region proximal method

Kim, D., Sra, S., Dhillon, I.

24th European Conference on Operational Research (EURO 2010), 24, pages: 278, April 2010 (poster)

Abstract
We present a method for sparse regression problems. Our method is based on the nonsmooth trust-region framework that minimizes a sum of smooth convex functions and a nonsmooth convex regularizer. By employing a separable quadratic approximation to the smooth part, the method enables the use of proximity operators, which in turn allow tackling the nonsmooth part efficiently. We illustrate our method by implementing it for three important sparse regression problems. In experiments with synthetic and real-world large-scale data, our method is seen to be competitive, robust, and scalable.

ei

PDF [BibTex]

PDF [BibTex]


no image
A toolbox for predicting G-quadruplex formation and stability

Wong, HM., Stegle, O., Rodgers, S., Huppert, J.

Journal of Nucleic Acids, 2010(564946):1-6, March 2010 (article)

Abstract
G-quadruplexes are four stranded nucleic acid structures formed around a core of guanines, arranged in squares with mutual hydrogen bonding. Many of these structures are highly thermally stable, especially in the presence of monovalent cations, such as those found under physiological conditions. Understanding of their physiological roles is expanding rapidly, and they have been implicated in regulating gene transcription and translation among other functions. We have built a community-focused website to act as a repository for the information that is now being developed. At its core, this site has a detailed database (QuadDB) of predicted G-quadruplexes in the human and other genomes, together with the predictive algorithm used to identify them. We also provide a QuadPredict server, which predicts thermal stability and acts as a repository for experimental data from all researchers. There are also a number of other data sources with computational predictions. We anticipate that the wide availability of this information will be of use both to researchers already active in this exciting field and to those who wish to investigate a particular gene hypothesis.

ei

PDF DOI [BibTex]

PDF DOI [BibTex]


no image
A Novel Protocol for Accuracy Assessment in Classification of Very High Resolution Images

Persello, C., Bruzzone, L.

IEEE Transactions on Geoscience and Remote Sensing, 48(3):1232-1244, March 2010 (article)

Abstract
This paper presents a novel protocol for the accuracy assessment of the thematic maps obtained by the classification of very high resolution images. As the thematic accuracy alone is not sufficient to adequately characterize the geometrical properties of high-resolution classification maps, we propose a protocol that is based on the analysis of two families of indices: 1) the traditional thematic accuracy indices and 2) a set of novel geometric indices that model different geometric properties of the objects recognized in the map. In this context, we present a set of indices that characterize five different types of geometric errors in the classification map: 1) oversegmentation; 2) undersegmentation; 3) edge location; 4) shape distortion; and 5) fragmentation. Moreover, we propose a new approach for tuning the free parameters of supervised classifiers on the basis of a multiobjective criterion function that aims at selecting the parameter values that result in the classification map that jointly optimize thematic and geometric error indices. Experimental results obtained on QuickBird images show the effectiveness of the proposed protocol in selecting classification maps characterized by a better tradeoff between thematic and geometric accuracies than standard procedures based only on thematic accuracy measures. In addition, results obtained with support vector machine classifiers confirm the effectiveness of the proposed multiobjective technique for the selection of free-parameter values for the classification algorithm.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
PAC-Bayesian Bounds for Discrete Density Estimation and Co-clustering Analysis

Seldin, Y., Tishby, N.

Workshop "Foundations and New Trends of PAC Bayesian Learning", 2010, March 2010 (poster)

Abstract
We applied PAC-Bayesian framework to derive gen- eralization bounds for co-clustering1. The analysis yielded regularization terms that were absent in the preceding formulations of this task. The bounds sug- gested that co-clustering should optimize a trade-off between its empirical performance and the mutual in- formation that the cluster variables preserve on row and column indices. Proper regularization enabled us to achieve state-of-the-art results in prediction of the missing ratings in the MovieLens collaborative filtering dataset. In addition a PAC-Bayesian bound for discrete den- sity estimation was derived. We have shown that the PAC-Bayesian bound for classification is a spe- cial case of the PAC-Bayesian bound for discrete den- sity estimation. We further introduced combinatorial priors to PAC-Bayesian analysis. The combinatorial priors are more appropriate for discrete domains, as opposed to Gaussian priors, the latter of which are suitable for continuous domains. It was shown that combinatorial priors lead to regularization terms in the form of mutual information.

ei

PDF Web [BibTex]

PDF Web [BibTex]


no image
On the Entropy Production of Time Series with Unidirectional Linearity

Janzing, D.

Journal of Statistical Physics, 138(4-5):767-779, March 2010 (article)

Abstract
There are non-Gaussian time series that admit a causal linear autoregressive moving average (ARMA) model when regressing the future on the past, but not when regressing the past on the future. The reason is that, in the latter case, the regression residuals are not statistically independent of the regressor. In previous work, we have experimentally verified that many empirical time series indeed show such a time inversion asymmetry. For various physical systems, it is known that time-inversion asymmetries are linked to the thermodynamic entropy production in non-equilibrium states. Here we argue that unidirectional linearity is also accompanied by entropy generation. To this end, we study the dynamical evolution of a physical toy system with linear coupling to an infinite environment and show that the linearity of the dynamics is inherited by the forward-time conditional probabilities, but not by the backward-time conditionals. The reason is that the environment permanently provides particles that are in a product state before they interact with the system, but show statistical dependence afterwards. From a coarse-grained perspective, the interaction thus generates entropy. We quantitatively relate the strength of the non-linearity of the backward process to the minimal amount of entropy generation. The paper thus shows that unidirectional linearity is an indirect implication of the thermodynamic arrow of time, given that the joint dynamics of the system and its environment is linear.

ei

PDF DOI [BibTex]

PDF DOI [BibTex]


no image
Derivatives of Logarithmic Stationary Distributions for Policy Gradient Reinforcement Learning

Morimura, T., Uchibe, E., Yoshimoto, J., Peters, J., Doya, K.

Neural Computation, 22(2):342-376, February 2010 (article)

Abstract
Most conventional policy gradient reinforcement learning (PGRL) algorithms neglect (or do not explicitly make use of) a term in the average reward gradient with respect to the policy parameter. That term involves the derivative of the stationary state distribution that corresponds to the sensitivity of its distribution to changes in the policy parameter. Although the bias introduced by this omission can be reduced by setting the forgetting rate γ for the value functions close to 1, these algorithms do not permit γ to be set exactly at γ = 1. In this article, we propose a method for estimating the log stationary state distribution derivative (LSD) as a useful form of the derivative of the stationary state distribution through backward Markov chain formulation and a temporal difference learning framework. A new policy gradient (PG) framework with an LSD is also proposed, in which the average reward gradient can be estimated by setting //!-- MFG_und--//amp;#947; = 0, so it becomes unnecessary to learn the value functions. We also test the performance of the proposed algorithms using simple benchmark tasks and show that these can improve the performances of existing PG methods.

ei

PDF Web DOI [BibTex]

PDF Web DOI [BibTex]


no image
Bayesian Online Multitask Learning of Gaussian Processes

Pillonetto, G., Dinuzzo, F., De Nicolao, G.

IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(2):193-205, February 2010 (article)

Abstract
Standard single-task kernel methods have recently been extended to the case of multitask learning in the context of regularization theory. There are experimental results, especially in biomedicine, showing the benefit of the multitask approach compared to the single-task one. However, a possible drawback is computational complexity. For instance, when regularization networks are used, complexity scales as the cube of the overall number of training data, which may be large when several tasks are involved. The aim of this paper is to derive an efficient computational scheme for an important class of multitask kernels. More precisely, a quadratic loss is assumed and each task consists of the sum of a common term and a task-specific one. Within a Bayesian setting, a recursive online algorithm is obtained, which updates both estimates and confidence intervals as new data become available. The algorithm is tested on two simulated problems and a real data set relative to xenobiotics administration in human patients.

ei

DOI [BibTex]

DOI [BibTex]


no image
The semigroup approach to transport processes in networks

Dorn, B., Fijavz, M., Nagel, R., Radl, A.

Physica D: Nonlinear Phenomena, 239(15):1416-1421, January 2010 (article)

Abstract
We explain how operator semigroups can be used to study transport processes in networks. This method is applied to a linear Boltzmann equation on a finite as well as on an infinite network and yields well-posedness and information on the long term behavior of the solutions to the presented problems.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Optimization of k-Space Trajectories for Compressed Sensing by Bayesian Experimental Design

Seeger, M., Nickisch, H., Pohmann, R., Schölkopf, B.

Magnetic Resonance in Medicine, 63(1):116-126, January 2010 (article)

Abstract
The optimization of k-space sampling for nonlinear sparse MRI reconstruction is phrased as a Bayesian experimental design problem. Bayesian inference is approximated by a novel relaxation to standard signal processing primitives, resulting in an efficient optimization algorithm for Cartesian and spiral trajectories. On clinical resolution brain image data from a Siemens 3T scanner, automatically optimized trajectories lead to significantly improved images, compared to standard low-pass, equispaced, or variable density randomized designs. Insights into the nonlinear design optimization problem for MRI are given.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Learning the Reward Model of Dialogue POMDPs

Boularias, A., Chinaei, H., Chaib-Draa, B.

NIPS Workshop on Machine Learning for Assistive Technology (MLAT-2010), 2010 (poster)

ei

[BibTex]

[BibTex]


no image
Consistent Nonparametric Tests of Independence

Gretton, A., Györfi, L.

Journal of Machine Learning Research, 11, pages: 1391-1423, 2010 (article)

ei

PDF [BibTex]

PDF [BibTex]


no image
Inferring latent task structure for Multitask Learning by Multiple Kernel Learning

Widmer, C., Toussaint, N., Altun, Y., Rätsch, G.

BMC Bioinformatics, 11 Suppl 8, pages: S5, 2010 (article)

Abstract
The lack of sufficient training data is the limiting factor for many Machine Learning applications in Computational Biology. If data is available for several different but related problem domains, Multitask Learning algorithms can be used to learn a model based on all available information. In Bioinformatics, many problems can be cast into the Multitask Learning scenario by incorporating data from several organisms. However, combining information from several tasks requires careful consideration of the degree of similarity between tasks. Our proposed method simultaneously learns or refines the similarity between tasks along with the Multitask Learning classifier. This is done by formulating the Multitask Learning problem as Multiple Kernel Learning, using the recently published q-Norm MKL algorithm.

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Erste Erfahrungen bei der Beurteilung hämato-onkologischer Krankheitsmanifestationen an den Extremitäten mit einem PET/MRT-Hybridsystem.

Sauter, A., Boss, A., Kolb, A., Mantlik, F., Bethge, W., Kanz, L., Pfannenberg, C., Stegger, L., Pichler, B., Claussen, C., Horger, M.

Thieme Verlag, Stuttgart, Germany, 91. Deutscher R{\"o}ntgenkongress, 2010 (poster)

ei

Web DOI [BibTex]

Web DOI [BibTex]


no image
Policy learning algorithmis for motor learning (Algorithmen zum automatischen Erlernen von Motorfähigkigkeiten)

Peters, J., Kober, J., Schaal, S.

Automatisierungstechnik, 58(12):688-694, 2010, clmc (article)

Abstract
Robot learning methods which allow au- tonomous robots to adapt to novel situations have been a long standing vision of robotics, artificial intelligence, and cognitive sciences. However, to date, learning techniques have yet to ful- fill this promise as only few methods manage to scale into the high-dimensional domains of manipulator robotics, or even the new upcoming trend of humanoid robotics. If possible, scaling was usually only achieved in precisely pre-structured domains. In this paper, we investigate the ingredients for a general ap- proach policy learning with the goal of an application to motor skill refinement in order to get one step closer towards human- like performance. For doing so, we study two major components for such an approach, i. e., firstly, we study policy learning algo- rithms which can be applied in the general setting of motor skill learning, and, secondly, we study a theoretically well-founded general approach to representing the required control structu- res for task representation and execution.

am

link (url) [BibTex]


no image
Gait planning based on kinematics for a quadruped gecko model with redundancy

Son, D., Jeon, D., Nam, W. C., Chang, D., Seo, T., Kim, J.

Robotics and Autonomous Systems, 58, 2010 (article)

pi

[BibTex]

[BibTex]


Thumb xl toc image
Molecular QED of coherent and incoherent sum-frequency and second-harmonic generation in chiral liquids in the presence of a static electric field

Fischer, P., Salam, A.

MOLECULAR PHYSICS, 108(14):1857-1868, 2010 (article)

Abstract
Coherent second-order nonlinear optical processes are symmetry forbidden in centrosymmetric environments in the electric-dipole approximation. In liquids that contain chiral molecules, however, and which therefore lack mirror image symmetry, coherent sum-frequency generation is possible, whereas second-harmonic generation remains forbidden. Here we apply the theory of molecular quantum electrodynamics to the calculation of the matrix element, transition rate, and integrated signal intensity for sum-frequency and second-harmonic generation taking place in a chiral liquid in the presence and absence of a static electric field, to examine which coherent and incoherent processes exist in the electric-dipole approximation in liquids. Third- and fourth-order time-dependent perturbation theory is employed in combination with single-sided Feynman diagrams to evaluate two contributions arising from static field-free and field-induced processes. It is found that, in addition to the coherent term, an incoherent process exists for sum-frequency generation in liquids. Surprisingly, in the case of dc-field-induced second-harmonic generation, the incoherent contribution is found to always vanish for isotropic chiral liquids even though hyper-Rayleigh second-harmonic generation and electric-field-induced second-harmonic generation are both independently symmetry allowed in any liquid.

pf

DOI [BibTex]


no image
Flat dry elastomer adhesives as attachment materials for climbing robots

Unver, O., Sitti, M.

IEEE transactions on robotics, 26(1):131-141, IEEE, 2010 (article)

pi

[BibTex]

[BibTex]


no image
A Bayesian approach to nonlinear parameter identification for rigid-body dynamics

Ting, J., DSouza, A., Schaal, S.

Neural Networks, 2010, clmc (article)

Abstract
For complex robots such as humanoids, model-based control is highly beneficial for accurate tracking while keeping negative feedback gains low for compliance. However, in such multi degree-of-freedom lightweight systems, conventional identification of rigid body dynamics models using CAD data and actuator models is inaccurate due to unknown nonlinear robot dynamic effects. An alternative method is data-driven parameter estimation, but significant noise in measured and inferred variables affects it adversely. Moreover, standard estimation procedures may give physically inconsistent results due to unmodeled nonlinearities or insufficiently rich data. This paper addresses these problems, proposing a Bayesian system identification technique for linear or piecewise linear systems. Inspired by Factor Analysis regression, we develop a computationally efficient variational Bayesian regression algorithm that is robust to ill-conditioned data, automatically detects relevant features, and identifies input and output noise. We evaluate our approach on rigid body parameter estimation for various robotic systems, achieving an error of up to three times lower than other state-of-the-art machine learning methods.

am

link (url) [BibTex]


no image
A first optimal control solution for a complex, nonlinear, tendon driven neuromuscular finger model

Theodorou, E. A., Todorov, E., Valero-Cuevas, F.

Proceedings of the ASME 2010 Summer Bioengineering Conference August 30-September 2, 2010, Naples, Florida, USA, 2010, clmc (article)

Abstract
In this work we present the first constrained stochastic op- timal feedback controller applied to a fully nonlinear, tendon driven index finger model. Our model also takes into account an extensor mechanism, and muscle force-length and force-velocity properties. We show this feedback controller is robust to noise and perturbations to the dynamics, while successfully handling the nonlinearities and high dimensionality of the system. By ex- tending prior methods, we are able to approximate physiological realism by ensuring positivity of neural commands and tendon tensions at all timesthus can, for the first time, use the optimal control framework to predict biologically plausible tendon tensions for a nonlinear neuromuscular finger model. METHODS 1 Muscle Model The rigid-body triple pendulum finger model with slightly viscous joints is actuated by Hill-type muscle models. Joint torques are generated by the seven muscles of the index fin-

am

PDF [BibTex]

PDF [BibTex]


no image
Dzyaloshinskii-Moriya interactions in systems with fabrication induced strain gradients: ab-initio study

Beck, P., Fähnle, M

{Journal of Magnetism and Magnetic Materials}, 322, pages: 3701-3703, 2010 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
On the nature of displacement bursts during nanoindentation of ultrathin Ni films on sapphire

Rabkin, E., Deuschle, J. K., Baretzky, B.

{Acta Materialia}, 58, pages: 1589-1598, 2010 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Nanospheres generate out-of-plane magnetization

Amaladass, E., Ludescher, B., Schütz, G., Tyliszczak, T., Lee, M., Eimüller, T.

{Journal of Applied Physics}, 107, 2010 (article)

mms

DOI [BibTex]

DOI [BibTex]


no image
Temperature dependence of the magnetic properties of L10-FePt nanostructures and films

Bublat, T., Goll, D.

{Journal of Applied Physics}, 108(11), 2010 (article)

mms

DOI [BibTex]

DOI [BibTex]