-
Geometric Decision Tree
In this paper, we present a new algorithm for learning oblique decision trees. Most of the current decision tree algorithms rely on impurity measures to assess the goodness of hyperplanes at each node while learning a decision tree in top-down fashion. These impurity measures do not properly capture the geometric structures in the data. Motivated by this, our algorithm uses a strategy for assessing the hyperplanes in such a way that the geometric structure in the data is taken into account. At each node of the decision tree, we find the clustering hyperplanes for both the classes and use their angle bisectors as the split rule at that node. We show through empirical studies that this idea leads to small decision trees and better performance. We also present some analysis to show that the angle bisectors of clustering hyperplanes that we use as the split rules at each node are solutions of an interesting optimization problem and hence argue that this is a principled method of learning a decision tree.
-
IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics publication information
-
Generalized Biased Discriminant Analysis for Content-Based Image Retrieval
Biased discriminant analysis (BDA) is one of the most promising relevance feedback (RF) approaches to deal with the feedback sample imbalance problem for content-based image retrieval (CBIR). However, the singular problem of the positive within-class scatter and the Gaussian distribution assumption for positive samples are two main obstacles impeding the performance of BDA RF for CBIR. To avoid both of these intrinsic problems in BDA, in this paper, we propose a novel algorithm called generalized BDA (GBDA) for CBIR. The GBDA algorithm avoids the singular problem by adopting the differential scatter discriminant criterion (DSDC) and handles the Gaussian distribution assumption by redesigning the between-class scatter with a nearest neighbor approach. To alleviate the overfitting problem, GBDA integrates the locality preserving principle; therefore, a smooth and locally consistent transform can also be learned. Extensive experiments show that GBDA can substantially outperform the original BDA, its variations, and related support-vector-machine-based RF algorithms.
-
A Comparison of Information Functions and Search Strategies for Sensor Planning in Target Classification
This paper investigates the comparative performance of several information-driven search strategies and decision rules using a canonical target classification problem. Five sensor models are considered: one obtained from classical estimation theory and four obtained from Bernoulli, Poisson, binomial, and mixture-of-binomial distributions. A systematic approach is presented for deriving information functions that represent the expected utility of future sensor measurements from mutual information, Rènyi divergence, Kullback–Leibler divergence, information potential, quadratic entropy, and the Cauchy–Schwarz distance. The resulting information-driven strategies are compared to direct-search, alert–confirm, task-driven (TS), and log-likelihood-ratio (LLR) search strategies. Extensive numerical simulations show that quadratic entropy typically leads to the most effective search strategy with respect to correct-classification rates. In the presence of prior information, the quadratic-entropy-driven strategy also displays the lowest rate of false alarms. However, when prior information is absent or very noisy, TS and LLR strategies achieve the lowest false-alarm rates for the Bernoulli, mixture-of-binomial, and classical sensor models.
-
Hybrid Ant Colony-Genetic Algorithm (GAAPI) for Global Continuous Optimization
Many real-life optimization problems often face an increased rank of nonsmoothness (many local minima) which could prevent a search algorithm from moving toward the global solution. Evolution-based algorithms try to deal with this issue. The algorithm proposed in this paper is called GAAPI and is a hybridization between two optimization techniques: a special class of ant colony optimization for continuous domains entitled API and a genetic algorithm (GA). The algorithm adopts the downhill behavior of API (a key characteristic of optimization algorithms) and the good spreading in the solution space of the GA. A probabilistic approach and an empirical comparison study are presented to prove the convergence of the proposed method in solving different classes of complex global continuous optimization problems. Numerical results are reported and compared to the existing results in the literature to validate the feasibility and the effectiveness of the proposed method. The proposed algorithm is shown to be effective and efficient for most of the test functions.
-
Initialization Independent Clustering With Actively Self-Training Method
The results of traditional clustering methods are usually unreliable as there is not any guidance from the data labels, while the class labels can be predicted more reliable by the semisupervised learning if the labels of partial data are given. In this paper, we propose an actively self-training clustering method, in which the samples are actively selected as training set to minimize an estimated Bayes error, and then explore semisupervised learning to perform clustering. Traditional graph-based semisupervised learning methods are not convenient to estimate the Bayes error; we develop a specific regularization framework on graph to perform semisupervised learning, in which the Bayes error can be effectively estimated. In addition, the proposed clustering algorithm can be readily applied in a semisupervised setting with partial class labels. Experimental results on toy data and real-world data sets demonstrate the effectiveness of the proposed clustering method on the unsupervised and the semisupervised setting. It is worthy noting that the proposed clustering method is free of initialization, while traditional clustering methods are usually dependent on initialization.
-
Table of Contents
-
Fully Automatic Recognition of the Temporal Phases of Facial Actions
Past work on automatic analysis of facial expressions has focused mostly on detecting prototypic expressions of basic emotions like happiness and anger. The method proposed here enables the detection of a much larger range of facial behavior by recognizing facial muscle actions [action units (AUs)] that compound expressions. AUs are agnostic, leaving the inference about conveyed intent to higher order decision making (e.g., emotion recognition). The proposed fully automatic method not only allows the recognition of 22 AUs but also explicitly models their temporal characteristics (i.e., sequences of temporal segments: neutral, onset, apex, and offset). To do so, it uses a facial point detector based on Gabor-feature-based boosted classifiers to automatically localize 20 facial fiducial points. These points are tracked through a sequence of images using a method called particle filtering with factorized likelihoods. To encode AUs and their temporal activation models based on the tracking data, it applies a combination of GentleBoost, support vector machines, and hidden Markov models. We attain an average AU recognition rate of 95.3% when tested on a benchmark set of deliberately displayed facial expressions and 72% when tested on spontaneous expressions.
-
A Dynamic Hybrid Framework for Constrained Evolutionary Optimization
Based on our previous work, this paper presents a dynamic hybrid framework, called DyHF, for solving constrained optimization problems. This framework consists of two major steps: global search model and local search model. In the global and local search models, differential evolution serves as the search engine, and Pareto dominance used in multiobjective optimization is employed to compare the individuals in the population. Unlike other existing methods, the above two steps are executed dynamically according to the feasibility proportion of the current population in this paper, with the purpose of reasonably distributing the computational resource for the global and local search during the evolution. The performance of DyHF is tested on 22 benchmark test functions. The experimental results clearly show that the overall performance of DyHF is highly competitive with that of a number of state-of-the-art approaches from the literature.
-
Coordination of Multiple Agents With Double-Integrator Dynamics Under Generalized Interaction Topologies
The problem of the convergence of the consensus strategies for multiple agents with double-integrator dynamics is studied in this paper. The investigation covers two kinds of different settings. In the setting with the interaction topologies for the position and velocity information flows being modeled by different graphs, some sufficient conditions on the fixed interaction topologies are derived for the agents to reach consensus. In the setting with the interaction topologies for the position and velocity information flows being modeled by the same graph, we systematically investigate the consensus algorithm for the agents under both fixed and dynamically changing directed interaction topologies. Specifically, for the fixed case, a necessary and sufficient condition on the interaction topology is established for the agents to reach (average) consensus under certain assumptions. For the dynamically changing case, some sufficient conditions are obtained for the agents to reach consensus, where the condition imposed on the dynamical topologies is shown to be more relaxed than that required in the existing literature. Finally, we demonstrate the usefulness of the theoretical findings through some numerical examples.
-
Stabilization of Nonlinear Systems Using Sampled-Data Output-Feedback Fuzzy Controller Based on Polynomial-Fuzzy-Model-Based Control Approach
This paper investigates the stability of sampled-data output-feedback (SDOF) polynomial-fuzzy-model-based control systems. Representing the nonlinear plant using a polynomial fuzzy model, an SDOF fuzzy controller is proposed to perform the control process using the system output information. As only the system output is available for feedback compensation, it is more challenging for the controller design and system analysis compared to the full-state-feedback case. Furthermore, because of the sampling activity, the control signal is kept constant by the zero-order hold during the sampling period, which complicates the system dynamics and makes the stability analysis more difficult. In this paper, two cases of SDOF fuzzy controllers, which either share the same number of fuzzy rules or not, are considered. The system stability is investigated based on the Lyapunov stability theory using the sum-of-squares (SOS) approach. SOS-based stability conditions are obtained to guarantee the system stability and synthesize the SDOF fuzzy controller. Simulation examples are given to demonstrate the merits of the proposed SDOF fuzzy control approach.
-
Robust Color Texture Features Under Varying Illumination Conditions
Under varying illumination, both the statistical and structural contents of color texture are modified, leading to changes in the observed texture surface. We model the effect of illumination as a perturbation on an ideal color texture and show that the spectra of the ambient light have a significant impact on the observed texture patterns in the individual color channels. Motivated by studies in human color constancy, we propose a correlation-based transformation that minimizes the effect of illumination variation in color texture analysis. Experimental results are included, which validate the performance of the proposed minvariance model in the analysis of color texture.
-
IEEE Systems, Man, and Cybernetics Society Information
-
Radial Basis Function Networks With Linear Interval Regression Weights for Symbolic Interval Data
This paper introduces a new structure of radial basis function networks (RBFNs) that can successfully model symbolic interval-valued data. In the proposed structure, to handle symbolic interval data, the Gaussian functions required in the RBFNs are modified to consider interval distance measure, and the synaptic weights of the RBFNs are replaced by linear interval regression weights. In the linear interval regression weights, the lower and upper bounds of the interval-valued data as well as the center and range of the interval-valued data are considered. In addition, in the proposed approach, two stages of learning mechanisms are proposed. In stage 1, an initial structure (i.e., the number of hidden nodes and the adjustable parameters of radial basis functions) of the proposed structure is obtained by the interval competitive agglomeration clustering algorithm. In stage 2, a gradient-descent kind of learning algorithm is applied to fine-tune the parameters of the radial basis function and the coefficients of the linear interval regression weights. Various experiments are conducted, and the average behavior of the root mean square error and the square of the correlation coefficient in the framework of a Monte Carlo experiment are considered as the performance index. The results clearly show the effectiveness of the proposed structure.
-
Evolutionary and Principled Search Strategies for Sensornet Protocol Optimization
Interactions between multiple tunable protocol parameters and multiple performance metrics are generally complex and unknown; finding optimal solutions is generally difficult. However, protocol tuning can yield significant gains in energy efficiency and resource requirements, which is of particular importance for sensornet systems in which resource availability is severely restricted. We address this multi-objective optimization problem for two dissimilar routing protocols and by two distinct approaches. First, we apply factorial design and statistical model fitting methods to reject insignificant factors and locate regions of the problem space containing near-optimal solutions by principled search. Second, we apply the Strength Pareto Evolutionary Algorithm 2 and Two-Archive evolutionary algorithms to explore the problem space, with each iteration potentially yielding solutions of higher quality and diversity than the preceding iteration. Whereas a principled search methodology yields a generally applicable survey of the problem space and enables performance prediction, the evolutionary approach yields viable solutions of higher quality and at lower experimental cost. This is the first study in which sensornet protocol optimization has been explicitly formulated as a multi-objective problem and solved with state-of-the-art multi-objective evolutionary algorithms.
-
Telepresence Index for Bilateral Teleoperations
This paper proposes a performance index called telepresence index for bilateral teleoperation, which can be used both for the performance evaluation of bilateral control architectures and for design purposes. This index is intended to represent a comprehensive performance objective consisting of transparency and kinematic correspondence, which are two major performance objectives of bilateral teleoperation. In order to quantify the performance objective, telepresence index has employed the error vector magnitude, which enables a seamless combination of magnitude and phase errors and the accommodation of time delay. In comparison with existing performance indices, it was observed that telepresence index possesses the comprehensiveness of performance objectives, magnitude/phase integrity, and the capacity to include time delay, which the others lack in one way or another. The index was applied to evaluate the performances of two widely known control architectures: PD-type bilateral control and Ueda's ideal control. In all cases, telepresence index has been compared favorably with the other indices in terms of clarity, convenience, and accuracy, thereby demonstrating its superiority.
-
Gaussian-Mixture-Model-Based Spatial Neighborhood Relationships for Pixel Labeling Problem
In this paper, we present a new algorithm for pixel labeling and image segmentation based on the standard Gaussian mixture model (GMM). Unlike the standard GMM where pixels themselves are considered independent of each other and the spatial relationship between neighboring pixels is not taken into account, the proposed method incorporates this spatial relationship into the standard GMM. Moreover, the proposed model requires fewer parameters compared with the models based on Markov random fields. In order to estimate model parameters from observations, instead of utilizing an expectation–maximization algorithm, we employ gradient method to minimize a higher bound on the data negative log-likelihood. The performance of the proposed model is compared with methods based on both standard GMM and Markov random fields, demonstrating the robustness, accuracy, and effectiveness of our method.
-
Enhanced Models for Expertise Retrieval Using Community-Aware Strategies
Expertise retrieval, whose task is to suggest people with relevant expertise on the topic of interest, has received increasing interest in recent years. One of the issues is that previous algorithms mainly consider the documents associated with the experts while ignoring the community information that is affiliated with the documents and the experts. Motivated by the observation that communities could provide valuable insight and distinctive information, we investigate and develop two community-aware strategies to enhance expertise retrieval. We first propose a new smoothing method using the community context for statistical language modeling, which is employed to identify the most relevant documents so as to boost the performance of expertise retrieval in the document-based model. Furthermore, we propose a query-sensitive AuthorRank to model the authors' authorities based on the community coauthorship networks and develop an adaptive ranking refinement method to enhance expertise retrieval. Experimental results demonstrate the effectiveness and robustness of both community-aware strategies. Moreover, the improvements made in the enhanced models are significant and consistent.
-
Enhancing Collaborative Filtering by User Interest Expansion via Personalized Ranking
Recommender systems suggest a few items from many possible choices to the users by understanding their past behaviors. In these systems, the user behaviors are influenced by the hidden interests of the users. Learning to leverage the information about user interests is often critical for making better recommendations. However, existing collaborative-filtering-based recommender systems are usually focused on exploiting the information about the user's interaction with the systems; the information about latent user interests is largely underexplored. To that end, inspired by the topic models, in this paper, we propose a novel collaborative-filtering-based recommender system by user interest expansion via personalized ranking, named iExpand. The goal is to build an item-oriented model-based collaborative-filtering framework. The iExpand method introduces a three-layer, user–interests–item, representation scheme, which leads to more accurate ranking recommendation results with less computation cost and helps the understanding of the interactions among users, items, and user interests. Moreover, iExpand strategically deals with many issues that exist in traditional collaborative-filtering approaches, such as the overspecialization problem and the cold-start problem. Finally, we evaluate iExpand on three benchmark data sets, and experimental results show that iExpand can lead to better ranking performance than state-of-the-art methods with a significant margin.
-
On Convergence of Differential Evolution Over a Class of Continuous Functions With Unique Global Optimum
Differential evolution (DE) is arguably one of the most powerful stochastic real-parameter optimization algorithms of current interest. Since its inception in the mid 1990s, DE has been finding many successful applications in real-world optimization problems from diverse domains of science and engineering. This paper takes a first significant step toward the convergence analysis of a canonical DE (DE/rand/1/bin) algorithm. It first deduces a time-recursive relationship for the probability density function (PDF) of the trial solutions, taking into consideration the DE-type mutation, crossover, and selection mechanisms. Then, by applying the concepts of Lyapunov stability theorems, it shows that as time approaches infinity, the PDF of the trial solutions concentrates narrowly around the global optimum of the objective function, assuming the shape of a Dirac delta distribution. Asymptotic convergence behavior of the population PDF is established by constructing a Lyapunov functional based on the PDF and showing that it monotonically decreases with time. The analysis is applicable to a class of continuous and real-valued objective functions that possesses a unique global optimum (but may have multiple local optima). Theoretical results have been substantiated with relevant computer simulations.
|
