Correntropy

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 54, NO. 6, JUNE 2006

2187

Generalized Correlation Function: Definition, Properties, and Application to Blind Equalization
Ignacio Santamaría, Senior Member, IEEE, Puskal P. Pokharel, Student Member, IEEE, and Jose C. Principe, Fellow, IEEE

Abstract—With an abundance of tools based on kernel methods and information theoretic learning, a voidstill exists in incorporating both the time structure and the statistical distribution of the time series in the same functional measure. In this paper, a new generalized correlation measure is developed that includes the information of both the distribution and that of the time structure of a stochastic process. It is shown how this measure can be interpreted from a kernel method as well as from aninformation theoretic learning points of view, demonstrating some relevant properties. To underscore the effectiveness of the new measure, a simple blind equalization problem is considered using a coded signal. Index Terms—Blind equalization, entropy, generalized correlation kernel, information theoretic learning, reproducing kernel Hilbert space (RKHS).

I. INTRODUCTION

N

ATURALprocesses of interest for engineering are composed of two basic characteristics: statistical distribution of amplitudes and time structure. Time in itself is very fundamental and is crucial to many real-world problems, and the instantaneous random variables are hardly ever independently distributed, i.e., stochastic processes possess a time structure. For this reason, there are widely used measures thatquantify the time structure like the autocorrelation function. On the other hand, there are a number of methods that are solely based on the statistical distribution, ignoring the time structure. A single measure that includes both of these important characteristics could greatly enhance the theory of stochastic random processes. The fact that reproducing kernels are covariance functions asdescribed by Aronszajn [1] and Parzen [2] explains their early role in inference problems. More recently, numerous algorithms using kernel methods, including support vector machines [3], kernel principal component analysis [4], kernel Fisher discriminant analysis [5], and kernel canonical correlation analysis [6],
Manuscript received December 22, 2004; revised June 25, 2005. The work of I. Santamaríawas supported in part by MEC (Ministerio de Educación) under grant PR2004-0364, and by MCYT (Ministerio de Ciencia y Tecnología) under grants TIC2001-0751-C04-03 and TEC2004-06451-C05-02/TCM. The work of P. P. Pokharel and J. C. Principe was supported by NSF grant ECS-0300340. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. David J. Miller. I.Santamaría is with the Communications Engineering Department (DICOM), University of Cantabria, Santander, 39005, Spain (e-mail: nacho@gtas.dicom.unican.es). P. P. Pokharel and J. C Principe are with the Computational NeuroEngineering Laboratory, Electrical and Computer Engineering Department, University of Florida, Gainesville, FL 32611 USA (e-mail: pokharel@cnel.ufl.edu; principe@cnel.ufl.edu).Digital Object Identifier 10.1109/TSP.2006.872524

[7] have been proposed. Likewise, advances in information theoretic learning (ITL), have brought out a number of applications, where entropy and divergence employ Parzen’s nonparametric estimation [8]–[11]. Many of these algorithms have given very elegant solutions to complicated nonlinear problems. Most of all these contemporary algorithms arebased on assumptions of independent distribution of data, which in many cases is not realistic. Obviously, an accurate description of a stochastic process requires both the information of the distribution and that of its time structure. A void still exists in incorporating both in the same functional measure, providing the insight and usefulness of similarity over time. Our new function is a step...