Recovery of Fine Resolution Information in Multispectral Processing1 Brian Zuerndorfer, Gregory H. Wakefield, and Anthony W. England Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor, MI 48109-2122 Email address: bwz @ caen.engin.umich.edu (None) = RL-2008 ABSTRACT In this paper, we consider multiple-sensor processing and develop a unified method forrepresenting multiple-sensor data. When resolution varies between sensors, such a multiple-sensor system can be viewed as samples of a scale-space signal representation. We show that if the spatial transfer function of the sensors are Gaussian, then scale-space filtering can be used to recover small scale (fine resolution) information through extrapolation in scale. As an example of multiple-sensor processing, we considermutispectal processing of remote sensing in which images of surfae scenes are simultaneously generated at different (enter) frequencies. INTRODUCTION Scale-space filtering was introduced in the early 1980's as a technique for signalnalysis over multiple scales [1,2]. The origins of scale-space filtering lie in the edge detection concerns of computer vision but have since been applied to other problems in computer vision [3,4] as well as a model for multiresolution systems [5]. In this latter context, scale-space analysis provides a mathematical framework for data integration in multiresolution systems that are characterized by classes of sensors varying lng ag a dimension called scale. For example, multispectrl analysis in remote sensing requires integration of data from constant-Q bndlimit sensors that vary in center frequency [6]. A simple approximation of such a multiresolution system is that of a multiscale system in which a single parameter, e.g., bandwidth, charaterizes the primary diffences among each of the bandlimited sensors. Under this approximation, the data from each sensor rpresents samples of the scale-space representation of the imaged object. Formally, scale-space filter thry describes the effects of filter scale on functions e(x,s) of the form [1,2,7], e(x,s) O{r(x,s)}, where O(. ) is alinear opeor and r(x,s) is a filter output signal given by, r(x,s) h(x,s)*i(x). Th function i(x) is the input signal and h(xs) is a family of filters which is paraeri by a continuous variable. This variable s is inversely p onal to filter bandwidth and denotes the sl of the o ilter 1 Worc d by NSF MIP-8657884 and the Shell Oil Compy Fon Scale-space filtering represents the signal i(x) by the twodimensional function r(x,s) and draws inferences about variations of i(x) from the threshold crossing contours of e(x,s). The location of these threshold crossings, x(s), is dependent on scale and calculated from e(C(s),s) = x(s)), (1) for some specified threshold function (.). The threshold crossings form contours in the x-s plane. In the nomenclature of scale-space theory, the x-s plane is the scale-space, the function e(.,.) is the scale-space image, and the threshold crossing contours, x(s), are thefingerprints. Although Zuemdorfcr and Wakefield [7] have shown that the requirements can be relaxed while still preserving major points of the theory, the strongest theorems of scale-space signal representation [2,8] assume Gaussian kernels of the form h(x,s) -=C xp(2 — s -,2xk — ' 2 sk ' (2) and a Laplacian operatorfor O {.}. In this case, the fingerprints present a continuous track of the inflection points of the signal i(.) as it is filtered over scale. The inflection points can be used for locating "edges" in the signal [9]. Given the use of Gaussian knels, Gaussian filtering can be used to degrade the resolution (broaden the PSF) of fineresolution sensors in multiresolution systems to match that of a coarse-resolution sensor. Alternatively, given the features of a signal meaed by a coarse-resolution sensor, it is useful to register their ocations with signal featres measured by fineresolution sensor Modelled as a multiscale system, these two problems represent interpolaton and extrapolation, respectively, of the sampled fingerprnts. In the following, we present a formal development of extrapolaton in scale-space and then apply extrapolation to multspectral processing of remote sensing [6]. EXTRAPOLATION IN SCALE-SPACE Exapolation concerns determining the threshold crossing contous x(s), given by, h(x(s),s)*i,(x(s)) r(x(s),s) = a(x(s)), (3) where a(.) is a threshold funcon. The function i,(.) is an indicaor composed of a linear combination of single sensor dms,

N j-l where i(.) is the ideal output signal from the j sensor, and the as ar coefficients. The function (.) is the image output from te j sensorfor a device having infnitesimal spatial resolution. Without loss of generality, the sensor data are ordered by increasing scale of the sensor. Comparing (3) with (1), the operator 0(,} in (3) is the identity. The finerprints x(s) in (3) are estimates of boundary locations in i(.). True boundaries occur between surface regions having different ii(.) values, so that boundaries occur at x values where i,(.) crosses the threshold function, X (s), S o) rCxorso) (9a) and (8b) can be written as, x"(s) = (9b) r,_(xOso) (x'(so))2+ 2rsOr,(xoo)'(so) + sr,.(Xo, so) A - r(xo, o) Repeating the steps above for u(s) yields, u(s) = u(s)+(s -sO)u'(s)+ 2 "()+..., (10) where, i,(x) = a(x). (4) The threshold function o(.) is a linear function of x. In general, i(.) cannot be processed directly due to the finite PSF of the sensor. However, in the absence of noise, the threshold crossings of r(.,s) approach those of i(.) as s - 0, since the kernel h(.,s) approximaes a delta function as s -- 0. To better aproximate boundary locations, we seek x(s) for as small a scale as possible. If sj is the finest scale at which data from the j channel is available, and s$ <... < SN, then x(s) can only be detertinedfors > sv. However I<sNN, and the boundary estimate is improved if there exists a threshold function P(.) such that, s) s= --- (uos p,(uo So) (1 la) (lib) "(so) =,x(uqo) (u'(s))2 + 2sp(uo, so)'(so) + s2p,(uo,so) B -p,(uoso) ) and uo u(so). The threshold function 3(.) is linear, so that (.) = B for some constant B. Thus, for u(s) to approximate x(s), it is necessary that, x'(so= u'(s and x"(s u"(so) or, h(u(s),s)*i,(u(s)) p(u(s),s) = p(u(s)), where u(s) are the fingerprints derived from (5), and (5) (6) h(x,s)* ix) A - h(x,s)*, i,(,x) (12) u(s) X(S) for SN S 2 1. Note that u(s) for a particular threshold function P(.) need only approximate x(s) over part of a single contour, and that diffnt threshold functions are used to apprximae x(s) for diff t contow To demoltrat the sipgal and threshold requi-ments o1 achieve (6), consider the Taylor expansion of x(s) about so, x(s) =x(S)+ (s -SO)x'(SO)+ 2 s x(s)+.... (7) By the implicit function theorem [10], h(x,s)*Z;i,(x) for n=1,2,3. B - hx,s)*il(x) EXAMPLES = r,(xooso) C16(xo - raxao o) x'(so) = We apply the results of the previous section to a multispectral system that interates data from N sensors, where each sensor perates at a diffrent (center) frequency. In this system, eachsensoreceiv energy using an apture, e.g., devices such as lenses in optical application, atennas in microwave application, and arrays in sonar applications that collect radiated energy that is emitt o fleed d from a subject of interest In this system, as the j sensoris scanned over a subject, the output signal is the convolution of the spatially varying radiation intensity of the subject with the radiation pattern (antenna patern) of the apertue The radiation intensity of the subject is equivalent o the scale-space input for the j' sensor, ). The radiation patn of the apert isequivalent to a scale-spacefilterimulse response jsensorh(.,s), where the scale of the implse reponse is the width (beamwidth) of the radiation patn; s is poporonal the wavelength of the sensor. In this system, tru reiom l boundaries occur at level crossings ofi(.), and e apprximated by the fingeprints x(s). A class of functions for which (12) holds is that where different sensor inputs, (.), are scaled versions of each other. In this case, the idicitr s given by, (r,(o so - a=(x) (x(s))2? + 2r,Cx so)S'(s) + r,,,so) 0(xo) - r,X, S o) where x, x($o); subscripted variables indicate prtial difrentiation. In (3) the threShold function a(.) is a linear, so that ck(.) = A for some constant A, and oa(.) = 0. By solution to the heat equation, the use of a Oaussian kenel yields, r,(XoJo) = SorCxS). As a result, (8a) can be written as,

i (xr) = ai)jK(x) (13a) and the finest scale (highest frequency) input signal is given by, i(x) = aK(x), (13b) for an arbitrary function K(.). The significance of such functions can be seen in multiplicative models, ij(x) =I(x)R(x), as are often used in image processing and remote sensing. In active remote sensing systems, I(.) and R1(.) are surface illumination and reflection functions, respectively. In passive systems, I(.) and R1(.) are surface temperature and surface emissivity, respectively. In both systems, R(.) is a function of the surface type and is dependent on frequency. As a result, the indicator is given by, i(x)= I(x)RCx). (14) An example of a surface satisfying (13a) and (13b) is one consisting of a single surface type, and a spatially varying illumination intensity or surface temperature. Another class of functions that satisfies (12) is quadratic functions. In remote sensing applications, such signals occur in passive systems where surface temperature and surface emissivity change linearly in the vicinity of a boundary [11]. In this case, I(.) and R(.) are linear, I(x) = ax + R,(x) = a,. + 3, so that, i,(x) = (ai)2 + (b)+(cI) (16a) and il(x) = ax2+ bx + c (16b) where, af = a,a~X bj = a,3ow + P13, Cj = =Sp. A simplified example of a one-dimensional surfaces model that satisfies (16a) and (16b) is shown in Figures 1-4 (this model is derived from [11]). In the figures, the indicator is composed of data from two sensors, and threshold function, and the corresponding fingerprint is shown in Figure 4. Comparing the fingerprints of Figures 2 and 4 shows that the fingerprint of il(.) (Figure 4) is a reasonable match to the fingerprint of iI(.) (Figure 2), particularly at smaller scales. Since the fingerprints of i,(.) cannot be calculated at small scales, the fingerprints of i(.) can be used to approximate the fingerprints of iX(.) at small scales. CONCLUSION For the cases of Gaussian filtering and linear threshold crossings, we've demonstrated that extrapolation of scale can be performed in multispectral processing for signals that satisfy (12). The fingerprints of extrapolated signals approximate the actual multispectral fingerprints at small scales, and can be used when the multispectral fingerprints are not available. In showing the approximation of extrapolation fingerprints to multispectral fingerprints, only three terms of the Taylor expansion were exploited (i.e., (12) was satisfied for 3 terms). It can be shown that N>3 terms of a Taylor expansion can be used if (12) is satisfied for N terms. As a result, in the absence of noise, extrapolation fingerprints that match the actual multispectral fingerprints at N>3 terms of a Taylor expansion will provide a better approximation at small scales. REFERENCES [1] A. Witkin; "Scale-space filtering," Proc. Int. Joint Conf. Artif.Intell.; Karsruhe, West Germany; 1983; PP. 1019-1021. [2] A. Yuille and T. Poggio; "Scaling theorems for zero crossings," IEEE Trans. Pat. Anal. Mach. Intell., Vol. PAMI-8, No. 1; Jan. 1986; PP. 15-25. [3] S. Barnard; "Stochastic stereo matching over scale," Proc. DARPA Image Understanding Workshop; 1988. [4] A. Witkin, D. Terzopoulis, and M Kass; "Signal matching through scale space," Int. J. Computer Vision, Vol. 1.; 1988; PP. 134-144. [5] B. Zuendorfer, A. England, and G. Wakefield; "The radiobrightness of freezing terrain," Proc. Int. Geosci. Remote Sensing Symp., Vancouver, B.C.; 1989; PP. 2748-2751. [6] J. Richards; Remote Sensing Digital Image Analysis, Springer-Verlag, Berlin; 1986. [7] B. Zuerndorfe and G. Wakefield; "Extensions of scalespace filtering to machine sensing," submitted to IEEE Trans. Part. Anal. Mach. Intell.. [8] J. Babaud, A. Witkin, M. Baudin, R. Duda; "Uniqueness of the Gaussian kernel for scale-space filtering", IEEE Trans. Patt. Anal. Mach. Intell., Vol. PAMI-8, No. 1; Jan 1986; PP. 26-33. [9] D. Marr and E. Hildreth; "Theory of edge detection," Proc. R. Soc. London B, Vol. 207; 1980; PP. 187-217. [10] C. Goffan; Calculs of Several Variables, Harper & Row; New York; 1965. [11] B. Zuerdorfer, A. England, F. Ulaby, and C. Dobson; "Mapping frzethaw oundaries with SMMR data'. submited to J. Agricultu and Forest Meteorology. i,@) = i@) - )2), (17) where the scale of sensor 1 is less than the scale of sensor 2. In Figures 1-4, the functions il(.) and ia(.) are quadratic in the vicinity of regional boundaries. The surface type changes at x=125, so that il(.) and i2(.) exhibit different behavior in the vicinity of boundaries for x<125 and x>125 (Figure 1). A threshold level is selectd to locate a boundary around the surface region with a low i(.) value. The coresponding fingerprint is shown in Figure 2. Figure 3 shows the i,(.) function

a *e In 3.0 -. 2.5 2.0 1.5 — hdeakx 1.0.Senorw ~......,ww2 0.5 0.0..........-1 0. 25. 50. 75. 100. 125. 150. 175. 200. 225. 250. X Positon Figure 1. Sensor input signals, indictor, and threshold. X Position Figure 2. Indicator fingerprints. CO) i In 3.0 2.5 2.0 1.5 1.0 0.5 0.0 _n s -— Sor1 0. 25. 50. 75. 100. 125. 150. 175. 200. 225. 250. X Positon Figure 3. High frequency (fine scale) sensor input signal and threshold. X Position Figure 4. High frequency sensor fingeprints.