{"title": "Non-iterative Estimation with Perturbed Gaussian Markov Processes", "book": "Advances in Neural Information Processing Systems", "page_first": 531, "page_last": 538, "abstract": "", "full_text": "Non-iterative Estimation with Perturbed\n\nGaussian Markov Processes\n\nYunsong Huang\nSignal and Image Processing Institute\n\nB. Keith Jenkins\n\nDepartment of Electrical Engineering-Systems\n\nUniversity of Southern California\n\nLos Angeles, CA 90089-2564\n\n{yunsongh,jenkins}@sipi.usc.edu\n\nAbstract\n\nWe develop an approach for estimation with Gaussian Markov processes\nthat imposes a smoothness prior while allowing for discontinuities. In-\nstead of propagating information laterally between neighboring nodes\nin a graph, we study the posterior distribution of the hidden nodes as a\nwhole\u2014how it is perturbed by invoking discontinuities, or weakening\nthe edges, in the graph. We show that the resulting computation amounts\nto feed-forward fan-in operations reminiscent of V1 neurons. Moreover,\nusing suitable matrix preconditioners, the incurred matrix inverse and\ndeterminant can be approximated, without iteration, in the same compu-\ntational style. Simulation results illustrate the merits of this approach.\n\n1 Introduction\n\nTwo issues, (i) ef\ufb01cient representation, and (ii) ef\ufb01cient inference, are of central importance\nin the area of statistical modeling of vision problems. For generative models, often the ease\nof generation and the ease of inference are two con\ufb02icting features. Factor Analysis [1]\nand its variants, for example, model the input as a linear superposition of basis functions.\nWhile the generation, or synthesis, of the input is immediate, the inference part is usually\nnot. One may apply a set of \ufb01lters, e.g., Gabor \ufb01lters, to the input image. In so doing,\nhowever, the statistical modeling is only deferred, and further steps, either implicit or ex-\nplicit, are needed to capture the \u2018code\u2019 carried by those \ufb01lter responses. By characterizing\nmutual dependencies among adjacent nodes, Markov Random Field (MRF) [2] and graph-\nical models [3] are other powerful ways for modeling the input, which, when continuous,\nis often conveniently assumed to be Gaussian. In vision applications, it\u2019s suitable to em-\nploy smoothness priors admitting discontinuities [4]. Examples include weak membranes\nand plates [5], formulated in the context of variational energy minimization. Typically, the\ninference for MRF or graphical models would incur lateral propagation of information be-\ntween neighboring units [6]. This is appealing in the sense that it consists of only simple,\nlocal operations carried out in parallel. However, the resulting latency could undermine the\nplausibility that such algorithms are employed in human early vision inference tasks [7].\n\nIn this paper we take the weak membrane and plate as instances of Gaussian processes\n(GP). We show that the effect of marking each discontinuity (hereafter termed as \u201cbond-\n\n\fbreaking\u201d) is to perturb the inverse of covariance matrix of the hidden nodes x by a matrix\nof rank 1. When multiple bonds are broken, the computation of the posterior mean and\ncovariance of x would involve the inversion of a matrix, which typically has large con-\ndition number, implying very slow convergence in straight-forward iterative approaches.\nWe show that there exists a family of preconditioners that can bring the condition number\nclose to 1, thereby greatly speeding up the iteration\u2014to the extent that a single step would\nsuf\ufb01ce in practice. Therefore, the predominant computation employed in our approach is\nnoniterative, of fan-in and fan-out style. We also devise ways to learn the parameters re-\ngarding state and observation noise non-iteratively. Finally, we report experimental results\nof applying the proposed algorithm to image-denoising.\n\n2 Perturbing a Gaussian Markov Process (GMP)\nConsider a spatially invariant GMP de\ufb01ned on a torus, x \u223c N (0, Q0), whose energy\u2014\nde\ufb01ned as xT Q\u22121\n0 x\u2014is the sum of energies of all edges1 in the graph, due to the Markovian\nproperty. In what follows, we perturb the potential matrix Q\u22121\n0 by reducing the coupling\nenergy of certain bonds2. This relieves the smoothness constraint on the nodes connected\nvia those bonds.\nSuppose the energy reduction of a bond connecting node i and j (whose state vectors are xi\nand xj, respectively) can be expressed as (xT\nj fj)2, where fi and fj are coef\ufb01cient\nvectors. This becomes (xT f)2, if f is constructed to be a vector of same size as x, with the\nonly non-zero entries fi and fj corresponding to node i and j. This manipulation can be\n0 \u2212 f f T , which is equivalent\nidenti\ufb01ed with a rank-1 perturbation of Q\u22121\n0 x \u2212 (xT f)2,\u2200x. We call this an elementary perturbation of Q\u22121\nto xT Q\u22121\n0 ,\nand f an elementary perturbation vector associated with the particular bond.\nWhen L such perturbations have taken place (cf. Fig. 1), we form the L perturbation vectors\ninto a matrix F1 = [f 1, . . . , f L], and then the collective perturbations yield\n\n1 x \u2190 xT Q\u22121\n\n1 \u2190 Q\u22121\n\n0 , as Q\u22121\n\ni fi + xT\n\n= Q\u22121\n\nQ\u22121\nQ1 = Q0 + Q0F1(I \u2212 F T\n\n0 \u2212 F1F T\n\n1\n\n1\n\n1 Q0F1)\u22121F T\nwhich follows from the Sherman-Morrison-Woodbury Formula (SMWF).\n\nand thus\n\n1 Q0,\n\n(1)\n(2)\n\n2.1 Perturbing a membrane and a plate\n\n(cid:112)\n(1 \u2212 \u03b72)/q and fj = \u2212fi.\n\nIn a membrane model [5], xi is scalar and the energy of the bond connecting xi and xj is\n(xi \u2212 xj)2/q, where q is a parameter denoting the variance of state noise. Upon perturba-\ntion, this energy is reduced to \u03b72(xi \u2212 xj)2/q, where 0 < \u03b7 (cid:28) 1 ensures positivity of the\nenergy. Then, the energy reduction is (1 \u2212 \u03b72)(xi \u2212 xj)2/q, from which we can identify\nfi =\nIn the case of a plate [5], xi = [ui, uhi, uvi]T , in which ui represents the intensity, while\nuhi and uvi represent its gradient in the horizontal and vertical direction, respectively.\nWe de\ufb01ne the energy of a horizontal bond connecting node j and i as E(\u2212,i)\n= (uvi \u2212\n(cid:21)\nuvj)2/q + d(\u2212,i)T O\u22121d(\u2212,i), where\n1 1\n0 1\n\n1/3 1/2\n1/2\n1\n\nand O = q\n\nd(\u2212,i) =\n\n(cid:21)(cid:20)\n\n(cid:21)\n\n(cid:20)\n\n(cid:20)\n\n(cid:20)\n\n(cid:21)\n\n\u2212\n\nuj\nuhj\n\n0\n\n,\n\nui\nuhi\n\n1Henceforth called bonds, as edge will refer to intensity discontinuity in an image.\n2The bond energy remains positive. This ensures the positive de\ufb01niteness of the potential matrix.\n\n\f1\n\nthe superscript (\u2212, i) representing horizontal bond to the left of node i. The \ufb01rst and sec-\nond term of E(\u2212,i) would correspond to (\u22022u(h, v)/\u2202h\u2202v)2/q and (\u22022u(h, v)/\u2202h2)2/q,\nrespectively, if u(h, v) is a continuous function of h and v (cf. [5]).\nis re-\n= [(uvi \u2212 uvj)2 + (uhi \u2212 uhj)2]/q, i.e., coupling between node i\nduced to E(\u2212,i)\n(cid:112)\nand j exists only through their gradient values, one can show that the energy reduction\n= [ui \u2212 uj \u2212 (uhi + uhj)/2]2\u00b7 12/q. Taking the actual energy reduction to\n\u2212 E(\u2212,i)\nE(\u2212,i)\n(cid:112)\n12(1 \u2212 \u03b72)/q[1,\u22121/2, 0]T\n\u2212 E(\u2212,i)\nbe (1 \u2212 \u03b72)(E(\u2212,i)\n12(1 \u2212 \u03b72)/q[\u22121, \u22121/2, 0]T , where 0 < \u03b7 (cid:28) 1 ensures the posi-\nand fj\ntive de\ufb01niteness of the resulting potential matrix. A similar procedure can be applied\nto a vertical bond in the plate, producing a perturbation vector f (|,i), whose compo-\n(cid:112)\n(|,i) =\nnents are zero everywhere except for fi\n\n(cid:112)\n12(1 \u2212 \u03b72)/q[1, 0,\u22121/2]T and fj\n12(1 \u2212 \u03b72)/q[\u22121, 0, \u22121/2]T , for which node j is the lower neighbor of node i.\n\n), we can identify fi\n\nIf E(\u2212,i)\n\n(\u2212,i) =\n\n(\u2212,i) =\n\n(|,i) =\n\n0\n\n1\n\n0\n\n1\n\n0\n\nOne can verify that xT f = 0 when the plate assumes the shape of a linear slope, mean-\ning that this perturbation produces no energy difference in such a case. (xT f)2 becomes\nsigni\ufb01cant when the perturbed, or broken, bond associated with f straddles across a step\ndiscontinuity of the image. Such an f is thus related to edge detection.\n\n2.2 Hidden state estimation\nStandard formulae exist for the posterior covariance K and mean \u02c6x of x, given a noisy\nobservation3 y = Cx + n, where n \u223c N (0, rI).\n\n\u03b1 + CT C/r]\u22121,\nfor either the unperturbed (\u03b1 = 0) or perturbed (\u03b1 = 1) process. Thus,\n\nand K\u03b1 = [Q\u22121\n\n\u02c6x\u03b1 = K\u03b1CT y/r,\n\n1 ]\u22121,\n\n0 + CT C/r \u2212 F1F T\nK1 = [Q\u22121\n0 \u2212 F1F T\n= [K\u22121\n=K 0 + W1H\u22121\n1 K0F1,\n\n1 ]\u22121,\n1 W T\n1 ,\nand W1 (cid:44) K0F1\n\napplying SMWF,\n\nwhere H1 (cid:44) I \u2212 F T\n\nfollowing Eq. 3 and 1\n\n(3)\n\n(4)\n(5)\n\n(6)\n\n\u2234 \u02c6x1 = K1CT y/r\n\n= K0CT y/r + W1H\u22121\n1 W T\n1 CT y/r,\n1 z1, where\n\n\u02c6xc (cid:44) W1H\u22121\n= W1H\u22121\n\nwhere\n\n1 W T\n\n1 CT y/r = \u02c6x0 + \u02c6xc,\n\nz1 = W T\n\n1 CT y/r\n\n(7)\nOn a digital computer, the above computation can be ef\ufb01ciently implemented in the Fourier\ndomain, despite the huge size of K\u03b1 and Q\u03b1. For example, K1 equals K0\u2014a circulant\nmatrix\u2014plus a rank-L perturbation (cf. Eq. 4). Since each column of W1 is a spatially\nshifted copy of a prototypical vector, arising from breaking either a horizontal or a vertical\nbond, convolution can be utilized in computing W T\nis\ndeferred to Section 3. On a neural substrate, however, the computation can be implemented\nby inner-products in parallel. For instance, z1r is the result of inner-products between\nthe input y and the feed-forward fan-in weights CW , coded by the dendrites of identical\nneurons, each situated at a broken bond. Let v1 = H\u22121\n1 z1 be the responses of another layer\nof neurons. Then C \u02c6xc = CW v1 amounts to the back-projection of layer v1 to the input\nplane with fan-out weights identical to the fan-in counterpart.\nWe can also apply the above procedure incrementally4, i.e., apply F1 and then F2, both\nconsisting of a set of perturbation vectors. Quantities resulting from the \u03b1\u2019th perturba-\n\n1 CT y. The computation of H\u22121\n\n1\n\n3The observation matrix C = I for a membrane, and C = I \u2297 [1, 0, 0] for a plate.\n4Latency considerations, however, preclude the practicability of fully incremental computation.\n\n\f0.01\n\n0.005\n\nl\n\ne\nu\na\nv\n \nt\n\n0\n\ni\n\nh\ng\ne\nW\n\u22120.005\n\n\u22120.01\n\n10\n\n15\n\n(c)\n\n20\n\n25\n\n(a)\n\n(b)\n\nFigure 1: A portion of\nMRF. Solid and broken\nlines denote intact and\nbroken bonds, respec-\ntively. Open circles de-\nnote hidden nodes xi\nand \ufb01lled circles denote\nobserved nodes yi.\n\nFigure 2: The resulting receptive \ufb01eld of the edge detector pro-\nduced by breaking the shaded bond shown in Fig. 1. The central\nvertical dashed line in (a) and (b) marks the location of the ver-\ntical streak of bonds shown as broken in Fig. 1. In (a), those\nbonds are not actually broken; in (b), they are. In (c), a central\nhorizontal slice of (a) is plotted as a solid curve and the coun-\nterpart of (b) as a dashed curve.\n\ny, \u02c6x1\n\u02c6xc\n\u02c6x0\n\nFigure 3: Estimation of x given input y. \u02c6x0: by unperturbed rod; \u02c6x1: coinciding per-\nfectly with y, is obtained by a rod whose two bonds at the step edges of y are broken; \u02c6xc:\ncorrection term, engendered by the perturbed rod.\n\n(cid:124)(cid:123)(cid:122)(cid:125)\n\n(cid:124)\n(cid:123)(cid:122)\n+ W1H\u22121\n\n1 W T\n\u03b4W2\n\n(cid:125)\n\ntion step can be obtained from those of the (\u03b1 \u2212 1)\u2019th step, simply by replacing the sub-\nscript/superscript \u20181\u2019 and \u20180\u2019 with \u03b1 and \u03b1 \u2212 1, respectively, in Eqs. 1 to 6. In particular,\n(8)\n\nW2 = K1F2 = K0F2\n\n1 F2\n\n,\n\ng\nW2\n\nwhere (cid:102)W2 refers to the weights due to F2 in the absence of perturbation F1, which, when\nmembrane model, wherein \u2018receptive \ufb01eld\u2019 refers to (cid:102)W2 and W2 in the case of panel (a)\n\nindeed existent, would exert a contextual effect on F2, thereby contributing to the term\n\u03b4W2.\nFigure 2 illustrates this effect on one perturbation vector (termed \u2018edge detector\u2019) in a\n\nand (b), respectively. Evidently, the receptive \ufb01eld of W2 across the contextual boundary\nis pinched off. Figure 3 shows the estimation of x, cf. Eq. 6 and 7, using a 1D plate, i.e.,\nrod. We stress that once the relevant edges are detected, \u02c6xc is computed almost instantly,\nwithout the need of iterative re\ufb01nement via lateral propagation. This could be related to the\nbrightness \ufb01lling-in signal[8].\n\n2.3 Parameter estimation\n\nAs edge inference/detection is outside the scope of this paper, we limit our attention to\n\ufb01nding optimal values for the parameters r and q. Although the EM algorithm is possible\n\n\f\u22121\n\ny)/r)/2\n\n\u22121y)/2\n\n= \u2212(N Ln(2\u03c0) + N Lnr + Ln|(cid:102)S\u03b1| + (yT(cid:102)S\u03b1\n\nfor that purpose, we strive for a non-iterative alternative. To that end, we reparameterize r\nwe have y \u223c N (0, S\u03b1), where S\u03b1 = rI + CQ\u03b1CT . Note that (cid:102)S\u03b1 (cid:44) S\u03b1/r does not\nand q into r and \u0001 = q/r. Given a possibly perturbed model M\u03b1, in which x \u223c N (0, Q\u03b1),\ndepend on r when \u0001 is \ufb01xed, as Q\u03b1 \u221d q \u221d r =\u21d2 S\u03b1 \u221d r. Next, we aim to maximize the\nlog-probability of y, which is a vector of N components (or pixels).\n\u02dcJ\u03b1 (cid:44) Lnp(y|M\u03b1) = \u2212(N Ln(2\u03c0) + Ln|S\u03b1| + yT S\u03b1\nSetting \u2202 \u02dcJ\u03b1/\u2202r = 0 \u21d2 \u02c6r =E \u03b1/N, where E\u03b1 (cid:44) yT(cid:102)S\u03b1\nDe\ufb01ne J (cid:44) N LnE\u03b1 + Ln|(cid:102)S\u03b1| = const. \u2212 2 \u02dcJ\u03b1|\u02c6r\n\n(9)\n(10)\nJ is a function of \u0001 only, and we locate the \u02c6\u0001 that minimizes J as follows. Prompted by the\nfact that \u0001 governs the spatial scale of the process [5] and scale channels exist in primate\nvisual system, we compute J(\u0001) for a preselected set of \u0001, corresponding to spatial scales\nhalf-octave apart, and then \ufb01t the resulting J\u2019s with a cubic polynomial, whose location of\nminimum suggests \u02c6\u0001. We use this approach in Section 4.\nComputing J in Eq. 10 needs two identities, which are included here without proof (the\nsecond can be proven by using SMWF and its associated determinant identity): E\u03b1 =\nyT (y \u2212 C \u02c6x\u03b1) (cf. Appendix A of [5]), and |S0|/|S\u03b1| = |B\u03b1|/|H\u03b1|, where\nis, E\u03b1 can be readily obtained once \u02c6x\u03b1 has been estimated, and |(cid:102)S\u03b1| =\n|(cid:102)S0||H\u03b1|/|B\u03b1|, in which |(cid:102)S0| can be calculated in the spectral domain, as S0 is circulant.\nThat\nThe computation of |H\u03b1| and |B\u03b1| is dealt with in the next section.\n\nand B\u03b1 (cid:44) I \u2212 F\u03b1\n\nH\u03b1 = I \u2212 F\u03b1\n\nT K0F\u03b1,\n\n\u22121\n\ny\n\nT Q0F\u03b1\n\n(11)\n\n3 Matrix Preconditioning\n\nSome of the foregoing computation necessitates matrix determinant and matrix inverse,\ne.g., H\u22121z1(cf. Eq. 7). Because H is typically poorly conditioned, plain iterative means to\nevaluate H\u22121za would converge very slowly. Methods exist in the literature for \ufb01nding a\nmatrix P ([9] and references therein) satisfying the following two criteria: (1) inverting P\nis easy; (2) the condition number \u03ba(P \u22121H) approaches 1. Ideally, \u03ba(P \u22121H) = 1 implies\nP = H. Here we summarize our \ufb01ndings regarding the best class of preconditioners when\nH arises from some prototypical con\ufb01gurations of bond breaking. We call the following\nprocedure Approximate Diagonalization (AD).\n\n(cid:81)\n\n(1) \u2018DFT\u2019. When a streak of broken bonds forms a closed contour, with a consistent polarity\nconvention (e.g., the excitatory region of the receptive \ufb01eld of the edge detector associated\nwith each bond lies inside the enclosed region), H and B (cf. Eq. 11) are approximately cir-\nii, then (cid:101)H = X\u039bHX\u2020\nculant. Let X be the unitary Fourier matrix of same size as H, then H e = X\u2020HX would\nbe approximately diagonal. Let \u039bH be diagonal: \u039bH ij = \u03b4ijH e\nis\ni \u039bH ii approximates |H|; X\u039bH\n\u22121X\u2020\na circulant matrix approximating H;\napprox-\n\u22121X\u2020z1, which\nimates H\u22121. In this way, a computation such as H\u22121z1 becomes X\u039bH\nweight vector. The quality of this preconditioner (cid:101)H can be evaluated by both the condition\namounts to simple fan-in and fan-out operations, if we regard each column of X as a fan-in\nnumber \u03ba((cid:101)H\u22121H) and the relative error between the inverse matrices:\n(12)\nwhere k (cid:5) kF denotes Frobenius norm. The same X can approximately diagonalize B, and\nthe product of the diagonal elements of the resulting matrix approximates |B|.\n\n\u0001 (cid:44) k(cid:101)H\u22121 \u2212 H\u22121kF /kH\u22121kF ,\n\n\f(2) \u2018DCST\u2019. One end of the streak of broken bonds (target contour) abuts another contour,\nand the other end is open (i.e., line-end). Imagine a vibrational mode of the membrane/plate\ngiven the con\ufb01guration of broken bonds. The vibrational contrast of the nodes across the\nbroken bond at a line-end has to be small, since in the immediate vicinity there exist paths\nof intact bonds linking the two nodes. This suggests a Dirichlet boundary condition at\nthe line-end. At the abutting end (i.e., a T-junction), however, the vibrational contrast can\nbe large, since the nodes on different sides of the contour are practically decoupled. This\nsuggests a von Neumann boundary condition. This analysis leads to using a transform\n(termed \u2018HSWA\u2019 in [10]) which we call \u2018DCST\u2019, denoting sine phase at the open end and\n\u221a\ncosine phase at the abutting end. The unitary transform matrix X is given by: Xi,j =\n2L + 1 cos(\u03c0(i \u2212 1/2)(j \u2212 1/2)/(L + 1/2)), 1 \u2264 i, j \u2264 L, where L is the number of\n2\nbroken bonds in the target contour.\n(3) \u2018DST\u2019. When the streak of broken bonds form an open-ended contour, H can be approx-\nimately diagonalized by Sine Transform (cf. the intuitive rationale stated in case (2)), of\nwhich the unitary transform matrix X is given by: Xi,j =\n2/(L + 1) sin(\u03c0ij/(L + 1)),\n1 \u2264 i, j \u2264 L.\nFor a \u2018clean\u2019 prototypical contour, the performance of such preconditioners is remarkable,\ntypically producing 1 \u2264 \u03ba < 1.2 and \u0001 < 0.05. When contours in the image are intercon-\nnected in a complex way, we \ufb01rst parse the image domain into non-overlapping enclosed\nregions, and then treat each region independently. A contour segment dividing two re-\ngions is shared between them, and thus would contribute two copies, each belonging to one\nregion[11].\n\n(cid:112)\n\n4 Experiment\n\nWe test our approach on a real image (Fig. 4a), which is corrupted with three increasing\nlevels of white Gaussian noise: SNR = 4.79db (Fig. 4b), 3.52db, and 2.34db. Our task is to\nestimate the original image, along with \ufb01nding optimal q and r. We used both membrane\nand plate models, and in each case we used both the \u2018direct\u2019 method, which directly com-\nputes H\u22121 in Eq. 7 and |H|/|B| required in Eq. 10, and the \u2018AD\u2019 method, as described in\nSection 3, to compute those quantities in approximation.\n\nWe \ufb01rst apply a Canny detector to generate an edge map (Fig. 4g) for each noisy image,\nwhich is then converted to broken bonds. The large number (over 104) of broken bonds\nmakes the direct method impractical. In order to attain a \u2018direct\u2019 result, we partition the\nimage domain into a 5 \u00d7 5 array of blocks (one such block is delineated by the inner\nsquare in Fig. 4g), and focus on each of them in turn by retaining edges not more than 10\npixels from the target block (this block\u2019s outer scope is delineated with the outer square in\nFig. 4g). When \u02c6x is inferred given this partial edge map, only its pixels within the block\nare considered valid and are retained. We mosaic up \u02c6x from all those blocks to get the\ncomplete inferred image. In \u2018AD\u2019, we parse the contours in each block and apply different\ndiagonalizers accordingly, as summarized in Section 3. The performance of the three types\nof AD is plotted in Fig. 5, from which it is evident that in majority of cases \u03ba < 1.5 and\n\u0001 \u2264 10%. Fig. 4e and f illustrate the procedure to \ufb01nd optimal q/r for a membrane and a\nplate, respectively, as explained in Section 2.3. Note how good the cubic polynomial \ufb01t is,\nand that the results of AD do not deviate much from those of the direct (rigorous) method.\nFig. 4c and 4d show \u02c6x by a perturbed and intact membrane model, respectively. Notice that\nthe edges, for instance around Lena\u2019s shoulder and her hat, in Fig. 4d are more smeared\nthan those in Fig. 4c (cf. Fig. 3). Table 1 summarizes the value of optimal q/r and Mean-\nSquared-Error (MSE). Our results compare favorably with those listed in the last column\nof the table, which is excerpted from [12].\n\n\f(a)\n\n(d)\n\ndirect\ncubic fit\nAD\ncubic fit\nextremum\n\n(b)\n\nx 104\n\n5\n\n4.8\n\n4.6\n\nJ\n\n4.4\n\n4.2\n\n4\n\n3.8\n\n0.1\n\nq/r\n(e)\n\n(c)\n\nx 104\n\n5.5\n\n5\n\nJ\n\n4.5\n\n4\n\n1\n\n0.01\n\n0.1\n\n1\n\nq/r\n(f)\n\nFigure 4: (a) Original image, (b) noisy image. Estimation by (c) a\nperturbed membrane, and (d) an intact membrane. The criterion func-\ntion of varying q/r for (e) perturbed membrane, and (f) perturbed plate,\nwhich shares the same legend as in (e). (g) Canny edge map.\n\n(g)\n\n\u03ba\n\n2.2\n\n2\n\n1.8\n\n1.6\n\n1.4\n\n0.2\n\n0.15\n\n0\n\n20\n\n40\n\n200\n\n400\n\n2\n\n1.5\n\n1\n\n0\n\n0.2\n\n0.15\n\n0.1\n\n0.05\n\n\u0001\n\n0.1\n\n0.05\n\n0\n\n0\n\n10\n\n20\n\n30\n\n0\n\n0\n\n(a)\nDFT\n\n100\n\n200\n\n(b)\nDST\n\n3\n\n2.5\n\n2\n\n1.5\n\n1\n\n0\n\n0.25\n\n0.2\n\n0.15\n\n0.1\n\n0.05\n\n0\n\n0\n\n50\n\n100\n\n100\n\n50\n(c)\n\nDCST\n\nafter\n\nFigure 5: Histograms\nof condition number\n\u03ba\nprecondi-\ntioning, and relative\nerror \u0001 as de\ufb01ned in\nillustrating\nEq. 12,\nperformance\nthe\npreconditioners,\nof\nand\nDFT,\nDCST, on their\nre-\nspective\ndatasets.\nHorizontal axes in-\ndicate the number of\noccurrences in each\nbin.\n\nDST,\n\n\fTable 1: Optimal q/r and MSE.\n\nmembrane model\nAD\n\ndirect\n\nq/r\n\n0.456\n0.299\n0.217\n\nMSE\n92\n104\n115\n\nq/r\n\n0.444\n0.311\n0.233\n\nMSE\n92\n104\n115\n\nSNR\n\n4.79\n3.52\n2.34\n\ndirect\n\nq/r\n\n0.067\n0.044\n0.033\n\nMSE\n100\n111\n119\n\nplate model\n\nAD\n\nImproved\n\nEntropic [12]\n\nq/r\n\n0.075\n0.049\n0.031\n\nMSE\n98\n108\n121\n\nMSE\n121\n138\n166\n\n5 Conclusions\n\nWe have shown how the estimation with perturbed Gaussian Markov processes\u2014hidden\nstate and parameter estimation\u2014can be carried out in non-iterative way. We have adopted\na holistic viewpoint. Instead of focusing on each individual hidden node, we have taken\neach process as an entity under scrutiny. This paradigm shift changes the way information\nis stored and represented\u2014from the scenario where the global pattern of the process is\nembodied entirely by local couplings to the scenario where fan-in and fan-out weigths, in\naddition to local couplings, re\ufb02ect the patterns of larger scales.\n\nAlthough edge detection has not been treated in this paper, our formulation is capable of\ndoing so, and our preliminary results are encouraging. It may be premature at this stage to\ntranslate the operations of our model to neural substrate; we speculate nevertheless that our\napproach may have relevance to understanding biological visual systems.\n\nAcknowledgments\n\nThis work was supported in part by the TRW Foundation, ARO (Grant Nos. DAAG55-98-\n1-0293 and DAAD19-99-1-0057), and DARPA (Grant No. DAAD19-0010356).\nReferences\n[1] Z. Ghahramani and M.J. Beal. Variational inference for Bayesian mixtures of factor analysers.\n\nIn Advances in Neural Information Processing Systems, volume 12. MIT Press, 2000.\n[2] S.Z. Li. Markov Random Field Modeling in Computer Vision. Springer-Verlag, 1995.\n[3] M.I. Jordan, Z. Ghahramani, T.S. Jaakkola, and L.K. Saul. An introduction to variational meth-\n\nods for graphical models. Machine Learning, 37:183\u2013233, 1999.\n\n[4] F. C. Jeng and J. W. Woods. Compound Gauss-Markov random \ufb01elds for image estimation.\n\nIEEE Trans. on Signal Processing, 39(3):683\u2013697, 1991.\n\n[5] A. Blake and A. Zisserman. Visual Reconstruction. MIT Press, 1987.\n[6] J.S. Yedidia, W.T. Freeman, and Y. Weiss. Bethe free energy, kikuchi approximations, and\n\nbelief propagation algorithms. Technical Report TR2001-16, MERL, May 2001.\n\n[7] S. Thorpe, D. Fize, and C. Marlot. Speed of processing in the human visual system. Nature,\n\n381:520\u2013522, 1996.\n\n[8] L. Pessoa and P. De Weerd, editors. Filling-in: From Perceptual Completion to Cortical Reor-\n\nganization. Oxford: Oxford University Press, 2003.\n\n[9] R. Chan, M. Ng, and C. Wong. Sine transform based preconditioners for symmetric toeplitz\n\nsystems. Linear Algebra and its Applications, 232:237\u2013259, 1996.\n\n[10] S. A. Martucci. Symmetric convolution and the discrete sine and cosine transforms.\n\nTrans. on Signal Processing, 42(5):1038\u20131051, May 1994.\n\nIEEE\n\n[11] H. Zhou, H. Friedman, and R. von der Heydt. Coding of border ownership in monkey visual\n\ncortex. J. Neuroscience, 20(17):6594\u20136611, 2000.\n\n[12] A. Ben Hamza, H. Krim, and G. B. Unal. Unifying probabilistic and variational estimation.\n\nIEEE Signal Processing Magazine, pages 37\u201347, September 2002.\n\n\f", "award": [], "sourceid": 2908, "authors": [{"given_name": "Yunsong", "family_name": "Huang", "institution": null}, {"given_name": "B.", "family_name": "Jenkins", "institution": null}]}