0 Up votes0 Down votes

5 views46 pagesAug 06, 2011

© Attribution Non-Commercial (BY-NC)

PDF, TXT or read online from Scribd

Attribution Non-Commercial (BY-NC)

5 views

Attribution Non-Commercial (BY-NC)

- 24 Emergency _Preparednes
- Character Extraction From Documents Using Wavelet Maxima
- File Serve
- j 041215358
- Slope Calculation From Contour Lines in a Topographic Map
- A Combination of Wavelet Artificial Neural Networks Integrated with Bootstrap Sampler in Time Series Prediction
- Lec - 15-16 Denoising, Shrinkage and Other Transforms v4.0
- 10.1.1.5
- Tsunami.warning.30.Marzo.2015
- Texture Segmentation and classification in Biomedical Engineering
- mathgen-662640205
- Wavelets Hancock2006
- Effects of Nonlinearity on WAVELETS
- Assignment-tsunami Risk Assessment_Farhana
- UNDERSTANDING+NATURAL+DISASTERS
- g.bumi
- ch3_unit2
- Pythagorean Theorem Worksheet
- A COMPARATIVE STUDY OF DCT, DWT & HYBRID (DCT-DWT) TRANSFORM
- news article on earthquakes and volcanoes

You are on page 1of 46

www.math.yale.edu/~mmm82

1) Relate learning and approximation under smoothness constraints, motivate need for good 'building blocks for approximation 2) Some very classical building blocks in Euclidean spaces: Fourier and wavelets (a comparison) 3) New multiscale building blocks for manifolds: for this we need to talk about about diffusion geometries: 3a) Global diffusion geometries, eigenfunctions of the Laplacian 3b) Multiscale diffusion wavelets 4) Stylized applications 5) Future work

In many cases learning algorithms (or learning itself?) can be viewed as an approximation problem, under smoothness constraints, on a set of points/state space represented as a manifold or graph. [Think of SMVs as an example] Approximation Smoothness constraint Manifold/graph

Belief: the structure of the state space has to do with the learning task.

Goal

Develop tools for multiscale analysis of functions on manifolds, varifolds, graphs, datasets.

Motivations

Analyze large amount of data, and functions on this data, intrinsically rather low-dimensional, embedded in high dimensions. Paradigm: we have a large number of documents (e.g.: web pages, gene array data, (hyper)spectral data, molecular dynamics data etc...) and a way of measuring similarity between pairs. Model: a graph (G,E,W) In important practical cases: vertices are points in high-dimensional Euclidean space, weights may be a function of Euclidean distance.

Difficulties

Data sets in high-dimensions are complicated, as are classes of interesting functions on them. We want to do approximation (learning) of such functions. Parametrize low dimensional subspaces of smooth functions on sets embedded in highdimension. Fast algorithms

Documents, web searching Customer databases Satellite imagery Transaction logs Social networks Gene arrays, proteomics data Art transactions data Traffic (automobilistic, network) statistics

0.8

0.6

0.4

0.2

-0.2

-0.4

-0.6

-0.8

50

100

150

200

250

120

100

80

60

40

20

6 4

-2

-4

-6

-8

-10

-12

-14

50

100

150

200

250

300

[RR Coifman, S. Lafon] Related previous and current work by M. Belkin, P. Niyogi on eigenmaps and applications to semi-supervised learning on manifolds and graphs. Part of the material in the next few slides is courtesy of Stephane Lafon. From local to global: diffusion distances Motto: Diffusion distance measures and averages connections of all lengths. It is more stable and finer than geodesic distance, uses a preponderance of evidence

X = { x1 , x2 ,..., xN } data set.

k ( x , y ) kernel defined on the data and being symmetric k ( x, y) = k ( x, y) positivity-preserving: k ( x , y ) 0 positive semi-definite: ( x) ( y)k ( x, y) 0

x X y X

The kernel k describes the geometry of X by defining the relationship between the data points.

Examples: If X lies in n-dimensional Euclidean space, k(x,y) could be: - exponentially weighted distance: exp(-(||x-y||/a)^2 - angle: <x,y>/(||x|| ||y||) - harmonic potential: 1/(a+||x-y||), or powers of it - feature distances: any of the above applied in the range f(X), f nonlinear, possibly mapping X to higher dimension - if a model for the data is available (probabilistic, or from a (stochastic) dynamical system), k may be chosen to be consistent with that model If X is an abstract graph: we need to be given some weights on edges, measuring similarity. This could wildly vary: from the graph obtained by discretizing a PDE, to ways of measuring similarity between two web pages, or two protein chains...

Diffusion Distances

x

y

Diffusion embedding mapping X with diffusion distance into Euclidean space with Euclidean distance:

Phi1

Phi2

Phi3

Diffusion embedding mapping X with diffusion distance into Euclidean space with Euclidean distance:

Recent kernel methods: LLE (Roweis, Saul 2000), Laplacian Eigenmaps (Belkin, Niyogi 2002) , Hessian Eigenmaps (Donoho, Grimes 2003), LTSA (Zhang, Zha 2002) ... all based on the following paradigm: minimize Q( f ) where Q( f ) = Qx ( f )

x X

Qx ( f ) : quadratic form measuring local variation of f i a neighborhood of x n Solution: compute eigenfunctions {l }of Q and map data points via x (0 ( x), 1 ( x),..., p ( x)) T

x X

Applications

- Classifiers in the semi-supervised learning context (M. Belkin, P. Nyogi) - fMRI data (F. Meyer, X. Shen) - Art data (W Goetzmann, PW Jones, MM, J Walden) - Hyperspectral Imaging in Pathology (MM, GL Davis, F Warner, F. Geshwind, A Coppi, R. DeVerse, RR Coifman) - Molecular dynamics simulations (RR. Coifman, G.Hummer, I. Kevrekidis, MM) - Text documents classification (RR. Coifman, MM)

Art data

[W Goetzmann, PW Jones, MM, J Walden] Given the sales of art works (paintings and drawings) of 58 artists in for 9 years. View them as 58 points in 9 dimensions, try to find clusters that indicate how the works of groups of artists are 'traded' together. Does this correlate to style?

3 2 1 0 -1 -2 -3 -4 5 0 -5 -10 -4 -2 0 2 4 6 8

[MM, GL Davis, F Warner, F. Geshwind, A Coppi, R. DeVerse, RR Coifman]

So far...

..we have seen that: - it seems useful to consider a framework in which, for a given data set, local similarities are given only between very similar points - it is possible to organize these local information by diffusion into global parametrizations, - these parametrizations can be found by looking at the eigenvectors of a diffusion operator, - these eigenvectors in turn yield a nonlinear embedding into low-dimensional Euclidean space, - the eigenvectors can be used for global Fourier analysis on the set/manifold PROBLEM: Either very local information or very global information: in many problems the intermediate scales are very interesting (think of web pages)! Would like MULTISCALE information! Solution 1: proceed bottom up: repeatedly cluster together in a multi-scale fashion, in a way that is faithful to the operator: diffusion wavelets. Solution 2: proceed top bottom: cut greedily according to global information, and repeat procedure on the pieces: recursive partitioning, local cosines... Solution 3: do both!

We are given a graph X with weights W. There is a natural random walk P on X induced by these weights. P maps probability distributions on X to probability distributions on X. The reversibility condition on P implies that P is conjugate to a selfadjoint operator T. We assume T is renormalized to have 2-norm 1. The spectra of T and its powers look like:

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1

8 (T ) 4 (T ) 2 (T )

(T

16)

0 5 10 V 15 20

0

...

25

30

[Coifman,MM] Multiscale compression scheme: - Random walk for 1=2^0 step - Collect together random walkers into representatives - Write a random walk on the representatives - Let the representatives random walk 2=2^1 steps...

The decay in the spectrum of T says powers of T are low-rank, hence compressible.

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1

16 (T ) 8 (T ) 4 (T ) 2 (T )

0 5 10 V 15 20

0

...

25

30

=

We have frequencies: the eigenvalues of the diffusion T. What about dilations , translations , downsampling? We may have minimal information about the geometry, and only locally. Let's think in terms of functions on the set X. Dilations: Use the diffusion operator T and its dyadic powers as dilations. Translations and downsampling: Idea: diffusing a basis of scaling functions at a certain scale by a power of T should yield a redundant set of coarser scaling functions at the next coarser scale: reduce this set to a Riesz (i.e. well-conditioned) or even orthonormal basis. This is downsampling in the function space, and corresponds to finding a well-conditioned subset of translates.

Connections...

Wavelets: - Lifting (W. Sweldens, I. Daubechies,...) - Continuous wavelets from group actions (Grossman, Morlet, Ali, Antoine, Gazeau, many physicists...) Classical Harmonic Analysis: - Littlewood-Paley on semigroups (Stein), Markov diffusion semigroups (large literature) - Martingales associated to the above, Brownian motion - Heat kernel estimates, generalized Heisenberg principles (A. Nahmod) - Harmonic Analysis of eigenfunctions of the Laplacian on domains, manifolds and graphs - Atomic decompositions (Coifman, Weiss, Rochberg...) Numerics: - Algebraic Multigrid (Brandt, large literature since the 80's...) - Kind of inverse FMM (Rohklin, Beylkin-Coifman-Rohklin, ...) - Multiscale matrix compression techniques (Gu-Eisenstat, Chaen-GimbutasMartinsson-Rohklin,...) - Randomizable - FFTs?

[JC Bremer, RR Coifman, MM, AD Szlam]

We can split the wavelet subspaces further, in a hierarchical dyadic fashion, very much like in the classical case. The splittings are generated by numerical kernel and numerical range operations.

Compression example I

Denoising

[a la Coifman-Wickerhauser & Donoho-Johnstone]

Given 1,000 documents, each of which is associated with 1,000 words, with a value indicating the relevance of each word in that document. View this as a set of 1,000 in 1,000 dimensions, construct a graph with 1,000 vertices, each of which with few selected edges to very close-by documents.

0.15 0.2 0.1 0 0 -0.1 0 -0.2 -0.05 -0.05 0 0.05 0.1 -0.1 -0.2 0.1 0.05 0.05 0 0 -0.05 -0.1 -0.05 0.1 -0.05 0.05 -0.1 -0.15 0.1 0.15 0.1 0.05

Physics

0.2

0.2

0.1

0.1

0

0

-0.1

-0.1

-0.2 0.1 0.05 0 -0.05 -0.1 -0.05 0

0.2

0.1 0.05

0.15

0.1

Paleontology Climate

0.2

-0.1

Astronomy, planets

-0.2 0.1 0.05 0 -0.05 0.05 0 -0.1 -0.05 0 0.1 0.2 0.15 0.1

0.1

-0.1

-0.1

-0.2 0.1

Comets, asteroids

Documents

- By Jessica Gorman--"Pan scrapings!" announced G. Kenneth Sams. "When you come down to it, that is -what has brought us all here tonight-pan scrapings."-- It was probably not the most appetizing introduction for a $150per-plate -museum fundraiser. Some 150 guests, many in black tie, had just sat down to -enjoy their first course at lavish banquet tables in the Upper Egyptian Gallery -of the University of Pennsylvania Museum of Archaeology and Anthropology in -Philadelphia. Did they r Can chemical analysis confirm a wine's authenticity?-Damaris Christensen-- There's a romance to wine unmatched by most agricultural products. Making -wine, with all its complex flavors, remains as much an art as a science.-- Hints of citrus, peach, raspberry, pear, oak, grass, or flowers may show up in -the taste or smell of wines. The catalog of factors that determine the flavor -and bouquet is almost as long as the list of adjectives that connoisseurs use -to describe them.--

'chemical'

'food'

'ancient'

Documents Words 'nitrogen' 'plant' 'ecologist' 'carbon' 'global' by J. Raloff-- Acid rain and agricultural pollution 'specy' 'native' 'change' 'pollution' both spew nitrogen into the air. Though -plants need 'university' 'growth' 'soil' 'human' 'activity' nitrogen to grow, a new study finds that even small 'nutrient' 'dioxide' 'fuel' 'acid' 'cycle' additions of -this fertilizing pollutant can perturb the 'natural' landscape.-- In plots of Minnesota prairie to which ecologists applied nitrogen for 12 -years, native grasses showed a dramatically impaired ability to compete against -weeds that had immigrated from Europe centuries ago.-- The nitrogen treatment triggered the terrestrial equ ans = by C. Mlot-- For most of agricultural history, nitrogen has been a precious commodity. Only -specialized bacteria and lightning could convert atmospheric nitrogen into -biologically usable forms. Today, however, fertilizers and fossil fuels have -made nitrogen so freely available that it has become too much of a good thing.-- In a review of nitrogen's effects across the environmental spectrum, a team of -ecologists headed by Peter M. Vitousek of Stanford University has concluded in -no

Documents Words 'earthquake' 'wave' 'fault' 'quake' 'tsunami' New technologies convert the motion of waves into 'california' 'southern' 'rock' 'seismologist' watts-- Peter Weiss-'ocean' 'lake' 'coast' 'tsunamis' 'north' R. Monastersky-- The underwater volcano growing 'region' 'island' 'seismic' 'plate' 'crust' off Hawaii's south coastline collapsed 'power' - The Mush Zone A slurpy layer lurks deep inside the planet-by I. Peterson-- In late 1942, carrying 15,000 U.S. soldiers bound for England, the Queen Mary -hit a storm about 700 miles off the coast of Scotland. Without warning amid the -tumult, a single, mountainous wave struck the ocean liner, rolling it over and -washing water across its upper decks. Luckily, the ship managed to right itself -and continue on its voyage.-- Gargantuan waves, which appear unexpectedly even under calm conditions in the -open ocean, have damaged and sunk numerous s - Racing the Waves Seismologists try to catch quake tremors quickly enough to -save lives-- geologists have been snapping a very different picture of -the lake lately. Far beneath Lake Tahoe's gentle surface, they say, several -hidden earthquake faults snake across the lake's flat bottom. How a middling quake made a giant tsunami Waves of Death-- Why the New Guinea tsunami

This is a wavelet analysis on manifolds, graphs, markov chains, certain fractals, while Laplacian eigenfunctions do the corresponding Fourier Analysis, adapted to a given diffusion operator. We are compressing: powers of the operator, functions of the operators, subspaces of the function subspaces on which its powers act. This yields sampling formulas, quadrature formulas. Does not require the diffusion to be self-adjoint, nor needs the eigenvectors. We are constructing a biorthogonal version of the transform (better adapted to studying Markov chains): this will allow efficient denoising, compression, discrimination on all the spaces mentioned above. Preliminary: n log(n) with good constants for symmetric problems. The multiscale spaces are a natural scale of complexity spaces for learning empirical functions on the data set. Diffusion scaling functions extend outside the set, in a natural multiscale fashion. Exploring ties with measure-geometric considerations used for embedding metric spaces in Euclidean spaces with small distortion. Study and compression of dynamical systems.

- Use for regularization and learning on graphs and manifolds: semisupervised and plain supervised. - Robustness to noise, perturbations, extension of the functions to test points. - Biorthogonal construction (better localization, better constants) - Multiscale embeddings for graphs, measure-geometric implications - Martingale aspects and fast Brownian motion simulation in nonhomogeneous media - Compression of data sets - Going nonlinear This talk, papers, Matlab code available at: www.math.yale.edu/~mmm82 (or Google for Mauro Maggioni) Thank you!

- 24 Emergency _PreparednesUploaded bySelvaPunitha
- Character Extraction From Documents Using Wavelet MaximaUploaded byhossein_kho
- File ServeUploaded byNithya Velam
- j 041215358Uploaded byIOSRJEN : hard copy, certificates, Call for Papers 2013, publishing of journal
- Slope Calculation From Contour Lines in a Topographic MapUploaded byzartol
- A Combination of Wavelet Artificial Neural Networks Integrated with Bootstrap Sampler in Time Series PredictionUploaded byAnonymous 7VPPkWS8O
- Lec - 15-16 Denoising, Shrinkage and Other Transforms v4.0Uploaded byNikesh Bajaj
- 10.1.1.5Uploaded bySameera Roshan
- Tsunami.warning.30.Marzo.2015Uploaded byMario Zavala
- Texture Segmentation and classification in Biomedical EngineeringUploaded byFauzi Imaduddin Adhim
- mathgen-662640205Uploaded bymdp anon
- Wavelets Hancock2006Uploaded byJulian Palacios Espinoza
- Effects of Nonlinearity on WAVELETSUploaded byseventhsensegroup
- Assignment-tsunami Risk Assessment_FarhanaUploaded byFarhana Farhath
- UNDERSTANDING+NATURAL+DISASTERSUploaded byRajaDurai Ramakrishnan
- g.bumiUploaded byammarsyahmi
- ch3_unit2Uploaded byRajesh Vungarala
- Pythagorean Theorem WorksheetUploaded byrofi
- A COMPARATIVE STUDY OF DCT, DWT & HYBRID (DCT-DWT) TRANSFORMUploaded byGJESR
- news article on earthquakes and volcanoesUploaded byapi-240485135
- Passage Www.zetatalk4.ComUploaded byDesligandoaMatrix
- anderson time theoryUploaded byapi-303245090
- 091002 Samoa TsunamiUploaded byEmad Mansy
- Using the Triangle Inequality to AccelerateUploaded bymanjula7
- 1.9780898719543.ch2.pdfUploaded bySuntood
- Biggy BigUploaded byDaffa Firmansyah
- Set Theory Chap2Uploaded byapi-3823071
- 2100backgroundguideUploaded byapi-284993615
- General Training Test4Uploaded by채원종
- ABCUploaded byShakthidhar Rao

- LaplaceUploaded byAjith Kumar Arumugham
- Boundary Element MethodsUploaded byGowrishetty Srinivas
- Chapter 7 - Laplace, 2010Uploaded byjason403
- The Nehari Manifold for Nonlocal Elliptic Operators Involving Concave–Convex NonlinearitiesUploaded byOmagu
- Vector Calculus WikibookUploaded byLynn Hollenbeck Breindel
- Msc Phy 2014Uploaded bySouvik Naskar
- CU CBCS Physics SyllabusUploaded bySuman Chowdhury
- Intro2FEM.pdfUploaded byJay
- The Electrostatic PotencialUploaded byTochtli Terolerolino
- armonicos esfericosUploaded byCris Mantilla Suárez
- Laplace OperatorUploaded byguest
- KAPPA_DDA_book.pdfUploaded byFranck Iparraguirre
- Vector Calculus 2Uploaded byHergene Faner
- Degeneracy..Uploaded bypradeep khanna
- Mesh Motion Alternatives in OpenFOAMUploaded bybinho58
- Kappa Dda Book, Feb 2011Uploaded byGabriel Colmont
- Numerical Methods for Partial Differential EquationsUploaded byAlexey Voronin
- De Rham-Hodge Theory Lecture NotesUploaded bySteven Scott
- Math 3Uploaded bySohom Das
- ch0aUploaded bybbteenager
- SYMBOLS.docxUploaded byHemprasad Badgujar
- SpectralTheory LNUploaded byRamón Marcelo Bustos Méndez
- web.iitd.ac.in_~vvksrini_Igor-PDEUploaded bySaiRakesh
- An Implicit Corrected SPH Formulation for Thermal Diffusion With Linear Free Surface BoundaryUploaded byAnonymous HijNGQtN
- BYUOpticsBookUploaded byLuke Seed
- Derivation of the Laplace-Operator Derivation of Coordinates by Partial Derivative.pdfUploaded bysami821111
- The Hydrogen Atom (Uploaded byDebora Paskarina
- 05Uploaded bysurvinderpal
- The Analytic Element Method - Analiticki Elementi i Superpozicija !!!!Uploaded bymarcograndi1975
- legendreUploaded by0410058

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.