You are on page 1of 14

Data-centric computing with Netezza Architecture

DISC reading group September 24, 2007

High Level Points • Supercomputer use model today: – Compile. submit. wait – Does a poor job of taking advantage of human insight available in interactive models • Large datasets can be interactively processed using Netezza .

What is Netezza? • Essentially: A big. fast SQL database .

What is Netezza? • Frontend provides SQL interface • Backend is a large rack of specialized blades .

e..Custom Backend Blades • Commodity CPU. i. disk • Custom FPGA replaces disk interface – Can do basic filtering in hardware. NIC. stream processing before data hits main memory .

Division of Data • Database distributed across multiple (100+) SPUs • Each SPU controls. manages its slice of DB • No info on data management. . replciation. etc.

parsing.Division of Labor • SPU FPGA handles basic filtering tasks • SPU CPU handles record level processing: filtering. projecting. • SPU CPU handles most operations on intermediate results: sorts. logging. aggregates • Frontend CPU handles remaining operation >>> Processing close to disk . joins. etc.

What can this be used for? • Paper gives 3 examples: – Citation graph processing – Search for particular structure in electrical netlist – Word meaning disambiguation through search of ontology .

. 80-100x speedup on smaller tests .Citation graph example • Look through large.g. 388 million edges) • Find both strong (direct edge) and weak couplings (e. sparse graph (16 million nodes. two papers cite the same work) • Essentially same code for workstation and Netezza – no need to expose parallel architecture • Workstation DNF.

10 million wires • Search for AND structure .IC netlist example • Flattened netlist of 3.5 million transistors.

as well as many other (1300+) identical structures generated through other means .IC example results • Combinatorial explosion makes directly joining all possibilities for each element impossible • Can constrain better using fanouts of signals internal to the circuit • Individual SQL queries for finding possible matches for the individual transistors took under 10 seconds • Found all uses of the AND macro.

concepts. IS-A relations. and constraints on concepts • Goal is to search the space.Ontology example • Expand out all possible interpretations of a phrase • Ontology specifies lexical elements. expand concepts to find all matches to given phrase .

• can pipeline multiple queries . vs. but reduces iterations / recursions • Recoded ontology triples as integers • 5.Ontology results • Partially unfolded ontology – Greatly expands database size.58 sec. 262 sec.

Issues • Works if you can reduce your problem to SQL queries • All of the problems were based on graph expansion / exploration – how about other domains? • Issues of database partitioning? How does arbitrary slicing across 108 blades affect performance / scalability. for non-sparse problems? • Strawman comparison to workstation class machine: how does a traditional DB server / storage cluster compare? . esp.