Professional Documents
Culture Documents
To cite this article: Govindarajan Natarajan & R. J. Bezama (2007) Microjet Cooler with Distributed
Returns, Heat Transfer Engineering, 28:8-9, 779-787, DOI: 10.1080/01457630701328627
This paper details the technology elements developed to design and manufacture a liquid microjet array cooling device for the
thermal management of very high power dissipating electronic chips. Multilayer ceramic technology (MLC) is used to build
the cooling device with micron-size jet arrays, which includes a distributed return network for the spent fluid. Intertwined
microchannel flow networks inside the cooler body distribute the flow in and out of the device. A cooler with 1600 jets and
1681 interstitial returns for the drains built using glass ceramic material is discussed. When tested with an 18 mm heated
silicon chip and an average convection coefficient of 0.052 MW/m2 K, the device demonstrated a cooling capability greater
than 2.5 MW/m2 with a water pressure drop of < 70 kPa. Further extension of the cooling capability to greater than 6 MW/m2 ,
as predicted by the simulation, is also discussed.
detailed below. Then, the flow manifold section that feeds the
jet array section of the cooler is designed by taking advantage of
the capabilities of MLC technology. Of course, the complexity
and effort required of the second section depends mainly on
whether the cooler does or does not include a distributed return
manifold, and so this design aspect must be defined together
with the selection of jet array geometry.
Many combinations of cooler geometric parameters will sat-
isfy the given requirements for thermal performance and cooler
pressure drop. Thus, the first step in the design effort is to select
a manufacturable jet pitch and diameter capable of delivering
the required thermal performance. The existing MLC manufac-
turing plant at IBM East Fishkill builds sophisticated micro-
electronic MLC modules for high-end computers with stringent
ground rules for features. This gives us flexibility to test the best
options possible. Considering that the allowed pressure drop is
relatively low, the jet velocity for cooler using water as fluid
was kept below 2.5–3.0 m/s. Under these conditions, a jet diam-
eter of 200 microns or less will operate with a low jet Reynolds
number, such as Re < 500, which is well within the laminar
flow regime. Nevertheless, an initial estimate for the jet-to-jet
pitch, jet diameter, and jet-to-chip gap can be obtained using the
following correlation provided by Martin [3], in spite of the fact
that the given correlations apply to jet arrays without distributed
return and relatively large jet Reynolds number, or Re > 2000,
which can get into the turbulent jet regime.
where
K2 = K3 (2 − 4.4K3 )/(1 + 0.2(H/D − 6)K3 ) Figure 2 (a) Convection coefficient for jet impingement using water at room
temperature as a function of jet diameter. In all cases, the jet velocity is 3 m/s,
K3 = 0.886/(P/D) P/D = 3.5, and H/D = 2. Martin’s correlation is used to extrapolate the convection
coefficient to lower Reynolds number. (b) Thermal coefficient K1 , from Eq. (1),
2000 < Re < 100,000; 0.004 < K23 < 0.040; 2 < H/D < 12 depends only on the geometric ratios P/D and H/D. Depending on H/D ratio,
the optimal P/D ratio falls in the 3-4 range.
To verify the applicability of using Martin’s correlation to pre-
dict jet array Nusselt number for low Reynolds number regime, jet diameter when the dimensionless jet pitch falls in the range of
we have run a limited number of simulations using CFD code us- 3 to 4 units, as per Figure 2b. Also, the correlation predicts an in-
ing jet array geometric parameters in a range that is compatible creasing thermal performance with a decreasing dimensionless
with MLC fabrication technology. The results of this compari- jet-to-chip gap size and projects even higher performance below
son, illustrated graphically in Figure 2a (Re = 75 to 600), show the given dimensionless gap limit of 2 units. However, this trend
that Martin’s correlation systematically underpredicts the jet is not supported by CFD simulations. Instead, CFD predicts op-
Nusselt number by about 10–20% in this low Reynolds regime; timal thermal performance with limited pressure drop when the
hence, it is suitable for use to get a preliminary design region of dimensionless gap is about 2 units. As the dimensionless gap
interest for the desired jet array. ratio gets below 2 units, frictional losses at the top and bottom
The most significant aspect of Martin’s correlation, which gap surfaces increase sufficiently to reduce the jet vortex circu-
makes it a powerful design tool, is that it decouples the jet lation speed and thus reduces the heat transfer rate. Using this
geometric parameters from the fluid operating conditions and approach, we estimate that the jet diameter and pitch must fall
thermal properties. In our analysis, all geometric parameters within 100–150 microns and 400–450 microns, respectively, to
are made dimensionless using the jet diameter as characteristic deliver the target cooling rate of 2 MW/m2 .
length. A detailed scrutiny of correlation dependency on jet ge- The second step is to decide if distributed return is needed
ometry suggests maximum thermal performance peaks for any or not. Because distributed return increases significantly the
heat transfer engineering vol. 28 nos. 8–9 2007
782 G. NATARAJAN AND R. J. BEZAMA
Figure 4 Contours of velocity under a submerged jet array with 625 jets. Jet
numbers start from 1 at center of array. Jet diameter is 100 microns, H/D = 3,
P/D = 4, V = 3 m/s, q = 4 MW/m2 .
Table 1 Parameters and CFD results for three different jet arrays with
distributed return
Setup parameters
Jet pitch [microns] 500 500 350
Gap height [microns] 400 400 200
Jet velocity [m/s] 2.5 7.0 25
q” [MW/m2 ] 2.0 5.0 10.0
Results
Re 250 700 2500
Maximum chip temp [◦ C] 59.7 78.8 97.5
P [kPa] 1.7 7.3 48
h [MW/m2 K] 0.059 0.070 0.352
Nu 9.8 11.7 59
Thermal resistance [m2 K/MW] 19 11 8
In all three cases, the jet diameter is 100 microns, the chip thickness is 700
microns, and the fluid is water at 22◦ C.
Figure 6 Single submerged jet model using axisymmetric 2D geometry of must connect to the main external fluid supply and return lines,
100 microns diameter jet and heated silicon solid. Fluid is water at 22◦ C, with only a few inlet and outlet ports are desired. But to keep the over-
V = 9 m/s, H/D = 5, P/D = 5, and q = 2 MW/m2 . all device pressure drop low, it is necessary to preserve the fluid
velocity approximately constant and at a value lower than the
The third and last step is to evaluate the expected thermal jet average fluid velocity. This is accomplished by varying the
performance of the selected cooler geometry. This work is done manifold microchannel cross-section to a given predetermined
using CFD simulation on a representative section of the cooler target value. We determined this value to be preferably about
that covers each section of the device. If the thermal performance 1.5–2 times the total area covered by all the jets, independent
falls short of the target, then one or more parameters of the design of the local manifold dimensions. With this scheme, the fluid
or the fluid operating conditions can be changed. The design does not experience multiple changes in average fluid veloc-
procedure should be restarted when the performance delta is ity within the manifold and minimize the cooler total pressure
large. drop.
To expedite the search for a suitable jet array geometry, a se- To facilitate the manifold design and provide a means for min-
ries of different cases are evaluated using a simple jet geometry, imizing the fluid pressure distribution between all inlets and/or
like one jet and a 2D-axisymmetric model, as shown schemati- all outlets, the cooler body is divided into two main sections
cally in Figure 6a. The simulation results for this particular case with different interconnection functionality. The bottom section
are shown in Figures 6b and 6c. This example shows a thermal (see Figure 7, section A), which includes at the bottom surface
performance that significantly exceeds the 2 MW/m2 design tar- both jet and distributed return arrays, is subdivided into 100
get because the maximum temperature shown in Figure 6b is independent macro-cells each with 16 jet cells and 16 drains.
only 55.3◦ C and the maximum temperature allowed for the chip On the top side of the bottom section, each macro-cell includes
is 85◦ C. Unfortunately, the pressure drop of 54 kPa in the cavity only one supply and one return port. To interconnect the top and
section alone is high, as it consumes most of the pressure drop bottom sides of each macro-cell, both inlet and outlet ports inter-
budget of 70 kPa. Ideally, the pressure drop in the jet cavity connect to four inlet and four outlet microchannels to distribute
should be less than 20 kPa for this section of the cooler. The the fluid sideways within the cell volume, and each microchan-
sensitivity of pressure drop to jet velocity and jet pitch can be nel is in turn connected to four jets or four return ports on the
seen in the data shown in Table 1. However, these results ver- cooler bottom surface. The top section of the cooler (see Figure 7,
ify that a jet array geometry with pitch below 500 microns is
capable of delivering cooling capability above the 2 MW/m2 K
target.
The design of the manifold section begins once the jet array
design parameters have been selected. These parameters fully
characterize only the surface of the cooler that faces the surface Figure 7 Back side of the cooler manifold. External water into the manifold
of the chip that needs cooling. As the other side of the cooler and spent fluid out of the manifold as single units.
section B) has two separate flow grids that interconnect all macro
cells to each other and connect to the cooler inlet and outlet ports
on the cooler top surface.
The manufacturing process needed to build the designed
cooler is based on a recent modification [7,8] of existing MLC Figure 8 Array of jet holes and drain holes (150 microns nominal diameter)
manufacturing technology [9], which now enables us to build on square grid (jet-to-jet pitch of 450 microns nominal). The corner holes on the
intricate microchannels and interlayer connecting vias inside a square grid are drains, and the center hole is the jet.
multilayer ceramic substrate. Table 2 outlines the conventional
MLC build process, starting with Al2 O3 –MgO–SiO2 glass par- The microchannels are formed using 150 micron-diameter
ticles mixed with organic binders and solvents to form glass punches. Even though the punch grid is still kept at 450 mi-
ceramic greensheets on casting. These thin cast greensheets, crons to match with jet grids, the movement of greensheets to
when dry and blanked, are relatively easy to handle and are form channels is about a quarter of the channel width per punch
machinable. Individual greensheets are then punched to form stroke. A portion of punch pattern shown in Figure 8 is over-
50 microns or larger circular via holes. These vias become the laid on the microchannel pattern in Figure 9 for understand-
vertical connectors in eventual circuits formed between lay- ing the layered build approach of the MLC technology used
ers. Copper metal powder mixed with organics, in the consis- for the cooler build. We continue to narrow down the num-
tency of a paste, can then be screened on the greensheets us- ber of stringed jet cold fluid conduits and the drain spent fluid
ing masks in the desired circuit pattern. In doing so, the vias conduits (like the ones shown in Figure 10, which is an in-
are also filled simultaneously. After drying or evaporation of termediate greensheet layer) to less than about 4, as shown in
solvents, several screened layers are then stacked precisely to Figure 7 (top view). The feature shapes and sizes are deliber-
align via to via in adjacent sheets. The stack then is laminated ate and verified by CFD to accomplish reduced fluid resistance.
using adequate heat and pressure for a given time. The lami- Figure 7 also shows the cross-section of the ceramic microjet
nates are subjected to proper sinter process to remove the or- fluid distribution manifold with intertwined cold and spent flu-
ganics and co-densify the glass particles and the metal powder idic networks.
with near zero pattern distortion. IBM’s unique sinter process
further crystallizes the glass-copper composite into a cordierite-
based multilayered ceramic with dense co-fired copper inter-
connects. Subsequently, surface finishing and a plating pro-
cess occur to yield electrically good modules capable of
having semiconductor chips mounted on them for use in com-
puter applications.
In order to build a cooler with a 3D microfluidic flow network,
based on the mature and reliable MLC processing, several new
processes had to be developed. The formations of long channels,
preservation of channel dimensions in the assembled green body,
and stability of open structures during sintering are all major
challenges. Figure 8 is an example of the array of holes formed
in a greensheet for jets and drains.
The punch size used to form these holes were 150 mi-
crons in diameter and sheets were moved 225 microns per
punch stroke. The adjacent greensheet layer that strings mul-
tiple jets together and the respective drains together is shown in
Figure 9. Figure 9 Array of microchannels that feeds fluid to multiple jets and drains.
Figure 10 Array of internal interconnect greensheet layer. section and the aluminum enclosure used to hold and feed the
cooler. To quantify the predictability of CFD code when applied
RESULTS AND DISCUSSION to this particular thermal situation, two 3D models were built
using the measured characteristics of the tested module. The
To determine thermal and fluid dynamic characteristic re- models included all 16 jets, 16 drains, and internal microchan-
sponse of this cooling device, the cooler was assembled with nels located in one macro-cell of the cooler bottom section, and
a heated silicon chip with calibrated temperature sensors into a also included a relevant section of the heated chip. One model
test module with appropriate and fully monitored inlet and outlet was built with a 300 micron gap, while the other was built with
fluid ports. The cooler substrates, characterized after sintering, a 165 micron gap. Results of the simulation are shown in Table
have jet diameter of 126 ± 2 microns and jet pitch of 438 ± 6 3, and the pressure and temperature distribution for Case A are
microns. The gap between the silicon chip and the cooler in the shown in Figures 12 and 13, respectively.
assembled module was determined to be 165 ± 5 microns. Inspection of the results for Case C show a very close agree-
The module pressure drop was measured with flowing room ment between the experimental data and the CFD code. Case
temperature water at different flow rates while monitoring the C data projects a convection coefficient of 0.064 MW/m2K
differential pressure between the inlet and outlet ports. The de- (Nu = 12.8) for jet velocity equal to 2 m/s (Re = 252). Scaling
vice pressure drop is made dimensionless by converting the ex- the Nusselt number prediction using Reynolds number with 0.73
perimental data into equivalent lengths, or Le, relative to the jet exponential dependency (reported [10] for equivalent fluid flow
diameter, using the following relationship: conditions) yields Nu = 10.7 for Re = 198 and validates the use
of this particular CFD code to project thermal performance for
Le = P Re/32ρV2J (2)
The resulting characteristic pressure drop curve for this device,
shown in Figure 11, projects a maximum operating jet velocity
of 1.8 m/s (Re = 239, Le = 148) for a given maximum operating
pressure drop of <70 kPa.
The thermal response of this cooling device was character-
ized using a heated chip, assembled into a module with a 165
micron gap. Analysis of the thermal data collected shows the
measured convection coefficient of 0.052 MW/m2 K (Nu = 10.4)
with an average jet velocity of 1.6 m/s (Re = 198) and a pres-
sure drop of 53 kPa (Le = 126). Because the pressure drop
calculated with the CFD model for this geometry predicts a
pressure drop of only < 17 kPa for the bottom section, we can
deduce that more than two-thirds of the total pressure drop in
this cooler is driven by the top section of the cooler and may
be related to the specific selection of manifold design on this
section of the cooler. This larger fraction of the total pressure
drop can be reduced significantly by careful redesign of this Figure 12 Pressure distribution for Case A in Table 3.
Table 3 Sensitivity analysis using CFD for built cooler jet array geometry,
with water at room temperature
Case A B C
Parameters
VJ m/s 2.0 3.0 2.0
Gap Microns 300 300 165
Results
P kPa 17 33 19
TJ ◦C 63.1 57.8 65.3
h MW/m2 K 0.069 0.084 0.064
q max MW/m2 3.1 3.5 2.9
Parameters are jet velocity and gap size. The power flux is 2 MW/m2 , chip
thickness is 780 microns, and jet diameter and pitch are 126 microns and
438 microns, respectively.
NOMENCLATURE
To fluid inlet temperature, K [9] Knickerbocker, J. U., Pompeo, F. L., Tai, A. F., Thomas, D.
VJ jet average velocity, m/s L., Weekly, R. D., Nealon, M. G., Hamel, H. C., Haridass, A.,
VD drainage velocity, m/s Humenik, J. N., Shelleman, R. A., Reddy, S. N., Prettyman, K.
M., Fasano, B. V., Ray, S. K., Lombardi, T. E., Marston, K. C.,
Coico, P. A., Brofman, P. J., Goldmann, L. S., Edwards, D. L., Zitz,
Greek Symbols
J. A., Iruvanti, S., Shinde, S. L., and Longworth, H. P., An Ad-
vanced Multichip Module (MCM) for High-Performance UNIX
P pressure drop, kPa Servers, IBM J. Res. & Dev., vol. 46, no. 6, pp. 779–804, 2002.
ρ fluid density, kg/m3 [10] Brunschwiler, T., Rothuizen, H., Fabbri, M., Kloter, U., Michel,
B., Bezama, R. J., and Natarajan, G., Direct Liquid Jet Impinge-
ment with Micron-Sized Nozzle Array and Distributed Return
REFERENCES
Architecture, 20th ITHERM Conference, San Diego, Calif., May
2006.
[1] Peterson, G. P., and Ma, H. B., Theoretical Analysis of the Maxi-
mum Heat Transport in Triangular Grooves: A Study of Idealized
Govindarajan Natarajan is a senior engineer at
Micro Heat Pipes, IMECE, HTD-Vol. 317-1, pp. 185–192, 1995. IBM Corporation, Server and Technology Group,
[2] Lu, M., Mok, L., and Bezama, R. J., A Vapor Chamber Using East Fishkill, New York, USA. He received his Ph.D.
Graphite Foams as Wicks for Cooling High Heat Flux Electronics, in chemical engineering in 1981 from the University
Proc. IPACK2005-73170, ASME InterPACK’05, J. of Electronic of Missouri, Columbia, Missouri, and subsequently
Packaging, vol. 128, pp. 427–431, 2006. 2005. worked at Washington University, St. Louis, Mis-
[3] Martin, H., Heat and Mass Transfer between Impinging Gas Jets souri, and Ames Laboratory, Ames, Iowa, as a re-
and Solid Surfaces, Adv. Heat Transfer, vol. 13, pp. 1–60, 1977. search associate before joining Advanced Packag-
[4] Webb, B. W., and Ma, C. F., Single-Phase Liquid Jet Impingement ing Organization at IBM in 1985. He holds 63 U.S.
Heat Transfer, Adv. Heat Transfer, vol. 26, pp. 105–217, 1995. patents and more from other regions, co-authored a
book, and published more than 25 papers. Currently, he is working on chip
[5] Berger, D., Bezama, R. J., Herron, W. L., Michel, B., and
cooling, micromachining, microfluidics, biochips, and MEMS.
Natarajan, G., High Performance Integrated MLC Cooling De-
vice for High Power Density IC’s and Method of Manufacturing,
US Patent Application, 2005. R. J. Bezama is a distinguished engineer at IBM Cor-
[6] Michel, B., Brunschwiler, T., Rothuizen, H. E., Kloter, U., and poration, Server and Technology Group, East Fishkill,
Linderman, R., Cooling Device, European Patent Application, New York, USA. He received his Ph.D. in chemical
2005. engineering in 1983 from the University of Utah, Salt
Lake City, Utah. He holds more than 30 U.S. patents
[7] Natarajan, G., and Humenik, J. N., 3D Ceramic Microfluidic De-
and has authored and co-authored approximately 10
vice Manufacturing, J. Physics: Conference Series, vol. 34, pp. technical papers. His current activities include SCM
533–539, 2006. and MCM microelectronic packaging research and
[8] Natarajan, G., Humenik, J. N., and Cranmer, M., Method and development, research and development of high per-
Structure to Enable Fine Grid MLC Technology, US Patent formance cooling devices, and providing CFD mod-
Application, 2006. eling support to both development and manufacturing engineering groups.