Numerical Simulations in Fluid Dynamics using GPU a ...

1 Numerical Simulations in Fluid Dynamics using GPU: a practical introduction Dr Tomasz P Bednarz, Con Caris, Dr John A Taylor 22 September 2010 Content (behind the scene) introduction What is Computational Fluid Dynamics (CFD) and where is it used? Governing equations Navier-Stokes equations for conservation of mass, momentum & energy equation for thermal Fluid flow Discretisation Rectangular and Boundary Fitted Coordinates Algorithms HSMAC methodology, UPWIND and UTOPIA schemes. Verification Driven Cavity and Natural Convection case. Migration to GPU Migration of the existing code to OpenCL. Applications Scaling Analysis, Magneto-thermal convection, exchange flows. Closing Remarks CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . CSIRO overview CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction .

2 6500+ staff over 55 locations CSIRO today: a snapshot 160+ active licences of CSIRO innovation 20+ spin-off companies in six years Ranked in top 1% in 14 research fields One of the largest & most diverse in the world Australia s national science agency Building national prosperity and wellbeing CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . Fluid Dynamics CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . introduction Experiments and CFD? CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . Courtesy of High Field Magnet Laboratory, NL Air Air N2 gas Wakayama Jet Exchange Flows in Reservoirs Cooling Case Water circulation in reservoirs is driven by thermal gradients changing during day and night cycles. 01234501020304050t [s]dT [mm]dT ~ (kt) at Gr = and Pr = dT ~ (kt) at Gr = and Pr = CSIRO.

3 Insert presentation title, do not remove CSIRO from start of footer dT ~ (k t)1/2 Scaling relation: Exchange Flows in Reservoirs Cooling Case t = 100[s] t = 200[s] t = 500[s] t = 700[s] t = 1000[s] t = 4000[s] 171819202122232407142128354249566370 Time [min]Temperature [oC]abcdefghijkExchange Flows in Reservoirs Diurnal Case Pr = , Gr = 104 9 Dt PIV result unsharp mask CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . DEM and SPH examples CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . Courtesy of Paul Cleary Computational Modelling Group CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . Equations Discretisation Implementation Verification Results Transfer CPU code to GPU code Verification Results! Process Governing Equations, Discretisation Procedure, Algorithms.

4 CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . Governing equations (dimensional) Continuity equation + =0 + + = 1 + 2 2+ 2 2+ _ _ Momentum equations + + = 1 + 2 2+ 2 2+ _ _ Energy equation + + = 2 2+ 2 2 CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . unsteady acceleration convective acceleration pressure gradient viscous term body force Continuity equation + =0 Governing equations (non-dimensional) Momentum equations CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . + + = + 2 2+ 2 2+ + + = + 2 2+ 2 2+ + + = 2 2+ 2 2 Energy equation = Where: = = = = = = Staggered grid , +1, , +1 +1, +1 , +1, +1, +1 , +1 , , +1 +1, +1 +1, + + ( , ) Different variables are located at different locations: scalar variables (pressure, temperature, concentration) are stored at the centre of the cell, the horizontal velocity component is sampled at the centre of the vertical cell face, the vertical velocity component is sampled at the centre of the horizontal cell face.

5 The discrete values of u, v and p can be thought as located on three separate grids, each shifted by half grid spacing to the bottom, to the left, and to the lower left respectively. The staggered arrangement of the unknowns prevents possible pressure oscillations. CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . Staggered grid In 3D, the grid is set up exactly the same way. CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . Finite Difference (FD) method The continuous problem domain is replaced by a finite-difference mesh or grid. , +1, 1, , +1 , 1 If we think about Ui,j as U(X0,Y0) then: +1, = ( 0+ , 0) , +1= ( 0, 0+ ) 1, = ( 0 , 0) , 1= ( 0, 0 ) The idea of FD representation for derivative can be introduced by recalling the definition of the derivative for the function U(X,Y) at X0 and Y0: =lim 0 0+ , 0 ( 0, 0) The difference approximation can be put on a formal basis through the use of a Taylor-series expansion for U(X0+ X,Y0) about (X0,Y0): 0+ , 0= 0, 0+ 0 + 2 2 0 22!

6 + 3 3 0 3 !+ CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . Computational domain = = = = = = = = = = = = = = ; = CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . boundary strip boundary of computational domain computational domain Computational domain = = = = = = = = = = = = = CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . = ; = Discretisation of the continuity equation Continuity equation: CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . + =0 = = + = , 1, + , , 1 =0 All derivatives are calculated at the centre of the cell. = , = 1, = , = , 1 Discretisation of the x-momentum equation CSIRO.

7 Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . All derivatives are calculated at the right vertical face of the cell. + + = + 2 2+ 2 2+ = +1 Time advancement. Discretisation of the x-momentum equation CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . All derivatives are calculated at the right vertical face of the cell. + + = + 2 2+ 2 2+ = +1 = = 2 This term can be improved using upwind scheme !!! Discretisation of the x-momentum equation CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . All derivatives are calculated at the right vertical face of the cell.

8 + + = + 2 2+ 2 2+ = +1 = = 2 = = + + + 4 2 This term can be improved using upwind scheme !!! Discretisation of the x-momentum equation CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . All derivatives are calculated at the right vertical face of the cell. + + = + 2 2+ 2 2+ = +1 = = 2 = = + + + 4 2 = = +1 +1 Discretisation of the x-momentum equation CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . All derivatives are calculated at the right vertical face of the cell.

9 + + = + 2 2+ 2 2+ = +1 = = 2 = = + + + 4 2 = = +1 +1 = 1 + 1 + + = + 2 2+ 2 2+ Discretisation of the y-momentum equation CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . = 1 + 1 = = +1 +1 = = 2 = = + + + 4 2 = +1 All derivatives are calculated at the upper horizontal face of the cell. + + = 2 2+ 2 2 Discretisation of the energy equation CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction .

10 = 1 + 1 = +1 All derivatives are calculated in the centre of the cell. = = + 2 2 = = + 2 2 Velocity iteration + + = + 2 2+ 2 2+ + + = + 2 2+ 2 2+ Momentum equations are rewritten to get velocities: , +1= , + + + , +1= , + + + , + , + CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction . + + = 2 2+ 2 2 Temperature iteration Energy equation is rewritten to get temperature field: , +1= , + + , + Energy equation: CSIRO. Numerical Simulations in Fluid Dynamics using GPU: a practical introduction .