Parallel Programming in Fortran 95 using OpenMP

1 Parallel Programming in Fortran 95 using OpenMPMiguel HermannsSchool of Aeronautical EngineeringDepartamento de Motopropulsi on y Termofluidodin amicaUniversidad Polit ecnica de April 2002 Contents1 OpenMP Fortran Application Program Historicalremarks .. Whoisparticipating .. The sentinels for OpenMP directives and conditional compilation . Theparallelregionconstructor .. 52 OpenMP Work-sharingconstructs .. !$OMP DO/!$OMP END !$OMP SECTIONS/!$OMP END !$OMP SINGLE/!$OMP END !$OMP WORKSHARE/!$OMP END !$OMP Parallel DO/!$OMP END Parallel !$OMP Parallel SECTIONS/!$OMP END Parallel !$OMP Parallel WORKSHARE/!$OMP END Parallel !$OMP MASTER/!$OMP END !$OMP CRITICAL/!$OMP END !$OMP !$OMP !$OMP !$OMP ORDERED/!$OMP END !$OMP THREADPRIVATE (list).. 323 PRIVATE,SHARED & Data scope attribute clauses.

2 (list).. (list).. ( PRIVATE | SHARED | NONE ).. (list).. (list).. (list).. (list).. (operator:list).. Otherclauses .. (scalarlogicalexpression).. (scalarintegerexpression).. (type, chunk).. 524 The OpenMP run-time Executionenvironmentroutines .. Lockroutines .. Examples .. Fortran 90 665 The environment 71 Chapter 1 OpenMP Fortran ApplicationProgram IntroductionIn the necessity of more and more computational power, the developers of computingsystems started to think on using several of their existing computing machines in a joinedmanner. This is the origin of Parallel machines and the start of a new field for programmersand Parallel computers are very common in research facilities as well as compa-nies all over the world and are extensively used for complex computations, like simulationsof atomic explosions, folding of proteins or turbulent challenge in Parallel machines is the development of codes able of using the capabil-ities of the available hardware in order to solve larger problems in less time.

3 But parallelprogramming is not an easy task, since a large variety of architectures exist. Mainly twofamilies of Parallel machines can be identified:Shared-memory architecture: these Parallel machines are build up on a set of proces-sors which have access to a common memory. Usually the name ofSMP machinesis used for computers based on this architecture, where SMP stands forSymmetricMulti architecture: in these Parallel machines each processor has itsown private memory and information is interchanged between the processors throughmessages. The name ofclustersis commonly used for this type of of the two families has its advantages and disadvantages and the actual parallelprogramming standards try to exploit these advantages by focusing only on one of the last years a new industry standard has been created with the aim to serveas a good basis for the development of Parallel programs on shared-memory OpenMP Fortran Application Program Historical remarksShared-memory machines exist nowadays for a long time.

4 In the past, each vendor was de-veloping its own standard of compiler directives and libraries, which allowed a programto make use of the capabilities of their specific Parallel earlier standardization effort, ANSI X3H5 was never formally adopted, since on onehand no strong support of the vendors was existing and on the other hand distributedmemory machines, with their ownmore standardmessage passing libraries PVM andMPI, appeared as a good alternative to shared-memory in 1996-1997, a new interest in a standard shared-memory Programming interfaceappeared, mainly due to:1. A renewed interest from the vendors side in shared-memory The opinion by a part of the vendors, that the parallelization of programs usingmessage passing interfaces is cumbersome and long and that a more abstract pro-gramming interface would be the result of a large agreement between hardware vendors and compilerdevelopers and is considered to be an industry standard : it specifies a set of compilerdirectives, library routines, and environment variables that can be used to specify shared-memory parallelism in Fortran and C/C++ consolidates all this into a single syntax and semantics and finally deliversthe long-awaited promise of single source portability for shared-memory parallelism.

5 ButOpenMP is even more: it also addresses the inability of previous shared-memory directivesets to deal withcoarse-grain parallelism2. In the past, limited support for coarsegrain work has led to developers to think that shared-memory Parallel Programming wasinherently limited tofine-grain Who is participatingThe OpenMP specification is owned, written and maintained by theOpenMP Archi-tecture Review Board, which is a join of the companies actively taking part in thedevelopment of the standard shared-memory Programming interface. In the year 2000,the permanent members of the OpenMP ARB were: US Department of Energy, through its ASCI program Compaq Computer forMulti ProcessingandOpenmeans that the standard is defined through a specifi-cation accessible to parallelismmeans that the parallelism in the program is achieved through a decom-position of the target domain into a set of subdomains that is distributed over the different processors ofthe parallelismmeans that the parallelism in the program is achieved by distributing thework of the do-loops over the different processors, so that each processor computes part of the Introduction3 Fujitsu Hewlett-Packard Company Intel Corp.

6 International Business Machines Kuck & Associates, Inc. Silicon Graphics Incorporate Sun MicrosystemsAdditionally to the OpenMP ARB, a large number of companies contribute to thedevelopment of OpenMP by using it in their programs and compilers and reporting prob-lems, comments and suggestions to the OpenMP About this documentThis document has been created to serve as a good starting point for Fortran 95 pro-grammers interested in learning OpenMP . Special importance has been given to graphicalinterpretations and performance aspects of the different OpenMP directives and clauses,since these are lacking in the OpenMP specifications released by the OpenMP to complement the present document with these OpenMP specifications, sincesome aspects and possibilities have not been addressed here for the Fortran 95 Programming language is considered in the present document,although most of the concepts and ideas are also applicable to the Fortran 77 programminglanguage.

7 Since the author believes in the superiority of Fortran 95 over Fortran77 and inthe importance of a good Programming methodology, the present document only presentsthose features of OpenMP which are in agreement with such a Programming is the reason why it is advisable to have also a look at the OpenMP specifications,since the selection of the concepts presented here are a personal choice of the the existing documentation about OpenMP is not very extensive, the presentdocument has been released for free distribution over the Internet, while the copyrightof it is kept by the author. Any comments regarding the content of this document arewelcome and the author encourages people to send constructive comments and suggestionsin order to improve the time of writing this document (winter 2001-spring 2002) two different OpenMPspecifications are used in compilers.

8 Version and version Since the latter enhancesthe capabilities of the former, it is necessary to differentiate what is valid for each is accomplished by using adifferent color for the text that only applies to theOpenMP Fortran Application Program Interface, version has no sense that performance issues are addressed in a specification, since they are implementationdependent and in general different for each OpenMP Fortran Application Program The basicsOpenMP represents a collection of compiler directives, library routines and environmentvariables meant for Parallel Programming in shared-memory machines. A chapter is goingto be devoted to each of these elements, but before starting with the review of the availablecompiler directives, it is necessary to have a look at some basic aspects of named as basic aspects , the information presented in this section is thefundamental part of OpenMP which allows the inclusion of OpenMP commands in pro-grams and the creation as well as destruction of Parallel running regions of The sentinels for OpenMP directives and conditional com-pilationOne of the aims of the OpenMP standard is to offer the possibility of using the samesource code lines with an OpenMP -compliant compiler and with a normal compiler.

9 Thiscan only be achieved by hiding the OpenMP directives and commands in such a way, thata normal compiler is unable to see them. For that purpose the following twodirectivesentinelsare introduced:!$OMP!$Since the first character is an exclamation mark ! , a normal compiler will interpretthe lines as comments and will neglect their content. But an OpenMP -compliant compilerwill identify the complete sequences and will proceed as follows:!$OMP: the OpenMP -compliant compiler knows that the following information in theline is an OpenMP directive. It is possible to extend an OpenMP directive overseveral lines by placing the same sentinel in front of the following lines and usingthe standard Fortran 95 method of braking source code lines:!$OMP Parallel DEFAULT(NONE) SHARED(A, B) PRIVATE(C, D) &!$OMP REDUCTION(+:A)It is mandatory to include a white space between the directive sentinel!

10 $OMPandthe following OpenMP directive, otherwise the directive sentinel is not correctlyidentified and the line is treated as a comment.!$: the corresponding line is said to be affected by aconditional that its content will only be available to the compiler in case of beingOpenMP-compliant. In such a case, the two characters of the sentinel are sub-stituted by two white spaces so that the compiler is taking into account the in the previous case, it is possible to extend a source code line over several linesas The basics5!$ interval = L * OMP_get_thread_num() / &!$ (OMP_get_num_threads() - 1)Again, it is mandatory to include a white space between the conditional compilationdirective!$and the following source code, otherwise the conditional compilationdirective is not correctly identified and the line is treated as a sentinels can appear in any column as long as they are preceded only by whitespaces; otherwise, they are interpreted as normal The Parallel region constructorThe most important directive in OpenMP is the one in charge of defining the so calledparallel regions.