OpenMP Application Programming Interface

1 OpenMPApplication ProgrammingInterfaceVersion November 2015 Copyrightc 1997-2015 OpenMP Architecture Review to copy without fee all or part of this material is granted, provided the OpenMPArchitecture Review Board copyright notice and the title of this document appear. Notice isgiven that copying is by permission of OpenMP Architecture Review page intentionally left blank in published Scope .. Glossary .. Threading Concepts .. OpenMP Language Terminology .. Loop Terminology .. Synchronization Terminology .. Tasking Terminology .. Data Terminology .. Implementation Terminology .. Execution Model .. Memory Model .. Structure of the OpenMP Memory Model .. Device Data Environments .. The Flush Operation .. OpenMP Memory Consistency .. OpenMP Compliance .. Normative References .. Organization of this Document .. 232 Directive Format .. Fixed Source Form Directives.

2 Free Source Form Directives .. Stand-Alone Directives .. Conditional Compilation .. Fixed Source Form Conditional Compilation Sentinels .. Free Source Form Conditional Compilation Sentinel .. Internal Control Variables .. ICV Descriptions .. ICV Initialization .. Modifying and Retrieving ICV Values .. How ICVs are Scoped .. How the Per-Data Environment ICVs Work .. ICV Override Relationships .. Array Sections .. Determining the Number of Threads for aparallelRegion .. Controlling OpenMP Thread Affinity .. Canonical Loop Form .. Worksharing Constructs .. Loop Construct .. Determining the Schedule of a Worksharing Loop .. SIMD Constructs .. simdConstruct .. Loop SIMD Construct .. Tasking Constructs .. simdConstruct .. Task Scheduling .. Device Constructs .. dataConstruct .. enter dataConstruct .. exit dataConstruct .. 100iiOpenMP API Version November.

3 UpdateConstruct .. targetDirective .. simdConstruct .. Distribute Parallel Loop Construct .. Distribute Parallel Loop SIMD Construct .. Combined Constructs .. Parallel Loop Construct .. sectionsConstruct .. workshareConstruct .. Parallel Loop SIMD Construct .. parallelConstruct .. Target Parallel Loop Construct .. Target Parallel Loop SIMD Construct .. simdConstruct .. teamsConstruct .. distributeConstruct .. distribute simdConstruct .. teams distributeConstruct .. teams distribute simdConstruct .. Teams Distribute Parallel Loop Construct .. Target Teams Distribute Parallel Loop Construct .. Teams Distribute Parallel Loop SIMD Construct .. Target Teams Distribute Parallel Loop SIMD Construct .. Master and Synchronization Constructs and Clauses .. Cancellation Constructs .. pointConstruct .. Data Environment .. Data-sharing Attribute Rules .. Data-sharing Attribute Rules for Variables Referenced in a Data-sharing Attribute Rules for Variables Referenced in a Regionbut not in a Construct.

4 Data-Sharing Attribute Clauses .. Data Copying Clauses .. Data-mapping Attribute Rules and Clauses .. reductionDirective .. Nesting of Regions .. 2273 Runtime Library Runtime Library Definitions .. Execution Environment Routines .. 231ivOpenMP API Version November .. Lock Routines .. Timing Routines .. Device Memory Routines .. 2894 Environment .. 300viOpenMP API Version November .. 303A Stubs for Runtime Library C/C++ Stub Routines .. Fortran Stub Routines .. 316B Interface Example of File .. Example of an Interface DeclarationincludeFile .. Example of a Fortran Interface Declarationmodule.. Example of a Generic Interface for a Library Routine .. 342C OpenMP Implementation-Defined Behaviors343D Features Version to Differences .. Version to Differences .. Version to Differences .. Version to Differences .. 352 Index355 ContentsviiCHAPTER 11 Introduction2 The collection of compiler directives, library routines, and environment variables described in this3document collectively define the specification of the OpenMP Application Program Interface4( OpenMP API) for parallelism in C, C++ and Fortran specification provides a model for parallel Programming that is portable across architectures6from different vendors.

5 Compilers from numerous vendors support the OpenMP API. More7information about the OpenMP API can be found at the following web site8 directives, library routines, and environment variables defined in this document allow users to10create and to manage parallel programs while permitting portability. The directives extend the C,11C++ and Fortran base languages with single program multiple data (SPMD) constructs, tasking12constructs, device constructs, worksharing constructs, and synchronization constructs, and they13provide support for sharing, mapping and privatizing data. The functionality to control the runtime14environment is provided by library routines and environment variables. Compilers that support the15 OpenMP API often include a command line option to the compiler that activates and allows16interpretation of all OpenMP Scope18 The OpenMP API covers only user-directed parallelization, wherein the programmer explicitly19specifies the actions to be taken by the compiler and runtime system in order to execute the program20in parallel.

6 OpenMP -compliant implementations are not required to check for data dependencies,21data conflicts, race conditions, or deadlocks, any of which may occur in conforming programs. In22addition, compliant implementations are not required to check for code sequences that cause a231program to be classified as non-conforming. Application developers are responsible for correctly1using the OpenMP API to produce a conforming program. The OpenMP API does not cover2compiler-generated automatic parallelization and directives to the compiler to assist Threading Concepts6threadAn execution entity with a stack and associated static memory, threadAthreadthat is managed by the OpenMP runtime routineA routine that performs the intended function even when executed concurrently (by10more than onethread).11processorImplementation defined hardware unit on which one or moreOpenMP implementation defined logical execution : Adevicecould have one or deviceThedeviceon which theOpenMP programbegins deviceA device onto which code and data may be offloaded from thehost OpenMP Language Terminology18base languageA Programming language that serves as the foundation of the OpenMP : See Section on page 21 for a listing of currentbase20languagesfor the OpenMP programA program written in abase API Version November 2015structured blockFor C/C++, an executable statement, possibly compound, with a single entry at the1top and a single exit at the bottom, or an Fortran, a block of executable statements with a single entry at the top and a3single exit at the bottom, or an :5 For allbase languages:6 Access to thestructured blockmust not be the result of a branch.

7 And7 The point of exit cannot be a branch out of thestructured C/C++:9 The point of entry must not be a call tosetjmp();10 longjmp()andthrow()must not violate the entry/exit criteria;11 Calls toexit()are allowed in astructured block; and12 An expression statement, iteration statement, selection statement, or try13block is considered to be astructured blockif the corresponding14compound statement obtained by enclosing it in{and}would be a15structured Fortran:17 STOP statements are allowed in astructured contextIn C/C++, the innermost scope enclosing an Fortran, the innermost scoping unit enclosing an C/C++, a#pragma, and in Fortran, a comment, that specifiesOpenMP : See Section on page 26 for a description of spaceA non-empty sequence of space and/or horizontal tab programA program that consists of abase program, annotated with OpenMPdirectivesand26runtime library programAnOpenMP programthat follows all rules and restrictions of the directiveAn OpenMPdirectivethat may only be placed in a declarative context.

8 Adeclarative30directiveresults in one or more declarations only; it is not associated with the31immediate execution of any user 1. INTRODUCTION3executable directiveAn OpenMPdirectivethat is not declarative. That is, it may be placed in an1executable directiveAn OpenMPexecutable directivethat has no associated executable user OpenMPexecutable directive(and for Fortran, the pairedenddirective, if any)4and the associated statement, loop orstructured block, if any, not including the code5in any called routines. That is, the lexical extent of anexecutable constructA construct that is a shortcut for specifying one construct immediately nested inside7another construct. A combined construct is semantically identical to that of explicitly8specifying the first construct containing one instance of the second construct and no9other constructA construct that is composed of two constructs but does not have identical semantics11to specifying one of the constructs immediately nested inside the other.

9 A composite12construct either adds semantics not included in the constructs from which it is13composed or the nesting of the one construct inside the other is not code encountered during a specific instance of the execution of a givenconstruct15or of an OpenMP library routine. Aregionincludes any code in called routines as16well as any implicit code introduced by the OpenMP implementation. The generation17of ataskat the point where atask generating constructis encountered is a part of the18regionof theencountering thread, but anexplicit task regionassociated with atask19generating constructis not unless it is anincluded task region. The point where a20targetorteamsdirective is encountered is a part of theregionof the21encountering thread, but theregionassociated with thetargetorteamsdirective22is :24 Aregionmay also be thought of as the dynamic or runtime extent of a25constructor of an OpenMP library the execution of anOpenMP program, aconstructmay give rise parallel regionAparallelregionthat is executed by ateamconsisting of more than parallel regionAparallelregionthat is executed by ateamof only partAll code encountered during the execution of aninitial task regionthat is not part of31aparallelregioncorresponding to aparallelconstructor ataskregion32corresponding to :34 Asequential partis enclosed by animplicit parallel API Version November 2015 Executable statements in called routines may be in both asequential part1and any number of explicitparallelregionsat different points in the2program threadAnOpenMP threadthat hasthreadnumber 0.

10 Amaster threadmay be aninitial4threador thethreadthat encounters aparallelconstruct, creates ateam,5generates a set ofimplicit tasks, and then executes one of thosetasksasthread6number threadThethreadthat encountered theparallelconstructand generated aparallel8regionis theparent threadof each of thethreadsin theteamof thatparallel9region. Themaster threadof aparallelregionis the samethreadas itsparent10threadwith respect to any resources associated with anOpenMP threadWhen a thread encounters aparallelconstruct, each of the threads in the12generatedparallelregion s team arechild threadsof the sinitial threadis not achild threadof the thread that14encountered threadFor a giventhread, itsparent threador one of itsparent thread s ancestor threadFor a giventhread, one of itschild threadsor one of itschild threads set of one or morethreadsparticipating in the execution of :20 For anactive parallel region, the team comprises themaster threadand at21least one aninactive parallel region, theteamcomprises only themaster set ofthread teamscreated by groupAn initialthreadand itsdescendent parallel regionAninactive parallel regionthat is not generated from parallel regionssurround the wholeOpenMP program, alltargetregions,27and threadAthreadthat executes animplicit parallel constructAconstruct(lexically) enclosed by nested constructAconstructnested inside anotherconstructwith no otherconstructnested regionAregion(dynamically) enclosed by anotherregion.