A Simple API for Grid Applications (SAGA) - Open Grid Forum

1 Goodale, CardiffSAGA-CORE-WGShantenu Jha, UCL1 Hartmut Kaiser, LSUT hilo Kielmann, VU1 Pascal Kleijer, NECA ndre Merzky, VU/LSU1 John Shalf, LBNLC hristopher Smith, PlatformJanuary 15, 2008 Version: September 12, 2013A Simple API for grid Applications ( saga )Status of This DocumentThis document provides information to the grid community, proposing the corecomponents for an extensible Simple API for grid Applications ( saga CoreAPI). It is supposed to be used as input to the definition of language specificbindings for this API, and by implementors of these bindings. Distribution 2010/2011, a number of errata have been applied to this document. A com-plete changelog can be found in the appendix. Note that the API specified inthis document version is thus labelled as version , and as such obsoletes theprevious API version Most changes should be backward compatible withthe original specification (for details see changelog).

2 Copyright NoticeCopyrightc Open grid Forum (2007-2011). All Rights document specifies the core components for the Simple API for grid Ap-plications ( saga Core API), a high level, application-oriented API for gridapplication development. The scope of this API is derived from the require-ments specified in ( A Requirements Analysis for a Simple API forGrid Applications ). It will in the future be extended by additional API 12, 2013 Contents1 How to read this Document .. Notational Conventions .. Security Considerations ..52 General Design API Scope and Design Process .. The SIDL Interface Definition Language .. Language Binding Issues .. Compliant Implementations .. Object Management .. Asynchronous Operations and Concurrency .. State Diagrams .. Execution Semantics and Consistency Model.

3 Optimizing Implementations, Latency Hiding .. Configuration Management .. The URL Problem .. Miscellaneous Issues .. 303 saga API Specification Look & saga Error Handling .. saga Base Object.. saga URL Class .. saga I/O Buffer .. saga Session Management.. 12, saga Context Management .. saga Permission Model .. saga Attribute Model .. saga Monitoring Model .. saga Task Model .. 1474 saga API Specification API saga Job Management.. saga Name Spaces .. saga File Management.. saga Replica Management.. saga Streams .. saga Remote Procedure Call .. 3125 Intellectual Property Contributors .. Intellectual Property Statement .. Disclaimer .. Full Copyright Notice .. 325A saga Code Examples326B 12, 20131 IntroductionThis document specifies saga CORE, the Core of theSimpleAPIforGridApplications.

4 saga is a high-level API that directly addresses the needs ofapplication developers. The purpose of saga is two-fold:1. Provide ansimpleAPI that can be used with much less effort compared tothe vanilla interfaces of existing grid middleware. A guiding principle forachieving this simplicity is the80 20rule: serve 80 % of the use cases with20 % of the effort needed for serving 100 % of all possible Provide a standardized, common interface across various grid middlewaresystems and their How to read this DocumentThis document is an APIspecification, and as such targetsimplementors of theAPI, rather than its end users. In particular, this document should not beconfused with a saga Users Guide. This document might be useful as an APIreference, but, in general, the API users guide and reference should be publishedas separate documents, and should accompany saga implementations.

5 Thelatest version of the users guide and reference can be found implementor of the saga API should read the complete document will very likely be insufficientunlikely be sufficient to extract the embeddedSIDL specification of the API and implement a saga -compliant API. In par-ticular, the general design considerations in Section 2 give essential, additionalinformation to be taken into account for any implementation in order to beSAGA document is structured as follows. This Section focuses on the formalaspects of an OGF recommendation document. Section 2 outlines the generaldesign considerations of the saga API. Sections 3 and 4 contain the saga APIspecification itself. Section 5 gives author contact information and provides dis-claimers concerning intellectual property rights and copyright issues, accordingto OGF policies. Finally, Appendix A gives illustrative, non-normative, codeexamples of using the saga 12, Notational ConventionsThe key wordsMUST,MUST NOT,REQUIRED,SHALL,SHALL NOT,SHOULD,SHOULD NOT,RECOMMENDED,MAY, andOPTIONALare to be interpreted asdescribed in RFC 2119 [6].

6 Security ConsiderationsAs the saga API is to be implemented on different types of grid (and non- grid )middleware, it does not specify a single security model, but rather provideshooks to interface to various security models see the documentation of thesaga::contextclass in Section for saga implementation is considered secure if and only if it fully supports( implements) the security models of the middleware layers it builds upon,and neither provides any (intentional or unintentional) means to by-pass thesesecurity models, nor weakens these security models policies in any Design ConsiderationsSeptember 12, 20132 General Design ConsiderationsThis section addresses those aspects of the saga API specification common tomost or all of the saga packages as defined in Sections 3 and API Scope and Design ProcessThe scope and requirements of the saga API have been defined by OGF sSimpleAPIforGridApplicationsResearchGro up ( saga -RG).

7 The saga -RGhas collected as broad as possible a set of use cases which has been published [17]. The requirements for the saga API were derived from this usecases document, an analysis of which has been published as [18]. Theformal specification and resulting document is the work of theSAGA-COREW orkingGroup which was spawned from the Requirements from the saga Requirement AnalysisThe saga Requirement Analysis [18] lists the following functional and non-functional requirements of the saga API:Functional Requirements Job submission and management should be supported by the saga API. Resource discovery should be supported by the saga API. Data management should be supported by the saga API. Efficient data access should be supported by the saga API. Data replication should be supported by the saga API. Persistent storage of application specific information should be supportedby the saga API.

8 Streaming of data should be supported by the saga API. Support for messages on top of the streaming API should be consideredby the saga API. Asynchronous notification should be supported by the saga API. Application level event generation and delivery should be supported bythe saga Design ConsiderationsSeptember 12, 2013 Application steering should be supported by the saga API, but moreuse cases would be useful. GridRPC should be supported by the saga API. Further communication schemes should be considered as additional usecases are submitted to the group. Access to data-bases does not currently require explicit support in theSAGA Requirements Asynchronous operations should be supported by the API. Bulk operations should be supported by the API. The exception handling of the API should allow forapplicationlevel errorrecovery strategies. The saga API should be implementable on a variety of security infras-tructures.

9 The saga API should expose only a minimum of security details, if anyat all. Auditing, logging and accounting should not be exposed in the API. Workflows do not require explicit support on API level. QoS does not require explicit support on API level. Transactions do not require explicit support on API Requirement Adoption StrategyThe use cases expressed the above requirements different levels of importanceor urgency. This reflects the fact that some functionality is considered moreimportant or even vital (like file access and job submission) while other func-tionality is seen as nice to have by many use cases (like application steering).Also, the group of active people in the saga specification process constitutesa specific set of expertise and interest and this set is, to some extent, reflectedin the selection of saga packages specified in this example, as there were no use cases from the enterprise user community,nor was there any active participation from that community in the saga stan-dardization process, no enterprise specific API package is included here.

10 Design ConsiderationsSeptember 12, 2013does not imply that we consider them unnecessary, but rather reflects the wishand need to derive the API on real use cases, and to avoid the creation of anAPI from perceived use cases, and half-baked of the saga APIAs various sides expressed their need for the availability of a useful ( imple-mentable and usable) API specification as quickly as possible, the saga -CORE-WG decided to follow a two-phase approach. The saga API, as described inthis document, covers all requirements that are considered both urgent and suffi-ciently well understood to produce an API. Addressing the other the less urgentor well understood requirements is deferred to future versions, or extensions, ofthe saga API. Based upon this reasoning, areas of functionality (from now onreferred to aspackages) that are included in saga API are the following: jobs files (and logical files) streams remote procedure calls [19] auxiliary API s for session handle and security context asynchronous method calls (tasks) access control lists attributes monitoring error handlingPossible extensions to be included in future saga versions or extensions are: steering and extended monitoring possibly combining logical/physical files (read on logical files) persistent information storage (see, the GAT Advert Service [2]) GridCPR [11] task dependencies ( Simple work flows and task batches) extensions to existing classes, based on new use casesThe packages as listed above do not imply a hierarchy of API interfaces: allpackages are motivated by their use cases.