An Analysis of Persistent Memory Use with WHISPER

1 An Analysis of Persistent Memory Use with WHISPER Sanketh Nalli, Swapnil Haria, Mark D. Hill, Michael M. Swift, Haris Volos , Kimberly Keeton University of Wisconsin-Madison, USA Hewlett Packard Labs (HP Labs), non-volatile Memory (NVM) technologies promisedurability with read and write latencies comparable tovolatile Memory (DRAM). We define Persistent Memory (PM) as NVM accessed with byte addressability at low la-tency via normal Memory instructions. Persistent -memoryapplications ensure the consistency of Persistent data byinserting ordering points between writes to PM allowingthe construction of higher-level transaction mechanisms.

2 Anepochis a set of writes to PM between ordering put systems research in PM on a firmer footing, we de-veloped and analyzed a PM benchmark suite called WHIS-PER that comprises ten PM applications we gathered tocover all current interfaces to PM. A quantitative analysisreveals several insights: (a) only 4% of writes in PM-awareapplications are to PM and the rest are to volatile Memory ,(b) software transactions are often implemented with 5 to 50ordering points (c) 75% of epochs update exactly one 64 Bcache line, (d) 80% of epochs from the same thread dependon previous epochs from the same thread, while few epochsdepend on epochs from other on our Analysis , we propose the Hands-off Persis-tence System (HOPS) to track updates to PM in hardware design requires applications to force datato PM as each epoch ends.

3 HOPS provides high-level ISAprimitives for applications to express durability and order-ing constraints separately and enforces them automatically,while achieving better performance over current ap-proaches to Concepts Information systems Storage classmemoryKeywordsPersistent Memory (PM); Non-volatile Memory (NVM); Storage-class Memory ; Caches; Benchmark WHISPER stands for Wisconsin-HP Labs Suite for PersistencePermission to make digital or hard copies of all or part of this work for personal or classroom use is granted withoutfee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this noticeand the full citation on the first page.

4 Copyrights for components of this work owned by others than the author(s) mustbe honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute tolists, requires prior specific permission and /or a fee. Request permissions from 17 April 8 12, 2017, Xi an, Shaanxi, Chinac 2017 Copyright held by the owner/author(s). Publication rights licensed to 978-1-4503-4465-4/17/04.. $ : IntroductionPersistent Memory (PM) has received significant attentionin software research [9, 18, 21, 39], hardware research[26, 36, 43], and industry [5, 23, 25, 37]. We define PMas non-volatile Memory (NVM) accessed with byte address-ability (not just blocks) at low latency (not I/O bus) via reg-ular Memory instructions (not system calls).

5 Prior PM re-search studied either existing applications targeting tradi-tional volatile Memory systems and disk storage technolo-gies or micro-benchmarks developed in isolation that exer-cise only a specific mechanism. Although this has been agreat first step, it is challenging to compare various PM sys-tems and be confident that proposed systems are optimizedfor actual time is ripe to consider a comprehensive benchmarksuite that captures important and unique properties expectedin PM applications. First, PM applications are expected tostore data directly in PM for fast persistence in addition toaccessing data through more traditional block-based filesys-tem and database interfaces that offer other useful function-ality such as sharing and global naming.

6 Second, heteroge-neous Memory systems which contain volatile DRAM andNVM will likely be the dominant model for main mem-ory in the near future [22] due to the performance, reliabil-ity and cost of candidate NVM technologies. This organi-zation precludes automatically makingallmemory persis-tent [34, 36], and instead requires applications to selectivelyallocate data in PM and ensure its consistency and a result, applications will likely include both volatile andpersistent Memory data structures, thus exhibiting a mix ofmemory traffic. Finally, PM applications have to be crash-recoverable. In contrast to volatile Memory programs, theyhave to carefully order and persist writes to Memory withrespect to failures.

7 At a low level, these properties are en-sured by (i) explicitly writing data back from the processorcache to PM, (ii) enforcing ordering between writes to de-pendent structures [15], and (iii) waiting for data to becomedurable in PM before continuing execution. The set of writesbetween ordering points constitute anepoch[18, 35] and canreach PM in any order, as long as they are durable beforeanywrites from following epochs are durable. Software can im-plement Persistent transactions with ACID semantics usingthe ordering guarantees of epochs [28].This paper seeks to put PM systems research on a firmerfooting by developing, analyzing and releasing a PM bench-mark suite called WHISPER : Wisconsin-HP Labs Suite forPersistence.

8 It comprises ten PM applications we gathered tocover a wide variety of PM interfaces such as databases, in- Memory data stores, and Persistent heaps. WHISPER cov-ers applications that access PM directly, those using a PMtransactional library such as Mnemosyne [39], and thoseaccessing PM through a filesystem interface. We modifiedthe applications to be recoverable and instrumented themfor our own and future Analysis . WHISPER is available trace-based and quantitative Analysis ofWHISPER yields several behavioral characteristics and in-sights into the design of future PM Systems. On average,(a) only 4% writes in PM-aware applications are to PM andthe rest are to volatile Memory , (b) software transactionsare often implemented with 5 to 50 ordering points and re-quire durability only after the last ordering point, (c) 75%of epochs update exactly one 64B cache line, not necessar-ily the same line (d) 80% of epochs from the same threaddepend on previous epochs from the same thread while fewepochs depend on epochs from other threads.

9 These obser-vations suggest there is value in handling PM writes in aspecial structure that does not disturb the writeback cachesused for the 96% of volatile accesses but one that supportsmultiple writes to the same lines from different our Analysis , we propose theHands-Off Persistence System (HOPS) design. HOPS re-alizes the special structure mentioned above with logi-cally per-thread Persist Buffers (PBs) that (i) support multi-versioning (same line stored from different epochs) (ii)separate a more common, light-weight ordering barrier(ofence) from a less common, heavy-weight durability bar-rier (dfence), and (iii) distribute state to per-thread/corestructures for scalability to larger processors.

10 Our results in-dicate that HOPS improves application performance by 25%over current approaches to rest of this paper is as follows. Section 2 presentsvarious programming models for PM. Section 3 discussesthe applications comprising WHISPER . Section 4 describesour methodology. Section 5 reports our results and insightsinto PM application behavior. Section 6 concludes with adescription of HOPS and its BackgroundThree central challenges exist in programming for PM. First,data is only durable when it reaches PM; data in volatileprocessor caches are currently lost on a power failure. Asa result, applications that require durability must ensure thatdata leaves the cache and wait for it to reach PM.