An Empirical Lower Bound on the Overheads of Production ...

1 An Empirical Lower Bound on the Overheads of Production Garbage Collectors Zixian Cai Stephen M. Blackburn Michael D. Bond Martin Maas Australian National University Australian National University Ohio State University Google Abstract Despite the long history of garbage collection (GC) language with GC cannot be easily reversed, as switching to and its prevalence in modern programming languages, there is an alternative, such as C, C++, or Rust, is typically a huge [ ] 15 Dec 2021. surprisingly little clarity about its true Overheads .

2 Even when undertaking. For GC developers, the importance of a clear evaluated using modern , well-established methodologies, crucial tradeoffs made by GCs can go unnoticed, which leads to misin- understanding of GC Overheads is twofold. First, a costly GC. technique presented as cheap can mislead the community into terpretation of evaluation results. In this paper, we 1) develop a methodology that allows us to place a Lower Bound on the absoluteoverusing it, discouraging future optimizations, and dismissing overhead (LBO) of GCs, and 2) expose key performance tradeoffs cheaper alternatives.

3 Second, identifying costly techniques can of five Production GCs in OpenJDK 17, a high-performance reveal opportunities for hardware acceleration and optimization Java runtime. We find that with a modest heap size and across a diverse suite of modern benchmarks, Production GCs incur across the stack [1], especially in datacenter settings. substantial Overheads , spending 7 82 % more wall-clock time We now offer more detail regarding these two problems and and 6 92 % more CPU cycles relative to a zero-cost GC scheme.

4 Outline our contributions. We show that these Overheads can be masked by concurrency a) Unclear absolute Overheads : When evaluating GCs and generous provision of memory/compute. In addition, we find ( , GC algorithms), it is rare to report the Overheads of GCs that newer low-pause GCs are significantly more expensive than older GCs, and sometimes even deliver worse application latency relative to a zero-cost baseline, presumably because of the lack than stop-the-world GCs. Our findings reaffirm that GC is by of a good methodology for establishing such a baseline.

5 Hertz no means a solved problem and that a low-cost, low-latency GC and Berger [2] attempted to measure these Overheads . However, remains elusive. We recommend adopting the LBO methodology their approach relies on simulation and it is unclear whether and using a wider range of cost metrics for future GC evaluations. the conclusions can be extrapolated to GCs running on real This will not only help the community more comprehensively understand the performance characteristics of different GCs, but hardware. also reveal opportunities for future GC optimizations.

6 To address this problem, we introduce a Lower - Bound overhead (LBO) methodology to empirically estimate the I. I NTRODUCTION absolute Overheads of GC by approximating a zero-cost GC. Garbage collection (GC) is ubiquitous in software systems. scheme. In contrast with Hertz and Berger [2], the LBO. Managed languages, such as C#, Java, and JavaScript, continue methodology can measure GCs running on real hardware, and to grow in popularity due to their productivity and safety does not require invasive runtime instrumentation.

7 Due to its benefits, which are in part provided by GC. On servers, many simplicity, the LBO methodology makes minimum assumptions widely used web services, such as Twitter, GitHub, Shopify, about the GC implementation, and should therefore be easy to and Alibaba, make extensive use of such languages. On clients, apply to different languages, runtimes, and GC algorithms. JavaScript engines are embedded in every web browser, and We systematically evaluate five Production GCs in OpenJDK. Java runtimes are embedded in every Android phone.

8 17, the latest release of an industrial-strength, high-performance Despite the ubiquity of GC, there is a surprising lack of JVM. The LBO methodology offers clarity with respect to the clarity regarding its costs. In this paper, we address two key true Overheads of Production GCs. We find that these GCs problems: a) understanding the Overheads of GC algorithms impose substantial Overheads to the program execution across relative to a notional zero-cost baseline, and b) mitigating the a diverse set of workloads. Using a modest heap size, our LBO.

9 Misinterpretation of GC evaluations, which is common even methodology estimates that by using GC, applications spend when modern , well-established methodologies are used. 7 82 % more wall-clock time and 6 92 % more CPU cycles A lack of clarity regarding the Overheads of a technology relative to a zero-cost GC scheme. such as garbage collection can have profound impacts. For b) Misinterpretation of evaluation results: Not seeing the potential GC users, the decision of which GC algorithm to full picture of different collectors' performance can lead to use or even to use GC at all hinges on a clear understanding misinterpretation.

10 Specifically, knowing the limitations of a GC. of the true cost of GC algorithms. Without such information, can change our verdict on its suitability in certain use cases. a user might not realize how much a GC algorithm costs; In Fig. 1 and Fig. 2, we investigate three Production such cost might not actually be acceptable to the user if they OpenJDK collectors Serial, G1, and Shenandoah running had known it. Furthermore, the decision of whether to use a the lusearch benchmark . We use four commonly reported 1. Normalized wall-clock time Normalized CPU cycles GC GC.