Faster Chrome - Let The Compiler do the work

Wednesday, December 1, 2021

Chrome is fast, but there's always room for improvement. Often, that's achieved by carefully crafting the algorithms that make up Chrome. But there's a lot of Chrome, so why not let computers do at least some part of our work? In this installment of The Fast And the Curious, we'll show you several changes in how we build Chrome to achieve a 25.8% higher score on Speedometer on Windows and a 22.0% increase in browser responsiveness.

Why speed?

So why do we care about performance benchmarks? It's not a simple "higher numbers is better" chasing of achievements - performance was so important to Chrome that we embedded in our core principles, the "4Ss" - Speed, Security, Stability, Simplicity. And speed matters because we want a browser that responds quickly. Speed matters so much because we want to build a faster and more responsive browser. And by improving the speed of the browser, there's the additional benefit of maximizing battery use, so you don't have to charge your laptop/devices as often.

Speed? Size? Something Else?

Let's look at a typical optimization.

int foo();

int fiver(int num) {

for(int j = 0; j < 5; j++)

num = num + foo();

return num;

}

The compiler can either compile this as a loop (smaller), or turn it into five additions in a row (faster, but bigger)

You save the cost of checking the end of the loop and incrementing a variable every time through the loop. But in exchange, you now have many repeated calls to foo(). If foo() is called a lot, that is a lot of space.

And while speed matters a lot, we also care about binary size. (Yes, we see your memes!) And that tradeoff - exchanging speed for memory, and vice versa, holds for a lot of compiler optimizations.

So how do you decide if the cost is worth it? One good way is to optimize for speed in areas that are run often, because your speed wins accumulate each time you run a function. You could just guess at what you inline (your compiler can do this, it's called "a heuristic", it's an educated guess), and then measure speed and code size.

The result: Likely faster. Likely larger. Is that good?

Ask any engineer a question like that, and they will answer “It depends”. So, how do you get an answer?

The More You Know… (profiling & PGO)

The best way to make a decision is with data. We collect data based on what gets run a lot, and what gets run a little. We do that for several different scenarios, because our users do lots of different things with Chrome and want them to be fast.

Our goal is collecting performance data in various scenarios, and using that to guide the compiler. There are 3 steps needed:

Instrument for profiling
Run that instrumented executable in various scenarios
Use the resulting performance profile to guide the compiler.

But we can do more (ThinLTO)

That's a good start, but we can do better. Let's look at inlining - the compiler takes the code of a called function and inserts all of it at the callsite.

inline int foo() { return 3; };

int fiver_inline(int num) {

for(int j = 0; j < 5; j++)

num = num + foo();

return num;

}

When the compiler inlines foo(), it turns into

int fiver_inline(int num) {

for(int j = 0; j < 5; j++)

num = num + 3;

return num;

}

Not bad - saves us the function call and all the setup that goes with having a function. But the compiler can in fact even do better - because now all the information is in one place. The compiler can apply that knowledge and deduce that fiver_inline() adds the number three and does so 5 times - and so the entire code is boiled down to

return num + 15;

Which is awesome! But the compiler can only do this if the source code for foo() and the location where it is called are in the same source file - otherwise, the compiler does not know what to inline. (That's the fiver() example). A trivial way around that is to combine all the source files into one giant source file and compile and link that in one go.

There's just one downside - that approach needs to generate all of the machine code for Chrome, all the time. Change one line in one file, compile all of Chrome. And there's a lot of Chrome. It also effectively disables caching build results and so makes remote compilation much less useful. (And we rely a lot on remote compilation & caching so we can quickly build new versions of Chrome.)

So, back to the drawing board. The core insight is that each source file only needs to include a few functions - it doesn't need to see every single other file. All that's needed is "cleverly" mixing the right inline functions into the right source files.

Now we're back to compiling individual source files. Distributed/cached compilation works again, small changes don't cause a full rebuild, and since "ThinLTO" does just inline a few functions, and it is relatively little overhead.

Of course, the question of "which functions should ThinLTO inline?" still needs to be answered. And the answer is still "the ones that are small and called a lot". Hey, we know those already - from the profiles we generated for Profile Guided Optimization (PGO). Talk about lucky coincidences!

But wait, there's more! (Callgraph Sorting)

We've done a lot for inlined function calls. Is there anything we can do to speed up functions that haven't been inlined, too? Turns out there is.

One important factor is that the CPU doesn't fetch data byte by byte, but in chunks. And so, if we could ensure that a chunk of data doesn't just contain the function we need right now, but ideally also the ones that we'll need next, we could ensure that we have to go out and get chunks of data less often.

In other words, we want functions that are called right after the other to live next to each other in memory also ("code locality"). And we already know which functions are called close to each other - because we ran our profiling and stored performance profiles for PGO.

We can then use that information to ensure that the right functions are next to each other when we link.

I.e.

g.c

extern int f1();

extern int f2();

extern int f3();

int g() {

f1();

for(..) {

f3();

}

f1();

f2();

}

could be interpreted as "g() calls f3() a lot - so keep that one really close. f1() is called twice, so… somewhat close. And if we can squeeze in f2, even better". The calling sequence is a "call graph", and so this sorting process is called "call graph sorting".

Just changing the order of functions in memory might not sound like a lot, but it leads to ~3% performance improvement. And to know which functions calls which other ones a lot… yep. You guessed it. Our profiles from the PGO work pay off again.

One more thing.

It turns out that the compiler can make even more use of that profile data for PGO. (Not a surprise - once you know where the slow spots are, exactly, you can do a lot to improve!). To make use of that, and enable further improvements, LLVM has something called the "new pass manager". In a nutshell, it's a new way to run optimizations within LLVM, and it helps a lot with PGO. For much more detail, I'd suggest reading the LLVM blog post.

Turning that on leads to another ~3% performance increase, and ~9MB size reduction.

Why Now?

Good question. One part of that is that PGO & profiling unlock an entire new set of optimizations, as you've seen above. It makes sense to do that all in one go.

The other reason is our toolchain. We used to have a colorful mix of different technologies for compilers and linkers on different platforms.

And since this work requires changes to compilers and linkers, that would mean changing the build - and testing it - across 5 compilers and 4 linkers. But, thankfully, we've simplified our toolchain (Simplicity - another one of the 4S's!). To be able to do this, we worked with the LLVM community to make clang a great Windows compiler, in addition to partnering with the LLVM community to create new ELF (Linux), COFF (Windows), and Mach-O (macOS, iOS) linkers.

And suddenly, it's only a single toolchain to fix. (Almost. LTO for lld on MacOS is being worked on).

Sometimes, the best way to get more speed is not to change the code you wrote, but to change the way you build the software.

Posted by Rachel Blum, Engineering Director, Chrome Desktop
Data source for all statistics: Speedometer 2.0.

Partitioning Chrome's Code for Faster Launch Times on Android

Tuesday, November 16, 2021

Mobile devices are generally more resource constrained than laptops or desktops. Optimizing Chrome’s resource usage is critical to give mobile users a faster Chrome experience. As we’ve added features to Chrome on Android, the amount of Java code packaged in the app has continued to grow. In this The Fast and the Curious post we show how our team improved the speed and memory usage of Chrome on Android with Isolated Splits. With these improvements, Chrome on Android now uses 5-7% less memory, and starts and loads pages even faster than before.

The Problem

For Android apps (including Chrome on Android), compiled Java code is stored in .dex files. The user's experience in Chrome on Android is particularly sensitive to increases in .dex size due to its multi-process architecture. On Android, Chrome will generally have 3+ processes running at all times: the browser process, the GPU process, and one or more renderer processes. The vast majority of Chrome’s Java code is used only in the browser process, but the performance and memory cost of loading the code is paid by all processes.

Bundles and Feature Modules

Ideally, we would load the smallest chunk of Java necessary for a process to run. We can get close to this by using Android App Bundles and splitting code into feature modules. Feature modules allow splitting code, resources, and assets into distinct APKs installed alongside the base APK, either on-demand or during app install.

Now, it seems like we have exactly what we want: a feature module could be created for the browser process code, which could be loaded when needed. However, this is not how Android loads feature modules. By default, all installed feature modules are loaded on startup. For an app with a base module and three feature modules “a”, “b”, and “c”, this gives us an Android Context with a ClassLoader that looks something like this:

Having a small minimum set of installed modules that are all immediately loaded at startup is beneficial in some situations. For example, if an app has a large feature that is needed only for a subset of users, the app could avoid installing it entirely for users who don't need it. However, for more commonly used features, having to download a feature at runtime can introduce user friction -- for example, additional latency or challenges if mobile data is unavailable. Ideally we'd be able to have all of our standard modules installed ahead of time, but loaded only when they're actually needed.

Isolated Splits to the Rescue

A few days of spelunking in the Android source code led us to the android:isolatedSplits attribute. If this is set to “true”, each installed split APK will not be loaded during start-up, and instead must be loaded explicitly. This is exactly what we want to allow our processes to use less resources! The ClassLoader illustrated above now looks like this:

In Chrome’s case, the small amount of code needed in the renderer and GPU processes can be kept in the base module, and the browser code and other expensive features can be split into feature modules to be loaded when needed. Using this method, we were able to reduce the .dex size loaded in child processes by 75% to ~2.5MB, making them start faster and use less memory.

This architecture also enabled optimizations for the browser process. We were able to improve startup time by preloading the majority of the browser process code on a background thread while the Application initializes leading to a 7.6% faster load time. By the time an Activity or other component which needed the browser code was launched, it would already be loaded. By optimizing how features are allocated into feature modules, features can be loaded on-demand which saves the memory and loading cost until the feature is used.

Results

Since Chrome shipped with isolated splits in M89 we now have several months of data from the field, and are pleased to share significant improvements in memory usage, startup time, page load speed, and stability for all Chrome on Android users running Android Oreo or later:

Median total memory usage improved by 5.2%
Median renderer process memory usage improved by 7.9%
Median GPU process memory usage improved by 7.6%
Median browser process memory usage improved by 1.2%
95th percentile startup time improved by 7.6%
95th percentile page load speed improved by 2.3%
Large improvements in both browser crash rate and renderer hang rate

Posted by Clark Duvall, Chrome Software Engineer

Data source for all statistics: Real-world data anonymously aggregated from Chrome clients.

Searching, browsing, and shutdown Chrome performance improvements

Monday, November 1, 2021

Chrome has long-term investments in performance improvement across many projects and we are pleased to share improvements across speed, memory, and unexpected hangs in today’s The Fast and the Curious series post. One in six searches is now as fast as a blink of an eye, Chrome OS browsing now uses up to 20% less memory thanks to our PartitionAlloc investment, and we’ve resolved some thorny Chrome OS and Windows shutdown experiences.OmniboxYou’ve probably noticed that potential queries are suggested to you as you type when you’re searching the web using Chrome’s omnibox (as long as the “Autocomplete searches and URLs” feature is turned on in Chrome settings.) This makes searching for information faster and easier, as you don’t have to type in the entire search query -- once you’ve entered enough text for the suggestion to be the one you want, you can quickly select it.

Searching in Chrome is now even faster, as search results are prefetched if a suggested query is very likely to be selected. This means that you see the search results more quickly, as they’ve been fetched from the web server before you even select the query. In fact, our experiments found that search results are now 4X more likely to be shown within 500 ms!

Currently, this only happens if Google Search is your default search engine. However, other search providers can trigger this feature by adding information to the query suggestions sent from their servers to Chrome, as described in this article.

Chrome OS PartitionAlloc

Chrome’s new memory allocator, PartitionAlloc, rolled out on Android and Windows in M89, bringing improved memory usage [up to 22% savings] and performance [up to 9% faster responsiveness]. Since then, we have also implemented PartitionAlloc on Linux in M92 and Chrome OS in M93. We are now pleased to announce that M93 field data from Chrome OS shows a total memory footprint reduction of 15% in addition to a 20% browser process memory reduction, improving the Chromebook browsing experience for both single and multi-tabs.

Resolving the #1 shutdown hang

Often software engineers add a cache to a system with the goal of improving performance. But a frequent corollary of caching is that the cache may introduce other problems (code complexity, stability, memory consumption, data consistency), and may even make performance worse. In this case, a local cache was added years ago to Chrome's history system with the goal of making startup faster. The premise at the time, which seemed to bear out in lab testing, was that caching Chrome's internal in-memory history index would be faster than reindexing the history at each startup.

Thanks to our continuing systematic investigation into real-world performance using crash data in conjunction with anonymized performance metrics, we uncovered that not only did this cache add code complexity and unnecessary memory usage, but it was also our top contributor to shutdown hangs in the browser. This is because on some OSes, background priority threads can be starved of I/O indefinitely while there is any other I/O happening elsewhere on the system. Moreover, the performance benefits to our users were minimal, based on analysis of field data. We've now removed the cache and resolved our top shutdown hang. This was a great illustration of the principle that caching is not always the answer!

Stay tuned for many more performance improvements to come!

Posted by Yana Yushkina, Product Manager, Chrome Browser

Data source for all statistics: Real-world data anonymously aggregated from Chrome clients.

RenderingNG: an architecture that makes and keeps Chrome fast for the long term

Wednesday, October 6, 2021

Our continual investments in the performance of Chrome have led to significant improvements in battery life, memory, and the speed of the web. This post in The Fast & the Curious series highlights the rendering journey of Chrome over the past eight years, a journey that has led to a browser that is better across the board. For example, Chrome 94, as compared with Chrome 93:

is up to 8% more responsive on real pages,

saves more than 1400 years of CPU time per day, and

improves battery life by up to 0.5%

In addition, recent versions of Chrome are much better than those of years past with:

150% or more faster graphics rendering, and

greater reliability, due to a 6x reduction in GPU driver crashes on problematic hardware

IntroductionRenderingNG

RenderingNGcoming to a conclusion

The pyramid of success

Performance

GPU & multi-core

A great way Chrome can render content faster is to take advantage of the multi-core CPUs and advanced GPUs present in today’s devices. Multi-core means we can do multiple kinds of work in parallel. For example, Chrome parallelizes running JavaScript, scrolling a web page, decoding an image or video, and rastering new content that will be on-screen soon. GPUs enable even more parallelization by rastering every pixel on-screen, rather than one-at-a-time, yielding a large speedup.

Earlier generations of browsers were not built to natively support these technologies, because at the time GPUs and multi-core computers were not widespread. It turned out that bolting on highly parallel features was extremely challenging, requiring the team to re-architect the entire Chrome rendering pipeline over time. This re-architecture is RenderingNG.

Scaling up and scaling down

To avoid exhausting available resources and potentially diminishing the user experience of the web, browsers must be respectful of efficiency. They should not slow down your ability to interact with your computer or phone, or keep you from multitasking between the browser and other apps. Likewise, browsers also need to maximize battery life of your phone and laptop, and prevent the device itself from getting too hot (here’s looking at you, CPU cooling fans!). And finally, the browser must always be smooth and responsive to the user.

This means that the browser needs to be able to scale up and down its use of the GPU and multi-threading to balance multitasking, battery, and device temperature. As one example of scaling up, it’s important that scrolling happen in parallel with JavaScript on all devices; otherwise the browser will have poor user responsiveness. On the other hand, there are times when the browser must scale down because battery life may be considered more important than maximum use of GPU and CPU resources to make rendering faster.

It turns out to be challenging to be able to scale up and scale down seamlessly, and in a way that doesn’t break web content or make the browser unresponsive in key situations. RenderingNG applies novel technologies throughout the stack to make this work, including a complete rewrite of how compositing works, enabling the flexible use of GPU or CPU computations for arbitrary pieces of web pages, in order to pick the right fit for each web page and device.

Performance isolation

The third key technique is performance isolation. Performance isolation is what allows you to have a nice experience reading your email while music and video is playing in the background: your computer and operating system (OS) architecture makes sure that they all share the CPU fairly and smoothly. (This feature has been around for native apps for so long, we just take it for granted, but it wasn’t always that way!)

Browsers already have good isolation from other apps on your computer (thanks to the OS), but within a web page, it can be difficult to isolate all the pieces, such as iframes for ads and other embedded content, video and audio, animations, scrolling, JavaScript, and various other browser tasks. In earlier eras of the web, it was a huge achievement to simply make all these features work at all. Now the bar has been raised, and they need to fit together seamlessly in order to create a reliably fast user experience on the web.

RenderingNG’s architecture implements performance isolation for all of these features.

Reliability

Most of our work has been focused on reliability of the user experience. This is especially true when rewriting the entire rendering stack of a web browser--a huge and potentially risky undertaking.

We did this with four key techniques:

Compat2021 improvements over time

Extensibility

Finally, we always have one eye toward what web apps will need now and in the future to unlock better and better experiences for users. These apps are always pushing at the boundary of what is possible to do well on the web, and sometimes beyond. When they go beyond what current web APIs or architectures were designed for, the web app may still work, but it’ll sometimes be slower and more cumbersome to use---and more taxing on your computer than necessary. We are filling these gaps with new APIs that allow web apps to continue doing all these things, but in a way that is much easier to implement performantly.

Extensibility has been part of the design of RenderingNG from the beginning, and is a big reason why the system is clean and modular.

Results

The first RenderingNG performance optimization shipped in 2015. Below we’ve highlighted a few of the many subsequent performance improvements since then.

CompositeAfterPaint: faster and more responsive on all pages [1]

In the M94 release, we will ship CompositeAfterPaint, a new compositing subsystem that allows for arbitrary scaling up and down of GPU use for web app content. Even better, due to more efficient, purpose-built algorithms in this subsystem, CompositeAfterPaint will substantially improve:

Scroll latency by up to 8%
Responsiveness by up to 3%
Peak rendering speed by up to 3%
GPU memory use by more than 3%
CPU time spent rendering and interacting with tabs, resulting in a savings of more than 1400 years of CPU time per day
Battery life by up to 0.5%

GPU Raster: dramatically faster than before [2]

When it shipped on Mac in 2016, GPU raster resulted in a 37% improvement on the overall MotionMark benchmark, and 150% on HTML categories. We subsequently brought similar wins to all other platforms and content types, concluding in 2020. In 2021 we shipped improvements focused on further GPU acceleration for 2D Canvas rendering, with up to a 1000% improvement in the Path rendering test, and a 130% improvement overall on the MotionMark 1.2 benchmark, as measured on an Apple M1 Mac.

Out-of-process Raster: much-improved reliability [1]

Out-of-process raster shipped on Android in 2018. This reduced the crash rate for problematic GPU drivers by 6x, bringing reliability in line with other drivers.

Learn more

If you’d like to learn more about RenderingNG, head over to the Chrome Developers Blog, where we are publishing a whole series about the project, with more to come over time.

Thanks for reading!

Posted by Chris Harrelson, Lead Rendering Software Engineer

[1] Data source: Real-world data anonymously aggregated from Chrome clients.
[2] Data source: MotionMark.

Faster and more efficient phishing detection in M92

Tuesday, July 20, 2021

Keeping Chrome users safe as they browse the web is crucially important to Chrome; in fact, security has always been one of our four core principles. In some cases, security can come at the expense of performance. In our next post in The Fast and the Curious series, we are excited to share how improvements to our phishing detection algorithms keeps users safe online. With these improvements, phishing detection is now 50 times faster and drains less battery.Phishing detection

What you will see if a phishing attempt is detected

To preserve your privacy, by default Chrome's Safe Browsing mode never sends any images outside the browser. While this is great for privacy, it means that your machine has to do all the work to analyze the image.

Image processing can often generate heavy workloads because analyzing the image requires an evaluation of each pixel in what is commonly known as a “pixel loop.” Some modern monitors display upwards of 14 million pixels, so even simple operations on each of those pixels can add up to a lot of CPU use! For phishing detection, the operation that takes place on each pixel is the counting of its basic colors.

Here is what this looks like. The counts are stored in an associative data structure called a hashmap. For each pixel, we extract its RGB color values and store the counts in one of 3 different hashmaps -- one for each color.

Making it more efficient

Adding one item to a hashmap is fast, but we have to do this for millions of pixels. We try to avoid reducing the number of pixels to avoid compromising the quality of the analysis. However, the computation itself can be improved.

Our improvements to the pipeline look like this:

The code now avoids keeping track of RGB channels in three different hashmaps and instead uses only one to index by color. Three times less counting!
Consecutive pixels are summed before being counted in the hashmap. For a site with a uniform background color, this can reduce the hashmap overhead to almost nothing.

Here is what the counting of the colors looks like now. Notice how there are significantly fewer operations on the hashmap:

How much faster did this get?

Starting with M92, Chrome now executes image-based phishing classification up to 50 times faster at the 50th percentile and 2.5 times faster at the 99th percentile. On average, users will get their phishing classification results after 100 milliseconds, instead of 1.8 seconds.

This benefits you in two ways as you use Chrome. First and foremost, using less CPU time to achieve the same work improves general performance. Less CPU time means less battery drain and less time with spinning fans.
Second, getting the results faster means Chrome can warn you earlier. The optimization brought the percentage of requests that took more than 5 seconds to process from 16.25% to less than 1.6%. This speed improvement makes a real difference in security - especially when it comes to stopping you from entering your password in a malicious site!
Overall, these changes achieve a reduction of almost 1.2% of the total CPU time used by all Chrome renderer processes and utility processes.
At Chrome’s scale, even minor algorithm improvements can result in major energy efficiency gains in aggregate. Here’s to many more centuries of CPU time saved!

Stay tuned for many more performance improvements to come!

Posted by Olivier Li Shing Tat-Dupuis, Chrome Developer

Data source for all statistics: Real-world data anonymously aggregated from Chrome clients.

Chrome is up to 23% faster in M91 and saves over 17 years of CPU time daily

Thursday, May 27, 2021

Since the launch of Chrome in 2008, speed has been one of the 4 core principles that shape the work we do to deliver a highly performant browser. The V8 JavaScript compiler is a critical part of delivering maximum speed for the JavaScript that’s shipped on practically every web page. In our next post in The Fast and the Curious series, we are excited to share how improvements to the V8 engine are delivering up to 23% faster performance.

An important component of delivering a fast browser is fast JavaScript execution. In Chrome, that job is done by the V8 engine which executes over 78 years worth of JavaScript code on a daily basis. In M91 Chrome is now up to 23% faster with the launch of a new Sparkplug compiler and short builtin calls, saving over 17 years of our users' CPU time each day! Sparkplug is a new JavaScript compiler that fills the gap between needing to start executing quickly and optimizing the code for maximum performance. Short builtin calls optimize where in memory we put generated code to avoid indirect jumps when calling functions.

Sparkplug

The V8 engine has multiple compilers which can make different tradeoffs throughout the various phases of executing JavaScript. Three years ago, we launched a new two-tier compiler system consisting of Ignition and Turbofan. Ignition is a bytecode interpreter whose job is to start executing the JavaScript with as little delay as possible. Turbofan is the optimizing compiler that generates high-performance machine code based on information gathered during JavaScript execution; as a result, it starts up more slowly than Ignition’s bytecode compiler. Sparkplug strikes a balance between Ignition and Turbofan in that it does generate native machine code but does not depend on information gathered while executing the JavaScript code. This lets it start executing quickly while still generating relatively fast code. For a complete technical deep dive into what it took to make this new engine, please see our V8 blog post.

Short builtins

Short builtins is a mechanism by which the V8 engine optimizes the location in memory of generated code. When V8 generates CPU-specific code from JavaScript, it lays that code out in memory. This generated code will frequently call builtin functions, which are small snippets of code for handling common routines --everything from basic operations like adding two variables, to full-fledged functions in the JavaScript standard library. For some CPUs, calling functions that are further away from your generated code can cause CPU-internal optimizations (such as branch prediction logic) to fail. The fix for this is to copy the builtin functions into the same memory region as the generated code. This change is especially impactful for the new Apple M1 chip. Please see our V8 blog post to learn more about the impact across platforms of this feature.

Stay turned for many more performance improvements to come!

Posted by Thomas Nattestad, Chrome Product Manager

Data source for all statistics: Speedometer 2.0.

Digging for performance gold: finding hidden performance wins

Thursday, April 22, 2021

We are fortunate that so many people choose Chrome as their browser to get things done, which is why we are continually investing in making Chrome more performant. But with software as complex as Chrome, there is a lot of performance left hidden in areas we aren’t actively working on. In our latest post in the The Fast and the Curious series, we investigate how to diagnose, find, and fix performance problems that normally go undetected.The 1%

Approach

As engineers, our training in optimization is to focus on improving the algorithmic performance of the components we own. The last 3 years of analyzing the immensely complex codebase of Chrome however have taught us that the real issue is often cross-cutting: multiple unrelated features’ long-tail performance issues, sharing the same systemic root cause(s). Applying local expertise and optimization is likely to miss the global optimum. It is necessary to disregard our initial intuition and assume ignorance, forcing us to dig beyond what is immediately apparent and find the underlying root cause by relentlessly exposing what we don’t know.

Chasing Invisible Bugs

How do we find bugs that are unforeseen, unreproducible, unowned, and essentially invisible?

First, define a scenario. For this work, we focus on user-visible Jank, which we measure in the field as a way to systematically identify moments where Chrome feels slow.

Second, gather high actionability bug reports in the field. For this we rely on Chrome’s BackgroundTracing infrastructure to generate what we call Slow Reports. A subset of Canary users who have opted in to sharing anonymized metrics have circular-buffer tracing enabled to examine specific scenarios. If a preconfigured threshold on a metric of interest is hit, the trace buffer is captured, anonymized, and uploaded to Google servers.

Such a bug report might look like this:

chrome://tracing view of a 2 seconds Jank on AutocompleteController::UpdateResult() on an otherwise healthy machine

We have a culprit! Let’s optimize AutocompleteController? No! We don’t know why yet: keep assuming ignorance!

By augmenting BackgroundTracing with stack sampling, we were able to find a recurring stack under stalled AutoComplete events:
RegEnumValueW

RegEnumValueWStub

base::win::RegistryValueIterator::Read()

gfx::`anonymous namespace\'::CachedFontLinkSettings::GetLinkedFonts

gfx::internal::LinkedFontsIterator::GetLinkedFonts()

gfx::internal::LinkedFontsIterator::NextFont(gfx::Font *)

gfx::GetFallbackFonts(gfx::Font const &)

gfx::RenderTextHarfBuzz::ShapeRuns(...)

gfx::RenderTextHarfBuzz::ItemizeAndShapeText(...)

gfx::RenderTextHarfBuzz::EnsureLayoutRunList()

gfx::RenderTextHarfBuzz::EnsureLayout()

gfx::RenderTextHarfBuzz::GetStringSizeF()

gfx::RenderTextHarfBuzz::GetStringSize()

OmniboxTextView::CalculatePreferredSize()

OmniboxTextView::ReapplyStyling()

OmniboxTextView::SetText...)

OmniboxResultView::Invalidate()

OmniboxResultView::SetMatch(AutocompleteMatch const &)

OmniboxPopupContentsView::UpdatePopupAppearance()

OmniboxPopupModel::OnResultChanged()

OmniboxEditModel::OnCurrentMatchChanged()

OmniboxController::OnResultChanged(bool)

AutocompleteController::UpdateResult(bool,bool)

AutocompleteController::Start(AutocompleteInput const &)

(...)

Ah ha! Autocomplete is not at fault. Time to optimize GetFallbackFonts()?! But wait… Why is GetFallbackFonts() even called in the first place?

And before we figure that out, how do we know this is the #1 root cause of our overall long-tail performance issue? We’ve only looked at one trace so far after all...

The Measurement Conundrum

The metrics tell us how many users are affected and how bad it is, but they do not highlight the root cause.

Slow Reports tell us what the problem is for a specific user but not how many users are affected. And while we can query our corpus of Slow Report traces, it comes with inherent biases that make it impossible to correlate 1:1 with metrics. For instance, because Chrome only reports the first instance of bad performance per-session and only for users of the Canary/Dev channel, there’s both a startup and a population bias.

This is the measurement conundrum. The more actionability (data) a tool provides, the fewer scenarios it captures and the more bias it incurs. Depth vs. breadth.

Tools that attempt to do both sit somewhere in the middle, where they use aggregation over a large dataset and risk showing aggregate results based on flawed input (e.g. circular buffer tracing having dropped the interesting portion and contributing to a biased aggregate).

Thus we scientifically opted for the least engineering-minded option: open a bunch of Slow Report traces manually. This gave us the most actionability over a top-level issue we’d already quantified.

After opening dozens of traces it turned out that a great majority showed variations of the aforementioned fonts issue. While this didn’t give us a precise #users-affected, it was enough for us to believe it was the main cause of user pain seen in the metrics.

Fallback Fonts

We dug into why GetFallbackFonts() had to be called in the first place. In the example above, the caller is trying to determine the size in pixels of a Unicode string rendered by a given font.

If a substring within it is from a Unicode Block that can’t be rendered by the given font, GetFallbackFont() is used to request the system recommended fallback font for it. If that fails, GetFallbackFonts() is invoked to try all the linked fonts and determine the one that can best render it; that second fallback is much slower.

GetFallbackFont() should never fail, but in practice it’s not that simple. The reliable way to do this on Windows is to query DirectWrite; however, DirectWrite was added in Windows 7, when Chrome still supported Windows XP. Therefore the GetFallbackFont() logic was forced to stick to a less reliable heuristic using Uniscribe+GDI in order to work on both versions of the OS. Since things worked most of the time, no one noticed that this could have been cleaned up when Chrome later dropped support for Windows XP. With new tooling to investigate long-tail performance, this turned out to be the number one cause of jank (unnecessarily invoking GetFallbackFonts()).

We fixed that, reducing the amount of calls to GetFallbackFonts() by 4x.

Still not zero though, and still seeing instances of the aforementioned AutoComplete issue in our Slow Reports. Keep digging. DirectWrite’s GetFallbackFont() failing was unexpected, but since Slow Reports are anonymized, no user-generated strings can be uploaded -- and therefore, finding which codepoints were problematic was tricky. We teamed up with our privacy experts to instrument Unicode Block and Script of text blocks going through HarfBuzz so that we could ensure no leakage of Personally Identifiable Information.

The Emoji Saga

With this new recording enabled, the next wave of Slow Reports came back. The vast majority of reports indicated that font fallback was failing when DirectWrite was being asked to find a font for a codepoint (Unicode character) in Miscellaneous Symbols and Pictographs. We wrote a local script trying all codepoints in that Unicode Block and quickly found out which ones could be problematic: U+1F3FB - U+1F3FF are modifiers added in Unicode 8.0 and are meaningful only when paired with another codepoint. For instance, U+1F9D7 (🧗) when paired with U+1F3FF is 🧗🏿. No font can render U+1F3FF on its own, and font fallback would correctly error out after scanning all linked fonts when asked to find one. The bug was in the browser-side Unicode segmentation logic which incorrectly broke down these two codepoints and asked DirectWrite to render them separately instead of keeping them as a single grapheme.

But wait, doesn’t Chrome support modern Unicode..?! Indeed, it does, in Blink which renders the web content. But the browser-side logic was not updated to support modern emojis (with modifiers) because it didn’t use to draw emojis at all. It’s only when the browser UI (tab strip, bookmark bar, omnibox, etc.) was modernized to support Unicode circa 2018 that the legacy segmentation logic became an (invisible) problem.

On top of that, the caching logic did not cache on error, so trying to render a modifier on its own caused a massive jank, every time, for users with a lot of fonts installed. Ironically, this cache had been added to amortize the cost of this misunderstood bottleneck when Unicode support was first added to browser UI. Diving deeper into the underlying implementation of our fonts logic, rather than stopping at the layer of the fonts APIs, not only fixed a major performance issue but also resulted in a correctness fix for other emojis. For instance, 🏳️‍🌈 is encoded as U+1F3F3( 🏳️) + U+1F308 (🌈); before the itemization fix, browser UI would incorrectly render this grapheme as 🏳️🌈.

And the journey continues...

Our journey keeps going into various components of Chrome but it always follows the same basic playbook: assume ignorance and relentlessly investigate unforeseen, unreproducible, and unowned bugs. And while stack ranking issues is nigh impossible (see: measurement conundrum), fixing the top 5 findings from any given tool and zooming in on the long tail has always addressed the majority of the user pain in practice.

Using this approach, we have reduced user-visible jank by a factor of 10X over the last 2.5 years and improved long-tail performance of many features caught in the cross-fire.

99th percentile of # of unresponsive 100ms intervals over a 30 seconds sample

Posted by Gabriel Charette 🤸🏼 and Etienne Bergeron 🕵🏻, Chrome Software Engineers

Data source for all statistics: Real-world data anonymously aggregated from Chrome clients.

Efficient And Safe Allocations Everywhere!

Monday, April 12, 2021

In our constant work to improve performance, our engineers sometimes have to seek optimizations in places that most software developers don’t venture. In this post in our series, The Fast and the Curious, a team of senior engineers showed how they approached replacing the system-level memory allocator with an optimized version, yielding significant memory savings -- up to 22% on Windows.
PartitionAlloc is Chromium’s memory allocator, designed for lower fragmentation, higher speed, and stronger security and has been used extensively within Blink (Chromium’s rendering engine). In Chrome 89 the entire Chromium codebase transitioned to using PartitionAlloc everywhere (by intercepting and replacing malloc() and new) on Windows 64-bit and Android. Data from the field demonstrates up to 22% memory savings, and up to 9% improvement in responsiveness and scroll latency of Chrome.

Here's a closer look at memory usage in the browser process for Windows as the M89 release began rolling out in early March:

Background

Chrome is a multi-platform, multi-process, multi-threaded application, serving a wide range of needs, from small embedded WebViews on Android to spacecraft. Performance and memory footprint are of critical importance, requiring a tight integration between Chrome and its memory allocator. But heterogeneity across platforms can be prohibitive with each platform having a different implementation such as tcmalloc on Linux and Chrome OS, jemalloc or scudo on Android, and LFH on Windows.

When we started this project, our goals were to: 1) unify memory allocation across platforms, 2) target the lowest memory footprint without compromising security and performance, and 3) tailor the allocator to optimize the performance of Chrome. Thus we made the decision to use Chromium’s cross-platform allocator, to optimize memory usage for client rather than server workloads and to focus on meaningful end user activities, not micro-benchmarks that wouldn’t really matter in real world usage.

Allocator Security

PartitionAlloc was designed to support multiple independent partitions, i.e. non-overlapping regions of memory. We use these partitions throughout Blink to thwart some forms of type confusion attacks, such as ensuring strings are separated from layout objects. However, this approach only avoids collisions between types that are allocated from different partitions. Furthermore, PartitionAlloc buckets allocations by their sizes, to help avoid type confusion when potentially-colliding objects are of dissimilar size. These techniques work because PartitionAlloc doesn’t re-use address space. Once PartitionAlloc dedicates a region of address space to a certain partition and size bucket, it will always belong to that partition and size bucket.

Additionally, PartitionAlloc protects some of its metadata with guard pages (inaccessible ranges) around memory regions. Not all metadata is equal, however: free-list entries are stored within previously allocated regions, and thus surrounded by other allocations. To detect corrupted free-list entries and off-by-one overflows from client code, we encode and shadow them.
Finally, having our own allocator enables advanced security features like MiraclePtr and *Scan.

Architecture Details

Each partition in PartitionAlloc uses a single, central, slab-based allocator to conserve memory, with a minimal per-thread cache in front for scaling to multi-threaded workloads. This simplicity also pays performance dividends: we’ve extensively profiled and aggressively trimmed the allocator’s fast path, improving thread-local storage access, locks, reducing cache line fetches, and removing branches.

PartitionAlloc pre-reserves slabs of virtual address space. They are gradually backed by physical memory, as allocation requests arrive. Small and medium-sized allocations are grouped in geometrically-spaced, size-segregated buckets, e.g. [241; 256], [257; 288]. Each slab is split into regions (called “slot spans”) that satisfy allocations (“slots”) from only one particular bucket, thereby increasing cache locality while lowering fragmentation. Conversely, larger allocations don’t go through the bucket logic and are fulfilled using the operating system’s primitives directly (mmap() on POSIX systems, and VirtualAlloc() on Windows).

This central allocator is protected by a single per-partition lock. To mitigate the scalability problem arising from contention, we add a small, per-thread cache of small slots in front, yielding a three-tiered architecture:

The first layer (Per-thread cache) holds a small amount of slots belonging to smaller and more commonly used buckets. Because these slots are stored per-thread, they can be allocated without a lock and only requiring a faster thread-local storage lookup, improving cache locality in the process. The per-thread cache has been tailored to satisfy the majority of requests by allocating from and releasing memory to the second layer in batches, amortizing lock acquisition, and further improving locality while not trapping excess memory.

The second layer (Slot span free-lists) is invoked upon a per-thread cache miss. For each bucket size, PartitionAlloc knows a slot span with free slots associated with that size, and captures a slot from the free-list of that span. This is still a fast path, but slower than per-thread cache as it requires taking a lock. However, this section is only hit for larger allocations not supported by per-thread cache, or as a batch to fill the per-thread cache.

Finally, if there are no free slots in the bucket, the third layer (Slot span management) either carves out space from a slab for a new slot span, or allocates an entirely new slab from the operating system, which is a slow but very infrequent operation.

The overall performance and space-efficiency of the allocator hinges on the many tradeoffs across its layers such as how much to cache, how many buckets, and memory reclaiming policy. Please refer to PartitionAlloc to learn more about the design.

All in all, we hope you will enjoy the additional memory savings and performance improvements brought by PartitionAlloc, ensuring a safer, leaner, and faster Chrome for users on Earth and in outer space alike. Stay tuned for further improvements, and support of more platforms coming in the near future.

Posted by Benoît Lizé and Bartek Nowierski, Chrome Software Engineers

Data source for all statistics: Real-world data anonymously aggregated from Chrome clients.
*The core metric measures jank -- delay handling user input -- every 30 seconds.

Don’t Copy That Surface

Monday, April 5, 2021

This post is part of a new series The Fast and the Curious we're bringing you of deeper dives into the careful trade-offs and complex engineering that goes into making Chrome fast and reliable. This debugging adventure by Chrome developer and blogger Bruce Dawson reduced CPU usage by about 3% when using a webcam - a real help for those of us relying on video calls.
Video conferencing took on elevated importance in 2020. I’m not on the Google Meet team but I do work on Chrome, so I fired up my favorite profiler during one of my daily meetings to see if I could find anything useful.

There is a lot going on during video conferencing, spread across multiple processes. With my usual dozens of tabs open there were 37 Chrome processes, with six of them actively participating in the video conference. In addition there were over 200 other processes running (87 copies of svchost.exe, for instance), with four of those involved in video conferencing. You may well wonder why it takes 10 processes to connect two people, so here is a list of the processes and their roles:

audiodg.exe - Windows Audio Device Graph Isolation, audio output
dwm.exe - Windows Desktop Window Manager, showing video
svchost.exe - Windows Camera Frame Server (webcam capture)
System - Windows system process, does miscellaneous tasks on behalf of processes
chrome.exe - browser process, the master control program
chrome.exe - renderer process, the Meet tab
chrome.exe - GPU process, in charge of rendering pages
chrome.exe - NetworkService utility process, talking to the network
chrome.exe - VideoCaptureService, talking to the Windows Camera Frame Server
chrome.exe - AudioService, controls audio input and output

These tasks are spread across different processes for security and stability. If one of them crashes it can be restarted without taking everything down. If one of them is compromised due to a security bug then it is isolated from the rest of the system and the damage may be contained.

This all makes good sense, but having this many processes involved can make performance-profiling challenging. It can be challenging to look through all of these processes to find areas of potential improvement. It is made more difficult by the fact that I know little about the Meet architecture.

Analyzing a profile

Video conferencing is CPU intensive - you have to record, compress, transmit, receive, decompress, and display both audio and video. The data below shows CPU samples recorded by Microsoft’s Event Tracing for Windows (ETW). This sampling profiler works by interrupting every running thread about 1,000 times a second and recording a call stack. I used Windows Performance Analyzer (WPA) to display the results. In the screenshot below I am looking at a 10-second period and over 16,000 samples (representing about 16 seconds of CPU time) were recorded across the 10 processes involved in video conferencing:

That’s a lot of samples to look through, but the call stacks are collated so that you can drill down on the busiest stacks. I didn’t find anything in the first Chrome process, but in the second one I did:

It doesn’t look like much, but I recognized immediately that the 124 samples in KiPageFault were worth investigating. Most of the CPU-intensive work in this trace was important and unavoidable work but I had a hunch that these samples represented avoidable work - something that I could fix. And, even though they represented just 0.75% of the samples I suspected that they indicated a somewhat greater cost.

I recognized their importance immediately because this is something that I have seen before. KiPageFault means that the processor touched some memory that had been allocated, but was not currently in the process. This could mean that the pages had been removed from the process to save memory, but in an active process on a machine with lots of memory, that didn’t make sense. What was more likely was that this represented recently allocated memory.

When a program allocates a small amount of memory, the local memory manager (sometimes called the “heap”) will usually have some available that it can give to the program. But if it doesn’t have an appropriate block of memory then it will ask the operating system (OS) for some. If a program allocates a large amount of memory (greater than a MB or so) then the heap will definitely ask for more memory. This is, in itself, a relatively cheap operation. The heap asks the OS for some memory, the OS says “sure”, then the OS makes note of the fact that it promised this memory, and that’s it. The OS does not, at that time, actually give the program any memory. This is the way of the world on Windows, Linux, Android and it is good but it can be confusing and surprising. If the process never touches the memory then the memory is never added to the process, but if the process does touch the memory then individual pages of zeroed memory are brought into the process. These are called demand-zero page faults because zeroed pages are “faulted” into the process on demand.

In other words, allocating a large block of memory is quite cheap, but doesn’t actually set up the promised memory. Then, when the program tries to use the memory and the CPU discovers that there is no memory at that address it triggers an exception, which wakes up the OS. The OS checks its records and realizes that it did in fact promise to put memory at that address so it then puts some there and restarts the program. This happens so quickly that if you’re not paying attention you will miss it, but it shows up when profiling as samples hitting in KiPageFault.

This bizarre dance happens again for every 4-KiB block in the allocation - 4 KiB is the size of the pages that the CPU and the OS work on.

The cost is small. Across this 10-second period only 124 samples - representing about 124 ms or 0.124 seconds - hit inside of KiPageFault. The total cost of the enclosing CopyImage_SSE4_1 function was about 240 ms, so the page faults accounted for more than half of this function, but barely a quarter of the cost of the OnSample function on line 15.

The total costs of these page faults is modest but they hint at many other costs:

If this memory is being allocated repeatedly (presumably every frame) then it must also be freed every frame. On line 26 we can see that the Release function which frees the memory uses another 64 samples.
When the pages are freed the operating system has to zero them (for security reasons) so that they are ready to be reused. This is done in the Windows System process - an almost entirely hidden cost. Sure enough when I looked in the System process I saw 138 samples in the MiZeroPageThread. I found that 87% of the KiPageFault samples in the entire system were in the CopyImage_SSE4_1 call so presumably 87% of the 138 samples in the MiZeroPageThread were due to this pattern.

I analyzed these hidden costs of memory allocation in a 2014 blog post. The basic memory architecture of Windows hasn’t changed since then so the hidden costs remain about the same.

In addition to CPU samples my ETW trace contained call stacks for every call to VirtualAlloc. This WPA screenshot shows a 10-second period where the OnSample function does 298 allocations that are each 1.320 MB, roughly 30 per second:

At this point we can estimate that the cost of these repeated allocations is 124 (faulting in) plus 64 (freeing) plus 124 (87% of the zeroing samples) for a total of 312 samples. This gets us up to 1.9% of the total CPU cost of video conferencing. Fixing this is not going to change the world, but this is a change worth doing.

But wait, there’s more!

We are locking this buffer so that we can look at the contents, but it turns out we don’t actually want the lock call to copy the buffer at all. We just want the lock call to describe the buffer to us so that we can look at it in place. Therefore the entire cost of the MFCopyImage call is waste! That’s another 116 samples. In addition, in the CMF2DMediaBuffer::Unlock call on line 26 there is another call to CMF2DMediaBuffer::ContiguousCopyFrom. That’s because the Unlock call assumes that we might have modified the copy of the buffer, so it copies it back. So the 101 samples there are all waste as well!

If we can examine this buffer without the alloc/copy/copy/free dance then we can save 312 samples plus 116 samples (the rest of the copying cost) plus 101 samples (the copying-it-back cost) for a total saving of 3.2%. This is getting better all the time.

Note that sampled data is only statistically valid, and the actual percentages vary significantly depending on the computer and the exact workload. But, the point remains - it is a non-dramatic but worthwhile change to investigate.

Despite spending years in the video-game business my knowledge of these graphics-buffer locking and unlocking APIs is weak. I ended up relying on the wisdom of my Twitter followers to come to the conclusion that the copying was entirely avoidable, and to get a rough pattern for how it could be fixed. After filing an overly verbose bug I delegated the task of actually fixing it. The fix landed in M85 and was deemed important enough that it was then backported to M84.

You’d have to be paying very close attention to see the difference - spread across a Chrome process and the system process - but I hope that this helped some computers run a bit cooler and last longer on their batteries. And, while this inefficiency was found by profiling Google Meet, the improvement actually benefits any product that uses the webcam inside Chrome (and other Chromium-based browsers).

Verification

After the fix landed I compared two 10-second ETW traces from Chrome Canary before and after the change, each taken with no other programs running except a single Chrome tab running the Google Meet pre-meeting page. In both cases I looked at a 10-second period of time in the profiler. This showed:

CPU time in OnSample:

Before: 458 ms (432 ms of which were in Lock/Unlock/KiPageFault)
After: 27 ms

Allocations:

Before: 30 allocations per second of 1.32 MB (one per frame, running at 30 fps - a higher framerate would mean more allocations), totalling 396 MB over 10 seconds
After: 0 allocations

CPU time in the System process's MiZeroPageThread:

Before: 36 ms
After: 0 ms

These measurements showed - in three different ways - that the performance problem was fixed. The memory copying in OnSample was gone, the repeated allocations were gone, and the system process was doing less work. Mission accomplished, bug closed.

Chromium Blog

Faster Chrome - Let The Compiler do the work

Partitioning Chrome's Code for Faster Launch Times on Android

Searching, browsing, and shutdown Chrome performance improvements

RenderingNG: an architecture that makes and keeps Chrome fast for the long term

Faster and more efficient phishing detection in M92

Chrome is up to 23% faster in M91 and saves over 17 years of CPU time daily

Digging for performance gold: finding hidden performance wins

Efficient And Safe Allocations Everywhere!

Don’t Copy That Surface

Labels

Archive

Feed