Chromium Blog
News and developments from the open source browser project
Distributed Reliability Testing
středa 25. února 2009
We want Google Chrome to be as stable as possible. No matter what site you browse to or what you do, Chrome should never crash. A system we call "distributed reliability testing" is one of the main tools we use to help turn that goal into reality.
One of the advantages to being associated with Google is that we have access to a lot of information about the Web, and a lot of computers to test on. About once an hour, our distributed test infrastructure takes the very latest version of Google Chrome in development and uses it to automatically load a large number of the pages that Google has seen are most popular around the world. When it's done, it produces a report like this on the
Buildbot waterfall
that all our developers (and anyone else) can see:
Results for top 500 web sites:
success: 499; crashes: 0; crash dumps: 0; timeout: 1
Results for top 500 web sites without sandbox:
success: 463; crashes: 0; crash dumps: 0; timeout: 2
Results for extended list of web sites:
success: 99768; crashes: 3; crash dumps: 3; timeout: 463
Here the final test got through a bit over 100,000 pages before stopping to make way for the next build to be tested. And before each
Dev, Beta, or Stable channel
release, we run with a much larger number of URLs.
In addition, we "fuzz-test" the user interface, automatically performing arbitrary sequences of actions (opening a new tab, pressing the spacebar, opening various dialogs, etc. — a total of more than 30 possible actions). These are also run in our distributed testing architecture, so we can exercise thousands of combinations for each new version of Google Chrome in progress. The same report that shows the page-load results above collects these UI test results too:
Results for automated UI test:
success: 64643; crashes: 0; crash dumps: 0; timeout: 0
This sort of large-scale testing is great for finding crashes that happen only rarely, or that only affect pages that developers wouldn't have visited as part of their haphazard manual testing. By catching a problem right away even if it's very rare, it's easier for developers to figure out what change caused the error and fix it before it ever gets close to showing up in Google Chrome itself.
Posted by Pamela Greene and Patrick Johnson, Software Engineers
Štítky
$200K
1
10th birthday
4
abusive ads
1
abusive notifications
2
accessibility
3
ad blockers
1
ad blocking
2
advanced capabilities
1
android
2
anti abuse
1
anti-deception
1
background periodic sync
1
badging
1
benchmarks
1
beta
83
better ads standards
1
billing
1
birthday
4
blink
2
browser
2
browser interoperability
1
bundles
1
capabilities
6
capable web
1
cds
1
cds18
2
cds2018
1
cloud print
1
coalition
1
coalition for better ads
1
contact picker
1
content indexing
1
cookies
1
core web vitals
2
csrf
1
css
1
cumulative layout shift
1
custom tabs
1
dart
8
dashboard
1
Data Saver
3
Data saver desktop extension
1
day 2
1
deceptive installation
1
declarative net request api
1
design
2
developer dashboard
1
Developer Program Policy
2
developer website
1
devtools
13
digital event
1
discoverability
1
DNS-over-HTTPS
4
DoH
4
emoji
1
emscriptem
1
enterprise
1
extensions
27
Fast badging
1
faster web
1
features
1
feedback
2
field data
1
first input delay
1
Follow
1
fonts
1
form controls
1
frameworks
1
fugu
2
fund
1
funding
1
gdd
1
google earth
1
google event
1
google io 2019
1
google web developer
1
googlechrome
12
harmful ads
1
html5
11
HTTP/3
1
HTTPS
4
chrome
35
chrome 81
1
chrome 83
2
chrome 84
2
chrome ads
1
chrome apps
5
Chrome dev
1
chrome dev summit
1
chrome dev summit 2018
1
chrome dev summit 2019
1
chrome developer
1
Chrome Developer Center
1
chrome developer summit
1
chrome devtools
1
Chrome extension
1
chrome extensions
3
Chrome Frame
1
Chrome lite
1
Chrome on Android
2
chrome on ios
1
Chrome on Mac
1
Chrome OS
1
chrome privacy
4
chrome releases
1
chrome security
10
chrome web store
32
chromedevtools
1
chromeframe
3
chromeos
4
chromeos.dev
1
chromium
9
iframes
1
images
1
incognito
1
insecure forms
1
intent to explain
1
ios
1
ios Chrome
1
issue tracker
3
jank
1
javascript
5
lab data
1
labelling
1
largest contentful paint
1
launch
1
lazy-loading
1
lighthouse
2
linux
2
Lite Mode
2
Lite pages
1
loading interventions
1
loading optimizations
1
lock icon
1
long-tail
1
mac
1
manifest v3
2
metrics
2
microsoft edge
1
mixed forms
1
mobile
2
na
1
native client
8
native file system
1
New Features
5
notifications
1
octane
1
open web
4
origin trials
2
pagespeed insights
1
pagespeedinsights
1
passwords
1
payment handler
1
payment request
1
payments
2
performance
20
performance tools
1
permission UI
1
permissions
1
play store
1
portals
3
prefetching
1
privacy
2
privacy sandbox
4
private prefetch proxy
1
profile guided optimization
1
progressive web apps
2
Project Strobe
1
protection
1
pwa
1
QUIC
1
quieter permissions
1
releases
3
removals
1
rlz
1
root program
1
safe browsing
2
Secure DNS
2
security
36
site isolation
1
slow loading
1
sms receiver
1
spam policy
1
spdy
2
spectre
1
speed
4
ssl
2
store listing
1
strobe
2
subscription pages
1
suspicious site reporter extension
1
TCP
1
the fast and the curious
23
TLS
1
tools
1
tracing
1
transparency
1
trusted web activities
1
twa
2
user agent string
1
user data policy
1
v8
6
video
2
wasm
1
web
1
web apps
1
web assembly
2
web developers
1
web intents
1
web packaging
1
web payments
1
web platform
1
web request api
1
web vitals
1
web.dev
1
web.dev live
1
webapi
1
webassembly
1
webaudio
3
webgl
7
webkit
5
WebM
1
webmaster
1
webp
5
webrtc
6
websockets
5
webtiming
1
writable-files
1
yerba beuna center for the arts
1
Archive
2024
srp
čvn
kvě
dub
bře
úno
2023
lis
říj
zář
srp
čvn
kvě
dub
úno
2022
pro
zář
srp
čvn
kvě
dub
bře
úno
led
2021
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2020
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2019
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2018
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2017
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2016
pro
lis
říj
zář
srp
čvn
kvě
dub
bře
úno
led
2015
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2014
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2013
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2012
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2011
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2010
pro
lis
říj
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2009
pro
lis
zář
srp
čvc
čvn
kvě
dub
bře
úno
led
2008
pro
lis
říj
zář
Feed
Follow @ChromiumDev
Give us feedback in our
Product Forums
.