As part of http://trac.webkit.org/changeset/121351, I landed LayoutTests/css3/calc/transitions.html that uses the transitions test framework. It is failing flakily, with the error "CONSOLE MESSAGE: line 263: TypeError: 'undefined' is not a function (evaluating 'callback()')" See http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&tests=css3%2Fcalc%2F for results on other platforms - it fails on Mac, Win and Linux. Line 263 is in this function in http://trac.webkit.org/browser/trunk/LayoutTests/transitions/resources/transition-test-helpers.js: function waitForAnimationStart(callback, delay) { var delayTimeout = delay ? 1000 * delay + 10 : 0; // Why the two setTimeouts? Well, for hardware animations we need to ensure that the hardware animation // has started before we try to pause it, and timers fire before animations get committed in the runloop. window.setTimeout(function() { window.setTimeout(function() { callback(); // <-- line 263 }, 0); }, delayTimeout); } which is called from one place: function startTest(expected, usePauseAPI, callback) { if (callback) callback(); waitForAnimationStart(function() { runTest(expected, usePauseAPI); }); } and the callback is always provided. More data from my local testing on MacOS 10.6.8: - using --run-singly with NRWT I can't make transitions.html flake - I've tried dumb things like renaming callback to callbackfoo - the error still flakily occurs, complaining about callbackfoo
Flakiness resolved in http://trac.webkit.org/changeset/125111 - it was a problem with another test, not the framework.