For mass rebaselines (Leopard -> Snow Leopard, SVG changes, Skia changes), etc. it would be nice to be able to update baselines locally en-masse.
Created attachment 70938 [details] Patch
nice! I've often thought about writing a script that did exactly this ...
I like the use of a local httpd to handle the file copies. I'm happy to review this once you're happy with it.
Created attachment 71568 [details] Patch Progress
Created attachment 71619 [details] Patch More progress
Created attachment 71621 [details] Patch
(notes for anyone interested in trying this) Current usage: 1. Make a change that requires lots of (pixel) rebaselines 2. Run new-run-webkit-tests --tolerance 0 --pixel-tests [path/to/tests] 3. Run webkit-patch local-rebaseline /path/to/tests/results (usually WebKitBuild/{Debug|Release}/layout-test-results 4. Go to http://localhost:8127/ In the future, it probably also makes sense to allow a results zip file to be pulled from a bot, so that you can do mass rebaselines for other platforms too. In the web UI (which requires a 30 inch display), you can use the menus at the top to select tests by failure type and directory. Once you have a set of tests selected, you can navigate between them with the « and » buttons, or by using the left and right arrow keys. For a particular test, the current and expected outputs will be displayed, along with a diff. For image diffs, you can click anywhere in the images to get a "loupe" that lets you see pixel values up close. You can use the two drop-downs at the bottom of the screen to choose what you want done with the baselines. The baseline target is where the test outputs should be copied to. If moving between platforms, you may wish to move current baselines to another directory (e.g. if updating current Mac baselines for Snow Leopard, you would set the target to "mac" and move current baselines to "mac-leopard"). Once you've determined that baselines should be updated, you can use the "Add to rebaseline queue" button (or the q key) to add it to the list. Once you've added enough tests to the queue (commits probably shouldn't get any bigger than ~250 baselines), you show the queue with the "Queue" link and use the "Rebaseline tests" button (or the r key). You can watch the progress in the log area (toggled by the link in the upper right). The tool uses scm.py to move baselines around and mark them for addition/move, but I've only tried it in git checkouts so far.
+Ojan since he's interested in making rebaselining less painful too.
Hum... I'm having trouble using the tool because I don't have a results.json. It doesn't seem to be included in the zip from the bot.
Created attachment 71799 [details] Patch
For anyone wanting to use this to help with rebaselining of pixel tests for Snow Leopard: 1. webkit-patch apply-attachment 71799 [details] 2. Claim a directory from https://spreadsheets.google.com/ccc?key=0AkV6W8Td7jGwdFpFdzJLNzZXNFZkVmhXYmlNVS1PZVE&hl=en&authkey=COyB-LwN 3. new-run-webkit-tests --tolerance=0.0 --pixel-tests directory/you/picked 4. webkit-patch local-rebaseline WebKitBuild/Debug/layout-test-results 5. Go to http://localhost:8127 6. Set the baseline target as "mac" and move current baselines to "mac-leopard" For now, rebaseline only tests that have text anti-aliasing differences or HTML color name differences. Have no more than 100 tests per patch. See comment 7 for details on how to use the tool UI.
Yay! I got it to work. Pretty.
I updated the fast/table Snow Leopard expectations with this: http://trac.webkit.org/changeset/70835. Worked beautifully!
Created attachment 72384 [details] Patch
Created attachment 72614 [details] Patch
Created attachment 72778 [details] Patch
Created attachment 72996 [details] Patch
Comment on attachment 72996 [details] Patch Oops, this ended up on the wrong bug.
The tool is now checked in (thanks to Tony doing lots of reviews). I've also written up https://trac.webkit.org/wiki/RebaselineServer with instructions on how to use it. Feel free to file bugs and cc me on them if you notice any bugs or limitations.