Bug 187039 - Add support for testdriver WPT tests
Summary: Add support for testdriver WPT tests
Status: RESOLVED DUPLICATE of bug 197011
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on: 187095
Blocks:
  Show dependency treegraph
 
Reported: 2018-06-26 06:28 PDT by Frédéric Wang (:fredw)
Modified: 2024-01-24 07:01 PST (History)
13 users (show)

See Also:


Attachments
Patch (1.56 KB, patch)
2018-10-02 05:42 PDT, darshan
no flags Details | Formatted Diff | Diff
Patch (11.77 KB, patch)
2018-10-02 10:15 PDT, darshan
no flags Details | Formatted Diff | Diff
Patch (15.60 KB, patch)
2019-02-19 10:13 PST, darshan
dkadu: review?
ews-watchlist: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews105 for mac-highsierra-wk2 (2.52 MB, application/zip)
2019-02-19 11:27 PST, EWS Watchlist
no flags Details
Archive of layout-test-results from ews124 for ios-simulator-wk2 (10.12 MB, application/zip)
2019-02-19 12:14 PST, EWS Watchlist
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Frédéric Wang (:fredw) 2018-06-26 06:28:06 PDT
Some new WPT tests upstream rely on testdriver.js to emulate user interaction (for now click and key events). Currently, it does not seem we have imported any one into WebKit repository.

For documentation, see
https://web-platform-tests.org/writing-tests/testdriver.html
https://github.com/web-platform-tests/wpt/blob/master/docs/_writing-tests/testdriver-tutorial.md
https://github.com/web-platform-tests/wpt/blob/master/resources/testdriver.js
https://github.com/web-platform-tests/wpt/blob/master/resources/testdriver-vendor.js
https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/testdriver-extra.js

I guess when running WebKit tests using WebDriver, we have to change the executor to support the new commands (see https://github.com/web-platform-tests/wpt/blob/master/docs/_writing-tests/testdriver-tutorial.md#other-browsers )

When running run-webkit-tests, I think we can just modify testdriver-vendor.js to implement click and key events. For example, on some WebKit ports this is possible via testRunner.runUIScript (see e.g. tapAt or keyDown in LayoutTests/resources/ui-helper.js).
Comment 1 youenn fablet 2018-06-27 07:54:38 PDT
I also believe writing our own testdriver-vendor.js using internals/testRunner APIs is the way to go for WPT tests run by WTR.
Comment 2 darshan 2018-10-02 05:42:02 PDT
Created attachment 351375 [details]
Patch
Comment 3 Frédéric Wang (:fredw) 2018-10-02 08:27:58 PDT
Comment on attachment 351375 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=351375&action=review

> LayoutTests/imported/w3c/ChangeLog:6
> +

Extra white spaces.

> LayoutTests/imported/w3c/ChangeLog:7
> +        Add support for testdriver WPT tess

tests

> LayoutTests/imported/w3c/ChangeLog:15
> +

I think you should unskip testdriver tests from LayoutTests/infrastructure in order to have something to test.

> LayoutTests/imported/w3c/web-platform-tests/resources/testdriver.js:3
> +

So talking with Youenn, the proper way to implement this is actually to import the existing testdriver.js from upstream WPT (so that it implements window.test_driver) and implement testdriver-vendor.js so that it overrides window.test_driver_internal (or whatever else must be modified).

For example infrastructure/testdriver/click.html loads both files:
<script src="/resources/testdriver.js"></script>
<script src="/resources/testdriver-vendor.js"></script>

BTW, there should be some doc on upstream WPT explaining that.
Comment 4 darshan 2018-10-02 10:15:40 PDT
Created attachment 351410 [details]
Patch
Comment 5 youenn fablet 2018-10-03 07:19:30 PDT
Comment on attachment 351410 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=351410&action=review

> LayoutTests/TestExpectations:-858
> -webkit.org/b/187039 imported/w3c/web-platform-tests/infrastructure/testdriver [ Skip ] # testdriver not supported yet.

If we are unskipping these tests, we should be running these tests, but they do not have -expected.txt so I am unsure what happens there.
Comment 6 Frédéric Wang (:fredw) 2018-10-03 08:27:17 PDT
Comment on attachment 351410 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=351410&action=review

> LayoutTests/ChangeLog:8
> +        * TestExpectations:

It would be nice to give details about the change in the changelog. e.g. IIUC you plan not to include any macOS/iOS implementation yet in this bug?

>> LayoutTests/TestExpectations:-858
>> -webkit.org/b/187039 imported/w3c/web-platform-tests/infrastructure/testdriver [ Skip ] # testdriver not supported yet.
> 
> If we are unskipping these tests, we should be running these tests, but they do not have -expected.txt so I am unsure what happens there.

Yes, please run the tests and add the expectation files.

> LayoutTests/imported/w3c/web-platform-tests/resources/testdriver-vendor.js:2
> +    window.test_driver_internal.click =  function(element, coords){

the spacing around { and = operators does not look correct in this file. Please check how it is done for testdriver.js
Comment 7 darshan 2019-02-19 10:13:13 PST
Created attachment 362391 [details]
Patch
Comment 8 EWS Watchlist 2019-02-19 11:27:06 PST
Comment on attachment 362391 [details]
Patch

Attachment 362391 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11205779

New failing tests:
imported/w3c/web-platform-tests/payment-request/payment-request-canmakepayment-method.https.html
imported/w3c/web-platform-tests/payment-request/payment-request-canmakepayment-method-protection.https.html
Comment 9 EWS Watchlist 2019-02-19 11:27:08 PST
Created attachment 362401 [details]
Archive of layout-test-results from ews105 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6
Comment 10 EWS Watchlist 2019-02-19 12:14:42 PST
Comment on attachment 362391 [details]
Patch

Attachment 362391 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/11205980

New failing tests:
imported/w3c/web-platform-tests/payment-request/payment-request-canmakepayment-method.https.html
imported/w3c/web-platform-tests/payment-request/payment-request-canmakepayment-method-protection.https.html
Comment 11 EWS Watchlist 2019-02-19 12:14:44 PST
Created attachment 362412 [details]
Archive of layout-test-results from ews124 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews124  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6
Comment 12 Radar WebKit Bug Importer 2024-01-16 13:45:37 PST
<rdar://problem/121074803>
Comment 13 Sam Sneddon [:gsnedders] 2024-01-24 07:01:43 PST
We started adding support back in bug 197011, so let's close this as a duplicate.

*** This bug has been marked as a duplicate of bug 197011 ***