WebDriver is Coming to Safari in iOS 13
As anyone who has developed a mobile-friendly site can tell you, mobile browsers and desktop browsers are different. As device capabilities continue to evolve and users move more of their browsing to mobile browsers, web developers are facing an increasing need to write automated tests for the mobile web in the same way that desktop-oriented site content is tested.
Starting in iOS 13, Safari now includes native support for the W3C WebDriver standard. This feature builds on existing support for WebDriver in desktop Safari, first introduced in Safari 10 and macOS Sierra. This blog post explains how to get started with WebDriver for iOS, how to port your existing desktop-oriented tests, and a few details regarding safeguards and changes in behavior.
Getting Started
Control via WebDriver is exposed to developers via the /usr/bin/safaridriver executable, which hosts a driver that handles REST API requests sent by WebDriver test clients. In order to run WebDriver tests on an iOS device, it must be plugged into a macOS host that has a new enough version of safaridriver. Support for hosting iOS-based WebDriver sessions is available in safaridriver included with Safari 13 and later. Older versions of safaridriver do not support iOS WebDriver sessions.
If you’ve never used safaridriver on macOS before, you’ll first need to run safaridriver --enable and authenticate as an administrator. Then, you must enable Remote Automation on every device that you intend to use for WebDriver. To do this, toggle the setting in Settings → Safari → Advanced → Remote Automation.
Lastly, to run tests on the device, you’ll need to plug it into the macOS machine, trust the host, and ensure that it is unlocked when you try to start the WebDriver session.
Since safaridriver is capable of starting WebDriver sessions using Safari on the macOS host or Safari on an attached iOS device, it needs a little extra hint to know that you want to use a device or a desktop machine. To facilitate this, safaridriver supports many new capabilities that may be provided with a new session request. The most important new capability to use is ‘platformName’: ‘ios’; this tells the driver you want an iOS host instead of a macOS host. You can also specify the host(s) you wish to use by specifying device type, device UDID, device name, OS version, OS build, and browser version via standardized and extension capabilities. Lastly, safaridriver supports running WebDriver tests on iOS Simulator devices whose runtime is iOS 13 or later. To use a simulator for testing, add the capability ‘safari:useSimulator’: true. For a full list of supported capabilities and their semantics, please consult the safaridriver(1) man page by executing man safaridriver.
Safeguards
While it’s awesome to be able to write automated tests that run in Safari, a REST API for browser automation introduces a potential vector for malicious software. To support a valuable automation API without sacrificing a user’s privacy or security, Safari’s driver implementation introduces extra safeguards that further restrict browsing in WebDriver sessions. These safeguards also double as extra insurance against “flaky tests” by providing enhanced test isolation. Some of these safeguards are described below.
Just like on macOS, Safari on iOS isolates WebDriver tests by using a separate set of windows, tabs, preferences, and persistent storage. When a WebDriver session is active, existing tabs are hidden and a distinctively-colored WebDriver window is shown instead. Automation windows are easy to recognize by their orange Smart Search field. Similar to browsing in a Private Browsing window, WebDriver tests that run in an Automation window always start from a clean slate and cannot access Safari’s normal browsing history, AutoFill data, or other sensitive information. Aside from the obvious privacy benefits, this restriction also helps to ensure that tests are not affected by a previous test session’s persistent state such as local storage or cookies. When the WebDriver session terminates, the orange window goes away, preexisting tabs are restored, and any state accumulated during the WebDriver session is destroyed.
As a WebDriver test is executing in an Automation window, any attempts to interact with the window or web content could derail the test by unexpectedly changing the state of the page. To prevent this from happening, Safari installs a “glass pane” over the Automation window while the test is running. This blocks any stray interactions (mouse, keyboard, resizing, and so on) from affecting the Automation window. If a running test gets stuck or fails, it’s possible to interrupt the running test by tapping on the screen and choosing to end the session. When an automation session is interrupted, the test’s connection to the browser is permanently severed and cannot be resumed. Notifications and system gestures (i.e., slide in from top/bottom) are also suppressed while WebDriver tests are running in order to avoid flakiness.
A non-goal of this release is to support general-purpose device automation via the WebDriver API. The W3C WebDriver API focuses on automating interactions with web content, and that is also the focus of the WebDriver support added in iOS 13. Other means exist to change system-level configuration with the XCTest Framework, and are complimentary to WebDriver. There are other ways in which web content interacts with the rest of iOS, and most of these are suppressed in order to provide a stable testing environment. Safari for iOS does not allow WebDriver-initiated navigations to be handled outside of Safari. In other words, clicking a tel:// link will not offer to place a phone call, and clicking an app store link will not redirect the user to the App Store. Similarly, non-Safari content displayed by the system–such as update dialogs, app notifications, incoming calls, etc.–are suppressed while a WebDriver session is active. Lastly, the software keyboard is suppressed when focusing text fields in WebDriver sessions. To simulate text input with WebDriver, use the existing Perform Actions endpoint or Element Send Keys endpoint.
Porting Tests
A major goal of iOS 13 is to make it easier to develop and consume desktop-like web experiences on mobile devices. Towards that end, a major goal for WebDriver on iOS is to make it easy to port existing desktop-oriented WebDriver tests to work with Desktop-class browsing in Safari on iPadOS. As much as we’d like to treat macOS and iOS devices the same in our tests, there are important differences that affect how you may want to use WebDriver.
The first notable difference is handling of user input, namely touch and clicks. In this initial version of WebDriver on iOS, simulated input sources include keyboard, mouse, and single-finger touches. The Perform Actions command and Element Click command synthesize touches using mouse inputs. If your existing test uses mouse clicks, double, clicks, and drags, safaridriver converts these interactions into equivalent single finger touches; no code needs to be changed. Multiple touch input sources (i.e., fingers) are not supported in this release. We experimented with offering this via the existing Perform Actions endpoint, but found that it was difficult to author WebDriver commands that performed well-known gestures such as flick, pinch, pan, and so on. This is partly due to the low-level nature of the Actions API, and partly due to the timing-sensitive nature of gesture recognizers used by Safari and WebKit. Apple is working within the W3C WebDriver working group to define a more expressive, intuitive, and stable API for performing gestures in WebDriver tests.
A less surprising difference between macOS and iOS WebDriver is that endpoints to manipulate the OS window are generally not supported by Safari on iOS. Specifically, the Set Window Rect, Minimize Window, and Maximize Window endpoints will return “unsupported operation”. For more detailed information concerning which endpoints are supported, see the W3C WebDriver Commands reference.
Summary
With the introduction of native WebDriver support in Safari on iOS 13, it’s now possible to run the same automated tests of desktop-oriented web content on desktop and mobile devices equally. Safari’s support comes with new, exclusive safeguards to simultaneously protect user security and privacy and also help you write more stable and consistent tests. You can try out Safari’s WebDriver support today by installing a beta of macOS Catalina and iOS 13.
If you encounter unexpected behavior in Safari’s WebDriver support, or have other suggestions for improvement, please file a bug at https://bugreport.apple.com/. For quick questions or feedback, email web-evangelist@apple.com or tweet @webkit on Twitter. Happy testing!