Bug 224197 - Safari not computing image aspect ratios from width and height attributes for lazy loaded images
Summary: Safari not computing image aspect ratios from width and height attributes for...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Images (show other bugs)
Version: Safari 14
Hardware: All All
: P2 Normal
Assignee: cathiechen
URL:
Keywords: InRadar
Depends on: 224748 224911
Blocks:
  Show dependency treegraph
 
Reported: 2021-04-05 12:25 PDT by Owen Sullivan
Modified: 2021-05-28 06:05 PDT (History)
15 users (show)

See Also:


Attachments
error-image-test (277 bytes, text/html)
2021-04-15 05:37 PDT, cathiechen
no flags Details
Patch (9.18 KB, patch)
2021-04-20 03:46 PDT, cathiechen
no flags Details | Formatted Diff | Diff
Patch (9.44 KB, patch)
2021-04-21 03:52 PDT, cathiechen
no flags Details | Formatted Diff | Diff
Patch (9.96 KB, patch)
2021-04-23 09:32 PDT, cathiechen
no flags Details | Formatted Diff | Diff
error-image-test (301 bytes, text/html)
2021-05-07 01:29 PDT, cathiechen
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Owen Sullivan 2021-04-05 12:25:21 PDT
Safari is not reserving space in the painted page for images with set width and height attributes, with or without a src attribute. This is despite the fact that AspectRatioOfImgFromWidthAndHeightEnabled is supported with Safari 14 and newer, and enabled by default. This causes reflow when images are loaded via lazy-loading plugins/features.

Codepen with a demonstration of the issue (click on "Nav 4" to see the issue in practice, the anchor link should take you to the "Nav 4" section but you end up closer to the "Nav 3" section due to image reflow): https://codepen.io/sulliops/full/QWdwRxx

Screen recording demonstrating issue: https://gyazo.com/da76046282965f7a32ed3d85cc7423df

Bug report created at the request of Simon Fraser (smfr) at https://bugs.webkit.org/show_bug.cgi?id=201641#c58.
Comment 1 Smoley 2021-04-07 12:22:00 PDT
Thanks, this reproduces on Safari 13.1.2 as well as TOT 14.2 using the provided codepen.
Comment 2 Radar WebKit Bug Importer 2021-04-07 12:23:28 PDT
<rdar://problem/76359912>
Comment 3 cathiechen 2021-04-15 05:37:09 PDT
Created attachment 426098 [details]
error-image-test

Thanks! I can reproduce it.

This seems related to error images.
Because the src attribute is empty, so <img> is treated as an error image.
The current implementation does not map width height attribute to aspect ratio for error image.
However, as this feature developed, looks like it should apply to the empty src <img> without alt. And the wpt test img-aspect-ratio.html has updated.
See, https://github.com/whatwg/html/pull/6032

So I think it makes sense to update the behavior of error images.
Comment 4 Owen Sullivan 2021-04-15 05:43:42 PDT
(In reply to cathiechen from comment #3)
> Created attachment 426098 [details]
> error-image-test
> 
> Thanks! I can reproduce it.
> 
> This seems related to error images.
> Because the src attribute is empty, so <img> is treated as an error image.
> The current implementation does not map width height attribute to aspect
> ratio for error image.
> However, as this feature developed, looks like it should apply to the empty
> src <img> without alt. And the wpt test img-aspect-ratio.html has updated.
> See, https://github.com/whatwg/html/pull/6032
> 
> So I think it makes sense to update the behavior of error images.

Interestingly, I still experienced the issue after including an alt for each image. Could be a cache issue on my end, as I had previously been testing on a production website before I whipped up that CodePen, but figured I ought to add that information.
Comment 5 cathiechen 2021-04-15 06:21:55 PDT
(In reply to Owen Sullivan from comment #4)
> Interestingly, I still experienced the issue after including an alt for each
> image. Could be a cache issue on my end, as I had previously been testing on
> a production website before I whipped up that CodePen, but figured I ought
> to add that information.

Yeah, I analyzed the page a bit.
After this page is loaded, <img>s don't have src attribute(use data-src instead). For instance,
`<img class="lazyload img-fluid" data-src="https://placehold.jp/1000x2000.png" width="1000" height="2000">`

So the browser think this <img> is an error image, and doesn't apply the feature (mapping width height attributes to aspect-ratio)

After clicking "Nav 4", <img> will get the src attribute.

`<img class="img-fluid ls-is-cached lazyloaded" data-src="https://placehold.jp/1000x2000.png" width="1000" height="2000" src="https://placehold.jp/1000x2000.png">`

So <img> is not an error image now.

And I don't think <img> should have alt in this case, because an error image with alt won't apply the feature.
Comment 6 Owen Sullivan 2021-04-16 07:33:07 PDT
Probably a dumb question, but simply because I'm new to WebKit bugs: 

Smoley mentioned that the issue reproduces on Safari 13.1.2 in addition to Safari 14, although I wasn't aware the feature existed before Safari 14. 

In any case, can the fix be retroactively merged into existing versions of Safari through a library update? Or does it require a full application update? I'm not at all aware of what processes you all have in place to apply fixes, but if it can somehow be done retroactively it would really help with browser cross-compatibility and web design for users that don't frequently update their browsers.
Comment 7 cathiechen 2021-04-20 03:46:23 PDT
Created attachment 426536 [details]
Patch
Comment 8 cathiechen 2021-04-20 06:47:07 PDT
Comment on attachment 426536 [details]
Patch

Hi,
I think this patch is ready for review:)
Comment 9 Rob Buis 2021-04-20 07:11:56 PDT
Comment on attachment 426536 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=426536&action=review

> Source/WebCore/rendering/RenderReplaced.cpp:477
> +bool RenderReplaced::getAspectRatioFromWidthHeight(double& intrinsicRatio) const

This should probably return Optional<double>.
Comment 10 Darin Adler 2021-04-20 08:04:02 PDT
Comment on attachment 426536 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=426536&action=review

This is OK, but I think it could be made better. Suggestions below.

> Source/WebCore/rendering/RenderImage.cpp:854
> +    if (!element() || !is<HTMLImageElement>(*element()))
> +        return false;
> +
> +    if (isShowingAltText())
> +        return false;
> +
> +    return true;

The is<> function includes built-in null checking, so this can be written like this:

    return is<HTMLIMageElement>(element()) && isShowingAltText();

I think that’s clearer than the multiple return statements.

>> Source/WebCore/rendering/RenderReplaced.cpp:477
>> +bool RenderReplaced::getAspectRatioFromWidthHeight(double& intrinsicRatio) const
> 
> This should probably return Optional<double>.

Yes!

And to follow WebKIt coding style we’d then remove the word "get" from the name. Not even sure that "FromWidthHeight" is a good way to name this. I think it should be named intrinsicAspectRatio.

> Source/WebCore/rendering/RenderReplaced.h:61
> +    virtual bool canMapWidthHeightAsAspectRatio() const { return false; }

I don’t understand the name of this function. I don’t know what "map width height as aspect ratio" means. Maybe this should be named hasIntrinsicAspectRatio?

I am not 100% sure what the concept is. For example, if this returns false does that mean we have no intrinsic aspect ratio at all? Or does it just mean it should be ignored in RenderImage::computeIntrinsicRatioInformation?

> LayoutTests/imported/w3c/ChangeLog:9
> +        The test cases for error images and images without src in img-aspect-ratio.html are passed. This patch
> +        doesn't change the behavior of the original aspect ratio case, so it's failed like before.

Before doing more work here, I think we should take the time to restructure this test so it does multiple PASS/FAIL and not just a single FAIL. I expect that won’t be too difficult.
Comment 11 cathiechen 2021-04-21 03:51:06 PDT
Comment on attachment 426536 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=426536&action=review

Hi Darin and Rob,
Thanks for the review!

>> Source/WebCore/rendering/RenderImage.cpp:854
>> +    return true;
> 
> The is<> function includes built-in null checking, so this can be written like this:
> 
>     return is<HTMLIMageElement>(element()) && isShowingAltText();
> 
> I think that’s clearer than the multiple return statements.

Done, thanks! Changed to:

return is<HTMLIMageElement>(element()) && !isShowingAltText();

>>> Source/WebCore/rendering/RenderReplaced.cpp:477
>>> +bool RenderReplaced::getAspectRatioFromWidthHeight(double& intrinsicRatio) const
>> 
>> This should probably return Optional<double>.
> 
> Yes!
> 
> And to follow WebKIt coding style we’d then remove the word "get" from the name. Not even sure that "FromWidthHeight" is a good way to name this. I think it should be named intrinsicAspectRatio.

Done, thanks!

intrinsicAspectRatioFromWidthHeight? I think maybe it's good to have "FromWidthHeight" to indicate that the aspect ratio is calculated by attributes width and height?

>> Source/WebCore/rendering/RenderReplaced.h:61
>> +    virtual bool canMapWidthHeightAsAspectRatio() const { return false; }
> 
> I don’t understand the name of this function. I don’t know what "map width height as aspect ratio" means. Maybe this should be named hasIntrinsicAspectRatio?
> 
> I am not 100% sure what the concept is. For example, if this returns false does that mean we have no intrinsic aspect ratio at all? Or does it just mean it should be ignored in RenderImage::computeIntrinsicRatioInformation?

Sorry for the terribly naming.
It means we can compute the aspect ratio from attribute width / attribute height before image loaded. It is defined here https://html.spec.whatwg.org/#map-to-the-aspect-ratio-property
Currently, only img is allowed. I think next step we can expend it to more elements, like canvas, video, input with image type and picture etc.
This function determines which element is allowed to get aspect ratio from width and height.
So canMapWidthHeightToAspectRatio?

I added some explanations to this function.

>> LayoutTests/imported/w3c/ChangeLog:9
>> +        doesn't change the behavior of the original aspect ratio case, so it's failed like before.
> 
> Before doing more work here, I think we should take the time to restructure this test so it does multiple PASS/FAIL and not just a single FAIL. I expect that won’t be too difficult.

Sure, I think I can work on this:)
Comment 12 cathiechen 2021-04-21 03:52:05 PDT
Created attachment 426667 [details]
Patch
Comment 13 cathiechen 2021-04-23 09:32:43 PDT
Created attachment 426917 [details]
Patch
Comment 14 EWS 2021-04-23 15:00:19 PDT
Committed r276521 (236975@main): <https://commits.webkit.org/236975@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 426917 [details].
Comment 15 Owen Sullivan 2021-04-28 07:56:05 PDT
Downloaded nightly build 276521 from https://s3-us-west-2.amazonaws.com/minified-archives.webkit.org/mac-bigsur-x86_64%20arm64-release/276521.zip (as well as the most recent nightly) to try and test this fix, and I'm still experiencing the original problem. Should I be expect this and wait for a later commit, or is this a problem?
Comment 16 cathiechen 2021-05-07 01:28:34 PDT
(In reply to Owen Sullivan from comment #15)
> Downloaded nightly build 276521 from
> https://s3-us-west-2.amazonaws.com/minified-archives.webkit.org/mac-bigsur-
> x86_64%20arm64-release/276521.zip (as well as the most recent nightly) to
> try and test this fix, and I'm still experiencing the original problem.
> Should I be expect this and wait for a later commit, or is this a problem?

Hi Owen,
Sorry, for some reason I can't install the nightly build in my Mac.
I tried it in my local build(commit: f630ca6238 May 3rd 2021), it works as expected in https://codepen.io/sulliops/full/QWdwRxx
As for error-image-test in Attachments, there's a nit(not `width="1000px" height="2000px"`, should be `width="1000" height="2000"` instead). After fixing this, it works as expected too.
Comment 17 cathiechen 2021-05-07 01:29:16 PDT
Created attachment 427979 [details]
error-image-test
Comment 18 Kyle Bavender 2021-05-28 06:05:52 PDT
(In reply to Owen Sullivan from comment #15)
> Downloaded nightly build 276521 ... to try and test this fix, and I'm still experiencing the original problem

I came here by way of Safari Technology Preview v125 release notes. I can verify that Safari TP 125 no longer has the error (while I can reproduce the error in Safari 14.1.1). Great CodePen test case! We'll have to look forward to a future Safari update to see this hit Safari proper.

Thank you @cathiechen for addressing this! 🙇‍♂️