UI Events - Mouse Events and iframe targetting

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

UI Events - Mouse Events and iframe targetting

Dave Tapuska-2
It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each rendering engine.

I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the web author.

Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different frames when the mouse moves outside the bounds of the iframe.

1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see http://crbug.com/269917; we need to fix this).
2) Firefox targets the mouse down frame unless you move into a sibling iframe
3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.

We need to get some clarity written into the specification. As the spec indicates the target is:

Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.

All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a mouse button is depressed; this can leave the iframe in an inconsistent state possibly.

So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to the web author point of view.

I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging something around you might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug references google maps embedded inside a page as a use case that occurs in the field today. 

With respect to hover processing:
1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down. (Seems incorrect).
2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
3) IE 11 & Edge generate hover on whatever is under the mouse.

The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.

The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is interesting. Perhaps I'm flawed in my reproduction step; but some explanation behind this would be appreciated.

The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like hover and prevent default. 

dave.

Reply | Threaded
Open this post in threaded view
|

Re: UI Events - Mouse Events and iframe targetting

Dave Tapuska-2
It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find it you'd need to dig into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html


On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email]> wrote:
It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each rendering engine.

I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the web author.

Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different frames when the mouse moves outside the bounds of the iframe.

1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see http://crbug.com/269917; we need to fix this).
2) Firefox targets the mouse down frame unless you move into a sibling iframe
3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.

We need to get some clarity written into the specification. As the spec indicates the target is:

Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.

All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a mouse button is depressed; this can leave the iframe in an inconsistent state possibly.

So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to the web author point of view.

I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging something around you might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug references google maps embedded inside a page as a use case that occurs in the field today. 

With respect to hover processing:
1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
3) IE 11 & Edge generate hover on whatever is under the mouse.

The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.

The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is interesting. Perhaps I'm flawed in my reproduction step; but some explanation behind this would be appreciated.

The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like hover and prevent default. 

dave.


Reply | Threaded
Open this post in threaded view
|

Re: UI Events - Mouse Events and iframe targetting

Rick Byers-2
Thanks Dave,
I've been aware of differences between browsers here for awhile, but only recently been convinced that it's important that we try to fix this. /cc Jacob and Olli in case they have specific experience/advice for EdgeHTML and Gecko respectively.

In particular, I had a great discussion with the Google Maps team about why this matters to them.  The Google Maps embedded API lets you host a map in an iframe on your page.  Here when you click on the map and drag to pan it, the user really expects that it continues to pan even if the cursor leaves the frame.  They've got some ugly UserAgent-specific hacks to make this work on all browsers and these hacks have gotten in the way of us fixing other interop problems in Chrome.

So, we need some standard interoperable way to enable such a scenario.  But this doesn't mean we have to do it by default if that's hard to agree on or likely to cause developer confusion.  Is this scenario possible today in IE / Gecko using the non-standard setCapture API?  Presumably setPointerCapture can definitely be used for this purpose (though the spec is not clear).  But I'm not sure we want to block a solution to these problems on Pointer Events (eg. there should be some path for Safari as well).

Has there been any discussion about standardizing setCapture?  If IE and Gecko already have an interoperable implementation here, perhaps we should consider that a defacto standard and add it to blink too, then change our mouse event behavior to match IE's?

But still, I think the UI Events spec should be updated to be clear on what the default behavior for mouse events should be in this regard.

Thoughts?
   Rick
 

On Mon, Jun 22, 2015 at 1:21 PM, Dave Tapuska <[hidden email]> wrote:
It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find it you'd need to dig into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html


On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email]> wrote:
It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each rendering engine.

I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the web author.

Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different frames when the mouse moves outside the bounds of the iframe.

1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see http://crbug.com/269917; we need to fix this).
2) Firefox targets the mouse down frame unless you move into a sibling iframe
3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.

We need to get some clarity written into the specification. As the spec indicates the target is:

Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.

All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a mouse button is depressed; this can leave the iframe in an inconsistent state possibly.

So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to the web author point of view.

I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging something around you might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug references google maps embedded inside a page as a use case that occurs in the field today. 

With respect to hover processing:
1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
3) IE 11 & Edge generate hover on whatever is under the mouse.

The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.

The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is interesting. Perhaps I'm flawed in my reproduction step; but some explanation behind this would be appreciated.

The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like hover and prevent default. 

dave.



Reply | Threaded
Open this post in threaded view
|

Re: UI Events - Mouse Events and iframe targetting

Olli Pettay-2
On 06/22/2015 11:00 AM, Rick Byers wrote:
> Thanks Dave,
> I've been aware of differences between browsers here for awhile, but only recently been convinced that it's important that we try to fix this. /cc
> Jacob and Olli in case they have specific experience/advice for EdgeHTML and Gecko respectively.

So the idea behind "2) Firefox targets the mouse down frame unless you move into a sibling iframe"
is that iframe and top level browsing context would get similar behavior when dealing with mouse event based dragging.
If you have mousedown, you flag the mousedown browsing context as the active and any mouse events in ancestors (or outside the browser window) are
forwarded to the active browsing context.
(This was perhaps slightly based on the single process Firefox architecture where 'chrome browsing context' hosts the top level web browsing contexts).
I do consider the 'move into a sibling iframe' a bug, since the idea was to deal nested iframes only in that way, not also siblings.
The relevant change was done in https://bugzilla.mozilla.org/show_bug.cgi?id=603550 (or more like, that fixed Gecko to behave as it had behaved for ages).

As I said in that bug (5 years ago ;)),
I think I'm leaning over to the behavior where mousedown frame would always get the events, in other words implicit capturing.


-Olli




>
> In particular, I had a great discussion with the Google Maps team about why this matters to them.  The Google Maps embedded API
> <https://developers.google.com/maps/documentation/embed/> lets you host a map in an iframe on your page.  Here when you click on the map and drag to
> pan it, the user really expects that it continues to pan even if the cursor leaves the frame.  They've got some ugly UserAgent-specific hacks to make
> this work on all browsers and these hacks have gotten in the way of us fixing other interop problems in Chrome.
>
> So, we need some standard interoperable way to enable such a scenario.  But this doesn't mean we have to do it by default if that's hard to agree on
> or likely to cause developer confusion.  Is this scenario possible today in IE / Gecko using the non-standard setCapture API
> <https://developer.mozilla.org/en-US/docs/Web/API/Element/setCapture>?  Presumably setPointerCapture
> <https://w3c.github.io/pointerevents/#pointer-capture> can definitely be used for this purpose (though the spec is not clear
> <https://github.com/w3c/pointerevents/issues/16>).  But I'm not sure we want to block a solution to these problems on Pointer Events (eg. there should
> be some path for Safari as well).
>
> Has there been any discussion about standardizing setCapture?  If IE and Gecko already have an interoperable implementation here, perhaps we should
> consider that a defacto standard and add it to blink too, then change our mouse event behavior to match IE's?
>
> But still, I think the UI Events spec should be updated to be clear on what the default behavior for mouse events should be in this regard.
>
> Thoughts?
>     Rick
>
> On Mon, Jun 22, 2015 at 1:21 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:
>
>     It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find it you'd need to dig
>     into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html
>     <http://dtapuska.github.io/iframe-mouse-target/iframe_outer.html>
>
>
>     On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:
>
>         It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each rendering engine.
>
>         I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the web author.
>
>         Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different frames when the mouse
>         moves outside the bounds of the iframe.
>
>         1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see
>         http://crbug.com/269917; we need to fix this).
>         2) Firefox targets the mouse down frame unless you move into a sibling iframe
>         3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.
>
>         We need to get some clarity written into the specification. As the spec indicates the target is:
>
>           * |Event.target| <http://www.w3.org/TR/DOM-Level-3-Events/#widl-Event-target>: topmost event target
>             <http://www.w3.org/TR/DOM-Level-3-Events/#glossary-topmost-event-target>
>
>
>         Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.
>
>         All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a mouse button is
>         depressed; this can leave the iframe in an inconsistent state possibly.
>
>         So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to the web author point
>         of view.
>
>         I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging something around you
>         might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug references google maps
>         embedded inside a page as a use case that occurs in the field today.
>
>         With respect to hover processing:
>         1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
>         2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
>         3) IE 11 & Edge generate hover on whatever is under the mouse.
>
>         The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.
>
>         The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is interesting. Perhaps I'm
>         flawed in my reproduction step; but some explanation behind this would be appreciated.
>
>         The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like hover and prevent default.
>
>         dave.
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: UI Events - Mouse Events and iframe targetting

Rick Byers-2
On Tue, Jun 23, 2015 at 12:01 PM, Olli Pettay <[hidden email]> wrote:
On 06/22/2015 11:00 AM, Rick Byers wrote:
Thanks Dave,
I've been aware of differences between browsers here for awhile, but only recently been convinced that it's important that we try to fix this. /cc
Jacob and Olli in case they have specific experience/advice for EdgeHTML and Gecko respectively.

So the idea behind "2) Firefox targets the mouse down frame unless you move into a sibling iframe"
is that iframe and top level browsing context would get similar behavior when dealing with mouse event based dragging.
If you have mousedown, you flag the mousedown browsing context as the active and any mouse events in ancestors (or outside the browser window) are
forwarded to the active browsing context.
(This was perhaps slightly based on the single process Firefox architecture where 'chrome browsing context' hosts the top level web browsing contexts).
I do consider the 'move into a sibling iframe' a bug, since the idea was to deal nested iframes only in that way, not also siblings.
The relevant change was done in https://bugzilla.mozilla.org/show_bug.cgi?id=603550 (or more like, that fixed Gecko to behave as it had behaved for ages).

As I said in that bug (5 years ago ;)),
I think I'm leaning over to the behavior where mousedown frame would always get the events, in other words implicit capturing.

Thanks Olli!  If it's sufficiently web compatible then I'd prefer implicit capturing too.  That would often just be the right thing and wouldn't require any additional APIs to solve all the important scenarios I know of.  If other engines were open to trying to move to this model, we could give it a shot in Chrome.

-Olli





In particular, I had a great discussion with the Google Maps team about why this matters to them.  The Google Maps embedded API
<https://developers.google.com/maps/documentation/embed/> lets you host a map in an iframe on your page.  Here when you click on the map and drag to
pan it, the user really expects that it continues to pan even if the cursor leaves the frame.  They've got some ugly UserAgent-specific hacks to make
this work on all browsers and these hacks have gotten in the way of us fixing other interop problems in Chrome.

So, we need some standard interoperable way to enable such a scenario.  But this doesn't mean we have to do it by default if that's hard to agree on
or likely to cause developer confusion.  Is this scenario possible today in IE / Gecko using the non-standard setCapture API
<https://developer.mozilla.org/en-US/docs/Web/API/Element/setCapture>?  Presumably setPointerCapture
<https://w3c.github.io/pointerevents/#pointer-capture> can definitely be used for this purpose (though the spec is not clear
<https://github.com/w3c/pointerevents/issues/16>).  But I'm not sure we want to block a solution to these problems on Pointer Events (eg. there should
be some path for Safari as well).

Has there been any discussion about standardizing setCapture?  If IE and Gecko already have an interoperable implementation here, perhaps we should
consider that a defacto standard and add it to blink too, then change our mouse event behavior to match IE's?

But still, I think the UI Events spec should be updated to be clear on what the default behavior for mouse events should be in this regard.

Thoughts?
    Rick

On Mon, Jun 22, 2015 at 1:21 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

    It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find it you'd need to dig
    into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html
    <http://dtapuska.github.io/iframe-mouse-target/iframe_outer.html>


    On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

        It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each rendering engine.

        I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the web author.

        Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different frames when the mouse
        moves outside the bounds of the iframe.

        1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see
        http://crbug.com/269917; we need to fix this).
        2) Firefox targets the mouse down frame unless you move into a sibling iframe
        3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.

        We need to get some clarity written into the specification. As the spec indicates the target is:

          * |Event.target| <http://www.w3.org/TR/DOM-Level-3-Events/#widl-Event-target>: topmost event target
            <http://www.w3.org/TR/DOM-Level-3-Events/#glossary-topmost-event-target>


        Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.

        All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a mouse button is
        depressed; this can leave the iframe in an inconsistent state possibly.

        So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to the web author point
        of view.

        I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging something around you
        might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug references google maps
        embedded inside a page as a use case that occurs in the field today.

        With respect to hover processing:
        1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
        2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
        3) IE 11 & Edge generate hover on whatever is under the mouse.

        The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.

        The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is interesting. Perhaps I'm
        flawed in my reproduction step; but some explanation behind this would be appreciated.

        The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like hover and prevent default.

        dave.





Reply | Threaded
Open this post in threaded view
|

RE: UI Events - Mouse Events and iframe targetting

Ted Dinklocker

On the Microsoft side we discussed this and here is a summary of those discussions:

 

Everyone on the EdgeHTML side prefers the behavior that we have today. The example that came up a couple of times was drag and drop – if events are implicitly captured to the iframe, then there is no easy way to hand off the interaction from one frame to another.

 

We generally thought that the current approach in EdgeHTML is straightforward – events/hover applies to whatever is under the mouse, regardless of movement over iframes or button presses.

 

We investigated whether we fire enter/leave when leaving the iframe, and confirmed that we do fire the events on the element, not the document.

 

We agree that developers likely want a setPointerCapture approach, which gives the web developer flexibility and more control.

 

Thoughts?

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Rick Byers
Sent: Tuesday, June 23, 2015 11:19 AM
To: Olli Pettay <[hidden email]>
Cc: Dave Tapuska <[hidden email]>; [hidden email]; Jacob Rossi <[hidden email]>
Subject: Re: UI Events - Mouse Events and iframe targetting

 

On Tue, Jun 23, 2015 at 12:01 PM, Olli Pettay <[hidden email]> wrote:

On 06/22/2015 11:00 AM, Rick Byers wrote:

Thanks Dave,
I've been aware of differences between browsers here for awhile, but only recently been convinced that it's important that we try to fix this. /cc
Jacob and Olli in case they have specific experience/advice for EdgeHTML and Gecko respectively.


So the idea behind "2) Firefox targets the mouse down frame unless you move into a sibling iframe"
is that iframe and top level browsing context would get similar behavior when dealing with mouse event based dragging.
If you have mousedown, you flag the mousedown browsing context as the active and any mouse events in ancestors (or outside the browser window) are
forwarded to the active browsing context.
(This was perhaps slightly based on the single process Firefox architecture where 'chrome browsing context' hosts the top level web browsing contexts).
I do consider the 'move into a sibling iframe' a bug, since the idea was to deal nested iframes only in that way, not also siblings.
The relevant change was done in https://bugzilla.mozilla.org/show_bug.cgi?id=603550 (or more like, that fixed Gecko to behave as it had behaved for ages).

As I said in that bug (5 years ago ;)),
I think I'm leaning over to the behavior where mousedown frame would always get the events, in other words implicit capturing.

 

Thanks Olli!  If it's sufficiently web compatible then I'd prefer implicit capturing too.  That would often just be the right thing and wouldn't require any additional APIs to solve all the important scenarios I know of.  If other engines were open to trying to move to this model, we could give it a shot in Chrome.

 

-Olli




In particular, I had a great discussion with the Google Maps team about why this matters to them.  The Google Maps embedded API
<https://developers.google.com/maps/documentation/embed/> lets you host a map in an iframe on your page.  Here when you click on the map and drag to
pan it, the user really expects that it continues to pan even if the cursor leaves the frame.  They've got some ugly UserAgent-specific hacks to make
this work on all browsers and these hacks have gotten in the way of us fixing other interop problems in Chrome.

So, we need some standard interoperable way to enable such a scenario.  But this doesn't mean we have to do it by default if that's hard to agree on
or likely to cause developer confusion.  Is this scenario possible today in IE / Gecko using the non-standard setCapture API
<https://developer.mozilla.org/en-US/docs/Web/API/Element/setCapture>?  Presumably setPointerCapture
<https://w3c.github.io/pointerevents/#pointer-capture> can definitely be used for this purpose (though the spec is not clear
<https://github.com/w3c/pointerevents/issues/16>).  But I'm not sure we want to block a solution to these problems on Pointer Events (eg. there should
be some path for Safari as well).

Has there been any discussion about standardizing setCapture?  If IE and Gecko already have an interoperable implementation here, perhaps we should
consider that a defacto standard and add it to blink too, then change our mouse event behavior to match IE's?

But still, I think the UI Events spec should be updated to be clear on what the default behavior for mouse events should be in this regard.

Thoughts?
    Rick

On Mon, Jun 22, 2015 at 1:21 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

    It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find it you'd need to dig
    into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html
    <http://dtapuska.github.io/iframe-mouse-target/iframe_outer.html>


    On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

        It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each rendering engine.

        I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the web author.

        Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different frames when the mouse
        moves outside the bounds of the iframe.

        1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see
        http://crbug.com/269917; we need to fix this).
        2) Firefox targets the mouse down frame unless you move into a sibling iframe
        3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.

        We need to get some clarity written into the specification. As the spec indicates the target is:

          * |Event.target| <http://www.w3.org/TR/DOM-Level-3-Events/#widl-Event-target>: topmost event target
            <http://www.w3.org/TR/DOM-Level-3-Events/#glossary-topmost-event-target>


        Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.

        All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a mouse button is
        depressed; this can leave the iframe in an inconsistent state possibly.

        So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to the web author point
        of view.

        I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging something around you
        might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug references google maps
        embedded inside a page as a use case that occurs in the field today.

        With respect to hover processing:
        1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
        2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
        3) IE 11 & Edge generate hover on whatever is under the mouse.

        The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.

        The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is interesting. Perhaps I'm
        flawed in my reproduction step; but some explanation behind this would be appreciated.

        The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like hover and prevent default.

        dave.


 

 

Reply | Threaded
Open this post in threaded view
|

Re: UI Events - Mouse Events and iframe targetting

Rick Byers-2
In reply to this post by Dave Tapuska-2
This seems like a reasonable model - certainly simpler to describe / spec than the other alternatives.

But before we can attempt to change Chrome's behavior here (and really find out how breaking such a change would be), we'd need some solution for apps relying on our current capturing behavior (such as Google maps).  Element.setCapture works for this in IE, right?  Perhaps we should just attempt to ship that (a "defacto standard" even without spec coverage) in blink.  Or perhaps we should just provide a polyfill for it in terms of pointer events and setPointerCapture (though that would mean we'd have to put solving this problem on hold until we've shipped pointer events).

Do you have any usage metrics for Element.setCapture in Trident?  Would you like to be able to deprecate / remove it some day, or do you expect you'd want to keep it forever anyway?

Rick

On Tue, Jun 30, 2015 at 4:21 PM, Ted Dinklocker <[hidden email]> wrote:

On the Microsoft side we discussed this and here is a summary of those discussions:

 

Everyone on the EdgeHTML side prefers the behavior that we have today. The example that came up a couple of times was drag and drop – if events are implicitly captured to the iframe, then there is no easy way to hand off the interaction from one frame to another.

 

We generally thought that the current approach in EdgeHTML is straightforward – events/hover applies to whatever is under the mouse, regardless of movement over iframes or button presses.

 

We investigated whether we fire enter/leave when leaving the iframe, and confirmed that we do fire the events on the element, not the document.

 

We agree that developers likely want a setPointerCapture approach, which gives the web developer flexibility and more control.

 

Thoughts?

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Rick Byers
Sent: Tuesday, June 23, 2015 11:19 AM
To: Olli Pettay <[hidden email]>
Cc: Dave Tapuska <[hidden email]>; [hidden email]; Jacob Rossi <[hidden email]>
Subject: Re: UI Events - Mouse Events and iframe targetting

 

On Tue, Jun 23, 2015 at 12:01 PM, Olli Pettay <[hidden email]> wrote:

On 06/22/2015 11:00 AM, Rick Byers wrote:

Thanks Dave,
I've been aware of differences between browsers here for awhile, but only recently been convinced that it's important that we try to fix this. /cc
Jacob and Olli in case they have specific experience/advice for EdgeHTML and Gecko respectively.


So the idea behind "2) Firefox targets the mouse down frame unless you move into a sibling iframe"
is that iframe and top level browsing context would get similar behavior when dealing with mouse event based dragging.
If you have mousedown, you flag the mousedown browsing context as the active and any mouse events in ancestors (or outside the browser window) are
forwarded to the active browsing context.
(This was perhaps slightly based on the single process Firefox architecture where 'chrome browsing context' hosts the top level web browsing contexts).
I do consider the 'move into a sibling iframe' a bug, since the idea was to deal nested iframes only in that way, not also siblings.
The relevant change was done in https://bugzilla.mozilla.org/show_bug.cgi?id=603550 (or more like, that fixed Gecko to behave as it had behaved for ages).

As I said in that bug (5 years ago ;)),
I think I'm leaning over to the behavior where mousedown frame would always get the events, in other words implicit capturing.

 

Thanks Olli!  If it's sufficiently web compatible then I'd prefer implicit capturing too.  That would often just be the right thing and wouldn't require any additional APIs to solve all the important scenarios I know of.  If other engines were open to trying to move to this model, we could give it a shot in Chrome.

 

-Olli




In particular, I had a great discussion with the Google Maps team about why this matters to them.  The Google Maps embedded API
<https://developers.google.com/maps/documentation/embed/> lets you host a map in an iframe on your page.  Here when you click on the map and drag to
pan it, the user really expects that it continues to pan even if the cursor leaves the frame.  They've got some ugly UserAgent-specific hacks to make
this work on all browsers and these hacks have gotten in the way of us fixing other interop problems in Chrome.

So, we need some standard interoperable way to enable such a scenario.  But this doesn't mean we have to do it by default if that's hard to agree on
or likely to cause developer confusion.  Is this scenario possible today in IE / Gecko using the non-standard setCapture API
<https://developer.mozilla.org/en-US/docs/Web/API/Element/setCapture>?  Presumably setPointerCapture
<https://w3c.github.io/pointerevents/#pointer-capture> can definitely be used for this purpose (though the spec is not clear
<https://github.com/w3c/pointerevents/issues/16>).  But I'm not sure we want to block a solution to these problems on Pointer Events (eg. there should
be some path for Safari as well).

Has there been any discussion about standardizing setCapture?  If IE and Gecko already have an interoperable implementation here, perhaps we should
consider that a defacto standard and add it to blink too, then change our mouse event behavior to match IE's?

But still, I think the UI Events spec should be updated to be clear on what the default behavior for mouse events should be in this regard.

Thoughts?
    Rick

On Mon, Jun 22, 2015 at 1:21 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

    It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find it you'd need to dig
    into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html
    <http://dtapuska.github.io/iframe-mouse-target/iframe_outer.html>


    On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

        It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each rendering engine.

        I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the web author.

        Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different frames when the mouse
        moves outside the bounds of the iframe.

        1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see
        http://crbug.com/269917; we need to fix this).
        2) Firefox targets the mouse down frame unless you move into a sibling iframe
        3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.

        We need to get some clarity written into the specification. As the spec indicates the target is:

          * |Event.target| <http://www.w3.org/TR/DOM-Level-3-Events/#widl-Event-target>: topmost event target
            <http://www.w3.org/TR/DOM-Level-3-Events/#glossary-topmost-event-target>


        Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.

        All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a mouse button is
        depressed; this can leave the iframe in an inconsistent state possibly.

        So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to the web author point
        of view.

        I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging something around you
        might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug references google maps
        embedded inside a page as a use case that occurs in the field today.

        With respect to hover processing:
        1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
        2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
        3) IE 11 & Edge generate hover on whatever is under the mouse.

        The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.

        The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is interesting. Perhaps I'm
        flawed in my reproduction step; but some explanation behind this would be appreciated.

        The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like hover and prevent default.

        dave.


 

 


Reply | Threaded
Open this post in threaded view
|

Re: UI Events - Mouse Events and iframe targetting

Dave Tapuska-3
If I could remove the thought about what exists out there in the world today... 

I personally prefer IE's approach with respect to this. 

There exists a capture API under pointer events for this; and the Event.setCapture is just a predecessor to that.

Agreeing with Ted in that If we did an implicit capture then how do two frames collude to allow drag and drop between them? You'd need to re-broadcast the events; but what about sandbox (or out of process) iframes; you might not want the collusion; but still want the ability to drag and drop so that should really come from the process owner for input.

Traditional windowing systems have the same semantics:


I don't know why we wouldn't treat an iframe like a different from a traditional window.

dave.


On Tue, Jun 30, 2015 at 4:40 PM, Rick Byers <[hidden email]> wrote:
This seems like a reasonable model - certainly simpler to describe / spec than the other alternatives.

But before we can attempt to change Chrome's behavior here (and really find out how breaking such a change would be), we'd need some solution for apps relying on our current capturing behavior (such as Google maps)..  Element.setCapture works for this in IE, right?  Perhaps we should just attempt to ship that (a "defacto standard" even without spec coverage) in blink.  Or perhaps we should just provide a polyfill for it in terms of pointer events and setPointerCapture (though that would mean we'd have to put solving this problem on hold until we've shipped pointer events).

Do you have any usage metrics for Element.setCapture in Trident?  Would you like to be able to deprecate / remove it some day, or do you expect you'd want to keep it forever anyway?

Rick

On Tue, Jun 30, 2015 at 4:21 PM, Ted Dinklocker <[hidden email]> wrote:

On the Microsoft side we discussed this and here is a summary of those discussions:

 

Everyone on the EdgeHTML side prefers the behavior that we have today. The example that came up a couple of times was drag and drop – if events are implicitly captured to the iframe, then there is no easy way to hand off the interaction from one frame to another.

 

We generally thought that the current approach in EdgeHTML is straightforward – events/hover applies to whatever is under the mouse, regardless of movement over iframes or button presses.

 

We investigated whether we fire enter/leave when leaving the iframe, and confirmed that we do fire the events on the element, not the document.

 

We agree that developers likely want a setPointerCapture approach, which gives the web developer flexibility and more control.

 

Thoughts?

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Rick Byers
Sent: Tuesday, June 23, 2015 11:19 AM
To: Olli Pettay <[hidden email]>
Cc: Dave Tapuska <[hidden email]>; [hidden email]; Jacob Rossi <[hidden email]>
Subject: Re: UI Events - Mouse Events and iframe targetting

 

On Tue, Jun 23, 2015 at 12:01 PM, Olli Pettay <[hidden email]> wrote:

On 06/22/2015 11:00 AM, Rick Byers wrote:

Thanks Dave,
I've been aware of differences between browsers here for awhile, but only recently been convinced that it's important that we try to fix this. /cc
Jacob and Olli in case they have specific experience/advice for EdgeHTML and Gecko respectively.


So the idea behind "2) Firefox targets the mouse down frame unless you move into a sibling iframe"
is that iframe and top level browsing context would get similar behavior when dealing with mouse event based dragging.
If you have mousedown, you flag the mousedown browsing context as the active and any mouse events in ancestors (or outside the browser window) are
forwarded to the active browsing context.
(This was perhaps slightly based on the single process Firefox architecture where 'chrome browsing context' hosts the top level web browsing contexts).
I do consider the 'move into a sibling iframe' a bug, since the idea was to deal nested iframes only in that way, not also siblings.
The relevant change was done in https://bugzilla.mozilla.org/show_bug.cgi?id=603550 (or more like, that fixed Gecko to behave as it had behaved for ages).

As I said in that bug (5 years ago ;)),
I think I'm leaning over to the behavior where mousedown frame would always get the events, in other words implicit capturing.

 

Thanks Olli!  If it's sufficiently web compatible then I'd prefer implicit capturing too.  That would often just be the right thing and wouldn't require any additional APIs to solve all the important scenarios I know of.  If other engines were open to trying to move to this model, we could give it a shot in Chrome.

 

-Olli




In particular, I had a great discussion with the Google Maps team about why this matters to them.  The Google Maps embedded API
<https://developers.google.com/maps/documentation/embed/> lets you host a map in an iframe on your page.  Here when you click on the map and drag to
pan it, the user really expects that it continues to pan even if the cursor leaves the frame.  They've got some ugly UserAgent-specific hacks to make
this work on all browsers and these hacks have gotten in the way of us fixing other interop problems in Chrome.

So, we need some standard interoperable way to enable such a scenario.  But this doesn't mean we have to do it by default if that's hard to agree on
or likely to cause developer confusion.  Is this scenario possible today in IE / Gecko using the non-standard setCapture API
<https://developer.mozilla.org/en-US/docs/Web/API/Element/setCapture>?  Presumably setPointerCapture
<https://w3c.github.io/pointerevents/#pointer-capture> can definitely be used for this purpose (though the spec is not clear
<https://github.com/w3c/pointerevents/issues/16>).  But I'm not sure we want to block a solution to these problems on Pointer Events (eg. there should
be some path for Safari as well).

Has there been any discussion about standardizing setCapture?  If IE and Gecko already have an interoperable implementation here, perhaps we should
consider that a defacto standard and add it to blink too, then change our mouse event behavior to match IE's?

But still, I think the UI Events spec should be updated to be clear on what the default behavior for mouse events should be in this regard.

Thoughts?
    Rick

On Mon, Jun 22, 2015 at 1:21 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

    It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find it you'd need to dig
    into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html
    <http://dtapuska.github.io/iframe-mouse-target/iframe_outer.html>


    On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

        It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each rendering engine.

        I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the web author.

        Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different frames when the mouse
        moves outside the bounds of the iframe.

        1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see
        http://crbug.com/269917; we need to fix this).
        2) Firefox targets the mouse down frame unless you move into a sibling iframe
        3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.

        We need to get some clarity written into the specification. As the spec indicates the target is:

          * |Event.target| <http://www.w3.org/TR/DOM-Level-3-Events/#widl-Event-target>: topmost event target
            <http://www.w3.org/TR/DOM-Level-3-Events/#glossary-topmost-event-target>


        Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.

        All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a mouse button is
        depressed; this can leave the iframe in an inconsistent state possibly.

        So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to the web author point
        of view.

        I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging something around you
        might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug references google maps
        embedded inside a page as a use case that occurs in the field today.

        With respect to hover processing:
        1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
        2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
        3) IE 11 & Edge generate hover on whatever is under the mouse.

        The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.

        The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is interesting. Perhaps I'm
        flawed in my reproduction step; but some explanation behind this would be appreciated.

        The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like hover and prevent default.

        dave.


 

 



Reply | Threaded
Open this post in threaded view
|

Re: UI Events - Mouse Events and iframe targetting

Robert Kroeger
[+fsamuel, sadrul]


On Tue, Jun 30, 2015 at 2:41 PM, Dave Tapuska <[hidden email]> wrote:
If I could remove the thought about what exists out there in the world today... 

I personally prefer IE's approach with respect to this. 

There exists a capture API under pointer events for this; and the Event.setCapture is just a predecessor to that.

Agreeing with Ted in that If we did an implicit capture then how do two frames collude to allow drag and drop between them? You'd need to re-broadcast the events; but what about sandbox (or out of process) iframes; you might not want the collusion; but still want the ability to drag and drop so that should really come from the process owner for input.

Traditional windowing systems have the same semantics:


I don't know why we wouldn't treat an iframe like a different from a traditional window.

Exactly implementing X11 cursor grabs seems very dangerous: a hostile frame could refuse to abandon its grab. SetCapture's "... function cannot be used to capture mouse input meant for another process." would seem desirable in a web context?

Rob.

 

dave.


On Tue, Jun 30, 2015 at 4:40 PM, Rick Byers <[hidden email]> wrote:
This seems like a reasonable model - certainly simpler to describe / spec than the other alternatives.

But before we can attempt to change Chrome's behavior here (and really find out how breaking such a change would be), we'd need some solution for apps relying on our current capturing behavior (such as Google maps)..  Element.setCapture works for this in IE, right?  Perhaps we should just attempt to ship that (a "defacto standard" even without spec coverage) in blink.  Or perhaps we should just provide a polyfill for it in terms of pointer events and setPointerCapture (though that would mean we'd have to put solving this problem on hold until we've shipped pointer events).

Do you have any usage metrics for Element.setCapture in Trident?  Would you like to be able to deprecate / remove it some day, or do you expect you'd want to keep it forever anyway?

Rick

On Tue, Jun 30, 2015 at 4:21 PM, Ted Dinklocker <[hidden email]> wrote:

On the Microsoft side we discussed this and here is a summary of those discussions:

 

Everyone on the EdgeHTML side prefers the behavior that we have today. The example that came up a couple of times was drag and drop – if events are implicitly captured to the iframe, then there is no easy way to hand off the interaction from one frame to another.

 

We generally thought that the current approach in EdgeHTML is straightforward – events/hover applies to whatever is under the mouse, regardless of movement over iframes or button presses.

 

We investigated whether we fire enter/leave when leaving the iframe, and confirmed that we do fire the events on the element, not the document.

 

We agree that developers likely want a setPointerCapture approach, which gives the web developer flexibility and more control.

 

Thoughts?

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Rick Byers
Sent: Tuesday, June 23, 2015 11:19 AM
To: Olli Pettay <[hidden email]>
Cc: Dave Tapuska <[hidden email]>; [hidden email]; Jacob Rossi <[hidden email]>
Subject: Re: UI Events - Mouse Events and iframe targetting

 

On Tue, Jun 23, 2015 at 12:01 PM, Olli Pettay <[hidden email]> wrote:

On 06/22/2015 11:00 AM, Rick Byers wrote:

Thanks Dave,
I've been aware of differences between browsers here for awhile, but only recently been convinced that it's important that we try to fix this. /cc
Jacob and Olli in case they have specific experience/advice for EdgeHTML and Gecko respectively.


So the idea behind "2) Firefox targets the mouse down frame unless you move into a sibling iframe"
is that iframe and top level browsing context would get similar behavior when dealing with mouse event based dragging.
If you have mousedown, you flag the mousedown browsing context as the active and any mouse events in ancestors (or outside the browser window) are
forwarded to the active browsing context.
(This was perhaps slightly based on the single process Firefox architecture where 'chrome browsing context' hosts the top level web browsing contexts).
I do consider the 'move into a sibling iframe' a bug, since the idea was to deal nested iframes only in that way, not also siblings.
The relevant change was done in https://bugzilla.mozilla.org/show_bug.cgi?id=603550 (or more like, that fixed Gecko to behave as it had behaved for ages).

As I said in that bug (5 years ago ;)),
I think I'm leaning over to the behavior where mousedown frame would always get the events, in other words implicit capturing.

 

Thanks Olli!  If it's sufficiently web compatible then I'd prefer implicit capturing too.  That would often just be the right thing and wouldn't require any additional APIs to solve all the important scenarios I know of.  If other engines were open to trying to move to this model, we could give it a shot in Chrome.

 

-Olli




In particular, I had a great discussion with the Google Maps team about why this matters to them.  The Google Maps embedded API
<https://developers.google.com/maps/documentation/embed/> lets you host a map in an iframe on your page.  Here when you click on the map and drag to
pan it, the user really expects that it continues to pan even if the cursor leaves the frame.  They've got some ugly UserAgent-specific hacks to make
this work on all browsers and these hacks have gotten in the way of us fixing other interop problems in Chrome.

So, we need some standard interoperable way to enable such a scenario.  But this doesn't mean we have to do it by default if that's hard to agree on
or likely to cause developer confusion.  Is this scenario possible today in IE / Gecko using the non-standard setCapture API
<https://developer.mozilla.org/en-US/docs/Web/API/Element/setCapture>?  Presumably setPointerCapture
<https://w3c.github.io/pointerevents/#pointer-capture> can definitely be used for this purpose (though the spec is not clear
<https://github.com/w3c/pointerevents/issues/16>).  But I'm not sure we want to block a solution to these problems on Pointer Events (eg. there should
be some path for Safari as well).

Has there been any discussion about standardizing setCapture?  If IE and Gecko already have an interoperable implementation here, perhaps we should
consider that a defacto standard and add it to blink too, then change our mouse event behavior to match IE's?

But still, I think the UI Events spec should be updated to be clear on what the default behavior for mouse events should be in this regard.

Thoughts?
    Rick

On Mon, Jun 22, 2015 at 1:21 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

    It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find it you'd need to dig
    into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html
    <http://dtapuska.github.io/iframe-mouse-target/iframe_outer.html>


    On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

        It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each rendering engine.

        I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the web author.

        Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different frames when the mouse
        moves outside the bounds of the iframe.

        1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see
        http://crbug.com/269917; we need to fix this).
        2) Firefox targets the mouse down frame unless you move into a sibling iframe
        3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.

        We need to get some clarity written into the specification. As the spec indicates the target is:

          * |Event.target| <http://www.w3.org/TR/DOM-Level-3-Events/#widl-Event-target>: topmost event target
            <http://www.w3.org/TR/DOM-Level-3-Events/#glossary-topmost-event-target>


        Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.

        All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a mouse button is
        depressed; this can leave the iframe in an inconsistent state possibly.

        So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to the web author point
        of view.

        I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging something around you
        might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug references google maps
        embedded inside a page as a use case that occurs in the field today.

        With respect to hover processing:
        1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
        2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
        3) IE 11 & Edge generate hover on whatever is under the mouse.

        The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.

        The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is interesting. Perhaps I'm
        flawed in my reproduction step; but some explanation behind this would be appreciated.

        The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like hover and prevent default.

        dave.


 

 




Reply | Threaded
Open this post in threaded view
|

Re: UI Events - Mouse Events and iframe targetting

Olli Pettay-2
On 07/07/2015 12:49 AM, Robert Kroeger wrote:

> [+fsamuel, sadrul]
>
>
> On Tue, Jun 30, 2015 at 2:41 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:
>
>     If I could remove the thought about what exists out there in the world today...
>
>     I personally prefer IE's approach with respect to this.
>
>     There exists a capture API under pointer events for this; and the Event.setCapture is just a predecessor to that.
>
>     Agreeing with Ted in that If we did an implicit capture then how do two frames collude to allow drag and drop between them? You'd need to
>     re-broadcast the events; but what about sandbox (or out of process) iframes; you might not want the collusion; but still want the ability to drag
>     and drop so that should really come from the process owner for input.
>
>     Traditional windowing systems have the same semantics:
>
>     X's grab pointer - http://linux.die.net/man/3/xgrabpointer
>     Win32's SetCapture - https://msdn.microsoft.com/en-us/library/windows/desktop/ms646262(v=vs.85).aspx
>
>     I don't know why we wouldn't treat an iframe like a different from a traditional window.
>
>
> Exactly implementing X11 cursor grabs seems very dangerous: a hostile frame could refuse to abandon its grab. SetCapture's "... function cannot be
> used to capture mouse input meant for another process." would seem desirable in a web context?
>
In Gecko, IIRC, setCapture is automatically released after mouseup.
And again IIRC, IE did this differently when we added support for setCapture, but allowing
setCapture to keep the capture forever was just too scary from security point of view.

https://bugzilla.mozilla.org/show_bug.cgi?id=503943#c3 also
https://bugzilla.mozilla.org/show_bug.cgi?id=503943#c7


-Olli


> Rob.
>
>
>     dave.
>
>
>     On Tue, Jun 30, 2015 at 4:40 PM, Rick Byers <[hidden email] <mailto:[hidden email]>> wrote:
>
>         This seems like a reasonable model - certainly simpler to describe / spec than the other alternatives.
>
>         But before we can attempt to change Chrome's behavior here (and really find out how breaking such a change would be), we'd need some solution
>         for apps relying on our current capturing behavior (such as Google maps)..  Element.setCapture works for this in IE, right?  Perhaps we should
>         just attempt to ship that (a "defacto standard" even without spec coverage) in blink.  Or perhaps we should just provide a polyfill for it in
>         terms of pointer events and setPointerCapture (though that would mean we'd have to put solving this problem on hold until we've shipped
>         pointer events).
>
>         Do you have any usage metrics for Element.setCapture in Trident?  Would you like to be able to deprecate / remove it some day, or do you
>         expect you'd want to keep it forever anyway?
>
>         Rick
>
>         On Tue, Jun 30, 2015 at 4:21 PM, Ted Dinklocker <[hidden email] <mailto:[hidden email]>> wrote:
>
>             On the Microsoft side we discussed this and here is a summary of those discussions:____
>
>             __ __
>
>             Everyone on the EdgeHTML side prefers the behavior that we have today. The example that came up a couple of times was drag and drop – if
>             events are implicitly captured to the iframe, then there is no easy way to hand off the interaction from one frame to another.____
>
>             __ __
>
>             We generally thought that the current approach in EdgeHTML is straightforward – events/hover applies to whatever is under the mouse,
>             regardless of movement over iframes or button presses.____
>
>             __ __
>
>             We investigated whether we fire enter/leave when leaving the iframe, and confirmed that we do fire the events on the element, not the
>             document.____
>
>             __ __
>
>             We agree that developers likely want a setPointerCapture approach, which gives the web developer flexibility and more control.____
>
>             __ __
>
>             Thoughts?____
>
>             __ __
>
>             *From:*[hidden email] <mailto:[hidden email]> [mailto:[hidden email] <mailto:[hidden email]>] *On Behalf Of *Rick Byers
>             *Sent:* Tuesday, June 23, 2015 11:19 AM
>             *To:* Olli Pettay <[hidden email] <mailto:[hidden email]>>
>             *Cc:* Dave Tapuska <[hidden email] <mailto:[hidden email]>>; [hidden email] <mailto:[hidden email]>; Jacob Rossi
>             <[hidden email] <mailto:[hidden email]>>
>             *Subject:* Re: UI Events - Mouse Events and iframe targetting____
>
>             __ __
>
>             On Tue, Jun 23, 2015 at 12:01 PM, Olli Pettay <[hidden email] <mailto:[hidden email]>> wrote:____
>
>                 On 06/22/2015 11:00 AM, Rick Byers wrote:____
>
>                     Thanks Dave,
>                     I've been aware of differences between browsers here for awhile, but only recently been convinced that it's important that we try
>                     to fix this. /cc
>                     Jacob and Olli in case they have specific experience/advice for EdgeHTML and Gecko respectively.____
>
>
>                 So the idea behind "2) Firefox targets the mouse down frame unless you move into a sibling iframe"
>                 is that iframe and top level browsing context would get similar behavior when dealing with mouse event based dragging.
>                 If you have mousedown, you flag the mousedown browsing context as the active and any mouse events in ancestors (or outside the browser
>                 window) are
>                 forwarded to the active browsing context.
>                 (This was perhaps slightly based on the single process Firefox architecture where 'chrome browsing context' hosts the top level web
>                 browsing contexts).
>                 I do consider the 'move into a sibling iframe' a bug, since the idea was to deal nested iframes only in that way, not also siblings.
>                 The relevant change was done in https://bugzilla.mozilla.org/show_bug.cgi?id=603550 (or more like, that fixed Gecko to behave as it
>                 had behaved for ages).
>
>                 As I said in that bug (5 years ago ;)),
>                 I think I'm leaning over to the behavior where mousedown frame would always get the events, in other words implicit capturing.____
>
>             __ __
>
>             Thanks Olli!  If it's sufficiently web compatible then I'd prefer implicit capturing too.  That would often just be the right thing and
>             wouldn't require any additional APIs to solve all the important scenarios I know of.  If other engines were open to trying to move to this
>             model, we could give it a shot in Chrome.____
>
>             __ __
>
>                 -Olli
>
>
>
>                 ____
>
>
>                     In particular, I had a great discussion with the Google Maps team about why this matters to them.  The Google Maps embedded API
>                     <https://developers.google.com/maps/documentation/embed/> lets you host a map in an iframe on your page.  Here when you click on
>                     the map and drag to
>                     pan it, the user really expects that it continues to pan even if the cursor leaves the frame.  They've got some ugly
>                     UserAgent-specific hacks to make
>                     this work on all browsers and these hacks have gotten in the way of us fixing other interop problems in Chrome.
>
>                     So, we need some standard interoperable way to enable such a scenario.  But this doesn't mean we have to do it by default if
>                     that's hard to agree on
>                     or likely to cause developer confusion.  Is this scenario possible today in IE / Gecko using the non-standard setCapture API
>                     <https://developer.mozilla.org/en-US/docs/Web/API/Element/setCapture>?  Presumably setPointerCapture
>                     <https://w3c.github.io/pointerevents/#pointer-capture> can definitely be used for this purpose (though the spec is not clear
>                     <https://github.com/w3c/pointerevents/issues/16>).  But I'm not sure we want to block a solution to these problems on Pointer
>                     Events (eg. there should
>                     be some path for Safari as well).
>
>                     Has there been any discussion about standardizing setCapture?  If IE and Gecko already have an interoperable implementation here,
>                     perhaps we should
>                     consider that a defacto standard and add it to blink too, then change our mouse event behavior to match IE's?
>
>                     But still, I think the UI Events spec should be updated to be clear on what the default behavior for mouse events should be in
>                     this regard.
>
>                     Thoughts?
>                          Rick
>
>                     On Mon, Jun 22, 2015 at 1:21 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]> <mailto:[hidden email]
>                     <mailto:[hidden email]>>> wrote:
>
>                          It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find
>                     it you'd need to dig
>                          into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html
>                          <http://dtapuska.github.io/iframe-mouse-target/iframe_outer.html>
>
>
>                          On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>
>                     <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
>
>                              It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each
>                     rendering engine.
>
>                              I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the
>                     web author.
>
>                              Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different
>                     frames when the mouse
>                              moves outside the bounds of the iframe.
>
>                              1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see
>                     http://crbug.com/269917; we need to fix this).
>                              2) Firefox targets the mouse down frame unless you move into a sibling iframe
>                              3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.
>
>                              We need to get some clarity written into the specification. As the spec indicates the target is:
>
>                                * |Event.target| <http://www.w3.org/TR/DOM-Level-3-Events/#widl-Event-target>: topmost event target
>                                  <http://www.w3.org/TR/DOM-Level-3-Events/#glossary-topmost-event-target>
>
>
>                              Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.
>
>                              All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a
>                     mouse button is
>                              depressed; this can leave the iframe in an inconsistent state possibly.
>
>                              So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to
>                     the web author point
>                              of view.
>
>                              I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging
>                     something around you
>                              might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug
>                     references google maps
>                              embedded inside a page as a use case that occurs in the field today.
>
>                              With respect to hover processing:
>                              1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
>                              2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
>                              3) IE 11 & Edge generate hover on whatever is under the mouse.
>
>                              The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.
>
>                              The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is
>                     interesting. Perhaps I'm
>                              flawed in my reproduction step; but some explanation behind this would be appreciated.
>
>                              The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like
>                     hover and prevent default.
>
>                              dave.
>
>
>                     ____
>
>                 __ __
>
>             __ __
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: UI Events - Mouse Events and iframe targetting

Dave Tapuska-3
I see on my Windows 10 preview image; Edge/Spartan has document.releaseCapture, element.setCapture and element.releaseCapture as undefined; were they not used widely?
Or that since it wasn't formally specified and now that element.setPointerCapture is, the goal is to move people onto that API?

What about people still using mouse events and not pointer events? Does pointer capture influence mouse events in Edge? Or that those there were using releaseCapture, setCapture need to move to pointer events wholesale if they want to be able to capture the events.

dave.

On Mon, Jul 6, 2015 at 6:10 PM, Olli Pettay <[hidden email]> wrote:
On 07/07/2015 12:49 AM, Robert Kroeger wrote:
[+fsamuel, sadrul]


On Tue, Jun 30, 2015 at 2:41 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>> wrote:

    If I could remove the thought about what exists out there in the world today...

    I personally prefer IE's approach with respect to this.

    There exists a capture API under pointer events for this; and the Event.setCapture is just a predecessor to that.

    Agreeing with Ted in that If we did an implicit capture then how do two frames collude to allow drag and drop between them? You'd need to
    re-broadcast the events; but what about sandbox (or out of process) iframes; you might not want the collusion; but still want the ability to drag
    and drop so that should really come from the process owner for input.

    Traditional windowing systems have the same semantics:

    X's grab pointer - http://linux.die.net/man/3/xgrabpointer
    Win32's SetCapture - https://msdn.microsoft.com/en-us/library/windows/desktop/ms646262(v=vs.85).aspx

    I don't know why we wouldn't treat an iframe like a different from a traditional window.


Exactly implementing X11 cursor grabs seems very dangerous: a hostile frame could refuse to abandon its grab. SetCapture's "... function cannot be
used to capture mouse input meant for another process." would seem desirable in a web context?

In Gecko, IIRC, setCapture is automatically released after mouseup.
And again IIRC, IE did this differently when we added support for setCapture, but allowing
setCapture to keep the capture forever was just too scary from security point of view.

https://bugzilla.mozilla.org/show_bug.cgi?id=503943#c3 also
https://bugzilla.mozilla.org/show_bug.cgi?id=503943#c7


-Olli


Rob.


    dave.


    On Tue, Jun 30, 2015 at 4:40 PM, Rick Byers <[hidden email] <mailto:[hidden email]>> wrote:

        This seems like a reasonable model - certainly simpler to describe / spec than the other alternatives.

        But before we can attempt to change Chrome's behavior here (and really find out how breaking such a change would be), we'd need some solution
        for apps relying on our current capturing behavior (such as Google maps)..  Element.setCapture works for this in IE, right?  Perhaps we should
        just attempt to ship that (a "defacto standard" even without spec coverage) in blink.  Or perhaps we should just provide a polyfill for it in
        terms of pointer events and setPointerCapture (though that would mean we'd have to put solving this problem on hold until we've shipped
        pointer events).

        Do you have any usage metrics for Element.setCapture in Trident?  Would you like to be able to deprecate / remove it some day, or do you
        expect you'd want to keep it forever anyway?

        Rick

        On Tue, Jun 30, 2015 at 4:21 PM, Ted Dinklocker <[hidden email] <mailto:[hidden email]>> wrote:

            On the Microsoft side we discussed this and here is a summary of those discussions:____

            __ __

            Everyone on the EdgeHTML side prefers the behavior that we have today. The example that came up a couple of times was drag and drop – if
            events are implicitly captured to the iframe, then there is no easy way to hand off the interaction from one frame to another.____

            __ __

            We generally thought that the current approach in EdgeHTML is straightforward – events/hover applies to whatever is under the mouse,
            regardless of movement over iframes or button presses.____

            __ __

            We investigated whether we fire enter/leave when leaving the iframe, and confirmed that we do fire the events on the element, not the
            document.____

            __ __

            We agree that developers likely want a setPointerCapture approach, which gives the web developer flexibility and more control.____

            __ __

            Thoughts?____

            __ __

            *From:*[hidden email] <mailto:[hidden email]> [mailto:[hidden email] <mailto:[hidden email]>] *On Behalf Of *Rick Byers
            *Sent:* Tuesday, June 23, 2015 11:19 AM
            *To:* Olli Pettay <[hidden email] <mailto:[hidden email]>>
            *Cc:* Dave Tapuska <[hidden email] <mailto:[hidden email]>>; [hidden email] <mailto:[hidden email]>; Jacob Rossi
            <[hidden email] <mailto:[hidden email]>>
            *Subject:* Re: UI Events - Mouse Events and iframe targetting____

            __ __

            On Tue, Jun 23, 2015 at 12:01 PM, Olli Pettay <[hidden email] <mailto:[hidden email]>> wrote:____

                On 06/22/2015 11:00 AM, Rick Byers wrote:____

                    Thanks Dave,
                    I've been aware of differences between browsers here for awhile, but only recently been convinced that it's important that we try
                    to fix this. /cc
                    Jacob and Olli in case they have specific experience/advice for EdgeHTML and Gecko respectively.____


                So the idea behind "2) Firefox targets the mouse down frame unless you move into a sibling iframe"
                is that iframe and top level browsing context would get similar behavior when dealing with mouse event based dragging.
                If you have mousedown, you flag the mousedown browsing context as the active and any mouse events in ancestors (or outside the browser
                window) are
                forwarded to the active browsing context.
                (This was perhaps slightly based on the single process Firefox architecture where 'chrome browsing context' hosts the top level web
                browsing contexts).
                I do consider the 'move into a sibling iframe' a bug, since the idea was to deal nested iframes only in that way, not also siblings.
                The relevant change was done in https://bugzilla.mozilla.org/show_bug.cgi?id=603550 (or more like, that fixed Gecko to behave as it
                had behaved for ages).

                As I said in that bug (5 years ago ;)),
                I think I'm leaning over to the behavior where mousedown frame would always get the events, in other words implicit capturing.____

            __ __

            Thanks Olli!  If it's sufficiently web compatible then I'd prefer implicit capturing too.  That would often just be the right thing and
            wouldn't require any additional APIs to solve all the important scenarios I know of.  If other engines were open to trying to move to this
            model, we could give it a shot in Chrome.____

            __ __

                -Olli



                ____


                    In particular, I had a great discussion with the Google Maps team about why this matters to them.  The Google Maps embedded API
                    <https://developers.google.com/maps/documentation/embed/> lets you host a map in an iframe on your page.  Here when you click on
                    the map and drag to
                    pan it, the user really expects that it continues to pan even if the cursor leaves the frame.  They've got some ugly
                    UserAgent-specific hacks to make
                    this work on all browsers and these hacks have gotten in the way of us fixing other interop problems in Chrome.

                    So, we need some standard interoperable way to enable such a scenario.  But this doesn't mean we have to do it by default if
                    that's hard to agree on
                    or likely to cause developer confusion.  Is this scenario possible today in IE / Gecko using the non-standard setCapture API
                    <https://developer.mozilla.org/en-US/docs/Web/API/Element/setCapture>?  Presumably setPointerCapture
                    <https://w3c.github.io/pointerevents/#pointer-capture> can definitely be used for this purpose (though the spec is not clear
                    <https://github.com/w3c/pointerevents/issues/16>)..  But I'm not sure we want to block a solution to these problems on Pointer
                    Events (eg. there should
                    be some path for Safari as well).

                    Has there been any discussion about standardizing setCapture?  If IE and Gecko already have an interoperable implementation here,
                    perhaps we should
                    consider that a defacto standard and add it to blink too, then change our mouse event behavior to match IE's?

                    But still, I think the UI Events spec should be updated to be clear on what the default behavior for mouse events should be in
                    this regard.

                    Thoughts?
                         Rick

                    On Mon, Jun 22, 2015 at 1:21 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]> <mailto:[hidden email]
                    <mailto:[hidden email]>>> wrote:

                         It was pointed out to me offline that I didn't include the sample page that demonstrates the issues with the vendors; to find
                    it you'd need to dig
                         into the chromium bug; for simplicity I've put it on github. See http://dtapuska.github.io/iframe-mouse-target/iframe_outer..html
                         <http://dtapuska.github.io/iframe-mouse-target/iframe_outer.html>


                         On Tue, Jun 16, 2015 at 4:16 PM, Dave Tapuska <[hidden email] <mailto:[hidden email]>
                    <mailto:[hidden email] <mailto:[hidden email]>>> wrote:

                             It seems that processing mouse [move|up] events have 3 different implementations when involving an iframe in each
                    rendering engine.

                             I'm soliciting comments and hope we can work on defining some behavior and converging our implementations to benefit the
                    web author.

                             Specifically when an mouse down event occurs inside an iframe; the subsequent mouse moves/ups are targeted at different
                    frames when the mouse
                             moves outside the bounds of the iframe.

                             1) Chrome targets the frame that generated the mouse down frame sometimes (but has side effects with prevent default; see
                    http://crbug.com/269917; we need to fix this).
                             2) Firefox targets the mouse down frame unless you move into a sibling iframe
                             3) IE 11 & Edge target the topmost frame under the mouse move regardless of the mouse down operation.

                             We need to get some clarity written into the specification. As the spec indicates the target is:

                               * |Event.target| <http://www.w3.org/TR/DOM-Level-3-Events/#widl-Event-target>: topmost event target
                                 <http://www.w3.org/TR/DOM-Level-3-Events/#glossary-topmost-event-target>


                             Likewise in Edge/IE 11 you can have a mouse down event in the iframe; but the mouse up event occurs in the main frame.

                             All browsers do not generate a mouse leave/enter on the iframe document when the mouse leaves or enters the iframe when a
                    mouse button is
                             depressed; this can leave the iframe in an inconsistent state possibly.

                             So I would expect that the IE 11/Edge implementation is spec compliant. However this doesn't necessarily seem correct to
                    the web author point
                             of view.

                             I can see why the target is defined is set to be the iframe that handled the mouse down event because if you are dragging
                    something around you
                             might not want to start processing mouse move events on the outer frame until the mouse is released.  The crbug
                    references google maps
                             embedded inside a page as a use case that occurs in the field today.

                             With respect to hover processing:
                             1) Chrome doesn't generate hover events for items outside of the iframe during a mouse down.. (Seems incorrect).
                             2) Firefox does the interesting behaviour of generating hover events for items not in the parent frame but in sibling frames.
                             3) IE 11 & Edge generate hover on whatever is under the mouse.

                             The IE 11/Edge implementation seems straight forward. Perhaps some folks can confirm that it is as I think it to be.

                             The Firefox implementation mystifies me why it behaves different between the main frame and a sibling iframe is
                    interesting. Perhaps I'm
                             flawed in my reproduction step; but some explanation behind this would be appreciated.

                             The Chrome/WebKit implementation makes sense a little bit of sense to me as well; granted it has some weird bugs like
                    hover and prevent default.

                             dave.


                    ____

                __ __

            __ __