Browser detection for shepherd/css test runner

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Browser detection for shepherd/css test runner

Christian Biesinger-3
Hi there,

looking at the icons on
http://dev.w3.org/csswg/css-flexbox/#abspos-items, and test results
like http://test.csswg.org/harness/details/css-flexbox-1_dev/flexbox-abspos-child-001b/engine/webkit/,
it seems that the test runner considers Chrome to be Webkit.

That is bad. Chrome does not use Webkit anymore, and there is
definitely divergence between the two. Conversely, Presto is not so
important anymore.

Any chance of getting this updated?

thanks,
-christian

Reply | Threaded
Open this post in threaded view
|

Re: Browser detection for shepherd/css test runner

Florian Rivoal-4

> On 27 Mar 2015, at 22:09, Christian Biesinger <[hidden email]> wrote:
>
> Hi there,
>
> looking at the icons on
> http://dev.w3.org/csswg/css-flexbox/#abspos-items, and test results
> like http://test.csswg.org/harness/details/css-flexbox-1_dev/flexbox-abspos-child-001b/engine/webkit/,
> it seems that the test runner considers Chrome to be Webkit.
>
> That is bad. Chrome does not use Webkit anymore, and there is
> definitely divergence between the two. Conversely, Presto is not so
> important anymore.
>
> Any chance of getting this updated?


Completely agree with you with regards to Chrome.

As for Presto, it is certainly of quickly declining relevance, but there are still a few areas where it supports features not supported in other browsers, or conforms better than other browsers. I think we should keep it for a while longer. As it provides useful information to the people working on the spec or the tests.

Besides, there's nothing wrong with a little tongue-in-cheek challenge to other browsers ("Even old Presto gets this right!").

 - Florian


Reply | Threaded
Open this post in threaded view
|

Re: Browser detection for shepherd/css test runner

Peter Linss

On Mar 27, 2015, at 2:42 PM, Florian Rivoal <[hidden email]> wrote:

>
>> On 27 Mar 2015, at 22:09, Christian Biesinger <[hidden email]> wrote:
>>
>> Hi there,
>>
>> looking at the icons on
>> http://dev.w3.org/csswg/css-flexbox/#abspos-items, and test results
>> like http://test.csswg.org/harness/details/css-flexbox-1_dev/flexbox-abspos-child-001b/engine/webkit/,
>> it seems that the test runner considers Chrome to be Webkit.
>>
>> That is bad. Chrome does not use Webkit anymore, and there is
>> definitely divergence between the two. Conversely, Presto is not so
>> important anymore.
>>
>> Any chance of getting this updated?
>
>
> Completely agree with you with regards to Chrome.
The problem is, while there’s divergence in some areas, there’s still shared code in many (most?), and it currently requires human judgement to determine if passes form both count as two independent implementations or not.

If someone wants to generate a map (keyed by spec section) where there’s divergence (or the converse would be better, since that set isn’t getting bigger), I’d be happy to have the test harness automatically differentiate the results.

For the record, each result is keyed to the full UA string that generated it, so we can always go back and re-assign results to different products.

>
> As for Presto, it is certainly of quickly declining relevance, but there are still a few areas where it supports features not supported in other browsers, or conforms better than other browsers. I think we should keep it for a while longer. As it provides useful information to the people working on the spec or the tests.
>
> Besides, there's nothing wrong with a little tongue-in-cheek challenge to other browsers ("Even old Presto gets this right!”).

No reason to remove old passing implementations… I’ll take passes from Lynx if it helps get a spec to CR.

signature.asc (506 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Browser detection for shepherd/css test runner

Chris Lilley
In reply to this post by Christian Biesinger-3
Hello Christian,

Friday, March 27, 2015, 10:09:10 PM, you wrote:

> Hi there,

> looking at the icons on
> http://dev.w3.org/csswg/css-flexbox/#abspos-items, and test results
> like
> http://test.csswg.org/harness/details/css-flexbox-1_dev/flexbox-abspos-child-001b/engine/webkit/,
> it seems that the test runner considers Chrome to be Webkit.

> That is bad. Chrome does not use Webkit anymore,

Certainly on Windows, Linux, and Android: Chrome is not Webkit any more
and the level of divergence is now reasonably large.  We should
re-evaluate whether we treat these as independent implementations in
terms of test suite passes.

(On OSX and iOS Chrome is indeed Apple's Safari Webkit because no
other web engine is allowed on those locked-down platforms).



--
Best regards,
 Chris Lilley, Technical Director, W3C Interaction Domain


Reply | Threaded
Open this post in threaded view
|

Re: Browser detection for shepherd/css test runner

Florian Rivoal-4

> On 28 Mar 2015, at 13:02, Chris Lilley <[hidden email]> wrote:
>
> (On OSX and iOS Chrome is indeed Apple's Safari Webkit because no
> other web engine is allowed on those locked-down platforms).

Chrome uses webkit on iOS only. OSX isn't locked down the same way.

 - Florian

Reply | Threaded
Open this post in threaded view
|

Re: Browser detection for shepherd/css test runner

Koji Ishii
In reply to this post by Peter Linss
On Mar 28, 2015, at 10:45, Peter Linss <[hidden email]> wrote:
>
> The problem is, while there’s divergence in some areas, there’s still shared code in many (most?), and it currently requires human judgement to determine if passes form both count as two independent implementations or not.
>
> If someone wants to generate a map (keyed by spec section) where there’s divergence (or the converse would be better, since that set isn’t getting bigger), I’d be happy to have the test harness automatically differentiate the results.
>
> For the record, each result is keyed to the full UA string that generated it, so we can always go back and re-assign results to different products.

I agree with you on this point, but human can always make a judge to count two columns as one if we think the two implementations still share the code for each specification. I think we should separate whether we’d like to count them as one or two from whether we’d like to have separate icons/columns. The former needs human judgement for each feature, but I do not see any down side for doing the latter.

I was actually thinking to propose the same when I’m working on writing-modes fixes and am troubled to figure out which tests fail on Blink.

Could we consider the change?

> No reason to remove old passing implementations… I’ll take passes from Lynx if it helps get a spec to CR.

Agreed.

/koji


Reply | Threaded
Open this post in threaded view
|

Re: Browser detection for shepherd/css test runner

Christian Biesinger-3
In reply to this post by Peter Linss
On Fri, Mar 27, 2015 at 9:45 PM, Peter Linss <[hidden email]> wrote:
> The problem is, while there’s divergence in some areas, there’s still shared code in many (most?), and it currently requires human judgement to determine if passes form both count as two independent implementations or not.

Yes of course. I guess what you are implying is that the intent of
these boxes is to see at a glance whether there's enough
implementations to go from CR to REC...?

To me, it was more useful to see how various browsers do, and in some
cases (Flexbox, Grid, probably other more recent CSS specs) the
implementations do diverge and may easily pass different tests;
whether or not they should count as independent implementations is a
different issue.

> If someone wants to generate a map (keyed by spec section) where there’s divergence (or the converse would be better, since that set isn’t getting bigger), I’d be happy to have the test harness automatically differentiate the results.
>
> For the record, each result is keyed to the full UA string that generated it, so we can always go back and re-assign results to different products.
>
>>
>> As for Presto, it is certainly of quickly declining relevance, but there are still a few areas where it supports features not supported in other browsers, or conforms better than other browsers. I think we should keep it for a while longer. As it provides useful information to the people working on the spec or the tests.
>>
>> Besides, there's nothing wrong with a little tongue-in-cheek challenge to other browsers ("Even old Presto gets this right!”).
>
> No reason to remove old passing implementations… I’ll take passes from Lynx if it helps get a spec to CR.

Fair enough. Is the UA detection good enough to not treat current
Opera as Presto?

-christian

Reply | Threaded
Open this post in threaded view
|

Re: Browser detection for shepherd/css test runner

Peter Linss

On Mar 31, 2015, at 2:43 PM, Christian Biesinger <[hidden email]> wrote:

> On Fri, Mar 27, 2015 at 9:45 PM, Peter Linss <[hidden email]> wrote:
>> The problem is, while there’s divergence in some areas, there’s still shared code in many (most?), and it currently requires human judgement to determine if passes form both count as two independent implementations or not.
>
> Yes of course. I guess what you are implying is that the intent of
> these boxes is to see at a glance whether there's enough
> implementations to go from CR to REC...?

Correct, that was the primary purpose of the test harness. You'll notice after the table counts of how many passes are required to reach CR exit criteria.

>
> To me, it was more useful to see how various browsers do, and in some
> cases (Flexbox, Grid, probably other more recent CSS specs) the
> implementations do diverge and may easily pass different tests;
> whether or not they should count as independent implementations is a
> different issue.

Understood, one a spec exit's CR, the focus of the test suite does switch to conformance testing. At some point I can take a look at augmenting the system to display WebKit and Blink in different columns, but count them as one implementation. Ideally I'd like a mechanism to know where they should be considered the same vs independent...

>
>> If someone wants to generate a map (keyed by spec section) where there’s divergence (or the converse would be better, since that set isn’t getting bigger), I’d be happy to have the test harness automatically differentiate the results.
>>
>> For the record, each result is keyed to the full UA string that generated it, so we can always go back and re-assign results to different products.
>>
>>>
>>> As for Presto, it is certainly of quickly declining relevance, but there are still a few areas where it supports features not supported in other browsers, or conforms better than other browsers. I think we should keep it for a while longer. As it provides useful information to the people working on the spec or the tests.
>>>
>>> Besides, there's nothing wrong with a little tongue-in-cheek challenge to other browsers ("Even old Presto gets this right!”).
>>
>> No reason to remove old passing implementations… I’ll take passes from Lynx if it helps get a spec to CR.
>
> Fair enough. Is the UA detection good enough to not treat current
> Opera as Presto?
Yes, it goes by UA string. Opera's current UA string only mentions WebKit so that's what it's counted as. When we treat WebKit and Blink differently I can add some special case code to consider that Blink (though it would be best if the Opera US string actually listed Blink, for matter if Chrome would too...)

signature.asc (506 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Browser detection for shepherd/css test runner

Christian Biesinger-3
So, Webkit and Blink are diverging more and more. Could we update the
icons on https://drafts.csswg.org/css-flexbox/ and probably elsewhere
to separate out Blink from Webkit? For the purpose of CR exit, humans
can always make the judgement on whether they should count as
different implementations for that purpose.

-Christian

On Tue, Mar 31, 2015 at 3:31 PM, Linss, Peter <[hidden email]> wrote:

>
> On Mar 31, 2015, at 2:43 PM, Christian Biesinger <[hidden email]> wrote:
>
>> On Fri, Mar 27, 2015 at 9:45 PM, Peter Linss <[hidden email]> wrote:
>>> The problem is, while there’s divergence in some areas, there’s still shared code in many (most?), and it currently requires human judgement to determine if passes form both count as two independent implementations or not.
>>
>> Yes of course. I guess what you are implying is that the intent of
>> these boxes is to see at a glance whether there's enough
>> implementations to go from CR to REC...?
>
> Correct, that was the primary purpose of the test harness. You'll notice after the table counts of how many passes are required to reach CR exit criteria.
>
>>
>> To me, it was more useful to see how various browsers do, and in some
>> cases (Flexbox, Grid, probably other more recent CSS specs) the
>> implementations do diverge and may easily pass different tests;
>> whether or not they should count as independent implementations is a
>> different issue.
>
> Understood, one a spec exit's CR, the focus of the test suite does switch to conformance testing. At some point I can take a look at augmenting the system to display WebKit and Blink in different columns, but count them as one implementation. Ideally I'd like a mechanism to know where they should be considered the same vs independent...
>
>>
>>> If someone wants to generate a map (keyed by spec section) where there’s divergence (or the converse would be better, since that set isn’t getting bigger), I’d be happy to have the test harness automatically differentiate the results.
>>>
>>> For the record, each result is keyed to the full UA string that generated it, so we can always go back and re-assign results to different products.
>>>
>>>>
>>>> As for Presto, it is certainly of quickly declining relevance, but there are still a few areas where it supports features not supported in other browsers, or conforms better than other browsers. I think we should keep it for a while longer. As it provides useful information to the people working on the spec or the tests.
>>>>
>>>> Besides, there's nothing wrong with a little tongue-in-cheek challenge to other browsers ("Even old Presto gets this right!”).
>>>
>>> No reason to remove old passing implementations… I’ll take passes from Lynx if it helps get a spec to CR.
>>
>> Fair enough. Is the UA detection good enough to not treat current
>> Opera as Presto?
>
> Yes, it goes by UA string. Opera's current UA string only mentions WebKit so that's what it's counted as. When we treat WebKit and Blink differently I can add some special case code to consider that Blink (though it would be best if the Opera US string actually listed Blink, for matter if Chrome would too...)

Reply | Threaded
Open this post in threaded view
|

Re: Browser detection for shepherd/css test runner

Christian Biesinger-3
Another ping on this... anyone have thoughts at least?!

-Christian

On Thu, Mar 31, 2016 at 4:47 PM, Christian Biesinger <[hidden email]> wrote:
So, Webkit and Blink are diverging more and more. Could we update the
icons on https://drafts.csswg.org/css-flexbox/ and probably elsewhere
to separate out Blink from Webkit? For the purpose of CR exit, humans
can always make the judgement on whether they should count as
different implementations for that purpose.

-Christian

On Tue, Mar 31, 2015 at 3:31 PM, Linss, Peter <[hidden email]> wrote:
>
> On Mar 31, 2015, at 2:43 PM, Christian Biesinger <[hidden email]> wrote:
>
>> On Fri, Mar 27, 2015 at 9:45 PM, Peter Linss <[hidden email]> wrote:
>>> The problem is, while there’s divergence in some areas, there’s still shared code in many (most?), and it currently requires human judgement to determine if passes form both count as two independent implementations or not.
>>
>> Yes of course. I guess what you are implying is that the intent of
>> these boxes is to see at a glance whether there's enough
>> implementations to go from CR to REC...?
>
> Correct, that was the primary purpose of the test harness. You'll notice after the table counts of how many passes are required to reach CR exit criteria.
>
>>
>> To me, it was more useful to see how various browsers do, and in some
>> cases (Flexbox, Grid, probably other more recent CSS specs) the
>> implementations do diverge and may easily pass different tests;
>> whether or not they should count as independent implementations is a
>> different issue.
>
> Understood, one a spec exit's CR, the focus of the test suite does switch to conformance testing. At some point I can take a look at augmenting the system to display WebKit and Blink in different columns, but count them as one implementation. Ideally I'd like a mechanism to know where they should be considered the same vs independent...
>
>>
>>> If someone wants to generate a map (keyed by spec section) where there’s divergence (or the converse would be better, since that set isn’t getting bigger), I’d be happy to have the test harness automatically differentiate the results.
>>>
>>> For the record, each result is keyed to the full UA string that generated it, so we can always go back and re-assign results to different products.
>>>
>>>>
>>>> As for Presto, it is certainly of quickly declining relevance, but there are still a few areas where it supports features not supported in other browsers, or conforms better than other browsers. I think we should keep it for a while longer. As it provides useful information to the people working on the spec or the tests.
>>>>
>>>> Besides, there's nothing wrong with a little tongue-in-cheek challenge to other browsers ("Even old Presto gets this right!”).
>>>
>>> No reason to remove old passing implementations… I’ll take passes from Lynx if it helps get a spec to CR.
>>
>> Fair enough. Is the UA detection good enough to not treat current
>> Opera as Presto?
>
> Yes, it goes by UA string. Opera's current UA string only mentions WebKit so that's what it's counted as. When we treat WebKit and Blink differently I can add some special case code to consider that Blink (though it would be best if the Opera US string actually listed Blink, for matter if Chrome would too...)