Deprecating the old pubrules on Aug 1st, 2016

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Deprecating the old pubrules on Aug 1st, 2016

Denis Ah-Kang
Hi,

When the 2014 process document was released [1], we agreed on a 2-year
transition period in which WGs could still operate under the 2005
process. Starting from August 1st, all the WGs must use the 2015
process document.
Since we no longer want to invest resources in maintaining the old
pubrules [2], we think the end of the transition period is a good
opportunity to make Specberus [3] the official TR document checker.

The latest version of Specberus is now available at:
https://www.w3.org/pubrules/

That version is still missing a few pieces (docs and rules) and contains
a couple of bugs but we will take care of fixing everything in the next
days.

The major changes you will see are:
- a new UI
- only the process document 2015 supported
- only HTML5 documents supported. Since 2015, only a handful of
documents are still being published in HTML4 or XHTML1.0, and since
supporting these doctypes is too costly in terms of resources, we
prefer to help people migrate their documents to HTML5
- the service getdocrdf (e.g. [4]) being replaced with an API producing
JSON. If you rely on that service, do let me know and I will show you
how to do the migration. Note, tr.rdf remains untouched.

In terms of new features, Specberus provides a REST API [5] you can use
to check your documents (e.g. [6]) or extract their metadata (e.g. [7]).

If you are planning to publish new documents on /TR in the upcoming
weeks, I strongly suggest you to run it through the new pubrules and
report bugs on github [8].

Marcos, Shane and Tab, Echidna is already using Specberus to check
documents before publishing them so I don't expect major problem
for Respec and Bikeshed but let me know if I overlooked something.

As usual, feel free to ping us on irc (#pub) if there's any problem
with the new pubrules.

Denis


[1] https://lists.w3.org/Archives/Member/chairs/2014JulSep/0048.html
[2] https://www.w3.org/2005/07/pubrules
[3] https://github.com/w3c/specberus
[4]
https://www.w3.org/2001/10/getdocrdf?docAddr=https://www.w3.org/TR/2016/WD-reporting-1-20160407/
[5] https://github.com/w3c/specberus/blob/master/README.md#5-rest-api
[6]
https://www.w3.org/pubrules/api/validate?url=http://www.w3.org/TR/2016/WD-charmod-norm-20160407/&profile=auto
[7]
https://www.w3.org/pubrules/api/metadata?url=http://www.w3.org/TR/2016/WD-charmod-norm-20160407/&profile=auto
[8] https://github.com/w3c/specberus/issues

Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Shane McCarron-6
Just to clarify something...

I assume the policy for document format is that the PRIMARY format must be HTML5.  It remains the case that if a document has alternative versions in whatever format, those will continue to be permitted.  We often include PDF or EPub versions of Recommendations.  Moreover, if we update the RDFa family of Recommendations again, we would of course include a version of XHTML+RDFa that is encoded in XHTML+RDFa.  


On Thu, Jun 2, 2016 at 6:41 AM, Denis Ah-Kang <[hidden email]> wrote:
Hi,

When the 2014 process document was released [1], we agreed on a 2-year
transition period in which WGs could still operate under the 2005
process. Starting from August 1st, all the WGs must use the 2015
process document.
Since we no longer want to invest resources in maintaining the old
pubrules [2], we think the end of the transition period is a good
opportunity to make Specberus [3] the official TR document checker.

The latest version of Specberus is now available at:
https://www.w3.org/pubrules/

That version is still missing a few pieces (docs and rules) and contains
a couple of bugs but we will take care of fixing everything in the next
days.

The major changes you will see are:
- a new UI
- only the process document 2015 supported
- only HTML5 documents supported. Since 2015, only a handful of
documents are still being published in HTML4 or XHTML1.0, and since
supporting these doctypes is too costly in terms of resources, we
prefer to help people migrate their documents to HTML5
- the service getdocrdf (e.g. [4]) being replaced with an API producing
JSON. If you rely on that service, do let me know and I will show you
how to do the migration. Note, tr.rdf remains untouched.

In terms of new features, Specberus provides a REST API [5] you can use
to check your documents (e.g. [6]) or extract their metadata (e.g. [7]).

If you are planning to publish new documents on /TR in the upcoming
weeks, I strongly suggest you to run it through the new pubrules and
report bugs on github [8].

Marcos, Shane and Tab, Echidna is already using Specberus to check
documents before publishing them so I don't expect major problem
for Respec and Bikeshed but let me know if I overlooked something.

As usual, feel free to ping us on irc (#pub) if there's any problem
with the new pubrules.

Denis


[1] https://lists.w3.org/Archives/Member/chairs/2014JulSep/0048.html
[2] https://www.w3.org/2005/07/pubrules
[3] https://github.com/w3c/specberus
[4]
https://www.w3.org/2001/10/getdocrdf?docAddr=https://www.w3.org/TR/2016/WD-reporting-1-20160407/
[5] https://github.com/w3c/specberus/blob/master/README.md#5-rest-api
[6]
https://www.w3.org/pubrules/api/validate?url=http://www.w3.org/TR/2016/WD-charmod-norm-20160407/&profile=auto
[7]
https://www.w3.org/pubrules/api/metadata?url=http://www.w3.org/TR/2016/WD-charmod-norm-20160407/&profile=auto
[8] https://github.com/w3c/specberus/issues



--
Shane McCarron
Projects Manager, Spec-Ops
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Philippe Le Hegaret


On 06/02/2016 11:08 AM, Shane McCarron wrote:
> Just to clarify something...
>
> I assume the policy for document format is that the PRIMARY format must
> be HTML5.  It remains the case that if a document has alternative
> versions in whatever format, those will continue to be permitted.  We
> often include PDF or EPub versions of Recommendations.

Correct. This change doesn't impact alternative versions.

>  Moreover, if we
> update the RDFa family of Recommendations again, we would of course
> include a version of XHTML+RDFa that is encoded in XHTML+RDFa.
>
> [1] https://www.w3.org/TR/xhtml-rdfa/

The main bottleneck/limitation for us is the validation. If you get your
primary document to validate, then you don't have to worry. If you don't
get it to validate, then it's a different story and we'd need to look at
the specifics. It might be that we could allow the exception and/or
conclude that the validator needs an upgrade. We refrain as much as
possible from granting exceptions because otherwise we might as well
give up on the rule, which would trigger a set of consequences.

Philippe

Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Shane McCarron-6


On Thu, Jun 2, 2016 at 10:20 AM, Philippe Le Hegaret <[hidden email]> wrote:


 Moreover, if we
update the RDFa family of Recommendations again, we would of course
include a version of XHTML+RDFa that is encoded in XHTML+RDFa.

[1] https://www.w3.org/TR/xhtml-rdfa/

The main bottleneck/limitation for us is the validation. If you get your primary document to validate, then you don't have to worry. If you don't get it to validate, then it's a different story and we'd need to look at the specifics. It might be that we could allow the exception and/or conclude that the validator needs an upgrade. We refrain as much as possible from granting exceptions because otherwise we might as well give up on the rule, which would trigger a set of consequences.


Of course. 

I do have a concern about HTML5 extension specifications as well as with bits  of HTML5 that are in WhatWG specs but NOT in W3C specs.

Does the validator have a mode that only permits W3C-approved HTML5?
 

Shane McCarron
Projects Manager, Spec-Ops
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Philippe Le Hegaret
On 06/02/2016 11:22 AM, Shane McCarron wrote:
> I do have a concern about HTML5 extension specifications as well as with
> bits  of HTML5 that are in WhatWG specs but NOT in W3C specs.
>
> Does the validator have a mode that only permits W3C-approved HTML5?

The first concern here is: we want to make sure that the community at
large can read the document.

So, it's not ok to use extensions or markup that are not widely
supported, since it impacts the readers. The validator helps us in that.
We also favor using markup from Recommendation rather than Working
Drafts but again, that's first a readability issue for us. We authorized
the use of HTML5 before it became a recommendation for example (ie it
wasn't "W3C-approved" yet). If your recommendation has some obscure
implementations and readers won't be able to read your specification,
then our advise is for you to use fallbacks as well (see MathML for
example).

If I remember correctly, the validator does have a W3C profile for HTML5.

Philippe

Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Michael[tm] Smith
In reply to this post by Shane McCarron-6
Shane McCarron <[hidden email]>, 2016-06-02 10:22 -0500:
> Archived-At: <http://www.w3.org/mid/CAJdbnOC4PycuM4LJQVMqKHVNh4gJ0sFhGs1JuouXmP3Y5UXhag@...>
> ...
> Does the validator have a mode that only permits W3C-approved HTML5?

The https://validator.w3.org/nu/ checker implements W3C-conforming checks
in any cases where the W3C HTML fork states requirements that differ from
the upstream HTML spec

  —Mike

--
Michael[tm] Smith https://people.w3.org/mike

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Shane McCarron-6


On Thu, Jun 2, 2016 at 10:46 AM, Michael[tm] Smith <[hidden email]> wrote:
Shane McCarron <[hidden email]>, 2016-06-02 10:22 -0500:
> Archived-At: <http://www.w3.org/mid/CAJdbnOC4PycuM4LJQVMqKHVNh4gJ0sFhGs1JuouXmP3Y5UXhag@...>
> ...
> Does the validator have a mode that only permits W3C-approved HTML5?

The https://validator.w3.org/nu/ checker implements W3C-conforming checks
in any cases where the W3C HTML fork states requirements that differ from
the upstream HTML spec

 
Good to know.   What about "extensions" that W3C has approved (e.g., longdesc) or things in the WhatWG version that are not in the W3C version (e.g., microdata)?


--
Shane McCarron
Projects Manager, Spec-Ops
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Michael[tm] Smith
Shane McCarron <[hidden email]>, 2016-06-02 10:50 -0500:

>
> On Thu, Jun 2, 2016 at 10:46 AM, Michael[tm] Smith <[hidden email]> wrote:
>
> > Shane McCarron <[hidden email]>, 2016-06-02 10:22 -0500:
> > > Archived-At: <
> > http://www.w3.org/mid/CAJdbnOC4PycuM4LJQVMqKHVNh4gJ0sFhGs1JuouXmP3Y5UXhag@...
> > >
> > > ...
> > > Does the validator have a mode that only permits W3C-approved HTML5?
> >
> > The https://validator.w3.org/nu/ checker implements W3C-conforming checks
> > in any cases where the W3C HTML fork states requirements that differ from
> > the upstream HTML spec
> >
> >
> Good to know.   What about "extensions" that W3C has approved (e.g.,
> longdesc)
The W3C HTML checker supports longdesc

> or things in the WhatWG version that are not in the W3C version
> (e.g., microdata)?

schema.org supports (and even promotes) the use of microdata, and many
authors and orgs are using it producively. I hope you’re not suggesting it
would be a good idea to start having the W3C HTML checker flag errors in
the millions of schema.org-conforming documents containing microdata that
authors have created based on the schema.org guidelines

  —Mike

--
Michael[tm] Smith https://people.w3.org/mike

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Shane McCarron-6


On Thu, Jun 2, 2016 at 10:57 AM, Michael[tm] Smith <[hidden email]> wrote:


> or things in the WhatWG version that are not in the W3C version
> (e.g., microdata)?

schema.org supports (and even promotes) the use of microdata, and many
authors and orgs are using it producively. I hope you’re not suggesting it
would be a good idea to start having the W3C HTML checker flag errors in
the millions of schema.org-conforming documents containing microdata that
authors have created based on the schema.org guidelines


Oh - personally I would be happy with that change as I think Microdata is pretty broken.  But no, what I am asking is if the W3C profile that we use for validating W3C publications restricts the checking to things that are actually approved by the W3C.  W3C recommendations should not be using microdata - at least not in the primary format that we are checking as part of pub rules.  I imagine there are lots of other things that are getting thrown into the WhatWG version of HTML that are also not included in HTML as Recommended by the W3C.  We should not be permitting those in our formal publications either.

--
Shane McCarron
Projects Manager, Spec-Ops
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Marcos Caceres-5


On 3 Jun 2016, at 2:31 AM, Shane McCarron <[hidden email]> wrote:



On Thu, Jun 2, 2016 at 10:57 AM, Michael[tm] Smith <[hidden email]> wrote:


> or things in the WhatWG version that are not in the W3C version
> (e.g., microdata)?

schema.org supports (and even promotes) the use of microdata, and many
authors and orgs are using it producively. I hope you’re not suggesting it
would be a good idea to start having the W3C HTML checker flag errors in
the millions of schema.org-conforming documents containing microdata that
authors have created based on the schema.org guidelines


Oh - personally I would be happy with that change as I think Microdata is pretty broken.  But no, what I am asking is if the W3C profile that we use for validating W3C publications restricts the checking to things that are actually approved by the W3C.  W3C recommendations should not be using microdata - at least not in the primary format that we are checking as part of pub rules.  I imagine there are lots of other things that are getting thrown into the WhatWG version of HTML that are also not included in HTML as Recommended by the W3C.  We should not be permitting those in our formal publications either.

That's silly: we should, as PLH, said, use what browsers can render. We shouldn't use PubRules for petty WHATWG vs W3C HTML battles as that only harms users. 


--
Shane McCarron
Projects Manager, Spec-Ops
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Michael[tm] Smith
In reply to this post by Shane McCarron-6
Shane McCarron <[hidden email]>, 2016-06-02 11:31 -0500:

> Archived-At: <http://www.w3.org/mid/CAJdbnOA8Ey4wMZSRjtV1hHA3yKVrP=ofwc4kRm0rqSF4b3PNKQ@...>
>
> On Thu, Jun 2, 2016 at 10:57 AM, Michael[tm] Smith <[hidden email]> wrote:
> >
> > schema.org supports (and even promotes) the use of microdata, and many
> > authors and orgs are using it producively. I hope you’re not suggesting it
> > would be a good idea to start having the W3C HTML checker flag errors in
> > the millions of schema.org-conforming documents containing microdata that
> > authors have created based on the schema.org guidelines
> >
> Oh - personally I would be happy with that change as I think Microdata is
> pretty broken.
While I can understand very well might be happy with it personally, I doubt
the tens or hundreds of thousands of authors out there who are using it
productively and are getting the results from it that solve problems for
them would be happy about it in the way you are personally.

> But no, what I am asking is if the W3C profile that we use
> for validating W3C publications restricts the checking to things that are
> actually approved by the W3C.  W3C recommendations should not be using
> microdata - at least not in the primary format that we are checking as part
> of pub rules.

Are you aware of any that actually are? Are there any real-world actual
cases of that? And in the long list of problems that we’re here to try to
solve together, is this really one that you want to prioritize?

I work on the W3C HTML checker on my own dime, and on my own time, and your
partisan agitation about this seriously bums me out and de-motivates me
from feeling enthusiastic about working with you on getting other actual
real problems solved together.

But if you want me to flip the bit completely, go ahead and keep it up.

> I imagine there are lots of other things that are getting
> thrown into the WhatWG version of HTML that are also not included in HTML
> as Recommended by the W3C.

Yeah? You imagine? Maybe rather than imagining you could find specific cases
of that. Ones that are actually causing any real problems from anybody?

Or better yet, maybe you can use your imagination to think about actual
real problems to spend your time trying to help solve.

  —Mike

> We should not be permitting those in our formal
> publications either.

--
Michael[tm] Smith https://people.w3.org/mike

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Shane McCarron-6


On Thu, Jun 2, 2016 at 12:00 PM, Michael[tm] Smith <[hidden email]> wrote:
Shane McCarron <[hidden email]>, 2016-06-02 11:31 -0500:
> Archived-At: <http://www.w3.org/mid/CAJdbnOA8Ey4wMZSRjtV1hHA3yKVrP=ofwc4kRm0rqSF4b3PNKQ@...>
>


> But no, what I am asking is if the W3C profile that we use
> for validating W3C publications restricts the checking to things that are
> actually approved by the W3C.  W3C recommendations should not be using
> microdata - at least not in the primary format that we are checking as part
> of pub rules.

Are you aware of any that actually are? Are there any real-world actual
cases of that? And in the long list of problems that we’re here to try to
solve together, is this really one that you want to prioritize?

Real-world cases of what?  Specs using features that are not part of the baseline W3C recommendations?  I have no idea.  I brought this up because the W3C pubrules have always been super restrictive in what can be included in a formal publication (at least once it is past the WD stage). Microdata was an example that is fresh in my mind.  I could have mentioned any of the new elements coming in via extensions or being added / changed in HTML 5.1.  It's the same problem.  

The stated (and probably reasonable) rationale for these restrictions in formal publications from the W3C has always been that we must ensure everyone can access our content.  So we cannot rely upon features that are not broadly supported.  That doesn't just mean in the latest user agents.  It also means in assistive technologies, tool chains, translation engines, search engines, what have you.  

There is no way I am saying anything here that should come as a surprise to anyone who has been involved in making standards.


I work on the W3C HTML checker on my own dime, and on my own time, and your
partisan agitation about this seriously bums me out and de-motivates me
from feeling enthusiastic about working with you on getting other actual
real problems solved together.

But if you want me to flip the bit completely, go ahead and keep it up.

Partisan agitation?  I am sorry that you feel that way.  I was just trying to understand the scope of the publication rules changes and how they are going to be enforced.  

And fwiw, I also work on this on my own dime. 
 

> I imagine there are lots of other things that are getting
> thrown into the WhatWG version of HTML that are also not included in HTML
> as Recommended by the W3C.

Yeah? You imagine? Maybe rather than imagining you could find specific cases
of that. Ones that are actually causing any real problems from anybody?

I don't believe for a minute that any of this nonsense causes any real problems for anyone (at least outside of ATs).  Pubrules has always been a draconian mess.  Basically the bane of my existence for the last 15 years.  Every. Single. Time. I go to publish a spec I run into some crap that makes me waste hours / days sorting it out.  
I am attempting to explore what sort of fresh hell the new changes will bring.  And ensure that the tools and specifications I maintain are in a good place to help me and others avoid spending too much time there.  I am sorry if exploring that offends you.  If you read back you will see that I was not advocating for anything, nor asking for any changes.  I was just asking how it will work. Because I honestly have no idea.
 

Or better yet, maybe you can use your imagination to think about actual
real problems to spend your time trying to help solve.


Seriously?  That was unworthy.   

 

--
Shane McCarron
Projects Manager, Spec-Ops
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Philippe Le Hegaret
On 06/02/2016 01:29 PM, Shane McCarron wrote:
> I was just
> trying to understand the scope of the publication rules changes and how
> they are going to be enforced.

We're changing from:

[[
  All normative representations MUST validate as one of the following:
HTML 4.x, or some version of XHTML or XHTML+RDFa that is a W3C
Recommendation. HTML5 is also permitted with the following limitations:

     Inline markup for SVG 1.1 or MathML 2.0 is permitted but only with
a (fallback) alternative.
     If the HTML5 validator issues content warnings, the publication
request must include rationale why the warning is not problematic.

Note: Please consider how your content will render in browsers that do
not support HTML5.
]]

to

[[
All normative representations MUST validate as HTML5 with the following
limitations:
   Inline markup for MathML is permitted but should use a (fallback)
alternative.
   If the HTML5 validator issues content warnings, the publication
request must include rationale why the warning is not problematic.
]]
https://github.com/w3c/specberus/pull/388

This check happens in:
  https://github.com/w3c/specberus/blob/master/lib/rules/validation/html.js

I will note that the current XHTML+RDFa recommendation is ok with check:

 
https://validator.w3.org/nu/?doc=https%3A%2F%2Fwww.w3.org%2FTR%2Fxhtml-rdfa%2F

Philippe


Reply | Threaded
Open this post in threaded view
|

Re: Deprecating the old pubrules on Aug 1st, 2016

Shane McCarron-6


On Thu, Jun 2, 2016 at 12:42 PM, Philippe Le Hegaret <[hidden email]> wrote:

[[
All normative representations MUST validate as HTML5 with the following limitations:
  Inline markup for MathML is permitted but should use a (fallback) alternative.
  If the HTML5 validator issues content warnings, the publication request must include rationale why the warning is not problematic.
]]

Sure.  So my follow-up (but non-specific) question was "which profile of html5"?  To which the answer was that there is a W3C profile of html5 in the validator. Which is great, and I assume what is used.  My follow-up question to that was if that profile includes things that are not part of the baseline HTML5.  

Obviously I can just poke at it to find out.  But I was hoping someone more clueful than I would  just have a definitive answer.

--
Shane McCarron
Projects Manager, Spec-Ops