where is the ratified sparql 1.1 test suite?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

where is the ratified sparql 1.1 test suite?

james anderson-19
good evening,

i find this solution in the document held by the w3c at <http://www.w3.org/2009/sparql/docs/tests/data-sparql11/functions/strbefore01a.srx> .

                <result>
                        <binding name="s"><uri>http://example.org/s2</uri></binding>
                        <binding name="prefix"><literal></literal></binding>
                </result>

i find this solution in the respective document contained in the tar archive, <http://www.w3.org/2009/sparql/docs/tests/sparql11-test-suite-20121023.tar.gz>, a link to which is included in the <http://www.w3.org/2009/sparql/docs/tests/> .

                <result>
                        <binding name="s"><uri>http://example.org/s2</uri></binding>
                        <binding name="prefix"><literal xml:lang="ja"></literal></binding>
                </result>

this latter depicts a result which does not agree with the definition in the recommendation. the same situation applies to the result document, "strafter01.srx".
there was discussion[1] related to this during the ratification process, but the messages indicate no conclusion with respect to the test documents themselves

is there any archive file which comprises the ratified test suite?

best regards, from berlin,


[1] https://lists.w3.org/Archives/Public/public-rdf-dawg-comments/2012Jun/0050.html
---
james anderson | [hidden email] | http://dydra.com






Reply | Threaded
Open this post in threaded view
|

Re: where is the ratified sparql 1.1 test suite?

Gregory Williams
On May 17, 2015, at 2:11 PM, james anderson <[hidden email]> wrote:

>
> good evening,
>
> i find this solution in the document held by the w3c at <http://www.w3.org/2009/sparql/docs/tests/data-sparql11/functions/strbefore01a.srx> .
>
> <result>
> <binding name="s"><uri>http://example.org/s2</uri></binding>
> <binding name="prefix"><literal></literal></binding>
> </result>
>
> i find this solution in the respective document contained in the tar archive, <http://www.w3.org/2009/sparql/docs/tests/sparql11-test-suite-20121023.tar.gz>, a link to which is included in the <http://www.w3.org/2009/sparql/docs/tests/> .
>
> <result>
> <binding name="s"><uri>http://example.org/s2</uri></binding>
> <binding name="prefix"><literal xml:lang="ja"></literal></binding>
> </result>
>
> this latter depicts a result which does not agree with the definition in the recommendation. the same situation applies to the result document, "strafter01.srx".
> there was discussion[1] related to this during the ratification process, but the messages indicate no conclusion with respect to the test documents themselves
>
> is there any archive file which comprises the ratified test suite?

James,

I believe that the problem you’re referring to here is the difference between “” and “”@ja in the results files? I’m not able to find the language-tagged (bad) version of that data in the approved test suite. The tests :strbefore01a and :strafter01a both reference seemingly-valid result files (strbefore01a.srx and strafter01a.srx).

However, the CVS repository and tarball of the test suite also contain old files that contain the invalid data (strbefore01.srx and strafter01.srx). This is unfortunate, but shouldn’t cause problems so long as you are using the manifest files to find approved tests and their associated files.

.greg


Reply | Threaded
Open this post in threaded view
|

Re: where is the ratified sparql 1.1 test suite?

james anderson-19
good evening,

thank you for your note.

On 2015-05-18, at 16:01, Gregory Williams <[hidden email]> wrote:

On May 17, 2015, at 2:11 PM, james anderson <[hidden email]> wrote:

good evening,

i find this solution in the document held by the w3c at <http://www.w3.org/2009/sparql/docs/tests/data-sparql11/functions/strbefore01a.srx> .

<result>
<binding name="s"><uri>http://example.org/s2</uri></binding>
<binding name="prefix"><literal></literal></binding>
</result>

i find this solution in the respective document contained in the tar archive, <http://www.w3.org/2009/sparql/docs/tests/sparql11-test-suite-20121023.tar.gz>, a link to which is included in the <http://www.w3.org/2009/sparql/docs/tests/> .

<result>
<binding name="s"><uri>http://example.org/s2</uri></binding>
<binding name="prefix"><literal xml:lang="ja"></literal></binding>
</result>

this latter depicts a result which does not agree with the definition in the recommendation. the same situation applies to the result document, "strafter01.srx".
there was discussion[1] related to this during the ratification process, but the messages indicate no conclusion with respect to the test documents themselves

is there any archive file which comprises the ratified test suite?

James,

I believe that the problem you’re referring to here is the difference between “” and “”@ja in the results files?

yes, in one. in the other there is also a french language tag which would appear to be anomalous.

I’m not able to find the language-tagged (bad) version of that data in the approved test suite.

you raise, in passing, the proximate question: what is “the approved test suite”? (see below)

The tests :strbefore01a and :strafter01a both reference seemingly-valid result files (strbefore01a.srx and strafter01a.srx).

in the files served directly on the site, yes.
in the tar archive no.


However, the CVS repository and tarball of the test suite also contain old files that contain the invalid data (strbefore01.srx and strafter01.srx). This is unfortunate, but shouldn’t cause problems so long as you are using the manifest files to find approved tests and their associated files.

if i were to run straight from the net, i might be persuaded to agree with you.
that practice suffers, however, from two deficiencies:
- it is quite circumstantial, in that one cannot point to an object and indicate compliance with it, but can only say “hey, that’s what was being served on dddd-dd-dd@tt:tt:tt”
- there have been innumerable occasions over the past days when w3c’s web front-end decided to no longer serve the content, which makes it difficult to run tests in that mode. sometimes for days.

i could always wget and set up our own git repository, but having observed any number of those already in the wild - each of unknown provenance and with unknown content, that does not seem to be a well-considered approach.

best regards, from berlin,

.greg


---
james anderson | [hidden email] | http://dydra.com





Reply | Threaded
Open this post in threaded view
|

Re: where is the ratified sparql 1.1 test suite?

Esko Nuutila-2
In reply to this post by Gregory Williams
Hi everyone,

I have had similar problems with the tests. It would be very nice, if someone would take care that the up-to-date tests (and just those) were always available in the tarballs.

/Esko

On Mon, May 18, 2015 at 5:28 PM, james anderson <[hidden email]> wrote:
good evening,

thank you for your note.

On 2015-05-18, at 16:01, Gregory Williams <[hidden email]> wrote:

On May 17, 2015, at 2:11 PM, james anderson <[hidden email]> wrote:

good evening,

i find this solution in the document held by the w3c at <http://www.w3.org/2009/sparql/docs/tests/data-sparql11/functions/strbefore01a.srx> .

<result>
<binding name="s"><uri>http://example.org/s2</uri></binding>
<binding name="prefix"><literal></literal></binding>
</result>

i find this solution in the respective document contained in the tar archive, <http://www.w3.org/2009/sparql/docs/tests/sparql11-test-suite-20121023.tar.gz>, a link to which is included in the <http://www.w3.org/2009/sparql/docs/tests/> .

<result>
<binding name="s"><uri>http://example.org/s2</uri></binding>
<binding name="prefix"><literal xml:lang="ja"></literal></binding>
</result>

this latter depicts a result which does not agree with the definition in the recommendation. the same situation applies to the result document, "strafter01.srx".
there was discussion[1] related to this during the ratification process, but the messages indicate no conclusion with respect to the test documents themselves

is there any archive file which comprises the ratified test suite?

James,

I believe that the problem you’re referring to here is the difference between “” and “”@ja in the results files?

yes, in one. in the other there is also a french language tag which would appear to be anomalous.

I’m not able to find the language-tagged (bad) version of that data in the approved test suite.

you raise, in passing, the proximate question: what is “the approved test suite”? (see below)

The tests :strbefore01a and :strafter01a both reference seemingly-valid result files (strbefore01a.srx and strafter01a.srx).

in the files served directly on the site, yes.
in the tar archive no.


However, the CVS repository and tarball of the test suite also contain old files that contain the invalid data (strbefore01.srx and strafter01.srx). This is unfortunate, but shouldn’t cause problems so long as you are using the manifest files to find approved tests and their associated files.

if i were to run straight from the net, i might be persuaded to agree with you.
that practice suffers, however, from two deficiencies:
- it is quite circumstantial, in that one cannot point to an object and indicate compliance with it, but can only say “hey, that’s what was being served on dddd-dd-dd@tt:tt:tt”
- there have been innumerable occasions over the past days when w3c’s web front-end decided to no longer serve the content, which makes it difficult to run tests in that mode. sometimes for days.

i could always wget and set up our own git repository, but having observed any number of those already in the wild - each of unknown provenance and with unknown content, that does not seem to be a well-considered approach.

best regards, from berlin,

.greg


---
james anderson | [hidden email] | http://dydra.com








--
Esko Nuutila
Aalto University School of Science and Technology
Department of Computer Science and Engineering
P.O.Box 15400 (T-building, Konemiehentie 2, room B218)
FI-00076 AALTO, FINLAND
tel. +358 50 5750278 mailto: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: where is the ratified sparql 1.1 test suite?

Gregory Williams
In reply to this post by james anderson-19
On May 18, 2015, at 7:28 AM, james anderson <[hidden email]> wrote:
>
>> I believe that the problem you’re referring to here is the difference between “” and “”@ja in the results files?
>
> yes, in one. in the other there is also a french language tag which would appear to be anomalous.

Could you point specifically at the problem you’re seeing? I can’t seem to find it in either strbefore01a.srx or strafter01a.srx. The only “french language tag” I see relating to these tests is in strbefore01a.srx, and that looks valid to me as the STRBEFORE function finds a match and so "returns a literal of the same kind as the first argument” (in this case, “françai”@fr).

> if i were to run straight from the net, i might be persuaded to agree with you.
> that practice suffers, however, from two deficiencies:
> - it is quite circumstantial, in that one cannot point to an object and indicate compliance with it, but can only say “hey, that’s what was being served on dddd-dd-dd@tt:tt:tt”
> - there have been innumerable occasions over the past days when w3c’s web front-end decided to no longer serve the content, which makes it difficult to run tests in that mode. sometimes for days.
>
> i could always wget and set up our own git repository, but having observed any number of those already in the wild - each of unknown provenance and with unknown content, that does not seem to be a well-considered approach.

I wasn’t suggesting that you run tests directly against the network-served files. Only that you begin the process of running tests by parsing the manifest files and using that data to find approved tests (as opposed to running all tests of type mf:QueryEvaluationTest, for example, which might lead you to run a non-approved test).

thanks,
.greg


Reply | Threaded
Open this post in threaded view
|

Re: where is the ratified sparql 1.1 test suite?

james anderson-19
good morning;

thank you for your note.

On 2015-05-19, at 09:51, Gregory Williams <[hidden email]> wrote:

On May 18, 2015, at 7:28 AM, james anderson <[hidden email]> wrote:

I believe that the problem you’re referring to here is the difference between “” and “”@ja in the results files?

yes, in one. in the other there is also a french language tag which would appear to be anomalous.

Could you point specifically at the problem you’re seeing? I can’t seem to find it in either strbefore01a.srx or strafter01a.srx. The only “french language tag” I see relating to these tests is in strbefore01a.srx, and that looks valid to me as the STRBEFORE function finds a match and so "returns a literal of the same kind as the first argument” (in this case, “françai”@fr).

if i were to run straight from the net, i might be persuaded to agree with you.
that practice suffers, however, from two deficiencies:
- it is quite circumstantial, in that one cannot point to an object and indicate compliance with it, but can only say “hey, that’s what was being served on dddd-dd-dd@tt:tt:tt”
- there have been innumerable occasions over the past days when w3c’s web front-end decided to no longer serve the content, which makes it difficult to run tests in that mode. sometimes for days.

i could always wget and set up our own git repository, but having observed any number of those already in the wild - each of unknown provenance and with unknown content, that does not seem to be a well-considered approach.

I wasn’t suggesting that you run tests directly against the network-served files. Only that you begin the process of running tests by parsing the manifest files and using that data to find approved tests (as opposed to running all tests of type mf:QueryEvaluationTest, for example, which might lead you to run a non-approved test).

i guess i might have been confused by what’s in the served directory, v/s what’s in the tar archive, v/s the mf:QueryEvaluationTest complement present in the respective manifest, v/s the respective dawgt:approval status v/s the content of the manifest’s mf:entries list. my confusion.

given my evidently limited capacity, combined with the eventual variations entailed by simple v/s typed strings, and by dydra's value v/s lexical semantics for domains such as dates and numbers, the approach, to set up a git repository does recommend itself.[1]
we can leave the “current” w3c state in master and work through variations and simplifications in our own branch, with the goal to create a clear record of what we actually do and of any variations wit respect to the standard.

in the process of constructing that repository with content consolidated from the 1.1 and 1.0 suites, i observe two cases, where the 1.1 test suite master manifest entry list does not appear to be conclusive:
- the served entailment directory contains an index.html file. this masks the content. the archive contains no index file. for the moment, i took the served index file and the archive content.
- the http-rdf-update directory reflects a different structure than the others. a reference appears among the entries, but there appears to be neither a manifest nor any other “standard-format” test information in the directory itself.

best regards, from berlin
---
james anderson | [hidden email] | http://dydra.com





Reply | Threaded
Open this post in threaded view
|

the sparql 1.1 test suite: result specifications

james anderson-19
In reply to this post by Gregory Williams
good afternoon;

as we continue to work through the sparql test suites, we observe that numerous result documents are encoded with an rdf schema specific to the dawg sparql suite.
the structure follows no sparql 1.1 recommendation, and its description appears just as a reference[1] in the “sparql 1.1: test case structure” document.[2]

does this document form provide any advantage over the recommended ones for the purpose of regression tests?
is there any reason not to replace those documents with ones which use some recommended result schema?

best regards, from berlin,

[1] : http://www.w3.org/2001/sw/DataAccess/tests/result-set.n3
[2] : http://www.w3.org/2009/sparql/docs/tests/
---
james anderson | [hidden email] | http://dydra.com