automated testing for IRI (and URI) Specs?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

automated testing for IRI (and URI) Specs?

masinter
For "test the web forward" I want to start gathering/generating reftests.

I *think* the test framework is based on the Mozilla framework
https://developer.mozilla.org/en-US/docs/Creating_reftest-based_unit_tests


The tests I'm starting with are

http://web.lookout.net/2012/03/unicode-normalization-in-urls.html
http://lists.w3.org/Archives/Public/public-html/2010Feb/0255.html
generated from http://greenbytes.de/tech/webdav/urldecomp.xml 
https://bugzilla.mozilla.org/show_bug.cgi?id=479520 and http://annevankesteren.nl/2012/09/idna 
 There are also some BIDI examples in http://tools.ietf.org/html/draft-ietf-iri-bidi-guidelines  once we get that far.

Anyone know of any other existing IRI/URI/URL/LEIRI tests?

For tests that push DNS we might need some other test framework
(including an DNS server which will accept requests  <anything>.site.com  where anything is an arbitrary string,
and send request with Host header) so that I can test the IDNA conversion bits.



Reply | Threaded
Open this post in threaded view
|

Re: automated testing for IRI (and URI) Specs?

Chris Weber-4
On 10/20/2012 1:08 AM, Larry Masinter wrote:
> Anyone know of any other existing IRI/URI/URL/LEIRI tests?
>
> For tests that push DNS we might need some other test framework
> (including an DNS server which will accept requests  <anything>.site.com  where anything is an arbitrary string,
> and send request with Host header) so that I can test the IDNA conversion bits.
>

I have a compiled collection of about 500+ tests from Webkit, Julian, as
well as other sources available at: https://github.com/cweb/iri-tests.

I'd be happy to convert these into a JSON or some other format if it was
helpful.  I definitely agree that a DNS server, as well as a Web server,
would be useful for testing.  In my own setup I used:

1. The test cases linked to above.
2. A DNS server which would respond to *.foo.bar where * was an
arbitrary subdomain string as you noted.
3. A Web server which could:
 a) Serve each test case individually;
 b) Capture incoming HTTP requests based on each test case, allowing me
to analyze both the Host header as well as the path component in the GET
request.

The test cases were served as both an href as well as an img src with
the goals being to see how Web browsers parsed these URLs in two ways:

1. How each test case was parsed and represented in the DOM properties
of the page.
2. How each test case resulted in an HTTP request, and how the
components of that HTTP request (host header and GET path) compared to
the parsed DOM (hostname and path) properties of the same test case.

Unfortunately my setup was not very portable, but could be made so with
some effort.  I actually did a little more than noted above, because I
ran a packet sniffer to see what the corresponding DNS requests looked
like as well.  I posted some of the early results up here:
https://docs.google.com/spreadsheet/ccc?key=0At1OFOiVqCrvdFo3aFc1elhXS2pnVkpxOFZORjQ1cUE#gid=8
as well as at Github.  I meant to clean things up, use a consistent DNS
scheme, define metadata better, test refs, define expected conditions,
etc. but didn't go that far.

I did notice some substantial differences across Web browsers, and also
found some instances where the HTTP request did not match the DOM
properties for the same URL.  But maybe these are mostly edge cases.

Best regards,
Chris Weber





Reply | Threaded
Open this post in threaded view
|

RE: automated testing for IRI (and URI) Specs?

masinter
I started to update http://www.w3.org/wiki/UriTesting with additional information and a few more thoughts.

I'm sure there's some duplication, missing material, and errors, please feel free to correct.


> -----Original Message-----
> From: Chris Weber [mailto:[hidden email]]
> Sent: Saturday, October 20, 2012 4:15 PM
> To: Larry Masinter
> Cc: [hidden email]; [hidden email]; Robin Berjon ([hidden email]); Rebecca
> Hauck
> Subject: Re: automated testing for IRI (and URI) Specs?
>
> On 10/20/2012 1:08 AM, Larry Masinter wrote:
> > Anyone know of any other existing IRI/URI/URL/LEIRI tests?
> >
> > For tests that push DNS we might need some other test framework
> > (including an DNS server which will accept requests  <anything>.site.com
> where anything is an arbitrary string,
> > and send request with Host header) so that I can test the IDNA conversion
> bits.
> >
>
> I have a compiled collection of about 500+ tests from Webkit, Julian, as
> well as other sources available at: https://github.com/cweb/iri-tests.
>
> I'd be happy to convert these into a JSON or some other format if it was
> helpful.  I definitely agree that a DNS server, as well as a Web server,
> would be useful for testing.  In my own setup I used:
>
> 1. The test cases linked to above.
> 2. A DNS server which would respond to *.foo.bar where * was an
> arbitrary subdomain string as you noted.
> 3. A Web server which could:
>  a) Serve each test case individually;
>  b) Capture incoming HTTP requests based on each test case, allowing me
> to analyze both the Host header as well as the path component in the GET
> request.
>
> The test cases were served as both an href as well as an img src with
> the goals being to see how Web browsers parsed these URLs in two ways:
>
> 1. How each test case was parsed and represented in the DOM properties
> of the page.
> 2. How each test case resulted in an HTTP request, and how the
> components of that HTTP request (host header and GET path) compared to
> the parsed DOM (hostname and path) properties of the same test case.
>
> Unfortunately my setup was not very portable, but could be made so with
> some effort.  I actually did a little more than noted above, because I
> ran a packet sniffer to see what the corresponding DNS requests looked
> like as well.  I posted some of the early results up here:
> https://docs.google.com/spreadsheet/ccc?key=0At1OFOiVqCrvdFo3aFc1elhXS2
> pnVkpxOFZORjQ1cUE#gid=8
> as well as at Github.  I meant to clean things up, use a consistent DNS
> scheme, define metadata better, test refs, define expected conditions,
> etc. but didn't go that far.
>
> I did notice some substantial differences across Web browsers, and also
> found some instances where the HTTP request did not match the DOM
> properties for the same URL.  But maybe these are mostly edge cases.
>
> Best regards,
> Chris Weber
>
>
>