I've committed an incomplete implementation of the -04 spec
in Python to svn along with test cases taken from the spec. http://code.google.com/p/uri-templates/source/detail?r=57 The tests themselves are taken directly from the spec and are stored in a JSON file, to make it easier to reuse them. (Well, almost taken verbatim from the spec, below I list the cases that were either inconsistent or looked wrong and how I changed them.) The code does not implement 'partial' modifiers. Here are my notes: 1. The spec is fiddly, by which I mean that while the rules for handling semicolon, query parameter, path and label expansions all make sense on their own, they are all slightly different from each other. It makes for a lot more code than I expected, and is fairly tedious to implement. I'm not sure that the amount of complexity is warranted. To simplify I would like to see label and semicolon path style expansions dropped. I also believe the '+' modifier could be dropped without loss of generality. 2. From the spec, this rule isn't consistent: ["{?x,y,empty}", "?x=1024&y=768&empty="], 3. One of these is inconsistent ["X{.empty}", "X."], ["x{?empty_list}", "x"], 4. Also inconsistent ["{;list}", ";val1,val2,val3"], ["x{;name=none}", "x;name=Fred,Wilma,Pebbles"], 5. I tried working through the empty and default cases (section 2.5) but came across so many cases that seemed to disagree with previous examples or the wording of the spec that I began to suspect the section was incomplete. Here is the list of test cases I had collected and believe are wrong, before I stopped: ["x{empty_list=_}y", "xy"], ["x{empty_list+=_}y", "xempty_list._y"], ["x{empty_keys=_}y", "xy"], ["x{empty_keys+=_}y", "xempty_keys._y"], ["x{?empty}", "x?empty="], ["x{?empty=none}", "x?empty="], ["x{?empty_list*=none}", "x?none"], ["x{?empty_list+=none}", "x?empty_list.none"], ["x{?empty_keys*=none}", "x?none"], ["x{?empty_keys+=none}", "x?empty_keys.none"], I believe many of these cases with defaults are ones that Mike Burrows raised earlier. 6. I've come across yet another case where going in reverse, from a template to matching, would be useful. How about carving out () at the end of an expression as a place where a regex could be stored for applications that want to use a template for matching against incoming requests, but just state that the allowed regex's and matching algorithms are outside the scope of the current spec? Thanks, -joe |
In implementing a -04 parser in PHP, I also found the expansion operators to treat modifiers dissimilarly, resulting in a large number of special cases.
http://lab.kevburnsjr.com/php-uri-template-parser I did, however, get the library to pass for all examples I could find in the -04 spec (except those relating to partial modifiers which I have yet to implement). http://php-uri-template-parser.hackyhack.net/tests.php KB - a - --- - > 1. [...] > > To simplify I would like to see label and semicolon path > style expansions dropped. I also believe the '+' modifier could > be dropped without loss of generality. In the step from -03 to -04 the syntax has become more succinct, but has also become more magical. Simpler is better IMHO. KB - b - --- - > 6. I've come across yet another case where going in reverse, from a > template to matching, would be useful [...] Reversibility was something that originally attracted me to URI-Templates. If a resource could use the same pattern to match a URI as it uses to generate them, the world could be a much simpler place. However, it could be argued that most templates are not designed to be reversible (you wouldn't try to reverse an HTML template expansion). The best solution I could come up with is a second format employing an extended expression syntax (adding support for RegEx) used for matching that could be easily reduced to a valid uri-template for expansion. A reversible-uri-template if you will. Keeping this as a separate format would ensure that uri-template parsers remain simple to implement (important for adoption IMHO). KB - c - --- - Lastly, I wondered about using multiple operators for a single expression, particularly with respect to the + operator. Should expressions with multiple operators such as {+;list1,list2} be allowed in order to add support for Reserved Expansion of expressions with delimiter operators? - KevBurnsJr c: +001 (650) 521-7791 On Tue, Jun 1, 2010 at 9:00 PM, Joe Gregorio <[hidden email]> wrote: I've committed an incomplete implementation of the -04 spec |
In reply to this post by Joe Gregorio-2
On 2 June 2010 05:00, Joe Gregorio <[hidden email]> wrote: I've committed an incomplete implementation of the -04 spec Cool (the JSON part especially). Here are my notes: To simplify I would like to see label and semicolon path Please don't drop label expansion. I added this to Routes (as used by Pylons) and it enables applications to simplify their routing quite significantly. It's the optionality that makes them not redundant (unless you want to revive the prefix operator!). I'm not bothered by the other features, but are there more consistent ways of specifying them? But yes, it was fiddly and annoying. 2. From the spec, this rule isn't consistent: Yes, it seems we have both concluded that the trailing '=' above is needed (missing in the spec). 3. One of these is inconsistent My implementation manages both of these. Perhaps (it was a while ago) it was at the cost of a special case in the code though. What would you change them to? 4. Also inconsistent Yes, I disabled the latter test. 5. I tried working through the empty and default cases I pass several of those; I got away with disabling out just these tests: ('x{empty_list=_}y', 'xy'), ('x{empty_keys=_}y', 'xy'), ('x{;name=none}', 'x;name=Fred,Wilma,Pebbles'), ('x{;favs=none}', 'x;favs=color,red,volume,high'), ('x{;empty_list=none}', 'x;empty_list=none'), ('x{;empty_keys=none}', 'x;empty_keys=none'), 6. I've come across yet another case where going in reverse, from a template That could be done without changing the spec at this stage. Routes (for example) has a syntax that allows a regex to be embedded - e.g. "{foo:[a-z]+}" - but it's usually much better to provide these as separate parameters to the route definition. Implementers are free to do something similar now with the URI Template spec as it stands, and meanwhile the spec can progress towards approval. It should be sufficient for servers to be able to generate URI Templates from their native formats (this is what described_routes does with templates from Routes or Rails); in the worst case they can be maintained separately. So far I've manage to avoid doing anything that will encourage clients to parse URIs. In summary, there are only a few broken tests, the complexity is slightly annoying but still manageable (assuming the broken tests can be fixed to something sensible), and I for one would still press on towards approval. Meanwhile, feel free to get what you can from http://bitbucket.org/asplake/described_routes/src/tip/described_routes/uri_template.py http://bitbucket.org/asplake/described_routes/src/tip/tests/test_uri_template.py When the spec is stable it would be nice to release either package (I really don't mind whose) to pypi.
Regards, Mike [hidden email] http://positiveincline.com http://twitter.com/asplake |
Further to point 5, these are the actual expansions given by my implementation:
x{empty_list=_}y -> x_y (was: xy) x{empty_keys=_}y -> x_y (was: xy) x{;name=none} -> x;Fred,Wilma,Pebbles (was: x;name=Fred,Wilma,Pebbles) x{;favs=none} -> x;color,red,volume,high (was: x;favs=color,red,volume,high) x{;empty_list=none} -> x;none (was: x;empty_list=none) x{;empty_keys=none} -> x;none (was: x;empty_keys=none) Mike [hidden email] http://positiveincline.com http://twitter.com/asplake On 2 June 2010 11:18, Mike Burrows <[hidden email]> wrote:
|
In reply to this post by Mike Burrows (@asplake)
Hi Joe, Re point 6, just to mention that there are examples of what I describe in the JSON description of the Buzz API. I am led to believe that you are familiar with this. That aside, where next with the spec, implementation issues etc? If there's anything I can do... Regards, Mike [hidden email]
http://positiveincline.com http://twitter.com/asplake On 2 June 2010 11:18, Mike Burrows <[hidden email]> wrote: <snip>
|
In reply to this post by Joe Gregorio-2
I am trying to reconcile URI Templates with the apparent lack
of consensus on the more complex features. I guess that confusion shouldn't be surprising, since I forgot that I hadn't included definitions for each of the operators in draft 04 (sections 3.7-10), and then all my spec plans were discarded when my son was born the following week. One of the big problems seems to be wherever template variables make reference to implementation language types, like lists or structures, where everyone seems to have a different opinion on how to handle empty (or zero member) values. Likewise, I thought that the explode syntax would be valuable for expressing many-variable templates in a very short string, but it seems to add too much complexity to the implementations. On Jun 1, 2010, at 9:00 PM, Joe Gregorio wrote: > I've committed an incomplete implementation of the -04 spec > in Python to svn along with test cases taken from the spec. > > http://code.google.com/p/uri-templates/source/detail?r=57 > > The tests themselves are taken directly from the spec and > are stored in a JSON file, to make it easier to reuse them. > (Well, almost taken verbatim from the spec, below I list > the cases that were either inconsistent or looked wrong > and how I changed them.) The code does not implement > 'partial' modifiers. Can you say why it does not implement partials? That should be one of the easiest things to implement, and yet two people didn't bother. > Here are my notes: > > 1. The spec is fiddly, by which I mean that while the rules > for handling semicolon, query parameter, path and label > expansions all make sense on their own, they are all > slightly different from each other. It makes for a lot > more code than I expected, and is fairly tedious to implement. > I'm not sure that the amount of complexity is warranted. The goal is to make them consistent with the way that the same syntax is handled on websites. Semicolon, query, labels, and paths have distinct parsing behavior on websites regardless of templates. Looking at your implementation, I think the tedious bit is due to the special handling of lists and modifiers, not the different behavior of operators. > To simplify I would like to see label and semicolon path > style expansions dropped. I also believe the '+' modifier could > be dropped without loss of generality. I don't think that makes any sense. Label and params are the two most common variations on path selectors (e.g., Apache Sling, Rails, etc.). Having them as operators allows the ";" or "." prefix to be elided when optional, and multiple values to be properly separated. Trying to define URI templates without them will just eliminate templates as a viable description for those resources. > 2. From the spec, this rule isn't consistent: > ["{?x,y,empty}", "?x=1024&y=768&empty="], It is consistent with HTML forms. When a form variable is empty, it is submitted as name= > 3. One of these is inconsistent > ["X{.empty}", "X."], > ["x{?empty_list}", "x"], I made a distinction between empty string and empty list. I should just remove lists and exclude empty labels. > 4. Also inconsistent > ["{;list}", ";val1,val2,val3"], > ["x{;name=none}", "x;name=Fred,Wilma,Pebbles"], Yes, the second was intended. > 5. I tried working through the empty and default cases > (section 2.5) but came across so many cases that > seemed to disagree with previous examples or the wording > of the spec that I began to suspect the section was incomplete. > Here is the list of test cases I had collected and believe are wrong, > before I stopped: > > ["x{empty_list=_}y", "xy"], > ["x{empty_list+=_}y", "xempty_list._y"], > ["x{empty_keys=_}y", "xy"], > ["x{empty_keys+=_}y", "xempty_keys._y"], > ["x{?empty}", "x?empty="], > ["x{?empty=none}", "x?empty="], > ["x{?empty_list*=none}", "x?none"], > ["x{?empty_list+=none}", "x?empty_list.none"], > ["x{?empty_keys*=none}", "x?none"], > ["x{?empty_keys+=none}", "x?empty_keys.none"], > > I believe many of these cases with defaults are ones that Mike Burrows > raised earlier. Yes, they are all due to the distinction between empty strings and empty lists, and when lists are expanded versus just treated as another string. The missing prose doesn't help, but the easier solution is to remove those as options. > 6. I've come across yet another case where going in reverse, from a template > to matching, would be useful. How about carving out () at the end of > an expression as a place where a regex could be stored for applications > that want to use a template for matching against incoming requests, but > just state that the allowed regex's and matching algorithms are outside > the scope of the current spec? I don't think so. A regex can be derived from the template if the expressions are delimited by reserved characters and reserved substitution is not used. If those two conditions are not true, then a matching regex would have to be ambiguous, which won't satisfy anyone. I think we should just say what people should limit their template to if they want it to be matching. I am also leaning toward making the reserved expansion "+" operator a prefix on any variable, since that would be backward compatible with simple expressions and yet give people the rope they want to hang other values on without complicating the implementations further. We can then move the encode-reserved question inside the value-substitution routine rather than being operator-specific. ....Roy |
Free forum by Nabble | Edit this page |