[Bug 23643] New: Convenient operator for transform expressions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[Bug 23643] New: Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

            Bug ID: 23643
           Summary: Convenient operator for transform expressions
           Product: XPath / XQuery / XSLT
           Version: Working drafts
          Hardware: PC
                OS: Windows NT
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Update 3.0
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]

The XQUF transform expression has turned out to be an essential operation when
it comes to updating XML documents in main-memory. From our perspective,
however, it has the following shortcomings:

* Its verbose syntax makes updatable expressions inconvenient to read. The
copy/modify/return construct is particularly bulky when being embedded in
another FLWOR expression.

* Our users frequently mix up the components of FLWOR and transform
expressions. For many, it is hard to grasp why "copy" cannot be used in row
with "for" or "let".

* Most transform expressions we have encountered so far contain a single copy
variable and return the modified value without further modifications.

This is why I would like to propose a "modify" operator for XQUF 3.0, which
would require only few modifications in the grammar:

  AndExpr        ::= ModifyExpr ( "and" ModifyExpr )*
  ModifyExpr     ::= ComparisonExpr ( "!!" ComparisonExpr )?
  ComparisonExpr ::= ...

The modify operator works similar to the XQuery 3.0 map operator (and the "!!"
token has been chosen for that reason): the value of the first expression is
bound as context item, and the second expression performs updates on this item.
The updated item is returned as result.

>From the implementation point of view, this extension is straightforward:

* It can completely be derived from the transform expression.
* Error codes are the same, too (XUST0002, XUTY0013).

This is a little example how an expression may look like before and after:

* Using the transform expression:

  for $e in //item[@status ne 'up-to-date']
  let $c :=
    copy $c := $e
    modify insert node <sub/> into $c
    return $c
  return element updated { $c }

* Using the modify operator:

  for $e in //item[@status ne 'up-to-date']
  let $c := $e !! insert node <sub/> into .
  return element updated { $c }

I’m looking forward to the discussion.

--
You are receiving this mail because:
You are the QA Contact for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug 23643] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #1 from John Snelson <[hidden email]> ---
Interesting idea, although I'm not sure I like the way you spell the operator
"!!". Maybe just using "modify" as an infix operator would be more clear.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #2 from Christian Gruen <[hidden email]> ---
Yes, I’m open for alternative spellings.

One way why I chose special characters over a string is that we would need to
introduce two keywords (The full-text keyword "contains" was changed to
"contains text" for the same reason), and something like "<a/> modify node
insert node <b/> into ." isn’t that catchy anymore.

--
You are receiving this mail because:
You are the QA Contact for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug 23643] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

Michael Kay <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #3 from Michael Kay <[hidden email]> ---
We (or the user) could define a function

modify($n as node(), $f as updating function($n as node())) {
  copy $m := $n
  modify $f($m)
  return $m
}

and then you could write

for $e in //item[@status ne 'up-to-date']
return element updated { modify($e, function($n) {insert node <sub/> into $n})
}

This would be even more attractive if we had a terser syntax for inline
functions. In FtanSkrit I used

{$1 + $2}

for a function that adds its two arguments (i.e. our "function ($n, $m) {$n +
$m}"). We've decided to use a bare "{" for maps, but we could still go with
something like {= $1 + $2 }, the leading '=' being familiar to spreadsheet
users - or even just a leading '=' without the curlies:

for-each($seq, =($1 + 1))

and then Christian's example becomes

for $e in //item[@status ne 'up-to-date']
return element updated { modify($e, =insert node <sub/> into $1)}

I think we should be trying to build new capability by exploiting higher-order
functions wherever we can, in preference to inventing new higher-order
operators.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #4 from Christian Gruen <[hidden email]> ---
Mike, thanks for the response. I also think that a higher-order approach could
offer a comparable solution, provided that the syntax gets more concise at some
stage (the FtanSkrit proposal looks compelling).

As the modification of nodes in main-memory has got a very basic operation,
looking at our user feedback and our use cases, and as the effort for
specifying and implementing the extension is minimal, I still believe it could
deserve its own operator, as the existing solution is just too clumsy. If one
single keyword can be used, "modify" may be the better choice?

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #5 from Christian Gruen <[hidden email]> ---
I noticed that "modify" is no viable alternative, as it would get in conflict
with the existing keyword, when being used in the transform expression:

  copy $c := <c/> modify insert node <X/> into $c return $c

Other alternatives for "!!" could be the keyword "update", the special
character "~" or "<<" (to indicate that the following expressions have affect
the lefthand operand):

  <c/> !! insert node <X/> into .
  <c/> << insert node <X/> into .
  <c/> ~  insert node <X/> into .
  <c/> update insert node <X/> into .

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #6 from Christian Gruen <[hidden email]> ---
A little addition: we believe that the "update" keyword is the best choice. It
is both well readable and connects better to the existing vocabulary of the
XQUF:

CURRENTLY:
  for $item in db:open('data')//item
  return copy $c := $item
         modify delete node $c/text()
         return $c

PROPOSED:
  for $item in db:open('data')//item
  return $item update delete node text()

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #7 from Christian Gruen <[hidden email]> ---
If the 'update' keyword should cause conflicts, and if we want to avoid special
characters, 'update node' or 'modify node' would probably be the most
consistent choice.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

Innovimax <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #8 from Innovimax <[hidden email]> ---
and what about "updated by" ?

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #9 from Christian Gruen <[hidden email]> ---
...yes, might be more readable. Another alternative (to align the wording with
the existing transform expression) could be "modified by":

  doc('input.xml')//item modified by (
    replace node . with element TEXT { text() }
  )

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

Jonathan Robie <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |[hidden email]
         Resolution|---                         |FIXED

--- Comment #10 from Jonathan Robie <[hidden email]> ---
The Working Group agrees. We are adding the following ExprSingle:

TransformExpr ::= PathExpr ("transform" "with" "{" Expr? "}")?

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

Ghislain Fourny <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #11 from Ghislain Fourny <[hidden email]> ---
and

SimpleMapExpr ::= TransformExpr ("!" TransformExpr)*

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #12 from Christian Gruen <[hidden email]> ---
Thanks for looking at this.

One more question: I was surprised about the addition of the curly brace, as we
now have three mandatory tokens. Wouldn't one of the following expression be
syntactical valid, and sufficient?

  a) <a/> transform with insert node <b/> into .
  b) <a/> transform { insert node <b/> into . }

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #13 from John Snelson <[hidden email]> ---
We need a terminating token for the TransformExpr as the insert expressions are
all directly under ExprSingle, which creates ambiguity in operators after the
TransformExpr.

I think "$E transform { ... }" would work syntactically.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

Christian Gruen <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |---

--- Comment #14 from Christian Gruen <[hidden email]> ---
Fine. In this case, I would prefer to get rid of the additional keyword. What
do others think?

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

Jonathan Robie <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #15 from Jonathan Robie <[hidden email]> ---
At the Face-to-Face in Prague, we decided as following:

DECISION: Adopt the simple TransformExpr as described by bug 23643 and
modified to use the following syntax:

SimpleMapExpr ::= TransformExpr ("!" TransformExpr)*            
TransformExpr ::= PathExpr ("transform" "with" "{" Expr? "}")?

We apparently forgot to record that decision here.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

Christian Gruen <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |---

--- Comment #16 from Christian Gruen <[hidden email]> ---
I just reopened the issue, as there is already a TransformExpr in XQuery 3.0
Update:

[206] TransformExpr ::= "copy" "$" VarName ":=" ExprSingle (","
      "$" VarName ":=" ExprSingle)* "modify" ExprSingle "return" ExprSingle

We could rename the new expression to ModifyExpr or something similar.

Next, I still wonder why/if we really need three tokens..

  "transform" "with" "{"

..or if we could get rid of "with" or "{"? In my initial proposal, I suggested
to move the expression within AndExpr and ComparisonExpr; would this help?

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

Michael Dyck <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #17 from Michael Dyck <[hidden email]> ---
(In reply to Christian Gruen from comment #16)
> I just reopened the issue, as there is already a TransformExpr in XQuery 3.0
> Update:
>
> [206] TransformExpr ::= "copy" "$" VarName ":=" ExprSingle (","
>       "$" VarName ":=" ExprSingle)* "modify" ExprSingle "return" ExprSingle
>
> We could rename the new expression to ModifyExpr or something similar.

So TransformExpr uses the keyword "modify",
whereas ModifyExpr uses the keyword "transform".

Any chance we could fix that?

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #18 from Jonathan Robie <[hidden email]> ---
(In reply to Michael Dyck from comment #17)

> So TransformExpr uses the keyword "modify",
> whereas ModifyExpr uses the keyword "transform".
>
> Any chance we could fix that?

The names of nonterminals can be changed without changing the language.  We may
need WG approval for this, but I think we should do this.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 23643] [UPD 3.0] Convenient operator for transform expressions

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23643

--- Comment #19 from Michael Kay <[hidden email]> ---
I've argued in the past for the copy/modify expression to be called something
other than TransformExpr (but for some reason, I failed to convince people).

When the name of an expression bears no relationship to the keywords used, it
makes it very difficult to construct error messages that will be meaningful to
the 95% of users who haven't read the W3C spec.

--
You are receiving this mail because:
You are the QA Contact for the bug.

12