[Bug 26585] New: Proposal: fn:apply

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 26585] New: Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

            Bug ID: 26585
           Summary: Proposal: fn:apply
           Product: XPath / XQuery / XSLT
           Version: Working drafts
          Hardware: PC
                OS: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: XQuery 3.1
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]

XQuery 3.0 introduced functions that are derived from functional programming.
Function items can be passed as arguments to functions like fn:fold-left().
Together they allow a more functional style of coding. Most of these functions
can be used to perform map, filter, reduce on sequences.

However, one important function that is missing and for which I do not know a
reasonable workaround for is "apply" [1]. The wikipedia page shows that many
languages have ways to achieve this and Clojure has a function with the same
name [2].

In XQuery we can bind a variable to a function item. But when we want to use
it we are limited to one argument.

  $f = function my-add($a,$b,$c) { $a + $b + $c }
  $args = (1,2,3)

  $f($args) == $f((1,2,3))

If we had an apply function this would do it.

  fn:apply($f, $args) == $f(1,2,3)

Of course we could rewrite the function my-add to something like:

  $f = function my-add($seq) { sum($seq) }

  $f($args) == 6

But this is only practical for a limited number of cases and the function body
would need to unpack the sequence, and we also lose the type checking.

This limitation also shows with the proposed arrow operator (=>).

Someone coming from a language like Clojure, Lisp or Scheme would interpret the
following

  (1,2,3) => $f

to mean

  $f(1,2,3)

but in XQuery it would mean

  $f((1,2,3))

The arrow operator is similar to Clojure's threading operator (->) [3].

Clojure:

  (-> (1 2 3) f1 f2) == (f2 (f1 1 2 3))

Note that in Clojure -> is a macro which re-writes the form before the
compiler will see it. But I digress.

XQuery:

  (1,2,3) => $f1 => $f2

  $f2($f1((1,2,3))

Maybe this is a consequence of the fact that sequences cannot be nested, and
are effectively flattened.

I cannot oversee all the consequences but like a map may contain other maps
arrays can be nested so they could be used for "apply" semantics.

Currently we have:

  (1,2,3) => $f

  $f((1,2,3))

With array apply semantics:

  [1,2,3] => $f1 => $f2

  == $f2($f1(1,2,3))

  [(1,2),(3,4)] => $f1

  == $f1((1,2),(3,4))

  == fn:apply($f1, [(1,2),(3,4)])

On the surface this looks like an elegant solution to me and is closer to
the apply semantics in other languages.

A particular use case for fn:apply arose when I tried implementing Clojure Ring
and Compojure libraries in XQuery. These are comparable to WSGI in Python and
Rack in Ruby.

Below I simplified things but I hope it illustrates a concrete use case.

I have a function for defining a routing handler which maps an HTTP GET request
to
a handler function. The def-route() signature is

  function(
    $method as xs:string,
    $url-template as xs:string,
    $params as array(*),
    $handler as function(*)
  ) as function(map(*)) as map(*)

This means that a route handler function is something that takes a request
map as input and produces a response map as output.

  $handler =
      function($name, $age) {
        'Person: ' || $name || ' (age ' || $age ')'
      }

  $route = def-route('GET', '/person/{name}/{age}', ['name', 'age'], $handler)

  $route(request('GET', '/person/Joe/10'))

When a GET request comes in on '/person/Joe/10'. The route-handler $route
matches and the 'name' and 'age' parameters are parsed from the URL and added
to
the request map. The third argument is an array that specifies which keys need
to be taken from the request map and builds the $params array: ['Joe', '10'].
Finally it can invoke the $handler using

  fn:apply($handler, $params)

or

  $params => $handler

  == ['Joe', '10'] => $handler

This results in a response('Person: Joe (age 10)') which is then rendered to
the
browser.

Note that the handler does not need to know where to find the parameters
because
this is set up by the $route handler thus achieving a clean separation of
concerns and the possibility of using $handler in other situations because
it is not tied to the plumbing/routing logic.

As I said earlier, I may not be able to oversee all the consequences as I'm not
super intimate with all the specs involved. Or, my pseudo code is confusing.
Mea
culpa.

--Marc

[1] http://en.wikipedia.org/wiki/Apply
[2] http://clojuredocs.org/clojure_core/clojure.core/apply
[3] http://clojuredocs.org/clojure_core/clojure.core/-%3E

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 26585] Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

Michael Kay <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #1 from Michael Kay <[hidden email]> ---
In the past it hasn't been possible to treat the list of arguments of a
function as a single value because sequences can't be nested, as you suggest.
That difficulty is partly solved by the introduction of arrays. But I think
there are considerable difficulties that remain, notably the fact that the
arity of a function in XQuery is so much part of its nature. Doing this would
be powerful, but a lot of work to do well.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 26585] Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

Rob <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #2 from Rob <[hidden email]> ---
Created attachment 1503
  --> https://www.w3.org/Bugs/Public/attachment.cgi?id=1503&action=edit
demo for apply (limited) functionality built in xquery

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 26585] Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

--- Comment #3 from Rob <[hidden email]> ---
Created attachment 1504
  --> https://www.w3.org/Bugs/Public/attachment.cgi?id=1504&action=edit
apply module built in xquery (limited functionality)

The 'APPLY:argument.constructor' function gives problems in xquery. But since
the module should be built in java these problems probably can be overcome.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 26585] Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

--- Comment #4 from Rob <[hidden email]> ---
for inspiration a apply-function built in xquery. The challenge in xquery is
the (re-)creation of the argumentlist for the evaluationstring. But if the
apply-function would be a part of the xquery-language itself it probably will
be built in java. I'm not a java-expert but I wander if the problem that exists
on xquery-level also exists on java-level.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 26585] Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

--- Comment #5 from Marc <[hidden email]> ---
Created attachment 1505
  --> https://www.w3.org/Bugs/Public/attachment.cgi?id=1505&action=edit
apply with array semantics

Rob's example almost worked for me. I have added some extra argument
serialization and handling of arrays as described in my proposal. Here's the
adapted code. Note that 1) it may now be BaseX specific due to use of map and
array modules and 2) I heavily reformatted the code so it matches the other
code in my little library and 3) I have done little testing on this so there
are probably many potholes.
Next attachment will have some unit tests.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 26585] Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

--- Comment #6 from Marc <[hidden email]> ---
Created attachment 1506
  --> https://www.w3.org/Bugs/Public/attachment.cgi?id=1506&action=edit
unit tests for apply with array semantics

Here are the BaseX specific unit tests.

--
You are receiving this mail because:
You are the QA Contact for the bug.

Reply | Threaded
Open this post in threaded view
|

[Bug 26585] Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

Leo Wörteler <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #7 from Leo Wörteler <[hidden email]> ---
The current implementation of the arrow operator in the Working Draft [1]
defines it as a syntactic macro:

> If `$i` is an item and `f()` is a function, then `$i=>f()` is equivalent
> to `f($i)`, and `$i=>f($j)` is equivalent to `f($i, $j)`.

I think this is not very elegant and possibly confusing. Since e.g.
`local:foo#0` and `local:foo#1` can be completely different functions in
XQuery, it is potentially dangerous that in

  1 => local:bar() => local:foo()

it is not immediately obvious which of them is called.

I would propose that the second argument of `=>` should instead be a function
item taking one argument. Then `$arg => $f` can be translated into `$f($arg)`
directly and the Spec can define it simply as equivalent to:

  function op:arrow-apply(
    $arg as item()*,
    $func as function(item()*) as item()*
  ) as item()* {
    $func($arg)
  };

As a nice bonus this also makes the feature more flexible because the argument
to be inserted does not have to be the first one in the function:

  $file-extension => csv:get-separator() => (tokenize($line, ?))()

could be written as

  $file-extension => csv:get-separator#1 => tokenize($line, ?)

Everything that was possible before should still work when adding a "?" at the
start of the ArgumentList of each right-hand side of `=>`. The example from the
Spec becomes

  $string => upper-case(?) => normalize-unicode(?) => tokenize(?, "\s+")

or (shorter and more elegant):

  $string => upper-case#1 => normalize-unicode#1 => tokenize(?, "\s+")

In conclusion, using function items is more flexible and less confusing, and
the syntactic translation scheme makes for only marginally less verbose tyntax.

[1] http://www.w3.org/TR/2014/WD-xquery-31-20140424/#id-arrow-operator

--
You are receiving this mail because:
You are the QA Contact for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug 26585] Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

--- Comment #8 from Christian Gruen <[hidden email]> ---
I copied Leo Wörteler's proposal in an extra issue:
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26889

--
You are receiving this mail because:
You are the QA Contact for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug 26585] Proposal: fn:apply

Bugzilla from bugzilla@jessica.w3.org
In reply to this post by Bugzilla from bugzilla@jessica.w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585

Liam R E Quin <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #9 from Liam R E Quin <[hidden email]> ---
The WG is considering comment 7 separately (thank you, Christian, for copying
it).

On the main subsance of this issue, it's possible (I think) that a function
like fn:apply($f as function, $args as array) could be defined by F & O in the
future, but the Working Groups have not made such a decision now.

--
You are receiving this mail because:
You are the QA Contact for the bug.