304 or 412 for If-Modified-Since?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

304 or 412 for If-Modified-Since?

Cyrus Daboo-2
Hi folks,
RFC 2616 in Section 14.25 "If-Modified-Since" states that a 304 should be
returned if the resource is not modified.

RFC 4918 in Section 12.1 "412 Precondition Failed" states that a 412 should
be returned when a conditional header fails to hold.

So what should clients expect from WebDAV servers?

Second, does If-Modified-Since make sense for PROPFIND or REPORT requests?

--
Cyrus Daboo


Reply | Threaded
Open this post in threaded view
|

Re: 304 or 412 for If-Modified-Since?

Julian Reschke
On 30.06.2010 03:29, Cyrus Daboo wrote:
> Hi folks,
> RFC 2616 in Section 14.25 "If-Modified-Since" states that a 304 should
> be returned if the resource is not modified.

Yes.

> RFC 4918 in Section 12.1 "412 Precondition Failed" states that a 412
> should be returned when a conditional header fails to hold.

That should be reported as an RFC 4918 erratum. WebDAV can't override
normative requirements from HTTP/1.1. (

> So what should clients expect from WebDAV servers?

They should expect that headers defined in RFC 2616 work as advertised
there.

> Second, does If-Modified-Since make sense for PROPFIND or REPORT requests?

If the resource being addressed responds to GET, and does to with a
Last-Modified header, then it makes sense for consistency. However, GET
in a collection *usually* only returns information based on the state of
the collection and its direct members, so it wouldn't be useful for
PROPFIND/Depth=Infinity, and might not be useful at all for REPORT.

Of course the same question could be asked about If-None-Match.

I think making PROPFIND/REPORT responses more Web-friendly (adding a way
to GET those) would help with this issue (hint, hint).

Best regards, Julian

Reply | Threaded
Open this post in threaded view
|

Re: 304 or 412 for If-Modified-Since?

Elias Sinderson-2
In reply to this post by Cyrus Daboo-2
Hi Everyone,

I had responded privately to Cyrus earlier, with the intention of writing a better response to the list. I see that Julian has responded in the meantime and, while we agree on most all points, I am hoping for some thoughtful discussion / clarification of the last issue.


Cyrus Daboo wrote:
RFC 2616 in Section 14.25 "If-Modified-Since" states that a 304 should be returned if the resource is not modified.
Basically, yes, a conditional request with a valid If-Modified-Since header is expected to result in a 304 response if the resource has not been modified.

There is further clarification in Section 14, where specific request headers are defined, for the other If-* headers and their interaction with If-Modified-Since. Whether the server returns 304 vs 412 in the case of If-None-Match is dependent on the requested method (GET / HEAD vs. others). WRT If-Unmodified-Since, the server is required to return a 412 if the requested variant has been modified, and If-Match behaves similarly.

Confusing, yes, and I suspect that there are a few broken implementations of this in the wild, both in actual server behavior and in what client libraries expect in various situations.

RFC 4918 in Section 12.1 "412 Precondition Failed" states that a 412 should be returned when a conditional header fails to hold.
So what should clients expect from WebDAV servers?
The text you reference from 4918 does not override the normative text in 2616. Note the introductory text in Section 12 --

"These HTTP codes are not redefined, but their use is somewhat extended by WebDAV methods and requirements."

The following subsections apply to the methods defined in the WebDAV spec, not the base HTTP methods...

Second, does If-Modified-Since make sense for PROPFIND or REPORT requests?
My reading of the specs is that yes, this makes sense -- For these methods, the If-* headers apply to the resources that are in the scope of the request, with the 412 appearing in the <D: status> element associated with a given <D:href>.


So, as above, Julian and I are in agreement on all of the points except for this last... I've referenced 4918 briefly to look for some clarification of this issue but not found anything. I'm hoping that someone can provide a reference or some logical explanation to put my mind at ease.



Cheers,
Elias
Reply | Threaded
Open this post in threaded view
|

Re: 304 or 412 for If-Modified-Since?

Julian Reschke
Hi Elias,

nice to see you back on the mailing list! :-)

On 30.06.2010 09:01, Elias Sinderson wrote:
> ...
> There is further clarification in Section 14, where specific request
> headers are defined, for the other If-* headers and their interaction
> with If-Modified-Since. Whether the server returns 304 vs 412 in the
> case of If-None-Match is dependent on the requested method (GET / HEAD
> vs. others). WRT If-Unmodified-Since, the server is required to return a
> 412 if the requested variant has been modified, and If-Match behaves
> similarly.
> ...

Indeed. In HTTPbis we still plan to add guidelines on defining new
headers, and we probably add a paragraph about what to do when defining
new conditional headers.

> ...
>> RFC 4918 in Section 12.1 "412 Precondition Failed" states that a 412
>> should be returned when a conditional header fails to hold.
>> So what should clients expect from WebDAV servers?
> The text you reference from 4918 does not override the normative text in
> 2616. Note the introductory text in Section 12 --
>
> /"These HTTP codes are not redefined, but their use is somewhat extended
> by WebDAV methods and requirements."/
> ...

Agreed. But then it *does* redefine them :-)

> The following subsections apply to the methods defined in the WebDAV
> spec, not the base HTTP methods...

That might have been the intent, but it doesn't say that, right?

>> Second, does If-Modified-Since make sense for PROPFIND or REPORT
>> requests?
> My reading of the specs is that yes, this makes sense -- For these
> methods, the If-* headers apply to the resources that are in the scope
> of the request, with the 412 appearing in the <D: status> element
> associated with a given <D:href>.

We're now talking about multistatus/depth processing, right?

So we need to distinguish two cases:

- applying the condition to the requested resource (which may cause the
whole request to fail with 412, and which needs to be in sync with HTTP/1.1)

- once the server *does* decide to execute the request, how the
condition is applied during Depth:1/infinity processing.

The second case indeed is interesting, I assume you're looking at it
because of collection syncing?

It's *tempting* to claim that RFC 4918 already defines this, but I think
it would overly optimistic. Better define it properly, and potentially
add a way for the server to signal that it does this (DAV header comes
to mind).

> ...

Best regards, Julian

Reply | Threaded
Open this post in threaded view
|

Re: 304 or 412 for If-Modified-Since?

Elias Sinderson-2
On 30.06.2010, Julian Reschke wrote:
> In HTTPbis we still plan to add guidelines on defining new headers,
> and we probably add a paragraph about what to do when defining new
> conditional headers.
I think this would be well received by the broader community.


> On 30.06.2010 09:01, Elias Sinderson wrote:
>> The following subsections apply to the methods defined in the WebDAV
>> spec, not the base HTTP methods...
> That might have been the intent, but it doesn't say that, right?
Correct, a clarification would help here. (Assuming my interpretation is
correct!)


>> On 30.06.2010 03:29, Cyrus Daboo wrote:
>>> Second, does If-Modified-Since make sense for PROPFIND or REPORT
>>> requests?
>> My reading of the specs is that yes, this makes sense -- For these  
>> methods, the If-* headers apply to the resources that are in the
>> scope of the request, with the 412 appearing in the <D: status>
>> element associated with a given <D:href>.
> We're now talking about multistatus/depth processing, right?
Yes ...

> So we need to distinguish two cases:
> - applying the condition to the requested resource (which may cause
> the whole request to fail with 412, and which needs to be in sync with
> HTTP/1.1)
Agreed. The depth 0 case is pretty well understood (or, at least, can be
inferred from the related text in 2616 and 4918).

> - once the server *does* decide to execute the request, how the
> condition is applied during Depth:1/infinity processing.
>
> The second case indeed is interesting, I assume you're looking at it
> because of collection syncing?
More or less -- there are obviously application / domain specific
scenarios, but they all seem to collapse to this basic idea. As you
said, it is interesting ... more details on this below.

> It's *tempting* to claim that RFC 4918 already defines this, but I
> think it would overly optimistic. Better define it properly, and
> potentially add a way for the server to signal that it does this (DAV
> header comes to mind).

Yes, well, that would explain why I couldn't find anything relevant when
I referenced the specs!   :)

With some further analysis, my earlier statement obviously does not
hold. Here are my current thoughts --

For If-Modified-Since, depth requests can only really be evaluated
correctly by iterating over the entire set of resources in scope. Given
the unfortunate diversity in how timestamps are managed by different
systems, the only reliable way to support this would be to specify that
the server must consider the entire set of resources in scope and treat
If-Modified-Since as if it applies to everything. That is, we can't
indicate any optimizations in the spec wrt evaluation of a collections
children / descendants. Given that, however, this is a rather
straightforward feature to support since a single date is used across
multiple comparisons and there may be some obvious performance
optimizations to be made for a specific architecture.

If-None-Match is more interesting. For this, we need something akin to
CTags. I thought this had been discussed on the dist-auth list
previously but a quick search of the archives came up empty. Cyrus has
written up a quickie draft proposal on this idea for CalDAV [1], so
maybe it was only discussed on that list? A critical behavior to address
is whether CTag changes 'bubble up' to parent containers. The existing
writeup doesn't specify this, but it is necessary if CTags are to
support the semantics necessary for efficient evaluation of PROPFIND /
REPORT type methods at depth. While obviously a performance hit for
servers to maintain, it would provide some (perhaps more?) performance
benefit downstream when evaluating requests with depth 1 or infinity.

Have you considered this at any length before?



Regards,
Elias
________________
[1]
<http://svn.calendarserver.org/repository/calendarserver/CalendarServer/trunk/doc/Extensions/caldav-ctag.txt>

Reply | Threaded
Open this post in threaded view
|

Re: 304 or 412 for If-Modified-Since?

Julian Reschke
On 01.07.2010 01:37, Elias Sinderson wrote:

> ...
> With some further analysis, my earlier statement obviously does not
> hold. Here are my current thoughts --
>
> For If-Modified-Since, depth requests can only really be evaluated
> correctly by iterating over the entire set of resources in scope. Given
> the unfortunate diversity in how timestamps are managed by different
> systems, the only reliable way to support this would be to specify that
> the server must consider the entire set of resources in scope and treat
> If-Modified-Since as if it applies to everything. That is, we can't
> indicate any optimizations in the spec wrt evaluation of a collections
> children / descendants. Given that, however, this is a rather
> straightforward feature to support since a single date is used across
> multiple comparisons and there may be some obvious performance
> optimizations to be made for a specific architecture.
> ...

I'm not completely sure I understand.

Are you saying that the overall response code (not the multistatus body)
for a PROPFIND with Depth:infinity and If-Modified must take the
last-modified dates for all resources in scope into account?

I don't think this is what the spec says, nor that it should (it would
be extremely expensive to implement, and to work reliably would require
internal atomicity that usually can't be guaranteed).

> If-None-Match is more interesting. For this, we need something akin to
> CTags. I thought this had been discussed on the dist-auth list

I'm not convinced we need those. We really should make all of this more
compatible with GET, and rely on ETags.

> previously but a quick search of the archives came up empty. Cyrus has
> written up a quickie draft proposal on this idea for CalDAV [1], so
> maybe it was only discussed on that list? A critical behavior to address
> is whether CTag changes 'bubble up' to parent containers. The existing
> writeup doesn't specify this, but it is necessary if CTags are to
> support the semantics necessary for efficient evaluation of PROPFIND /
> REPORT type methods at depth. While obviously a performance hit for
> servers to maintain, it would provide some (perhaps more?) performance
> benefit downstream when evaluating requests with depth 1 or infinity.
>
> Have you considered this at any length before?

I'm currently not in the business of writing servers. That being said,
change tags "bubbling up" will be very expensive to implement in many
servers.

It would probably be a good idea to look at the problem that needs to be
solved first. Maybe defining a related resource that aggregates all this
hierarchy state (and which as a proper ETag) is all that's needed.

Best regards, Julian






Reply | Threaded
Open this post in threaded view
|

Re: 304 or 412 for If-Modified-Since?

Elias Sinderson-2
Julian Reschke wrote:
> On 01.07.2010 01:37, Elias Sinderson wrote:
>> [...] For If-Modified-Since, depth requests can only really be evaluated
>> correctly by iterating over the entire set of resources in scope. [...]
> I'm not completely sure I understand.
>
> Are you saying that the overall response code (not the multistatus
> body) for a PROPFIND with Depth:infinity and If-Modified must take the
> last-modified dates for all resources in scope into account?
Sorry, I should have been more specific -- I would expect to see 412's
in the <D:status> elements of the multistatus response, without any
corresponding propertied returned. Assuming, of course, that the server
would evaluate the entire resource set at all... This would additionally
require some mechanism for the client to indicate that it wanted the
header applied beyond the target resource. (Along the lines of the DAV
header, as you previously indicated, or otherwise.)

Yes, expensive, but perhaps not much more than the evaluation of the
PROPFIND itself, without any If- headers.


>> If-None-Match is more interesting. For this, we need something akin
>> to CTags. I thought this had been discussed on the dist-auth list [...]
> I'm not convinced we need those. We really should make all of this
> more compatible with GET, and rely on ETags.
That would certainly simplify things, if possible.   :)


>> [...] Have you considered this at any length before?
> I'm currently not in the business of writing servers. That being said,
> change tags "bubbling up" will be very expensive to implement in many
> servers.
Yes, agreed, especially where there is a 1:1 correspondence between
[E/C]Tag updates and file I/O operations. I'm generally more willing to
take this hit on the authoring side if it translates into comparable
savings on the read side, and doubly so in domains that have better than
an order of magnitude more reads than writes. IMO, this covers a the
lions share of web apps.


> It would probably be a good idea to look at the problem that needs to
> be solved first. Maybe defining a related resource that aggregates all
> this hierarchy state (and which as a proper ETag) is all that's needed.
Possibly -- do you think such a resource could be generated more cheaply
than maintaining hierarchies of [E/C]Tags? I think this depends largely
on the implementation details of a given server architecture.



Regards,
Elias

Reply | Threaded
Open this post in threaded view
|

Re: 304 or 412 for If-Modified-Since?

Julian Reschke
On 01.07.2010 10:53, Elias Sinderson wrote:

> Julian Reschke wrote:
>> On 01.07.2010 01:37, Elias Sinderson wrote:
>>> [...] For If-Modified-Since, depth requests can only really be evaluated
>>> correctly by iterating over the entire set of resources in scope. [...]
>> I'm not completely sure I understand.
>>
>> Are you saying that the overall response code (not the multistatus
>> body) for a PROPFIND with Depth:infinity and If-Modified must take the
>> last-modified dates for all resources in scope into account?
> Sorry, I should have been more specific -- I would expect to see 412's
> in the <D:status> elements of the multistatus response, without any
> corresponding propertied returned. Assuming, of course, that the server
> would evaluate the entire resource set at all... This would additionally
> require some mechanism for the client to indicate that it wanted the
> header applied beyond the target resource. (Along the lines of the DAV
> header, as you previously indicated, or otherwise.)

The Depth header? I think this is what the definition of the Depth
header already requires.

> Yes, expensive, but perhaps not much more than the evaluation of the
> PROPFIND itself, without any If- headers.

PROPFIND/Depth infinity is already expensive (and thus many servers do
not allow it anyway). When they do, they better *stream* the result to
the client. So, if there was a requirement to test all resources in
scope against a condition *before* starting to send the multistatus this
would be bad.

>>> If-None-Match is more interesting. For this, we need something akin
>>> to CTags. I thought this had been discussed on the dist-auth list [...]
>> I'm not convinced we need those. We really should make all of this
>> more compatible with GET, and rely on ETags.
> That would certainly simplify things, if possible. :)
>
>
>>> [...] Have you considered this at any length before?
>> I'm currently not in the business of writing servers. That being said,
>> change tags "bubbling up" will be very expensive to implement in many
>> servers.
> Yes, agreed, especially where there is a 1:1 correspondence between
> [E/C]Tag updates and file I/O operations. I'm generally more willing to
> take this hit on the authoring side if it translates into comparable
> savings on the read side, and doubly so in domains that have better than
> an order of magnitude more reads than writes. IMO, this covers a the
> lions share of web apps.
>
>
>> It would probably be a good idea to look at the problem that needs to
>> be solved first. Maybe defining a related resource that aggregates all
>> this hierarchy state (and which as a proper ETag) is all that's needed.
> Possibly -- do you think such a resource could be generated more cheaply
> than maintaining hierarchies of [E/C]Tags? I think this depends largely
> on the implementation details of a given server architecture.

I'd think of it as a different way to put things on the wire, not really
changing what the server needs to do in the end.

It would be great if we can avoid adding even more stuff to WebDAV, and
re-use more of plain HTTP instead.

See, for instance:
<http://greenbytes.de/tech/webdav/draft-reschke-http-get-location-latest.html>

Best regards, Julian

Reply | Threaded
Open this post in threaded view
|

Re: 304 or 412 for If-Modified-Since?

Cyrus Daboo-2
In reply to this post by Elias Sinderson-2
Hi Elias,

--On July 1, 2010 6:53:08 PM +1000 Elias Sinderson <[hidden email]>
wrote:

>> It would probably be a good idea to look at the problem that needs to
>> be solved first. Maybe defining a related resource that aggregates all
>> this hierarchy state (and which as a proper ETag) is all that's needed.
> Possibly -- do you think such a resource could be generated more cheaply
> than maintaining hierarchies of [E/C]Tags? I think this depends largely
> on the implementation details of a given server architecture.
>

So the WebDAV collection Sync REPORT is meant to take over the role of a
ctag and provide a true "what changed" synchronization capability. Please
review past messages on this mailing list.

The one open issue on that REPORT (that I need to follow up on) is whether
a true Depth:infinity capability should be added to the current Depth:1
behavior.

--
Cyrus Daboo