HTTP/2 Upgrade with content?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

HTTP/2 Upgrade with content?

Greg Wilkins-3


Section 3.2 describes the upgrade to HTTP/2 and it allows support for upgrade requests with bodies:
   Requests that contain an entity body MUST be sent in their entirety
   before the client can send HTTP/2 frames. This means that a large
   request entity can block the use of the connection until it is
   completely sent.
Servers will need to protect themselves from DoS attacks via such requests as buffering arbitrary large content in their entirety is a commitment that servers cannot generally give.

Thus servers will have to limit the size of the entities they are prepared to hold in this situation (and the size of a single normal request buffers is probably the memory commitment they are prepared to make for any given connection).

My question is, what should a server do if it receives an otherwise valid upgrade request that it could handle but with content that exceeds this memory limit?      Should it respond with a 413 REQUEST_ENTITY_TOO_LARGE or should it just ignore the upgrade and let the request be handled via HTTP/1.1 (which can stream the content into the request handler and it becomes somebody else's problem to limit memory usage).

My problem with ignoring the upgrade is that it is an arbitrary limit and it will be hard for clients to tell why some upgrades work and others do not.

Alternately my problem with 413 is that some servers might wish to avoid the whole upgrade with content path and thus send a 413 for any upgrade with content, which may break some clients that could otherwise proceed with HTTP/1.1

thoughts?

PS. in hindsight, I would rather that we had not allowed upgrades with content and instead told clients to upgrade with an OPTION request prior to any PUT/POST request.... gallop... gallop... gallop.... SLAM!



--
Greg Wilkins <[hidden email]>  @  Webtide - an Intalio subsidiary
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.
Reply | Threaded
Open this post in threaded view
|

RE: HTTP/2 Upgrade with content?

Mike Bishop

A server is blocked on the connection, just as it would be in HTTP/1.1.  The situation for the server is no worse than if the client weren’t offering Upgrade.  Servers will enforce the same limits of large bodies they’re not willing to handle.  Issuing a 413, telling the client that the reason their request isn’t being serviced is due to the size of the body, is confusing if the actual way to make it be serviced is to omit the Upgrade header.  That’s a bizarre use of 413.

 

My inclination would be for servers to ignore the Upgrade header if they don’t want to be blocked.  Clients will always need to handle Upgrade headers being ignored, since they can legitimately be stripped by intermediaries.  Clients will never understand all the reasons why some Upgrades work and some don’t – they have to handle both cases cleanly.

 

From: Greg Wilkins [mailto:[hidden email]]
Sent: Thursday, March 12, 2015 4:10 PM
To: HTTP Working Group
Subject: HTTP/2 Upgrade with content?

 

 

Section 3.2 describes the upgrade to HTTP/2 and it allows support for upgrade requests with bodies:

   Requests that contain an entity body MUST be sent in their entirety
   before the client can send HTTP/2 frames. This means that a large
   request entity can block the use of the connection until it is
   completely sent.

Servers will need to protect themselves from DoS attacks via such requests as buffering arbitrary large content in their entirety is a commitment that servers cannot generally give.

Thus servers will have to limit the size of the entities they are prepared to hold in this situation (and the size of a single normal request buffers is probably the memory commitment they are prepared to make for any given connection).

 

My question is, what should a server do if it receives an otherwise valid upgrade request that it could handle but with content that exceeds this memory limit?      Should it respond with a 413 REQUEST_ENTITY_TOO_LARGE or should it just ignore the upgrade and let the request be handled via HTTP/1.1 (which can stream the content into the request handler and it becomes somebody else's problem to limit memory usage).

My problem with ignoring the upgrade is that it is an arbitrary limit and it will be hard for clients to tell why some upgrades work and others do not.

Alternately my problem with 413 is that some servers might wish to avoid the whole upgrade with content path and thus send a 413 for any upgrade with content, which may break some clients that could otherwise proceed with HTTP/1.1

thoughts?

 

PS. in hindsight, I would rather that we had not allowed upgrades with content and instead told clients to upgrade with an OPTION request prior to any PUT/POST request.... gallop... gallop... gallop.... SLAM!



--

Greg Wilkins <[hidden email]>  @  Webtide - an Intalio subsidiary
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Greg Wilkins-3

Mike,

the requirement of 3.2 is different to a normal HTTP/1.1 request.   Normally there is no requirement for a server to buffer the entire request body before commencing handing of a normal HTTP/1.1 request.    Typically the application is called and a streaming API used to provide the content to the request handler.   Note that this is not a block or not to block question - modern HTTP/1.1 servers are perfectly capable of consuming content via asynchronous IO.

Thus from the servers point of view, the memory commitment required to service a single connection is the buffer size that it uses to read the request content.   Now applications might then aggregate those buffers and attempt to hold the entire content in memory, but that is an application responsibility and the server cannot do much about that.

This requirement is different.  It is allowing an arbitrary sized body content to be sent in the HTTP/1.1 request that must be held by the server so that it can be fed to the HTTP/2 handling of the request that takes place after the upgrade and after the 101 has been sent.

I guess technically the new HTTP2 connection could work in a mode where it receives content as HTTP/1.1 but sends the response as HTTP/2.... but that is a) stupidly complex for a mechanism that browser say they will not implement anyway; b) just asking for a deadlock if the response is large and requires flow control frames to be received that cannot be sent until the entire body is consumed.

But you point about upgrades failing for all sorts of reasons is a valid one.  So I agree that 413 is not the right response and that simply ignoring the upgrade on requests that have too large bodies (or perhaps any body at all for simplicity), is probably the right thing to do.

cheers






On 13 March 2015 at 10:47, Mike Bishop <[hidden email]> wrote:

A server is blocked on the connection, just as it would be in HTTP/1.1.  The situation for the server is no worse than if the client weren’t offering Upgrade.  Servers will enforce the same limits of large bodies they’re not willing to handle.  Issuing a 413, telling the client that the reason their request isn’t being serviced is due to the size of the body, is confusing if the actual way to make it be serviced is to omit the Upgrade header.  That’s a bizarre use of 413.

 

My inclination would be for servers to ignore the Upgrade header if they don’t want to be blocked.  Clients will always need to handle Upgrade headers being ignored, since they can legitimately be stripped by intermediaries.  Clients will never understand all the reasons why some Upgrades work and some don’t – they have to handle both cases cleanly.

 

From: Greg Wilkins [mailto:[hidden email]]
Sent: Thursday, March 12, 2015 4:10 PM
To: HTTP Working Group
Subject: HTTP/2 Upgrade with content?

 

 

Section 3.2 describes the upgrade to HTTP/2 and it allows support for upgrade requests with bodies:

   Requests that contain an entity body MUST be sent in their entirety
   before the client can send HTTP/2 frames. This means that a large
   request entity can block the use of the connection until it is
   completely sent.

Servers will need to protect themselves from DoS attacks via such requests as buffering arbitrary large content in their entirety is a commitment that servers cannot generally give.

Thus servers will have to limit the size of the entities they are prepared to hold in this situation (and the size of a single normal request buffers is probably the memory commitment they are prepared to make for any given connection).

 

My question is, what should a server do if it receives an otherwise valid upgrade request that it could handle but with content that exceeds this memory limit?      Should it respond with a 413 REQUEST_ENTITY_TOO_LARGE or should it just ignore the upgrade and let the request be handled via HTTP/1.1 (which can stream the content into the request handler and it becomes somebody else's problem to limit memory usage).

My problem with ignoring the upgrade is that it is an arbitrary limit and it will be hard for clients to tell why some upgrades work and others do not.

Alternately my problem with 413 is that some servers might wish to avoid the whole upgrade with content path and thus send a 413 for any upgrade with content, which may break some clients that could otherwise proceed with HTTP/1.1

thoughts?

 

PS. in hindsight, I would rather that we had not allowed upgrades with content and instead told clients to upgrade with an OPTION request prior to any PUT/POST request.... gallop... gallop... gallop.... SLAM!



--

Greg Wilkins <[hidden email]>  @  Webtide - an Intalio subsidiary
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.




--
Greg Wilkins <[hidden email]>  @  Webtide - an Intalio subsidiary
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.
Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Stefan Eissing
Thanks for bringing this up. I am at the same branch in the road in my mod_h2 implementation. 

So far, I decided to silently skip the upgrade on any request with a body, to make it predictable for a client. Not to succeed in a small test case and fail in production. 

It would be good to
- hear from clients what they prefer/expect. there could be a response header "upgrade: no-conten" or such
- get the same server behaviour across implementations for interop purposes

Cheers,

  Stefan



Am 13.03.2015 um 02:04 schrieb Greg Wilkins <[hidden email]>:


Mike,

the requirement of 3.2 is different to a normal HTTP/1.1 request.   Normally there is no requirement for a server to buffer the entire request body before commencing handing of a normal HTTP/1.1 request.    Typically the application is called and a streaming API used to provide the content to the request handler.   Note that this is not a block or not to block question - modern HTTP/1.1 servers are perfectly capable of consuming content via asynchronous IO.

Thus from the servers point of view, the memory commitment required to service a single connection is the buffer size that it uses to read the request content.   Now applications might then aggregate those buffers and attempt to hold the entire content in memory, but that is an application responsibility and the server cannot do much about that.

This requirement is different.  It is allowing an arbitrary sized body content to be sent in the HTTP/1.1 request that must be held by the server so that it can be fed to the HTTP/2 handling of the request that takes place after the upgrade and after the 101 has been sent.

I guess technically the new HTTP2 connection could work in a mode where it receives content as HTTP/1.1 but sends the response as HTTP/2.... but that is a) stupidly complex for a mechanism that browser say they will not implement anyway; b) just asking for a deadlock if the response is large and requires flow control frames to be received that cannot be sent until the entire body is consumed.

But you point about upgrades failing for all sorts of reasons is a valid one.  So I agree that 413 is not the right response and that simply ignoring the upgrade on requests that have too large bodies (or perhaps any body at all for simplicity), is probably the right thing to do.

cheers






On 13 March 2015 at 10:47, Mike Bishop <[hidden email]> wrote:

A server is blocked on the connection, just as it would be in HTTP/1.1.  The situation for the server is no worse than if the client weren’t offering Upgrade.  Servers will enforce the same limits of large bodies they’re not willing to handle.  Issuing a 413, telling the client that the reason their request isn’t being serviced is due to the size of the body, is confusing if the actual way to make it be serviced is to omit the Upgrade header.  That’s a bizarre use of 413.

 

My inclination would be for servers to ignore the Upgrade header if they don’t want to be blocked.  Clients will always need to handle Upgrade headers being ignored, since they can legitimately be stripped by intermediaries.  Clients will never understand all the reasons why some Upgrades work and some don’t – they have to handle both cases cleanly.

 

From: Greg Wilkins [mailto:[hidden email]]
Sent: Thursday, March 12, 2015 4:10 PM
To: HTTP Working Group
Subject: HTTP/2 Upgrade with content?

 

 

Section 3.2 describes the upgrade to HTTP/2 and it allows support for upgrade requests with bodies:

   Requests that contain an entity body MUST be sent in their entirety
   before the client can send HTTP/2 frames. This means that a large
   request entity can block the use of the connection until it is
   completely sent.

Servers will need to protect themselves from DoS attacks via such requests as buffering arbitrary large content in their entirety is a commitment that servers cannot generally give.

Thus servers will have to limit the size of the entities they are prepared to hold in this situation (and the size of a single normal request buffers is probably the memory commitment they are prepared to make for any given connection).

 

My question is, what should a server do if it receives an otherwise valid upgrade request that it could handle but with content that exceeds this memory limit?      Should it respond with a 413 REQUEST_ENTITY_TOO_LARGE or should it just ignore the upgrade and let the request be handled via HTTP/1.1 (which can stream the content into the request handler and it becomes somebody else's problem to limit memory usage).

My problem with ignoring the upgrade is that it is an arbitrary limit and it will be hard for clients to tell why some upgrades work and others do not.

Alternately my problem with 413 is that some servers might wish to avoid the whole upgrade with content path and thus send a 413 for any upgrade with content, which may break some clients that could otherwise proceed with HTTP/1.1

thoughts?

 

PS. in hindsight, I would rather that we had not allowed upgrades with content and instead told clients to upgrade with an OPTION request prior to any PUT/POST request.... gallop... gallop... gallop.... SLAM!



--

Greg Wilkins <[hidden email]>  @  Webtide - an Intalio subsidiary
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.




--
Greg Wilkins <[hidden email]>  @  Webtide - an Intalio subsidiary
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.
Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Daniel Stenberg
On Fri, 13 Mar 2015, Stefan Eissing wrote:

> - hear from clients what they prefer/expect. there could be a response
> header "upgrade: no-conten" or such

Not accepting a POST immediately will be surprising to users. There are quite
a few users in the wild today that run curl like this:

  $ curl -d lotsofdata [URL]

which thus sends a POST immediately with that data over HTTP/1.1, and the same
thing for HTTP/2 is almost identical, just adding asking for http2:

  $ curl --http2 -d lotsofdata [URL]

which will make a POST with an Upgrade: h2c header and Expect: 100-continue.

If a server then suddenly doesn't handle the POST + upgrade it puts a rather
arbitrary (from the client's perspective) block on when a client can or
cannot switch to HTTP/2 on such a server.

--

  / daniel.haxx.se

Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Jason Greene
In reply to this post by Greg Wilkins-3
What about draining the content and after the 101 the h2 response sends a 307?

On Mar 12, 2015, at 6:14 PM, Greg Wilkins <[hidden email]> wrote:



Section 3.2 describes the upgrade to HTTP/2 and it allows support for upgrade requests with bodies:
   Requests that contain an entity body MUST be sent in their entirety
   before the client can send HTTP/2 frames. This means that a large
   request entity can block the use of the connection until it is
   completely sent.
Servers will need to protect themselves from DoS attacks via such requests as buffering arbitrary large content in their entirety is a commitment that servers cannot generally give.

Thus servers will have to limit the size of the entities they are prepared to hold in this situation (and the size of a single normal request buffers is probably the memory commitment they are prepared to make for any given connection).

My question is, what should a server do if it receives an otherwise valid upgrade request that it could handle but with content that exceeds this memory limit?      Should it respond with a 413 REQUEST_ENTITY_TOO_LARGE or should it just ignore the upgrade and let the request be handled via HTTP/1.1 (which can stream the content into the request handler and it becomes somebody else's problem to limit memory usage).

My problem with ignoring the upgrade is that it is an arbitrary limit and it will be hard for clients to tell why some upgrades work and others do not.

Alternately my problem with 413 is that some servers might wish to avoid the whole upgrade with content path and thus send a 413 for any upgrade with content, which may break some clients that could otherwise proceed with HTTP/1.1

thoughts?

PS. in hindsight, I would rather that we had not allowed upgrades with content and instead told clients to upgrade with an OPTION request prior to any PUT/POST request.... gallop... gallop... gallop.... SLAM!



--
Greg Wilkins <[hidden email]>  @  Webtide - an Intalio subsidiary
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.
Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Cory Benfield
In reply to this post by Daniel Stenberg
On 13 March 2015 at 07:40, Daniel Stenberg <[hidden email]> wrote:

> Not accepting a POST immediately will be surprising to users. There are
> quite a few users in the wild today that run curl like this:
>
>  $ curl -d lotsofdata [URL]
>
> which thus sends a POST immediately with that data over HTTP/1.1, and the
> same thing for HTTP/2 is almost identical, just adding asking for http2:
>
>  $ curl --http2 -d lotsofdata [URL]
>
> which will make a POST with an Upgrade: h2c header and Expect: 100-continue.
>
> If a server then suddenly doesn't handle the POST + upgrade it puts a rather
> arbitrary (from the client's perspective) block on when a client can or
> cannot switch to HTTP/2 on such a server.

Agreed from hyper's perspective as well.

We can always add logic to only send HTTP upgrade headers on requests
without bodies, but it feels like a weird restriction.

Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Stefan Eissing
In reply to this post by Daniel Stenberg
Daniel,

do you mean curl will, after seeing a 101 response, send the body on the HTTP2 stream 1? That could be made to work...

//Stefan

> Am 13.03.2015 um 08:40 schrieb Daniel Stenberg <[hidden email]>:
>
> On Fri, 13 Mar 2015, Stefan Eissing wrote:
>
>> - hear from clients what they prefer/expect. there could be a response header "upgrade: no-conten" or such
>
> Not accepting a POST immediately will be surprising to users. There are quite a few users in the wild today that run curl like this:
>
> $ curl -d lotsofdata [URL]
>
> which thus sends a POST immediately with that data over HTTP/1.1, and the same thing for HTTP/2 is almost identical, just adding asking for http2:
>
> $ curl --http2 -d lotsofdata [URL]
>
> which will make a POST with an Upgrade: h2c header and Expect: 100-continue.
>
> If a server then suddenly doesn't handle the POST + upgrade it puts a rather arbitrary (from the client's perspective) block on when a client can or cannot switch to HTTP/2 on such a server.
>
> --
>
> / daniel.haxx.se

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782




Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Daniel Stenberg
On Fri, 13 Mar 2015, Stefan Eissing wrote:

> do you mean curl will, after seeing a 101 response, send the body on the
> HTTP2 stream 1? That could be made to work...

No, that won't work.

First, without "Expect: 100-continue" in the request the client will send the
body immediately without waiting for anything so there will be no chance to
send the 101 until after the request-body.

And _with_ "Expect: 100-continue" we've already concluded that 100 must be
sent back before the 101 and that will then make the body get sent as well
before the 101 will be considered!

The Expect+Upgrade discussion thread I'm referring to can be found here:

   https://lists.w3.org/Archives/Public/ietf-http-wg/2013JulSep/1047.html

--

  / daniel.haxx.se

Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Ilari Liusvaara
In reply to this post by Jason Greene
On Fri, Mar 13, 2015 at 04:28:46AM -0400, Jason T. Greene wrote:
> What about draining the content and after the 101 the h2 response
> sends a 307?

Might work if you never send a 100 (including implicitly by waiting
too long). 100-Continue may mean that the data transfer can't
be retried after it starts.

And if the client app is has simplistic redirect checking (just
count redirects), one could even 307 to self...


I.e, as flow:
- POST to foo (with upgrade to h2c and 100-expect)
- 101 to h2c
- 307 to foo
- POST to foo (with 100-expect)
- 100
- <payload>
- 200


-Ilari

Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Stefan Eissing
In reply to this post by Daniel Stenberg
Thanks for the link to the discussion, very helpful. I might be able to create a request in the server that reads a HTTP/1.1 body and produces a HTTP/2 response. Since the body can be read without any flow control needing to happen, that might just work. But there may be monsters lurking here, for example the server omitting a RST_STREAM on 1 while the http/1.1 body is still incomplete...

//Stefan

> Am 13.03.2015 um 10:34 schrieb Daniel Stenberg <[hidden email]>:
>
> On Fri, 13 Mar 2015, Stefan Eissing wrote:
>
>> do you mean curl will, after seeing a 101 response, send the body on the HTTP2 stream 1? That could be made to work...
>
> No, that won't work.
>
> First, without "Expect: 100-continue" in the request the client will send the body immediately without waiting for anything so there will be no chance to send the 101 until after the request-body.
>
> And _with_ "Expect: 100-continue" we've already concluded that 100 must be sent back before the 101 and that will then make the body get sent as well before the 101 will be considered!
>
> The Expect+Upgrade discussion thread I'm referring to can be found here:
>
>  https://lists.w3.org/Archives/Public/ietf-http-wg/2013JulSep/1047.html
>
> --
>
> / daniel.haxx.se
>

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782




Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Amos Jeffries-2
In reply to this post by Ilari Liusvaara
On 13/03/2015 10:42 p.m., Ilari Liusvaara wrote:

> On Fri, Mar 13, 2015 at 04:28:46AM -0400, Jason T. Greene wrote:
>> What about draining the content and after the 101 the h2 response
>> sends a 307?
>
> Might work if you never send a 100 (including implicitly by waiting
> too long). 100-Continue may mean that the data transfer can't
> be retried after it starts.
>
> And if the client app is has simplistic redirect checking (just
> count redirects), one could even 307 to self...
>
>
> I.e, as flow:
> - POST to foo (with upgrade to h2c and 100-expect)
> - 101 to h2c
> - 307 to foo

missing:
 - <complete payload in 1.x format dumped into the pipe>

> - POST to foo (with 100-expect)
> - 100
> - <payload>
> - 200
>
>
> -Ilari
>

No matter what you do the client has started sending a request specifid
as having a payload in HTTP/1 format. It must finish that request,
including the payload it promised, before any HTTP/2 may happen.
 - if the client is smart it would use chunked encoding then abandon
with 0-sized chunk on seeing the 30x. But that has other nasty problems,
and servers cant rely on it.

All these 30x tricks do is allow you to bit-bucket the 1.x payload
before moving on to the hopefully repeated 2.0 format one. It still
"blocks the connection" while the drain is happening - which may be GB
or TB of bandwidth and duration.

We have some prior painful experience applicable from NTLM vs POST
requests. In all cases there is large bandwidth wastage and latency.


However, that "blocking the connection" only means that the *client*
can't send other requests in parallel or h2 control frames. The server
is still free to PUSH_PROMISE etc. about what will follow once that 1.x
payload is done.


IMHO its actually simpler to implement a 1.1->2.0 converter and use it
as if it were a bit-bucket handler for the initial payload than to
handle all the edges cases involved when avoiding 1.x. Still not nice,
but simpler than any fancy dances with a mandatory bit-bucket anyway.


There is also the possibility of just using 2.0 natively from the start.
Manual tools and web apps with server knowledge should have no problems
just using 2.0 when they know the other end is.


Amos


Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Amos Jeffries-2
In reply to this post by Stefan Eissing
On 13/03/2015 11:03 p.m., Stefan Eissing wrote:
>
> Thanks for the link to the discussion, very helpful. I might be able
> to create a request in the server that reads a HTTP/1.1 body and
> produces a HTTP/2 response. Since the body can be read without any flow
> control needing to happen, that might just work. But there may be
> monsters lurking here, for example the server omitting a RST_STREAM on 1
> while the http/1.1 body is still incomplete...

RST_STREAM in 1.1 terms is a TCP RST on the connection. The client
sending 1.1 needs to treat it that way while it is still sending the 1.1
message.

If the client has finished sending it can (should? will?) treat it as
2.0 RST_STREAM on a closed stream. Eventually the end of the payload
will arrive at the server and things are good.

A server sending RST_STREAM 1 before its finished reading that 1.1
payload is shooting itself in the foot potentially and needs to handle
both those possibilities. Implementation choice.

Amos


Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Ilari Liusvaara
In reply to this post by Amos Jeffries-2
On Fri, Mar 13, 2015 at 11:43:11PM +1300, Amos Jeffries wrote:
>
> No matter what you do the client has started sending a request specifid
> as having a payload in HTTP/1 format. It must finish that request,
> including the payload it promised, before any HTTP/2 may happen.
>  - if the client is smart it would use chunked encoding then abandon
> with 0-sized chunk on seeing the 30x. But that has other nasty problems,
> and servers cant rely on it.

Sigh.

And if the client did 100-Continue, starts sending and gets 307, there
is real risk that client has no choice but treat that as a fatal error
(will not retry, report failure back to user/caller).

> We have some prior painful experience applicable from NTLM vs POST
> requests. In all cases there is large bandwidth wastage and latency.
 
And in some cases, things just plain won't work...


-Ilari

Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Jason Greene
In reply to this post by Daniel Stenberg

> On Mar 13, 2015, at 2:44 AM, Daniel Stenberg <[hidden email]> wrote:
>
> $ curl --http2 -d lotsofdata [URL]
>
> which will make a POST with an Upgrade: h2c header and Expect: 100-continue.

Why not attempt a PRI with the explicit http2 option?
Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Greg Wilkins-3


On 14 March 2015 at 00:09, Jason T. Greene <[hidden email]> wrote:

> On Mar 13, 2015, at 2:44 AM, Daniel Stenberg <[hidden email]> wrote:
>
> $ curl --http2 -d lotsofdata [URL]
>
> which will make a POST with an Upgrade: h2c header and Expect: 100-continue.

Why not attempt a PRI with the explicit http2 option?

Yes that would be a lot better and we already support it in jetty.
The issue with the curl usecase using upgrade is that all the upgrade work is essentially wasted effort - as the bulk upload is done with HTTP/1.1, you then upgrade to a fully multiplexed HTTP/2 connection, negotiate a couple of settings, send a single 2xx response and then get a goaway frame to tear the whole lot down.    What value did the --http2 provide other than make work!

Failing the direct PRI approach, it would be far better to try the upgrade on an OPTION request before commencing on the big post/put.

So currently implemented in jetty as an ignore of any h2c upgrade in a request with a body.

cheers


--
Greg Wilkins <[hidden email]>  @  Webtide - an Intalio subsidiary
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.
Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Daniel Stenberg
On Sat, 14 Mar 2015, Greg Wilkins wrote:

>> Why not attempt a PRI with the explicit http2 option?
>
> Yes that would be a lot better and we already support it in jetty.

Sure, curl will support that too soon but I think that is beside the point. In
this case users can use this option on virtually any server without knowing
whether it supports version 2 or not. Like Upgrade: is supposed to work!

> The issue with the curl usecase using upgrade is that all the upgrade work
> is essentially wasted effort - as the bulk upload is done with HTTP/1.1,

Yes, but... for example, what if that is the first operation out of several,
and then once the first operation has updated to version 2 all the subsequent
ones can continue using HTTP/2. And since the upgrade, it would just work with
1.1 servers as we all as 2 servers without the client having to know
before-hand.

And in the end, we usually allow curl to exercise every possible protocol
option that is valid (and sometimes also a few that aren't valid) to allow
people like readers of this list to test out and torture our own and others
servers!

--

  / daniel.haxx.se

Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Stefan Eissing
Well, this discussion quickly dove into the quantum foam between specification and deployed software.

I take away from it:
- that protocol switching happens immediately after the 101 response - for the downstream. On the upstream it switches after the request has been read. Pretty obvious in hindsight, as those things are.
- servers should be predictable in upgrading, either always with content or only without.
- the common use case for upgrade on POST does not benefit from h2c and this it seems not worth the effort to implement. Fullfilling the request on HTTP/1.1 format seems to work fine.
- maybe a mixed 1.1 up and h2c down request could be made to work, but there seems no gain for anyone in creating this mythical beast.

//Stefan

> Am 13.03.2015 um 15:04 schrieb Daniel Stenberg <[hidden email]>:
>
> On Sat, 14 Mar 2015, Greg Wilkins wrote:
>
>>> Why not attempt a PRI with the explicit http2 option?
>>
>> Yes that would be a lot better and we already support it in jetty.
>
> Sure, curl will support that too soon but I think that is beside the point. In this case users can use this option on virtually any server without knowing whether it supports version 2 or not. Like Upgrade: is supposed to work!
>
>> The issue with the curl usecase using upgrade is that all the upgrade work
>> is essentially wasted effort - as the bulk upload is done with HTTP/1.1,
>
> Yes, but... for example, what if that is the first operation out of several, and then once the first operation has updated to version 2 all the subsequent ones can continue using HTTP/2. And since the upgrade, it would just work with 1.1 servers as we all as 2 servers without the client having to know before-hand.
>
> And in the end, we usually allow curl to exercise every possible protocol option that is valid (and sometimes also a few that aren't valid) to allow people like readers of this list to test out and torture our own and others servers!

I think they call it "enhanced interrogation" nowadays.

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782




Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Roland Zink
On 13.03.2015 15:21, Stefan Eissing wrote:

> Well, this discussion quickly dove into the quantum foam between specification and deployed software.
>
> I take away from it:
> - that protocol switching happens immediately after the 101 response - for the downstream. On the upstream it switches after the request has been read. Pretty obvious in hindsight, as those things are.
> - servers should be predictable in upgrading, either always with content or only without.
> - the common use case for upgrade on POST does not benefit from h2c and this it seems not worth the effort to implement. Fullfilling the request on HTTP/1.1 format seems to work fine.
> - maybe a mixed 1.1 up and h2c down request could be made to work, but there seems no gain for anyone in creating this mythical beast.
>
> //Stefan
>
Another thing which the client may do if it needs to send a POST request
first and doesn't know if the server supports HTTP2 is to send an
OPTIONS request in front of the POST. Either the upgrade already works
for the OPTIONS request or it is then known the server will only support
HTTP/1.1.

Roland


Reply | Threaded
Open this post in threaded view
|

Re: HTTP/2 Upgrade with content?

Tatsuhiro Tsujikawa


On Fri, Mar 13, 2015 at 11:35 PM, Roland Zink <[hidden email]> wrote:
On 13.03.2015 15:21, Stefan Eissing wrote:
Well, this discussion quickly dove into the quantum foam between specification and deployed software.

I take away from it:
- that protocol switching happens immediately after the 101 response - for the downstream. On the upstream it switches after the request has been read. Pretty obvious in hindsight, as those things are.
- servers should be predictable in upgrading, either always with content or only without.
- the common use case for upgrade on POST does not benefit from h2c and this it seems not worth the effort to implement. Fullfilling the request on HTTP/1.1 format seems to work fine.
- maybe a mixed 1.1 up and h2c down request could be made to work, but there seems no gain for anyone in creating this mythical beast.

//Stefan

Another thing which the client may do if it needs to send a POST request first and doesn't know if the server supports HTTP2 is to send an OPTIONS request in front of the POST. Either the upgrade already works for the OPTIONS request or it is then known the server will only support HTTP/1.1.


In nghttp2 project, ​nghttp client does this approach: Use OPTIONS with POST for h2c upgrade.
nghttpx proxy only performs HTTP/2 upgrade if request has no request body, which I think has some popularity in this thread.  The reason why we chose this approach is... well we just defer the complicated effort to the future, but in retrospect, I think it is a good compromise for simplicity.

​Best regards,
Tatsuhiro Tsujikawa​

12