Connection limits

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Connection limits

Mark Nottingham-4

RFC2616 Section 8.1.4, Practical Considerations:

> Clients that use persistent connections SHOULD limit the number of  
> simultaneous connections that they maintain to a given server. A  
> single-user client SHOULD NOT maintain more than 2 connections with  
> any server or proxy. A proxy SHOULD use up to 2*N connections to  
> another server or proxy, where N is the number of simultaneously  
> active users. These guidelines are intended to improve HTTP response  
> times and avoid congestion.


I'm not sure I want to suggest that these limits should be changed,  
but I think it's worth a discussion. I've seen a number of cases where;
    * resources have been spread across multiple servers, just to work  
around this limitation
    * client-side frameworks have been designed to work around this  
limitation by batching multiple representations together (removing the  
possibility of caching)
    * because of low adoption of pipelining, two slow responses  
blocking an entire Web page
    * servers purposefully answer HTTP/1.0, because some clients will  
use four connections with them

Also, considering the wider availability of event-driven Web servers  
and intermediaries, resource limitations on servers isn't necessarily  
the problem it once was.

What do people think? Should we consider upping this to 4? There's  
also the possibility of negotiating a higher number of connections hop-
by-hop (but that would be outside of HTTPBIS).

Cheers,


--
Mark Nottingham       [hidden email]



Reply | Threaded
Open this post in threaded view
|

Re: Connection limits

David Morris-4


I think that in the contect of where the web has evolved, we should move
this to the category of a recommendation. The nature of the beast I
observe is absurd numbers of objects retrieved to compose invividual pages
(e.g., I recently counted 190+ objects on a single major new site page).

Given the 'two' rule, all it takes is two slow objects to seriously block
presentation of the page to the user. If I were the author of an end user
user agent (aka browser) the motivation would be to get many more than
2 parallel retrievals going. Based on this paragraph, the alternative is
to use discrete connections??? Not really better. A resource constrained
server can always opt to close connections either on the basis of too many
from a given client or too many total.

These limits were established in the days when an end user was luck to
have a 128kbps connection to the internet. With the latest 10mbps and
higher consumer grade connections and servers with many GB of memory,
defining limits in the HTTP protocol doesn't make sense to me.

Our mythical implementation guide could address the principles here
much more productively than the specification.

Dave Morris

On Wed, 5 Mar 2008, Mark Nottingham wrote:

>
> RFC2616 Section 8.1.4, Practical Considerations:
>
> > Clients that use persistent connections SHOULD limit the number of
> > simultaneous connections that they maintain to a given server. A
> > single-user client SHOULD NOT maintain more than 2 connections with
> > any server or proxy. A proxy SHOULD use up to 2*N connections to
> > another server or proxy, where N is the number of simultaneously
> > active users. These guidelines are intended to improve HTTP response
> > times and avoid congestion.
>
>
> I'm not sure I want to suggest that these limits should be changed,
> but I think it's worth a discussion. I've seen a number of cases where;
>     * resources have been spread across multiple servers, just to work
> around this limitation
>     * client-side frameworks have been designed to work around this
> limitation by batching multiple representations together (removing the
> possibility of caching)
>     * because of low adoption of pipelining, two slow responses
> blocking an entire Web page
>     * servers purposefully answer HTTP/1.0, because some clients will
> use four connections with them
>
> Also, considering the wider availability of event-driven Web servers
> and intermediaries, resource limitations on servers isn't necessarily
> the problem it once was.
>
> What do people think? Should we consider upping this to 4? There's
> also the possibility of negotiating a higher number of connections hop-
> by-hop (but that would be outside of HTTPBIS).
>
> Cheers,
>
>
> --
> Mark Nottingham       [hidden email]
>
>

Reply | Threaded
Open this post in threaded view
|

RE: Connection limits

Josh Cohen (MIG)

I would agree.

The connection limit, in addition to the bandwidth limitations of the time, also helped the servers themselves.  Back then, maintaining many simultaneous but short lived TCP connections was inefficient for many operating systems TCP stacks.  By switching to fewer, longer standing connections, and the hope of the use of pipelining, we thought we'd address that.
Nowadays with stacks much more efficiently tuned to handle these types of connections efficiently, and the non-occurrence of pipelining, I think this could be relaxed a bit.

(speaking for myself, not an official company position)


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of David Morris
Sent: Wednesday, March 05, 2008 12:30 AM
To: Mark Nottingham
Cc: [hidden email] Group
Subject: Re: Connection limits



I think that in the contect of where the web has evolved, we should move
this to the category of a recommendation. The nature of the beast I
observe is absurd numbers of objects retrieved to compose invividual pages
(e.g., I recently counted 190+ objects on a single major new site page).

Given the 'two' rule, all it takes is two slow objects to seriously block
presentation of the page to the user. If I were the author of an end user
user agent (aka browser) the motivation would be to get many more than
2 parallel retrievals going. Based on this paragraph, the alternative is
to use discrete connections??? Not really better. A resource constrained
server can always opt to close connections either on the basis of too many
from a given client or too many total.

These limits were established in the days when an end user was luck to
have a 128kbps connection to the internet. With the latest 10mbps and
higher consumer grade connections and servers with many GB of memory,
defining limits in the HTTP protocol doesn't make sense to me.

Our mythical implementation guide could address the principles here
much more productively than the specification.

Dave Morris

On Wed, 5 Mar 2008, Mark Nottingham wrote:

>
> RFC2616 Section 8.1.4, Practical Considerations:
>
> > Clients that use persistent connections SHOULD limit the number of
> > simultaneous connections that they maintain to a given server. A
> > single-user client SHOULD NOT maintain more than 2 connections with
> > any server or proxy. A proxy SHOULD use up to 2*N connections to
> > another server or proxy, where N is the number of simultaneously
> > active users. These guidelines are intended to improve HTTP response
> > times and avoid congestion.
>
>
> I'm not sure I want to suggest that these limits should be changed,
> but I think it's worth a discussion. I've seen a number of cases where;
>     * resources have been spread across multiple servers, just to work
> around this limitation
>     * client-side frameworks have been designed to work around this
> limitation by batching multiple representations together (removing the
> possibility of caching)
>     * because of low adoption of pipelining, two slow responses
> blocking an entire Web page
>     * servers purposefully answer HTTP/1.0, because some clients will
> use four connections with them
>
> Also, considering the wider availability of event-driven Web servers
> and intermediaries, resource limitations on servers isn't necessarily
> the problem it once was.
>
> What do people think? Should we consider upping this to 4? There's
> also the possibility of negotiating a higher number of connections hop-
> by-hop (but that would be outside of HTTPBIS).
>
> Cheers,
>
>
> --
> Mark Nottingham       [hidden email]
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Connection limits

Roy T. Fielding

On Mar 5, 2008, at 4:52 AM, Josh Cohen (MIG) wrote:

> I would agree.
>
> The connection limit, in addition to the bandwidth limitations of  
> the time, also helped the servers themselves.  Back then,  
> maintaining many simultaneous but short lived TCP connections was  
> inefficient for many operating systems TCP stacks.  By switching to  
> fewer, longer standing connections, and the hope of the use of  
> pipelining, we thought we'd address that.
> Nowadays with stacks much more efficiently tuned to handle these  
> types of connections efficiently, and the non-occurrence of  
> pipelining, I think this could be relaxed a bit.

Just out of curiosity, why do people keep saying things like "non-
occurrence
of pipelining" when the vast majority of connections I've looked at over
the past six years contain obviously pipelined GET requests and  
occasional
pipelines of POST (in spite of the bogus requirement).  Is this just  
because
you do all your work behind a corporate firewall proxy?  Is this just
another myth?  Or am I the only one on the planet who looks at traces to
servers using both a client and a server that support pipelining?

Er, in regards to the topic, I see no reason for the connection limits.
They should be replaced with a simple statement of why too many  
connections
results in counterproductive collision/bandwidth effects.

....Roy


Reply | Threaded
Open this post in threaded view
|

Re: Connection limits

Adrien de Croy


I agree

If pipelining is sending another request on a connection before you get
a response, then as far as I can tell, all the major browsers by default
do it.

And I don't think it makes any sense to try and come up with a magic
number for connections.  The TCP overhead is a lot lower now with
persistent connections which are now ubiquitous, so maybe the
requirement for many connections is reduced, but policy decisions like
that should be left to server or proxy admin/management (people), not
hard coded into software or written into specs.

Adrien


Roy T. Fielding wrote:

>
> On Mar 5, 2008, at 4:52 AM, Josh Cohen (MIG) wrote:
>> I would agree.
>>
>> The connection limit, in addition to the bandwidth limitations of the
>> time, also helped the servers themselves.  Back then, maintaining
>> many simultaneous but short lived TCP connections was inefficient for
>> many operating systems TCP stacks.  By switching to fewer, longer
>> standing connections, and the hope of the use of pipelining, we
>> thought we'd address that.
>> Nowadays with stacks much more efficiently tuned to handle these
>> types of connections efficiently, and the non-occurrence of
>> pipelining, I think this could be relaxed a bit.
>
> Just out of curiosity, why do people keep saying things like
> "non-occurrence
> of pipelining" when the vast majority of connections I've looked at over
> the past six years contain obviously pipelined GET requests and
> occasional
> pipelines of POST (in spite of the bogus requirement).  Is this just
> because
> you do all your work behind a corporate firewall proxy?  Is this just
> another myth?  Or am I the only one on the planet who looks at traces to
> servers using both a client and a server that support pipelining?
>
> Er, in regards to the topic, I see no reason for the connection limits.
> They should be replaced with a simple statement of why too many
> connections
> results in counterproductive collision/bandwidth effects.
>
> ....Roy
>
>

--
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com


Reply | Threaded
Open this post in threaded view
|

Re: Connection limits

Jamie Lokier
In reply to this post by Mark Nottingham-4

Mark Nottingham wrote:
>    * resources have been spread across multiple servers, just to work  
> around this limitation

Yes.  Or multiple DNS names to the same server - anyone know if that's
effective at getting more connections?

>    * servers purposefully answer HTTP/1.0, because some clients will  
> use four connections with them

Ooh, that's interesting, thanks.

I have a problem with the 2 connections advice in RFC 2616 (it's
theoretical since I haven't implemented this).  I would like to
produce a web page which is fetching server-driven events using a
"delayed response" model, which needs 3 connections:

    1. One for ordinary content on the page: images etc.

    2. One for the delayed response stream, using the optimal method
       detected for that client.

    3. One for a second delayed response stream used to probe the
       optimal method for that client, without knowledge of specific
       clients, and without knowing the unknowable intermediaries, by
       trying methods which may fail (timeouts, intermediaries
       timeout, intermediaries buffer partial response without
       forwarding, client bugs), and upgrading the primary delayed
       response stream to use better methods when they are detected as
       reliable for this client and its intermediaries.

Should I ever implement this, I will try using multiple DNS names for
the same server and see if that is effective.  If not, I will have to
use multiple IPs as well as multiple DNS names.

More connections are also required for web pages which contain
_multiple_ applications of this style at the same time, running
independently on the same web page (perhaps written by different
vendors), but connecting to the same servers.

> Also, considering the wider availability of event-driven Web servers  
> and intermediaries, resource limitations on servers isn't necessarily  
> the problem it once was.
>
> What do people think? Should we consider upping this to 4? There's  
> also the possibility of negotiating a higher number of connections hop-
> by-hop (but that would be outside of HTTPBIS).

If a new revision of HTTP still has a recommended small limit, I think
it would be useful to have a request or response header meaning "don't
count this connection towards the usual limit for the duration of
this request".

I'm under the impression that the 2 connection limit is more
rigorously enforced by IE than Mozilla, as I have seen comments about
needing to work around it in discussions about delayed-GET techniques,
but I haven't tested this.

-- Jamie

Reply | Threaded
Open this post in threaded view
|

Re: Connection limits

Jamie Lokier
In reply to this post by David Morris-4

David Morris wrote:

> I think that in the contect of where the web has evolved, we should move
> this to the category of a recommendation. The nature of the beast I
> observe is absurd numbers of objects retrieved to compose invividual pages
> (e.g., I recently counted 190+ objects on a single major new site page).
>
> Given the 'two' rule, all it takes is two slow objects to seriously block
> presentation of the page to the user. If I were the author of an end user
> user agent (aka browser) the motivation would be to get many more than
> 2 parallel retrievals going. Based on this paragraph, the alternative is
> to use discrete connections??? Not really better. A resource constrained
> server can always opt to close connections either on the basis of too many
> from a given client or too many total.
>
> These limits were established in the days when an end user was luck to
> have a 128kbps connection to the internet. With the latest 10mbps and
> higher consumer grade connections and servers with many GB of memory,
> defining limits in the HTTP protocol doesn't make sense to me.
>
> Our mythical implementation guide could address the principles here
> much more productively than the specification.

>From a network efficiency point of view, the optimal way to send 190+
objects is to multiplex them into a single TCP/IP connection.  That's
better for lots and lots of reasons, affecting latency, bandwidth, and
interaction with TCP's backoff heuristics.

But, in HTTP, if an object takes time to generate, it will block the
delivery of other objects over few connections.  This does happen.

>From an efficiency and response POV, I suspect the best network
performance on all axes is to multiplex responses in any order onto
few or one stream(s), as they are generated at the server, including
splitting chunks (partial message multiplexing).  Same with requests,
in the case of large requests.  And with appropriate heuristics at the
endpoint, not just random order.

If we ever develop a multiplexing HTTP protocol (e.g. see SCTP and/or
BEEP), then one connection might be enough and optimal for most
applications. :-)

-- Jamie

Reply | Threaded
Open this post in threaded view
|

RE: Connection limits

Eric Lawrence-4
In reply to this post by Jamie Lokier
<<Should I ever implement this, I will try using multiple DNS names for
the same server and see if that is effective.  If not, I will have to
use multiple IPs as well as multiple DNS names.>>
 
Multiple DNS names alone should be completely effective.
 
<<If a new revision of HTTP still has a recommended small limit, I think
it would be useful to have a request or response header meaning "don't
count this connection towards the usual limit for the duration of
this request".>>
 
Agreed.  This is something that has come up in other areas, particularly those trying to build COMET-style applications.  Also suggested was enabling the server to communicate, via a response header, the desired maximum connections. 
 
<<I'm under the impression that the 2 connection limit is more
rigorously enforced by IE than Mozilla, as I have seen comments about
needing to work around it in discussions about delayed-GET techniques,
but I haven't tested this.>>
 
Firefox2+ has a very cool feature whereby they respect the 2-connections suggestion, unless a request has been queued for more than maxStartDelay seconds (10, by default, IIRC).  At that time, the request just goes out of the wire, ignoring the limit.  They then allow up to a maximum of 8 connections per server.  Opera simply defaults to a max of 8 connections per server.
 
For IE8, we have raised the default connections per host limit to 6 if the user is on a non-modem connection.  http://almaer.com/blog/death-of-www1-www2-thanks-to-connection-limit-raising
 
Eric Lawrence
Program Manager
Internet Explorer - Security

From: [hidden email] [[hidden email]] On Behalf Of Jamie Lokier [[hidden email]]
Sent: Wednesday, March 05, 2008 4:01 PM
To: Mark Nottingham
Cc: [hidden email] Group
Subject: Re: Connection limits


Mark Nottingham wrote:
>    * resources have been spread across multiple servers, just to work
> around this limitation

Yes.  Or multiple DNS names to the same server - anyone know if that's
effective at getting more connections?

>    * servers purposefully answer HTTP/1.0, because some clients will
> use four connections with them

Ooh, that's interesting, thanks.

I have a problem with the 2 connections advice in RFC 2616 (it's
theoretical since I haven't implemented this).  I would like to
produce a web page which is fetching server-driven events using a
"delayed response" model, which needs 3 connections:

    1. One for ordinary content on the page: images etc.

    2. One for the delayed response stream, using the optimal method
       detected for that client.

    3. One for a second delayed response stream used to probe the
       optimal method for that client, without knowledge of specific
       clients, and without knowing the unknowable intermediaries, by
       trying methods which may fail (timeouts, intermediaries
       timeout, intermediaries buffer partial response without
       forwarding, client bugs), and upgrading the primary delayed
       response stream to use better methods when they are detected as
       reliable for this client and its intermediaries.

Should I ever implement this, I will try using multiple DNS names for
the same server and see if that is effective.  If not, I will have to
use multiple IPs as well as multiple DNS names.

More connections are also required for web pages which contain
_multiple_ applications of this style at the same time, running
independently on the same web page (perhaps written by different
vendors), but connecting to the same servers.

> Also, considering the wider availability of event-driven Web servers
> and intermediaries, resource limitations on servers isn't necessarily
> the problem it once was.
>
> What do people think? Should we consider upping this to 4? There's
> also the possibility of negotiating a higher number of connections hop-
> by-hop (but that would be outside of HTTPBIS).

If a new revision of HTTP still has a recommended small limit, I think
it would be useful to have a request or response header meaning "don't
count this connection towards the usual limit for the duration of
this request".

I'm under the impression that the 2 connection limit is more
rigorously enforced by IE than Mozilla, as I have seen comments about
needing to work around it in discussions about delayed-GET techniques,
but I haven't tested this.

-- Jamie

Reply | Threaded
Open this post in threaded view
|

RE: Connection limits

Eric Lawrence-4
In reply to this post by Adrien de Croy
I believe the state of the mainstream browsing world today is:
 
Opera enables pipelining by default.  They have a TON of code which attempts to detect incompatibility on the route to the server and disables pipelining if such problems are detected.
Firefox supports pipelining but disables it by default.  For the latest build of Firefox3, they have enabled pipelining to only HTTPS servers by default, under the theory that doing so is less likely to break compatibility as the CONNECT tunnel prevents proxies from detecting/breaking on pipelined requests.
Internet Explorer does not support pipelining.
All major browsers support persistent connections for HTTP/1.1, and for HTTP/1.0 with appropriate Keep-Alive directives.
 
Other tidbits:
IIS and Apache both support pipelining.  ISA proxies accept pipelined requests but queues them locally and does not pipeline to origin servers.  The System.NET HTTP stack supports pipelining (WinINET and WinHTTP do not).
 
Eric Lawrence
Program Manager
Internet Explorer - Security

From: [hidden email] [[hidden email]] On Behalf Of Adrien de Croy [[hidden email]]
Sent: Wednesday, March 05, 2008 1:05 PM
To: Roy T. Fielding
Cc: Josh Cohen (MIG); David Morris; Mark Nottingham; [hidden email] Group
Subject: Re: Connection limits



I agree

If pipelining is sending another request on a connection before you get
a response, then as far as I can tell, all the major browsers by default
do it.

And I don't think it makes any sense to try and come up with a magic
number for connections.  The TCP overhead is a lot lower now with
persistent connections which are now ubiquitous, so maybe the
requirement for many connections is reduced, but policy decisions like
that should be left to server or proxy admin/management (people), not
hard coded into software or written into specs.

Adrien


Roy T. Fielding wrote:

>
> On Mar 5, 2008, at 4:52 AM, Josh Cohen (MIG) wrote:
>> I would agree.
>>
>> The connection limit, in addition to the bandwidth limitations of the
>> time, also helped the servers themselves.  Back then, maintaining
>> many simultaneous but short lived TCP connections was inefficient for
>> many operating systems TCP stacks.  By switching to fewer, longer
>> standing connections, and the hope of the use of pipelining, we
>> thought we'd address that.
>> Nowadays with stacks much more efficiently tuned to handle these
>> types of connections efficiently, and the non-occurrence of
>> pipelining, I think this could be relaxed a bit.
>
> Just out of curiosity, why do people keep saying things like
> "non-occurrence
> of pipelining" when the vast majority of connections I've looked at over
> the past six years contain obviously pipelined GET requests and
> occasional
> pipelines of POST (in spite of the bogus requirement).  Is this just
> because
> you do all your work behind a corporate firewall proxy?  Is this just
> another myth?  Or am I the only one on the planet who looks at traces to
> servers using both a client and a server that support pipelining?
>
> Er, in regards to the topic, I see no reason for the connection limits.
> They should be replaced with a simple statement of why too many
> connections
> results in counterproductive collision/bandwidth effects.
>
> ....Roy
>
>

--
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com