Re: [tcpm] TCP Tuning for HTTP - update

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
42 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch
Hi, Mark, et al.,

I posted a review of this document to both to TCPM and HTTP WGs.

This update fails to address the issues I raised - notably that many of
the issues therein are known *and published*.

So first, can we discuss the issue of PLAGIARISM?

Not only of two of my works, but of many others that pointed out most of
the information summarized in this doc.

Second, the step of "adoption" needs to wait until there's something new
here that wasn't known 20 years ago and the issue of plagiarism is
addressed.

Joe

On 8/16/2016 7:02 PM, Mark Nottingham wrote:

> Hi TCPM,
>
> Just a quick note; Daniel and Tim have made an update to the TCP Tuning for HTTP draft:
>   https://tools.ietf.org/html/draft-stenberg-httpbis-tcp
>
> We've had a Call for Adoption open for this draft for a while, and will likely adopt it soon. However, we'd like to get feedback from this community first -- both about the latest version of the input document, and to see if there's interest in helping out.
>
> You can give feedback on the HTTP WG mailing list <https://lists.w3.org/Archives/Public/ietf-http-wg/>, or  by responding to this e-mail (Please leave the CC line; Patrick and I will try to summarise the feedback to the WG).
>
> Cheers,
>
> --
> Mark Nottingham   https://www.mnot.net/
>
>
>
>
> _______________________________________________
> tcpm mailing list
> [hidden email]
> https://www.ietf.org/mailman/listinfo/tcpm

Hi, all,

This doc was noted on the TCPM list.

See my observations below.

Joe


-------- Forwarded Message --------
Subject: Re: [tcpm] FW: Call for Adoption: TCP Tuning for HTTP
Date: Wed, 2 Mar 2016 12:25:07 -0800
From: Joe Touch <[hidden email]>
To: Scharf, Michael (Nokia - DE) <[hidden email]>,
[hidden email] Extensions <[hidden email]>
CC: [hidden email]



On 3/2/2016 1:39 AM, Scharf, Michael (Nokia - DE) wrote:
> I assume this could be of interest to the TCPM community.

I have doubts:

- it reads like a Linux manual page

        All Linux-specific references and commands would need to be
        moved to an appendix to be useful as an RFC.

- this repeats (sometimes correctly, sometimes in error) existing advice

J. Heidemann, K. Obraczka, J. Touch, “Modeling the Performance of HTTP
Over Several Transport Protocols,” IEEE/ACM Transactions on Networking,
V5, N5, Oct. 1997, pp.616-630.

T. Faber, J. Touch, and W. Yue, “The TIME-WAIT state in TCP and Its
Effect on Busy Servers,” in Proc. IEEE Infocom, 1999, pp. 1573-1583.

- it has significant errors

        TIME-WAIT issues apply to servers, not clients.

        Nagle has been known to perform poorly for multibyte
        interactive traffic for a very long time, including not
        only web traffic but also multi-byte character or keyboard
        signals.

        Disabling slow-start after idle is safe only with pacing.
        Without pacing, the resulting traffic can generate a burst
        that was never experienced and result in both poor performance
        for the current connection and potential impact to competing
        traffic.

(those are just a few)

Overall, I think a man page might be useful, but this summary isn't
useful for the IETF.

Joe

> -----Original Message-----
> From: Scharf, Michael (Nokia - DE)
> Sent: Wednesday, March 02, 2016 10:37 AM
> To: 'Mark Nottingham'; HTTP WG
> Cc: [hidden email]; Daniel Stenberg
> Subject: RE: Call for Adoption: TCP Tuning for HTTP
>
> The document refers to several TCPM RFCs with experimental status, e.g., in Section 3. That may have to be taken into account when heading towards BCP status.
>
> Michael
> (TCPM co-chair)
>
>
> -----Original Message-----
> From: Mark Nottingham [mailto:[hidden email]]
> Sent: Wednesday, March 02, 2016 6:47 AM
> To: HTTP WG
> Cc: [hidden email]; Daniel Stenberg
> Subject: Call for Adoption: TCP Tuning for HTTP
>
> [ copying Alison as our Transport Tech Advisor ]
>
> Daniel has kindly started a document about how HTTP uses TCP, both for /1 and /2:
>   <https://tools.ietf.org/html/draft-stenberg-httpbis-tcp>
>
> We haven't explicitly discussed this at a meeting, but I have heard interest in this topic from a variety of folks.
>
> What do people think about adopting this with a target of Best Current Practice?
>
> Please comment on-list.
>
> Regards,
>
> --
> Mark Nottingham   https://www.mnot.net/
>
>
> _______________________________________________
> tcpm mailing list
> [hidden email]
> https://www.ietf.org/mailman/listinfo/tcpm
>



On 3/2/2016 1:39 AM, Scharf, Michael (Nokia - DE) wrote:
> I assume this could be of interest to the TCPM community.

I have doubts:

- it reads like a Linux manual page

        All Linux-specific references and commands would need to be
        moved to an appendix to be useful as an RFC.

- this repeats (sometimes correctly, sometimes in error) existing advice

J. Heidemann, K. Obraczka, J. Touch, “Modeling the Performance of HTTP
Over Several Transport Protocols,” IEEE/ACM Transactions on Networking,
V5, N5, Oct. 1997, pp.616-630.

T. Faber, J. Touch, and W. Yue, “The TIME-WAIT state in TCP and Its
Effect on Busy Servers,” in Proc. IEEE Infocom, 1999, pp. 1573-1583.

- it has significant errors

        TIME-WAIT issues apply to servers, not clients.

        Nagle has been known to perform poorly for multibyte
        interactive traffic for a very long time, including not
        only web traffic but also multi-byte character or keyboard
        signals.

        Disabling slow-start after idle is safe only with pacing.
        Without pacing, the resulting traffic can generate a burst
        that was never experienced and result in both poor performance
        for the current connection and potential impact to competing
        traffic.

(those are just a few)

Overall, I think a man page might be useful, but this summary isn't
useful for the IETF.

Joe

> -----Original Message-----
> From: Scharf, Michael (Nokia - DE)
> Sent: Wednesday, March 02, 2016 10:37 AM
> To: 'Mark Nottingham'; HTTP WG
> Cc: [hidden email]; Daniel Stenberg
> Subject: RE: Call for Adoption: TCP Tuning for HTTP
>
> The document refers to several TCPM RFCs with experimental status, e.g., in Section 3. That may have to be taken into account when heading towards BCP status.
>
> Michael
> (TCPM co-chair)
>
>
> -----Original Message-----
> From: Mark Nottingham [mailto:[hidden email]]
> Sent: Wednesday, March 02, 2016 6:47 AM
> To: HTTP WG
> Cc: [hidden email]; Daniel Stenberg
> Subject: Call for Adoption: TCP Tuning for HTTP
>
> [ copying Alison as our Transport Tech Advisor ]
>
> Daniel has kindly started a document about how HTTP uses TCP, both for /1 and /2:
>   <https://tools.ietf.org/html/draft-stenberg-httpbis-tcp>
>
> We haven't explicitly discussed this at a meeting, but I have heard interest in this topic from a variety of folks.
>
> What do people think about adopting this with a target of Best Current Practice?
>
> Please comment on-list.
>
> Regards,
>
> --
> Mark Nottingham   https://www.mnot.net/
>
>
> _______________________________________________
> tcpm mailing list
> [hidden email]
> https://www.ietf.org/mailman/listinfo/tcpm
>
Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Mark Nottingham-2
Joe,

> On 17 Aug 2016, at 2:21 PM, Joe Touch <[hidden email]> wrote:
>
> Hi, Mark, et al.,
>
> I posted a review of this document to both to TCPM and HTTP WGs.
>
> This update fails to address the issues I raised - notably that many of
> the issues therein are known *and published*.

I'm assuming you're talking about <https://lists.w3.org/Archives/Public/ietf-http-wg/2016JanMar/0314.html>.


> So first, can we discuss the issue of PLAGIARISM?

Sure, let's discuss it. That's a very serious accusation. Are you saying that your material has been intentionally used without proper acknowledgement?

Personally, I doubt that. What may have happened is that the text brushes up against things that you've written about in the past, and you feel that you're not adequately acknowledged.

If that's the case, I'd observe that the IETF isn't an academic publisher, and acknowledging all prior work in an area is neither practical, nor required, nor current practice.

On the other hand, if it turns out that directing readers toward other documents (including yours) adds value, a reference might make sense.


> Not only of two of my works, but of many others that pointed out most of
> the information summarized in this doc.
>
> Second, the step of "adoption" needs to wait until there's something new
> here that wasn't known 20 years ago and the issue of plagiarism is
> addressed.

Other people in the HTTP *and* TCP communities have commented that such a document would be very useful, whether or not it's something "new that wasn't known 20 years ago".

Cheers,


>
> Joe
>
> On 8/16/2016 7:02 PM, Mark Nottingham wrote:
>> Hi TCPM,
>>
>> Just a quick note; Daniel and Tim have made an update to the TCP Tuning for HTTP draft:
>>  https://tools.ietf.org/html/draft-stenberg-httpbis-tcp
>>
>> We've had a Call for Adoption open for this draft for a while, and will likely adopt it soon. However, we'd like to get feedback from this community first -- both about the latest version of the input document, and to see if there's interest in helping out.
>>
>> You can give feedback on the HTTP WG mailing list <https://lists.w3.org/Archives/Public/ietf-http-wg/>, or  by responding to this e-mail (Please leave the CC line; Patrick and I will try to summarise the feedback to the WG).
>>
>> Cheers,
>>
>> --
>> Mark Nottingham   https://www.mnot.net/
>>
>>
>>
>>
>> _______________________________________________
>> tcpm mailing list
>> [hidden email]
>> https://www.ietf.org/mailman/listinfo/tcpm
>
> <Attached Message.eml><Attached Message.eml>

--
Mark Nottingham   https://www.mnot.net/





Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch
Hi, Mark,


On 8/16/2016 9:43 PM, Mark Nottingham wrote:

> Joe,
>
>> On 17 Aug 2016, at 2:21 PM, Joe Touch <[hidden email]> wrote:
>>
>> Hi, Mark, et al.,
>>
>> I posted a review of this document to both to TCPM and HTTP WGs.
>>
>> This update fails to address the issues I raised - notably that many of
>> the issues therein are known *and published*.
> I'm assuming you're talking about <https://lists.w3.org/Archives/Public/ietf-http-wg/2016JanMar/0314.html>.

Yes.
>
>
>> So first, can we discuss the issue of PLAGIARISM?
> Sure, let's discuss it. That's a very serious accusation. Are you saying that your material has been intentionally used without proper acknowledgement?

Yes.

> Personally, I doubt that. What may have happened is that the text brushes up against things that you've written about in the past, and you feel that you're not adequately acknowledged.
Plagiarism requires only that the material was published elsewhere
before. Intent has no bearing.

In addition, I informed the author - and both lists - about this over 5
months ago. You might claim that the first two versions were issued out
of ignorance, but you cannot claim that of the update.

> If that's the case, I'd observe that the IETF isn't an academic publisher, and acknowledging all prior work in an area is neither practical, nor required, nor current practice.
Plagiarism isn't an issue limited to academic environments. Publication
of a document on the web is still publication.

> On the other hand, if it turns out that directing readers toward other documents (including yours) adds value, a reference might make sense.

Those docs explain the issues more correctly and in more detail. That
should add enough value.

The real question is whether this draft adds value to those - which are
*already published*.


>
>> Not only of two of my works, but of many others that pointed out most of
>> the information summarized in this doc.
>>
>> Second, the step of "adoption" needs to wait until there's something new
>> here that wasn't known 20 years ago and the issue of plagiarism is
>> addressed.
> Other people in the HTTP *and* TCP communities have commented that such a document would be very useful, whether or not it's something "new that wasn't known 20 years ago".

We don't need to issue new documents for people who don't read old ones.

Joe


>
> Cheers,
>
>
>> Joe
>>
>> On 8/16/2016 7:02 PM, Mark Nottingham wrote:
>>> Hi TCPM,
>>>
>>> Just a quick note; Daniel and Tim have made an update to the TCP Tuning for HTTP draft:
>>>  https://tools.ietf.org/html/draft-stenberg-httpbis-tcp
>>>
>>> We've had a Call for Adoption open for this draft for a while, and will likely adopt it soon. However, we'd like to get feedback from this community first -- both about the latest version of the input document, and to see if there's interest in helping out.
>>>
>>> You can give feedback on the HTTP WG mailing list <https://lists.w3.org/Archives/Public/ietf-http-wg/>, or  by responding to this e-mail (Please leave the CC line; Patrick and I will try to summarise the feedback to the WG).
>>>
>>> Cheers,
>>>
>>> --
>>> Mark Nottingham   https://www.mnot.net/
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> tcpm mailing list
>>> [hidden email]
>>> https://www.ietf.org/mailman/listinfo/tcpm
>> <Attached Message.eml><Attached Message.eml>
> --
> Mark Nottingham   https://www.mnot.net/
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Mark Nottingham-2
Joe,

I've pinged some ADs to weigh in on this.

In the meantime -

> On 17 Aug 2016, at 3:23 PM, Joe Touch <[hidden email]> wrote:
>
> Plagiarism requires only that the material was published elsewhere
> before. Intent has no bearing.

So to paraphrase, you're accusing Daniel of failure to cite a relevant source out of ignorance in the first case...


> In addition, I informed the author - and both lists - about this over 5
> months ago. You might claim that the first two versions were issued out
> of ignorance, but you cannot claim that of the update.

... and then continuing to do so after being notified.

Correct?

Looking over <http://www.isi.edu/touch/pubs/ton97.pdf> and <http://www.isi.edu/touch/pubs/infocomm99/infocomm99-web/>, I'm having trouble identifying passages that are plagiarised; can you show us?

In particular, the latter paper proposes TIME-WAIT negotiation using a TCP option, sending a RST from clients, and a HTTP extension, whereas Daniel's draft only says this about TIME_WAIT:

"""
2.5.  Lower the TCP FIN timeout


   High connection completion rates will consume ephemeral ports
   quickly.  Lower the time during which connections are in FIN-WAIT-2/
   TIME_WAIT states so that they can be purged faster and thus maintain
   a maximal number of available sockets.  The primitives for the
   assignment of these values were described in [RFC0793], however
   significantly lower values are commonly used.


2.6.  Reuse sockets in TIME_WAIT state


   When running backend servers on a managed, low latency network you
   might allow the reuse of sockets in TIME_WAIT state for new
   connections when a protocol complete termination has occurred.  There
   is no RFC that covers this behaviour.
"""

So, where is the plagiarism?

Also, how do you reconcile that with the statement in your original e-mail that the draft repeats your material "sometimes correctly, sometimes in error"? Have you considered that what you consider to be an error in his plagiarism might in fact be a different idea (no matter whose is ultimately correct)?


>> If that's the case, I'd observe that the IETF isn't an academic publisher, and acknowledging all prior work in an area is neither practical, nor required, nor current practice.
> Plagiarism isn't an issue limited to academic environments. Publication
> of a document on the web is still publication.

Sure. It also isn't a legal issue in this form (unless you're asserting copyright?). Effectively, it's a cultural norm. Again, I will point out that in the culture of the IETF, we historically have not cited the complete provenance of every idea, both because it's impractical and because it doesn't benefit the reader.

As far as I know, the IETF does not have a stated position about what you regard as PLAGIARISM. Hopefully we can get some clarity about that from the ADs, as well as some definitive evidence of what you're asserting.


>> On the other hand, if it turns out that directing readers toward other documents (including yours) adds value, a reference might make sense.
>
> Those docs explain the issues more correctly and in more detail. That
> should add enough value.
>
> The real question is whether this draft adds value to those - which are
> *already published*.
>
>
>>
>>> Not only of two of my works, but of many others that pointed out most of
>>> the information summarized in this doc.
>>>
>>> Second, the step of "adoption" needs to wait until there's something new
>>> here that wasn't known 20 years ago and the issue of plagiarism is
>>> addressed.
>> Other people in the HTTP *and* TCP communities have commented that such a document would be very useful, whether or not it's something "new that wasn't known 20 years ago".
>
> We don't need to issue new documents for people who don't read old ones.



--
Mark Nottingham   https://www.mnot.net/


Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Willy Tarreau-3
In reply to this post by Joe Touch
Hi Joe,

On Tue, Aug 16, 2016 at 10:23:01PM -0700, Joe Touch wrote:
> > Other people in the HTTP *and* TCP communities have commented that such a
> > document would be very useful, whether or not it's something "new that
> > wasn't known 20 years ago".
>
> We don't need to issue new documents for people who don't read old ones.

I've just checked the two documents you referenced. They seem to be very
well detailed but they are *scientific* research. What Daniel created are
configuration advises for people who need to configure their servers. Yes
as you mentionned they look like man pages precisely because the purpose
is to ensure they're easily understood by people who are just seeking some
help to improve their configuration. You cannot expect a server admin to
read scientific papers explaining some TCP models with some formulas to
know what to do on their servers.

Also, I don't know if there have been any update, but these documents use
SunOS 4.1.3 running on a sparc 20 as a reference. While I used to love
working on such systems 20 years ago, they predate the web era and systems
have evolved a lot since to deal with high traffic. I remember by then I
used to be impressed when seeing a server sustain 50 connections per second.
Now when someone tells me he's limited to 100000 connections per second on
a single server, I ask how they bound their interrupts to see if there's
some headroom to go further.

So you need to expect that only researchers and maybe TCP stack developers
will find your work useful these days, server admins can hardly use this
anymore. However it is very possible that some TCP stacks have taken benefit
of your work to reach the level of performance they achieve right now, I
don't know. Thus I think that Daniel's work completes quite well what you've
done in that it directly addresses people's concerns without requiring the
scientific background.

Regards,
Willy

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch
In reply to this post by Mark Nottingham-2


On 8/16/2016 11:42 PM, Mark Nottingham wrote:

> Joe,
>
> I've pinged some ADs to weigh in on this.
>
> In the meantime -
>
>> On 17 Aug 2016, at 3:23 PM, Joe Touch <[hidden email]> wrote:
>>
>> Plagiarism requires only that the material was published elsewhere
>> before. Intent has no bearing.
> So to paraphrase, you're accusing Daniel of failure to cite a relevant source out of ignorance in the first case...
I'm being generous in assuming ignorance, but yes.

>
>> In addition, I informed the author - and both lists - about this over 5
>> months ago. You might claim that the first two versions were issued out
>> of ignorance, but you cannot claim that of the update.
> ... and then continuing to do so after being notified.
>
> Correct?
Yes.

> Looking over <http://www.isi.edu/touch/pubs/ton97.pdf> and <http://www.isi.edu/touch/pubs/infocomm99/infocomm99-web/>, I'm having trouble identifying passages that are plagiarised; can you show us?
It's the content - turning off Nagle, slow-start restart after idle, etc.

> In particular, the latter paper proposes TIME-WAIT negotiation using a TCP option, sending a RST from clients, and a HTTP extension,

It also outlines the whole problem for the first time and explains the
choice and impact of lowering TW timeout.

>  ...
>
> Also, how do you reconcile that with the statement in your original e-mail that the draft repeats your material "sometimes correctly, sometimes in error"? Have you considered that what you consider to be an error in his plagiarism might in fact be a different idea (no matter whose is ultimately correct)?
That might be a reasonable conclusion if there were discussion
explaining why the "different idea" was correct in comparison.

But that's missing. Saying to lower the timer for TW without discussing
the potential impact isn't a new idea. It's an incomplete discussion of
an old idea already discussed in detail elsewhere.

>
>>> If that's the case, I'd observe that the IETF isn't an academic publisher, and acknowledging all prior work in an area is neither practical, nor required, nor current practice.
>> Plagiarism isn't an issue limited to academic environments. Publication
>> of a document on the web is still publication.
> Sure. It also isn't a legal issue in this form (unless you're asserting copyright?). Effectively, it's a cultural norm. Again, I will point out that in the culture of the IETF, we historically have not cited the complete provenance of every idea, both because it's impractical and because it doesn't benefit the reader.
Although that's true in the smallest cases, the IETF does have two
concepts that support this norm: an author list and a set of references.

Can you explain how it helps the reader to not cite two documents that
are both squarely in the same area as this doc (interaction between HTTP
and TCP and the impact of running many small connections closed at the
client as for HTTP)?

> As far as I know, the IETF does not have a stated position about what you regard as PLAGIARISM. Hopefully we can get some clarity about that from the ADs, as well as some definitive evidence of what you're asserting.

You can if you want, but my primary point here is to have this work
corrected - and to stop the myth that "it doesn't matter" whether
*reasonable* citations are included.

Joe

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch
In reply to this post by Willy Tarreau-3


On 8/16/2016 11:45 PM, Willy Tarreau wrote:

> Hi Joe,
>
> On Tue, Aug 16, 2016 at 10:23:01PM -0700, Joe Touch wrote:
>>> Other people in the HTTP *and* TCP communities have commented that such a
>>> document would be very useful, whether or not it's something "new that
>>> wasn't known 20 years ago".
>> We don't need to issue new documents for people who don't read old ones.
> I've just checked the two documents you referenced. They seem to be very
> well detailed but they are *scientific* research. What Daniel created are
> configuration advises for people who need to configure their servers. Yes
> as you mentionned they look like man pages precisely because the purpose
> is to ensure they're easily understood by people who are just seeking some
> help to improve their configuration. You cannot expect a server admin to
> read scientific papers explaining some TCP models with some formulas to
> know what to do on their servers.
Actually, my docs are the tip of the iceberg, addressing the reasons why
some of these configuration recommendations make sense.

There are many other sites - and books - that already indicate how to
configure systems efficiently.

So if your argument is that a man page summary is needed, sure - but
again, is a new one needed? And why is this then needed as an RFC?

> Also, I don't know if there have been any update, but these documents use
> SunOS 4.1.3 running on a sparc 20 as a reference. While I used to love
> working on such systems 20 years ago, they predate the web era and systems
> have evolved a lot since to deal with high traffic. ...
Yes, and discussing those issues would be useful - but not in this
document either.

> ...
> So you need to expect that only researchers and maybe TCP stack developers
> will find your work useful these days, server admins can hardly use this
> anymore. However it is very possible that some TCP stacks have taken benefit
> of your work to reach the level of performance they achieve right now, I
> don't know. Thus I think that Daniel's work completes quite well what you've
> done in that it directly addresses people's concerns without requiring the
> scientific background.
Let me see if I get your complete argument:

    - the appropriate refs are 20 years old
    - server admins need a doc

What exactly do server admins need regarding Nagle (which is configured
inside the app already), socket sizing (configured inside the app), etc?

I.e., at the most this is a man page (specific to an OS). At the least,
this isn't useful at all.

Joe

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Alexey Melnikov
In reply to this post by Joe Touch
Joe,

On 17/08/2016 16:08, Joe Touch wrote:
> On 8/16/2016 11:42 PM, Mark Nottingham wrote:
>>> On 17 Aug 2016, at 3:23 PM, Joe Touch <[hidden email]> wrote:
>>>
  [snip]

>>>> If that's the case, I'd observe that the IETF isn't an academic publisher, and acknowledging all prior work in an area is neither practical, nor required, nor current practice.
>>> Plagiarism isn't an issue limited to academic environments. Publication
>>> of a document on the web is still publication.
>> Sure. It also isn't a legal issue in this form (unless you're asserting copyright?). Effectively, it's a cultural norm. Again, I will point out that in the culture of the IETF, we historically have not cited the complete provenance of every idea, both because it's impractical and because it doesn't benefit the reader.
> Although that's true in the smallest cases, the IETF does have two
> concepts that support this norm: an author list and a set of references.
>
> Can you explain how it helps the reader to not cite two documents that
> are both squarely in the same area as this doc (interaction between HTTP
> and TCP and the impact of running many small connections closed at the
> client as for HTTP)?
Instead of starting your discussion with words like "plagiarism", you
could have just asked for information to be clarified and a
citation/acknowledgement added? With your current introduction you
pissed off lots of people.
>> As far as I know, the IETF does not have a stated position about what you regard as PLAGIARISM. Hopefully we can get some clarity about that from the ADs, as well as some definitive evidence of what you're asserting.
> You can if you want, but my primary point here is to have this work
> corrected - and to stop the myth that "it doesn't matter" whether
> *reasonable* citations are included.
Noted.


Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch


> On Aug 17, 2016, at 8:28 AM, Alexey Melnikov <[hidden email]> wrote:
>
> Joe,
>
>> On 17/08/2016 16:08, Joe Touch wrote:
>> ...
>> Can you explain how it helps the reader to not cite two documents that
>> are both squarely in the same area as this doc (interaction between HTTP
>> and TCP and the impact of running many small connections closed at the
>> client as for HTTP)?
> Instead of starting your discussion with words like "plagiarism", you could have just asked for information to be clarified and a citation/acknowledgement added?
...

I did that 5 months ago, as noted in the attachments I included yesterday.

Joe

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Eliot Lear
In reply to this post by Alexey Melnikov
Perhaps we can agree that the reasonable course of action here is for
Joe to (re)-recommend a compact set of citations to the authors, perhaps
even in some easily consumable form to them (kramdown-2629 or XML)?

Eliot


On 8/17/16 5:28 PM, Alexey Melnikov wrote:

> Joe,
>
> On 17/08/2016 16:08, Joe Touch wrote:
>> On 8/16/2016 11:42 PM, Mark Nottingham wrote:
>>>> On 17 Aug 2016, at 3:23 PM, Joe Touch <[hidden email]> wrote:
>>>>
>  [snip]
>>>>> If that's the case, I'd observe that the IETF isn't an academic
>>>>> publisher, and acknowledging all prior work in an area is neither
>>>>> practical, nor required, nor current practice.
>>>> Plagiarism isn't an issue limited to academic environments.
>>>> Publication
>>>> of a document on the web is still publication.
>>> Sure. It also isn't a legal issue in this form (unless you're
>>> asserting copyright?). Effectively, it's a cultural norm. Again, I
>>> will point out that in the culture of the IETF, we historically have
>>> not cited the complete provenance of every idea, both because it's
>>> impractical and because it doesn't benefit the reader.
>> Although that's true in the smallest cases, the IETF does have two
>> concepts that support this norm: an author list and a set of references.
>>
>> Can you explain how it helps the reader to not cite two documents that
>> are both squarely in the same area as this doc (interaction between HTTP
>> and TCP and the impact of running many small connections closed at the
>> client as for HTTP)?
> Instead of starting your discussion with words like "plagiarism", you
> could have just asked for information to be clarified and a
> citation/acknowledgement added? With your current introduction you
> pissed off lots of people.
>>> As far as I know, the IETF does not have a stated position about
>>> what you regard as PLAGIARISM. Hopefully we can get some clarity
>>> about that from the ADs, as well as some definitive evidence of what
>>> you're asserting.
>> You can if you want, but my primary point here is to have this work
>> corrected - and to stop the myth that "it doesn't matter" whether
>> *reasonable* citations are included.
> Noted.
>
>
>


signature.asc (492 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Tim Wicinski

That sounds like a fine idea.  I'll be glad to go through those.

tim

On 8/17/16 12:26 PM, Eliot Lear wrote:

> Perhaps we can agree that the reasonable course of action here is for
> Joe to (re)-recommend a compact set of citations to the authors, perhaps
> even in some easily consumable form to them (kramdown-2629 or XML)?
>
> Eliot
>
>
> On 8/17/16 5:28 PM, Alexey Melnikov wrote:
>> Joe,
>>
>> On 17/08/2016 16:08, Joe Touch wrote:
>>> On 8/16/2016 11:42 PM, Mark Nottingham wrote:
>>>>> On 17 Aug 2016, at 3:23 PM, Joe Touch <[hidden email]> wrote:
>>>>>
>>  [snip]
>>>>>> If that's the case, I'd observe that the IETF isn't an academic
>>>>>> publisher, and acknowledging all prior work in an area is neither
>>>>>> practical, nor required, nor current practice.
>>>>> Plagiarism isn't an issue limited to academic environments.
>>>>> Publication
>>>>> of a document on the web is still publication.
>>>> Sure. It also isn't a legal issue in this form (unless you're
>>>> asserting copyright?). Effectively, it's a cultural norm. Again, I
>>>> will point out that in the culture of the IETF, we historically have
>>>> not cited the complete provenance of every idea, both because it's
>>>> impractical and because it doesn't benefit the reader.
>>> Although that's true in the smallest cases, the IETF does have two
>>> concepts that support this norm: an author list and a set of references.
>>>
>>> Can you explain how it helps the reader to not cite two documents that
>>> are both squarely in the same area as this doc (interaction between HTTP
>>> and TCP and the impact of running many small connections closed at the
>>> client as for HTTP)?
>> Instead of starting your discussion with words like "plagiarism", you
>> could have just asked for information to be clarified and a
>> citation/acknowledgement added? With your current introduction you
>> pissed off lots of people.
>>>> As far as I know, the IETF does not have a stated position about
>>>> what you regard as PLAGIARISM. Hopefully we can get some clarity
>>>> about that from the ADs, as well as some definitive evidence of what
>>>> you're asserting.
>>> You can if you want, but my primary point here is to have this work
>>> corrected - and to stop the myth that "it doesn't matter" whether
>>> *reasonable* citations are included.
>> Noted.
>>
>>
>>
>
>
>
>
> _______________________________________________
> tcpm mailing list
> [hidden email]
> https://www.ietf.org/mailman/listinfo/tcpm
>

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch
All,

It sounds like the job of an author or a search engine (or both).

I've given you a place to start.

Joe


On 8/17/2016 10:00 AM, Tim Wicinski wrote:

>
> That sounds like a fine idea.  I'll be glad to go through those.
>
> tim
>
> On 8/17/16 12:26 PM, Eliot Lear wrote:
>> Perhaps we can agree that the reasonable course of action here is for
>> Joe to (re)-recommend a compact set of citations to the authors, perhaps
>> even in some easily consumable form to them (kramdown-2629 or XML)?
>>
>> Eliot
>>
>>
>> On 8/17/16 5:28 PM, Alexey Melnikov wrote:
>>> Joe,
>>>
>>> On 17/08/2016 16:08, Joe Touch wrote:
>>>> On 8/16/2016 11:42 PM, Mark Nottingham wrote:
>>>>>> On 17 Aug 2016, at 3:23 PM, Joe Touch <[hidden email]> wrote:
>>>>>>
>>>  [snip]
>>>>>>> If that's the case, I'd observe that the IETF isn't an academic
>>>>>>> publisher, and acknowledging all prior work in an area is neither
>>>>>>> practical, nor required, nor current practice.
>>>>>> Plagiarism isn't an issue limited to academic environments.
>>>>>> Publication
>>>>>> of a document on the web is still publication.
>>>>> Sure. It also isn't a legal issue in this form (unless you're
>>>>> asserting copyright?). Effectively, it's a cultural norm. Again, I
>>>>> will point out that in the culture of the IETF, we historically have
>>>>> not cited the complete provenance of every idea, both because it's
>>>>> impractical and because it doesn't benefit the reader.
>>>> Although that's true in the smallest cases, the IETF does have two
>>>> concepts that support this norm: an author list and a set of
>>>> references.
>>>>
>>>> Can you explain how it helps the reader to not cite two documents that
>>>> are both squarely in the same area as this doc (interaction between
>>>> HTTP
>>>> and TCP and the impact of running many small connections closed at the
>>>> client as for HTTP)?
>>> Instead of starting your discussion with words like "plagiarism", you
>>> could have just asked for information to be clarified and a
>>> citation/acknowledgement added? With your current introduction you
>>> pissed off lots of people.
>>>>> As far as I know, the IETF does not have a stated position about
>>>>> what you regard as PLAGIARISM. Hopefully we can get some clarity
>>>>> about that from the ADs, as well as some definitive evidence of what
>>>>> you're asserting.
>>>> You can if you want, but my primary point here is to have this work
>>>> corrected - and to stop the myth that "it doesn't matter" whether
>>>> *reasonable* citations are included.
>>> Noted.
>>>
>>>
>>>
>>
>>
>>
>>
>> _______________________________________________
>> tcpm mailing list
>> [hidden email]
>> https://www.ietf.org/mailman/listinfo/tcpm
>>


Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Willy Tarreau-3
In reply to this post by Joe Touch
On Wed, Aug 17, 2016 at 08:14:08AM -0700, Joe Touch wrote:
> There are many other sites - and books - that already indicate how to
> configure systems efficiently.
>
> So if your argument is that a man page summary is needed, sure - but
> again, is a new one needed? And why is this then needed as an RFC?

The difference is that a man page is OS-specific while an RFC gets a
unique number and serves as a reference. It can be cited in new RFCs
to justify certain choices. It can receive errata. You would probably
say that the current document is very linux-centric for now but I
understood it as a beginning of something more generic where advices
are given based on principles before sysctls and that the resulting
sysctls are just examples of applications of the principles in the
document.

> > Also, I don't know if there have been any update, but these documents use
> > SunOS 4.1.3 running on a sparc 20 as a reference. While I used to love
> > working on such systems 20 years ago, they predate the web era and systems
> > have evolved a lot since to deal with high traffic. ...
> Yes, and discussing those issues would be useful - but not in this
> document either.

Why ? Lots of admins don't understand why the time_wait timeout remains
at 240 seconds on Solaris with people saying "if you want to be conservative
don't touch it but if you want to be modern simply shrink it to 30 seconds
or so". People need to understand why advices have changed over 3 decades.

> > So you need to expect that only researchers and maybe TCP stack developers
> > will find your work useful these days, server admins can hardly use this
> > anymore. However it is very possible that some TCP stacks have taken benefit
> > of your work to reach the level of performance they achieve right now, I
> > don't know. Thus I think that Daniel's work completes quite well what you've
> > done in that it directly addresses people's concerns without requiring the
> > scientific background.
> Let me see if I get your complete argument:
>
>     - the appropriate refs are 20 years old
>     - server admins need a doc
>
> What exactly do server admins need regarding Nagle (which is configured
> inside the app already), socket sizing (configured inside the app), etc?

Lots of things :
  - time_wait tuning (which everybody gives different advices on, I've
    even seen firewall vendors recommend to shrink it to one second because
    it allowed their product to perform better in benchmarks)

  - TCP timestamps: what they provide, what are the risks (some people in
    banking environments refuse to enable them so that they cannot be used
    as an oracle to help in timing attacks).

  - window scaling : how much is needed.

  - socket sizing : contrary to what you write, there's a lot of tuning
    on the web where people set the default buffer sizes to 16MB without
    understanding the impacts when dealing with many sockets

  - SACK : why it's better. DSACK what it adds on top of SACK.

  - ECN : is it needed ? does it really work ? where does it cause issues ?

  - SYN cookies : benefits, risks

  - TCP reuse/recycling : benefits, risks

  - dealing with buffer bloat : tradeoffs between NIC-based acceleration
    and pacing

  - what are orphans and why you should care about them in HTTP close mode

  - TCP fastopen : how does it work, what type of workload is improved,
    what are the risks (ie: do not enable socket creation without cookie
    by default just because you find it reduces your server load)

  - whether to choose a short or a large SYN backlog depending on your
    workload (ie: do you prefer to process everything even if the dequeuing
    is expensive or to drop early in order to recover fast).

... and probably many other that don't immediately come to my mind. None
of these ones was a real issue 20 years ago. All of them became issues for
many web server admins who just copy-paste random settings from various
blogs found on the net who just copy the same stupidities over and over
resulting in the same trouble being caused to each of their reader.

> I.e., at the most this is a man page (specific to an OS). At the least,
> this isn't useful at all.

As you can see above, nothing I cited was OS-specific but only workload
specific. That's why I think that an informational RFC is much more suited
to this than an OS-specific man page. The OS man page may rely on the RFC
to propose various tuning profiles for different workloads however.

Regards,
Willy

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch



On 8/17/2016 11:08 AM, Willy Tarreau wrote:
On Wed, Aug 17, 2016 at 08:14:08AM -0700, Joe Touch wrote:
There are many other sites - and books - that already indicate how to
configure systems efficiently.

So if your argument is that a man page summary is needed, sure - but
again, is a new one needed? And why is this then needed as an RFC?
The difference is that a man page is OS-specific while an RFC gets a
unique number and serves as a reference. 
That means the OS-specific stuff here needs to be removed...

It can be cited in new RFCs
to justify certain choices. 
Hmm. Like the refs I gave could be cited in this doc to justify *its* choices? :-)
...

      
Also, I don't know if there have been any update, but these documents use
SunOS 4.1.3 running on a sparc 20 as a reference. While I used to love
working on such systems 20 years ago, they predate the web era and systems
have evolved a lot since to deal with high traffic. ...
Yes, and discussing those issues would be useful - but not in this
document either.
Why ? Lots of admins don't understand why the time_wait timeout remains
at 240 seconds on Solaris with people saying "if you want to be conservative
don't touch it but if you want to be modern simply shrink it to 30 seconds
or so". People need to understand why advices have changed over 3 decades.

The advice hasn't really changed - the advice was given in the 99 ref, which includes some cases where it can still be appropriate to decrease that timer.

Again, though - the 99 ref explains the implications of both decisions - why it's 240 seconds and what happens when it's lowered.


So you need to expect that only researchers and maybe TCP stack developers
will find your work useful these days, server admins can hardly use this
anymore. However it is very possible that some TCP stacks have taken benefit
of your work to reach the level of performance they achieve right now, I
don't know. Thus I think that Daniel's work completes quite well what you've
done in that it directly addresses people's concerns without requiring the
scientific background.
Let me see if I get your complete argument:

    - the appropriate refs are 20 years old
    - server admins need a doc

What exactly do server admins need regarding Nagle (which is configured
inside the app already), socket sizing (configured inside the app), etc?
Lots of things : 
  - time_wait tuning (which everybody gives different advices on, I've
    even seen firewall vendors recommend to shrink it to one second because
    it allowed their product to perform better in benchmarks)
Again, the 99 ref gives that detail.
  - TCP timestamps: what they provide, what are the risks (some people in
    banking environments refuse to enable them so that they cannot be used
    as an oracle to help in timing attacks).
That's already covered in the security considerations of RFC 7323. How is HTTP different, if at all, from any other app?
  - window scaling : how much is needed.
Same issue here, same ref - how is HTTP different?

  - socket sizing : contrary to what you write, there's a lot of tuning
    on the web where people set the default buffer sizes to 16MB without
    understanding the impacts when dealing with many sockets
There's a whole book that encompasses that and some related issues:
http://www.onlamp.com/pub/a/onlamp/2005/11/17/tcp_tuning.html

Some advice is also given in Sec 6.3.3 of this:
J. Sterbenz, J. Touch, High Speed Networking: A Systematic Approach to High-Bandwidth Low-Latency Communication, Wiley, May 2001.

  - SACK : why it's better. DSACK what it adds on top of SACK.
That's in the SACK docs, which aren't cited. Again, how is HTTP different from any app?
  - ECN : is it needed ? does it really work ? where does it cause issues ?
That's in the ECN docs, which aren't cited. Again, how is HTTP different from any app?
  - SYN cookies : benefits, risks
That's in the RFC 4987, which at least IS cited. Again, how is HTTP different from any app?

  - TCP reuse/recycling : benefits, risks
Not sure what you mean here. There are a lot of docs on the issues with persistent-HTTP vs per-connection HTTP.

  - dealing with buffer bloat : tradeoffs between NIC-based acceleration
    and pacing
Bufferbloat typically involves large *uplink* transfers and how they interact with other uplink connections. Neither TCP nor HTTP is involved in this really.

  - what are orphans and why you should care about them in HTTP close mode
Orphaned TCP connections or orphaned HTTP processes?

  - TCP fastopen : how does it work, what type of workload is improved,
    what are the risks (ie: do not enable socket creation without cookie
    by default just because you find it reduces your server load)

Another doc that exists.

  - whether to choose a short or a large SYN backlog depending on your
    workload (ie: do you prefer to process everything even if the dequeuing
    is expensive or to drop early in order to recover fast).

... and probably many other that don't immediately come to my mind. None
of these ones was a real issue 20 years ago.

See above. Many were known around that time, but weren't documented in detail (it took a while for a proper ref to SYN cookies, and the book I wrote with Sterbenz came about because we'd seen wheels being rediscovered for 15 years).

 All of them became issues for
many web server admins who just copy-paste random settings from various
blogs found on the net who just copy the same stupidities over and over
resulting in the same trouble being caused to each of their reader.

This doc is all over the place.

If you want a doc to advise web admins, do so. But most of the items above aren't under admin control; they're buried in app and OS implementations, and most have already evolved do to the right thing.

I agree that a summary of a *focused set* of these might be useful *as a review* (note that review discussions usually include lots of refs). The key question is "what is the focus":

    - HTTP/TCP interactions
    - server administration advice
    - ?

IMO, RFCs should focus on issues specific to the protocols and their deployment - not general knowledge that exists in courses and textbooks.

I.e., at the most this is a man page (specific to an OS). At the least,
this isn't useful at all.
As you can see above, nothing I cited was OS-specific but only workload
specific. That's why I think that an informational RFC is much more suited
to this than an OS-specific man page. The OS man page may rely on the RFC
to propose various tuning profiles for different workloads however.

You have a good point that this is general info, but OS issues are not in the scope of the IETF and there are courses and books that already provide good advice on efficiently running web (and other) servers.

Joe

    

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Willy Tarreau-3
On Wed, Aug 17, 2016 at 11:31:33AM -0700, Joe Touch wrote:
> > It can be cited in new RFCs
> > to justify certain choices.
> Hmm. Like the refs I gave could be cited in this doc to justify *its*
> choices? :-)

I think it would be nice that this is cited, but to be clear on one
point, I've never heard about your papers before you advertised them
here in this thread, and yet I've been dealing with timewait issues
for 15 years like many people facing moderate to large web sites
nowadays. It just happens that you probably identified these issues
very early at low connection rates, but today anyone dealing with
more than 500-1000 connections per second on a server or worse on
a gateway quickly discovers that he has to make a choice.

> >> Yes, and discussing those issues would be useful - but not in this
> >> document either.
> > Why ? Lots of admins don't understand why the time_wait timeout remains
> > at 240 seconds on Solaris with people saying "if you want to be conservative
> > don't touch it but if you want to be modern simply shrink it to 30 seconds
> > or so". People need to understand why advices have changed over 3 decades.
>
> The advice hasn't really changed - the advice was given in the 99 ref,
> which includes some cases where it can still be appropriate to decrease
> that timer.

Most people see it the other way around : they see no valid case to *increase*
it beyond a few seconds, because for them the default value should be extremely
low (ie this firewall vendor several years ago trying to insist on one second).
Yes that's really sad but that's reality. And you can tell them to read 6191
they won't care.

> >   - TCP timestamps: what they provide, what are the risks (some people in
> >     banking environments refuse to enable them so that they cannot be used
> >     as an oracle to help in timing attacks).
> That's already covered in the security considerations of RFC 7323. How
> is HTTP different, if at all, from any other app?

HTTP is special in that it is fairly common to have to deal with tens of
thousands of connections per second between one client and one server when
you are on the server side, because you place a number of gateways (also
called reverse-proxies) which combine all of the possible issues you can
think of at a single place. Timestamps are one way to improve fast connection
recycling between the client and the server without having to cheat on
timewaits. But since they consume 12 bytes per packet, it's often advised
to disable them in benchmarks to get the highest throughput...

> >   - window scaling : how much is needed.
> Same issue here, same ref - how is HTTP different?

Same as above.

> >   - socket sizing : contrary to what you write, there's a lot of tuning
> >     on the web where people set the default buffer sizes to 16MB without
> >     understanding the impacts when dealing with many sockets
> There's a whole book that encompasses that and some related issues:
> http://www.onlamp.com/pub/a/onlamp/2005/11/17/tcp_tuning.html

Looks fine, could be added to the list of references.

> Some advice is also given in Sec 6.3.3 of this:
> J. Sterbenz, J. Touch, /High Speed Networking: A Systematic Approach to
> High-Bandwidth Low-Latency Communication/
> <http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471330361.html>,
> Wiley, May 2001.
>
> >   - SACK : why it's better. DSACK what it adds on top of SACK.
> That's in the SACK docs, which aren't cited. Again, how is HTTP
> different from any app?
> >   - ECN : is it needed ? does it really work ? where does it cause issues ?
> That's in the ECN docs, which aren't cited. Again, how is HTTP different
> from any app?
> >   - SYN cookies : benefits, risks
> That's in the RFC 4987, which at least IS cited. Again, how is HTTP
> different from any app?

So probably you're starting to see the benefit of having a single doc
to concentrate all this. You provided at least 3 different articles
to read and 2 or 3 different RFCs in addition to the original ones,
of course. A hosting provider whose web sites are down due to a lack
of tuning doesn't start to read many very long articles and even less
the most scientific ones, they need to find quick responses that they
can apply immediately (matter of minutes). So they launch google, they
type "web site dead, time-wait overflow" and they get plenty of
responses on stackoverflow and serverfault, many from people having
done the same in the past and repeating the same mistakes over and over.

A document validated by several people and giving links for further
reading can help improve this situation.

> >   - TCP reuse/recycling : benefits, risks
> Not sure what you mean here. There are a lot of docs on the issues with
> persistent-HTTP vs per-connection HTTP.

Often on a gateway you cannot completely chose. You have a mix of both.

> >   - dealing with buffer bloat : tradeoffs between NIC-based acceleration
> >     and pacing
> Bufferbloat typically involves large *uplink* transfers and how they
> interact with other uplink connections. Neither TCP nor HTTP is involved
> in this really.

Maybe in the end Daniel is not the only one not to read all articles
published on the web :-)

     http://www.cringely.com/2012/03/25/linux-3-3-finally-a-little-good-news-for-bufferbloat/
     https://www.ietf.org/proceedings/86/slides/slides-86-iccrg-0.pdf
     https://lwn.net/Articles/564978/

In short, by sending 64kB segments to your NIC and counting on it to
cut them into small pieces for you and sending the resulting packets
very close together, you increase the risk of losses on many network
equipments which run with not-that-large buffers. It's easy to observe
even on some 10Gbps switches. When you switch mixes 10G and 40G, it's
horrible, really.

> >   - what are orphans and why you should care about them in HTTP close mode
> Orphaned TCP connections or orphaned HTTP processes?

TCP connections. The server sends a response, performs a close() on the
socket, the data remain in the kernel buffers for the time it takes to
deliver these data to the client and to get them ACKed. From this point
the socket is called orphaned on the system because it doesn't belong to
any process anymore. But immediately after this, a process which runs with
a limited connection count can accept a new connection, which will in turn
be fed with large chunks of data and closed. And this runs over and over,
eating a huge amount of socket memory despite a small limit imposed on the
server's connection concurrency. At some point the kernel doesn't have
enough socket buffers anymore (especially after admins have set them to
16MB as instructed on $random tuning guide) and starts to kill some orphaned
connections to get some memory back. The not-so-nice effect that the admin
cannot detect is that the client gets truncated responses. Only a small part
of the tail is missing but in the logs, it's said that everything was sent.
But sent by the process means sent to the system only.

> >   - TCP fastopen : how does it work, what type of workload is improved,
> >     what are the risks (ie: do not enable socket creation without cookie
> >     by default just because you find it reduces your server load)
>
> Another doc that exists.
>
> >   - whether to choose a short or a large SYN backlog depending on your
> >     workload (ie: do you prefer to process everything even if the dequeuing
> >     is expensive or to drop early in order to recover fast).
> >
> > ... and probably many other that don't immediately come to my mind. None
> > of these ones was a real issue 20 years ago.
>
> See above. Many were known around that time, but weren't documented in
> detail (it took a while for a proper ref to SYN cookies, and the book I
> wrote with Sterbenz came about because we'd seen wheels being
> rediscovered for 15 years).

People rediscover wheels because it's hard to find simple and accurate
information on the net. Basically you have the choice :
  - either uneducated blog posts saying "how I saved my web site using 2
    sysctls"
  - or academic papers which are only understandable by scientific people
    having enough time

At least the first ones have the merit of being easy to test, and since
they appear to work they are viral.

> >  All of them became issues for
> > many web server admins who just copy-paste random settings from various
> > blogs found on the net who just copy the same stupidities over and over
> > resulting in the same trouble being caused to each of their reader.
>
> This doc is all over the place.
>
> If you want a doc to advise web admins, do so.

That's *exactly* what Daniel started to do when you told him he shouldn't
do it.

> But most of the items
> above aren't under admin control; they're buried in app and OS
> implementations, and most have already evolved do to the right thing.

That's not true anymore. There was a time where Solaris had one of the
most configurable TCP stack, you could do ndd /dev/tcp with a lot of
things. Nowadays all modern operating systems let you fiddle with a ton
of stuff. And even for the missing parts you can very easily find patches
all over the web because opensource has changed this considerably.

> I agree that a summary of a *focused set* of these might be useful *as a
> review* (note that review discussions usually include lots of refs).

The purpose as I understood it was precisely to gather knowledge from
people here operating such systems and adding some elements found from
various sources. I also expressed my interest in sharing such experience
which some will find valid and others not, which is perfect because it
means there's a choice to make depending on a use case.

> The key question is "what is the focus":
>
>     - HTTP/TCP interactions
>     - server administration advice
>     - ?

Those two go hand-in-hand nowadays. You probably know that the difficulty
for HTTP implementors to find a properly tuned TCP stack is what makes them
consider UDP-based alternatives, so that they can totally control the
behaviour from user-space and provide quick updates for their protocol.
Want an example ?

    https://www.chromium.org/quic

If it was possible to always have a perfectly tuned TCP stack I'm pretty
sure we wouldn't even hear about such initiatives. And this is not something
new, I happened to discuss about this subject with some people at IETF83 in
2012 already. By then I thought "they have no idea what they're trying to
reinvent" and now I'm thinking "well, they have their reasons and they
might be right after all given all the resistance they may be facing on
the TCP side to get some default timers changed".

> IMO, RFCs should focus on issues specific to the protocols and their
> deployment - not general knowledge that exists in courses and textbooks.

Courses and textbooks are totally outdated when they're not simply wrong.
I've first been tought that it was normal to fork a process after performing
an accept(), resulting in my very first proxy working this way. Then I've
been taught that it was mandatory to send an empty ACK after a SYN-ACK to
validate a connection (which is totally wrong otherwise it would not allow
this ACK to be lost). I've been taught that in order to accept an incoming
connection you had to have your socket in listen mode exclusively. This is
false as well, two clients can connect together during their connect()
phase, it even has some security implications that are often overlooked.
Like it or not, the HTTP protocol has brought TCP to an area it was not
initially intended for and I find it fantastic to see that this protocol
still scales so well. But we need to consider modern usages of this protocol
for the web, and not just academic research and e-mail.

> >> I.e., at the most this is a man page (specific to an OS). At the least,
> >> this isn't useful at all.
> > As you can see above, nothing I cited was OS-specific but only workload
> > specific. That's why I think that an informational RFC is much more suited
> > to this than an OS-specific man page. The OS man page may rely on the RFC
> > to propose various tuning profiles for different workloads however.
>
> You have a good point that this is general info, but OS issues are not
> in the scope of the IETF and there are courses and books that already
> provide good advice on efficiently running web (and other) servers.

Well, you want to run a quick test ? Google "tcp tuning for http server".
Skip the first two responses which are Daniel's document, take the next one :

    https://gist.github.com/kgriffs/4027835

Read a bit... Not bad, could have been much worse. OK found : 5 seconds
time_wait timeout for conntrack. Dig a little bit more... suggests 2M
time_wait sockets. At 5s per socket, that's an expected 400k connections
per second limit. This with buffers that can be as large as 16MB on both
sides (one could ask why to put 16MB buffers on the Rx path for a web
server). So basically the guy probably thinks that there's a need to
fill up to 6.4 TB/s and despite this the max-orphans was not set according
to the time-wait value, so users of his doc will start to cause some data
truncation before they run into connection failures, thus only their visitors
will know there are problems.

And this is the highest ranked doc on google after Daniel's. And overall
it's really not bad, much better than what we easily find everywhere.

The fact that Daniel's doc found before is a good hope that this sad state
of affairs can reach an end. That's why I think we should encourage him to
continue and give him all the reference he needs to have an undiscutably
good reference doc.

Regards,
Willy

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch


On 8/17/2016 2:13 PM, Willy Tarreau wrote:
> On Wed, Aug 17, 2016 at 11:31:33AM -0700, Joe Touch wrote:
>>> It can be cited in new RFCs
>>> to justify certain choices.
>> Hmm. Like the refs I gave could be cited in this doc to justify *its*
>> choices? :-)
> I think it would be nice that this is cited, but to be clear on one
> point, I've never heard about your papers before you advertised them
> here in this thread,

A search engine on the terms "TCP HTTP interaction" would have popped
them up rather quickly.

> and yet I've been dealing with timewait issues
> for 15 years like many people facing moderate to large web sites
> nowadays.

"timewait issues" and we're the 5th hit in Google.


> It just happens that you probably identified these issues
> very early at low connection rates, but today anyone dealing with
> more than 500-1000 connections per second on a server or worse on
> a gateway quickly discovers that he has to make a choice.

The issue was very common when the doc was written in 99, when even at
that time there were two issues - running out of the number space and
running out of kernel memory.

The number space issue of running out of ports was the basis of the IETF
port names doc in 2006
(https://tools.ietf.org/html/draft-touch-tcp-portnames-00) that became
the current proposal for a TCP "service number option" in 2013 (which
has been discussed at various IETFs in TCPM since then).

>>>> Yes, and discussing those issues would be useful - but not in this
>>>> document either.
>>> Why ? Lots of admins don't understand why the time_wait timeout remains
>>> at 240 seconds on Solaris with people saying "if you want to be conservative
>>> don't touch it but if you want to be modern simply shrink it to 30 seconds
>>> or so". People need to understand why advices have changed over 3 decades.
>> The advice hasn't really changed - the advice was given in the 99 ref,
>> which includes some cases where it can still be appropriate to decrease
>> that timer.
> Most people see it the other way around : they see no valid case to *increase*
> it beyond a few seconds, because for them the default value should be extremely
> low (ie this firewall vendor several years ago trying to insist on one second).
> Yes that's really sad but that's reality. And you can tell them to read 6191
> they won't care.

Most people's servers don't need to run fast enough to care (note that
nearly everyone runs some sort of web server on nearly every device,
whether for control or configuration). The only issue are high-volume
servers (the kind sysadmins deal with), and those people tend to already
know what the tradeoffs are and accept the risks.

>
>>>   - TCP timestamps: what they provide, what are the risks (some people in
>>>     banking environments refuse to enable them so that they cannot be used
>>>     as an oracle to help in timing attacks).
>> That's already covered in the security considerations of RFC 7323. How
>> is HTTP different, if at all, from any other app?
> HTTP is special in that it is fairly common to have to deal with tens of
> thousands of connections per second between one client and one server when
> you are on the server side, because you place a number of gateways (also
> called reverse-proxies) which combine all of the possible issues you can
> think of at a single place.

There are lots of services that have that many transactions - DNS
servers (even local ones), remote databases, etc.

The point is that HTTP doesn't make the problem different, so this isn't
an HTTP issue. It's a high rate server issue.


>  Timestamps are one way to improve fast connection
> recycling between the client and the server without having to cheat on
> timewaits. But since they consume 12 bytes per packet, it's often advised
> to disable them in benchmarks to get the highest throughput...
>
>>>   - window scaling : how much is needed.
>> Same issue here, same ref - how is HTTP different?
> Same as above.
>
>>>   - socket sizing : contrary to what you write, there's a lot of tuning
>>>     on the web where people set the default buffer sizes to 16MB without
>>>     understanding the impacts when dealing with many sockets
>> There's a whole book that encompasses that and some related issues:
>> http://www.onlamp.com/pub/a/onlamp/2005/11/17/tcp_tuning.html
> Looks fine, could be added to the list of references.
>
>> Some advice is also given in Sec 6.3.3 of this:
>> J. Sterbenz, J. Touch, /High Speed Networking: A Systematic Approach to
>> High-Bandwidth Low-Latency Communication/
>> <http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471330361.html>,
>> Wiley, May 2001.
>>
>>>   - SACK : why it's better. DSACK what it adds on top of SACK.
>> That's in the SACK docs, which aren't cited. Again, how is HTTP
>> different from any app?
>>>   - ECN : is it needed ? does it really work ? where does it cause issues ?
>> That's in the ECN docs, which aren't cited. Again, how is HTTP different
>> from any app?
>>>   - SYN cookies : benefits, risks
>> That's in the RFC 4987, which at least IS cited. Again, how is HTTP
>> different from any app?
> So probably you're starting to see the benefit of having a single doc
> to concentrate all this.
The same reason it's useful to have this all in one place is the reason
we already do - there are books and courses on this.

>  You provided at least 3 different articles
> to read and 2 or 3 different RFCs in addition to the original ones,
> of course. A hosting provider whose web sites are down due to a lack
> of tuning doesn't start to read many very long articles and even less
> the most scientific ones, they need to find quick responses that they
> can apply immediately (matter of minutes). So they launch google, they
> type "web site dead, time-wait overflow" and they get plenty of
> responses on stackoverflow and serverfault, many from people having
> done the same in the past and repeating the same mistakes over and over.

These people don't read RFCs to fix problems. They take online courses
or read "how to" books - which do already exist in this space.

> A document validated by several people and giving links for further
> reading can help improve this situation.
Those are the books and courses I'm talking about already.

>
>>>   - TCP reuse/recycling : benefits, risks
>> Not sure what you mean here. There are a lot of docs on the issues with
>> persistent-HTTP vs per-connection HTTP.
> Often on a gateway you cannot completely chose. You have a mix of both.

Sure... but again, that's not in this doc either.

>
>>>   - dealing with buffer bloat : tradeoffs between NIC-based acceleration
>>>     and pacing
>> Bufferbloat typically involves large *uplink* transfers and how they
>> interact with other uplink connections. Neither TCP nor HTTP is involved
>> in this really.
> Maybe in the end Daniel is not the only one not to read all articles
> published on the web :-)
>
>      http://www.cringely.com/2012/03/25/linux-3-3-finally-a-little-good-news-for-bufferbloat/
>      https://www.ietf.org/proceedings/86/slides/slides-86-iccrg-0.pdf
>      https://lwn.net/Articles/564978/
>
> In short, by sending 64kB segments to your NIC and counting on it to
> cut them into small pieces for you and sending the resulting packets
> very close together, you increase the risk of losses on many network
> equipments which run with not-that-large buffers.

Bufferbloat describes what happens when the buffers are too large, not
too small.

The problem you're describing is the interaction between burstiness and
tail-drop, which is addressed by ECN.

>  It's easy to observe
> even on some 10Gbps switches. When you switch mixes 10G and 40G, it's
> horrible, really.
RFCs aren't typically used as context in switch design...


>
>>>   - what are orphans and why you should care about them in HTTP close mode
>> Orphaned TCP connections or orphaned HTTP processes?
> TCP connections. The server sends a response, performs a close() on the
> socket, the data remain in the kernel buffers for the time it takes to
> deliver these data to the client and to get them ACKed. From this point
> the socket is called orphaned on the system because it doesn't belong to
> any process anymore. But immediately after this, a process which runs with
> a limited connection count can accept a new connection, which will in turn
> be fed with large chunks of data and closed. And this runs over and over,
> eating a huge amount of socket memory despite a small limit imposed on the
> server's connection concurrency. At some point the kernel doesn't have
> enough socket buffers anymore (especially after admins have set them to
> 16MB as instructed on $random tuning guide) and starts to kill some orphaned
> connections to get some memory back. The not-so-nice effect that the admin
> cannot detect is that the client gets truncated responses. Only a small part
> of the tail is missing but in the logs, it's said that everything was sent.
> But sent by the process means sent to the system only.

That's an implementation issue of the OS (or a bug, depending on whether
you consider TCP a reliable transport or not, IMO).

>
>>>   - TCP fastopen : how does it work, what type of workload is improved,
>>>     what are the risks (ie: do not enable socket creation without cookie
>>>     by default just because you find it reduces your server load)
>> Another doc that exists.
>>
>>>   - whether to choose a short or a large SYN backlog depending on your
>>>     workload (ie: do you prefer to process everything even if the dequeuing
>>>     is expensive or to drop early in order to recover fast).
>>>
>>> ... and probably many other that don't immediately come to my mind. None
>>> of these ones was a real issue 20 years ago.
>> See above. Many were known around that time, but weren't documented in
>> detail (it took a while for a proper ref to SYN cookies, and the book I
>> wrote with Sterbenz came about because we'd seen wheels being
>> rediscovered for 15 years).
> People rediscover wheels because it's hard to find simple and accurate
> information on the net.
Nobody looks to RFCs to solve that problem...

>  Basically you have the choice :
>   - either uneducated blog posts saying "how I saved my web site using 2
>     sysctls"
>   - or academic papers which are only understandable by scientific people
>     having enough time

... that's what net FAQs are for, as well as courses and books.

> At least the first ones have the merit of being easy to test, and since
> they appear to work they are viral.
>
>>>  All of them became issues for
>>> many web server admins who just copy-paste random settings from various
>>> blogs found on the net who just copy the same stupidities over and over
>>> resulting in the same trouble being caused to each of their reader.
>> This doc is all over the place.
>>
>> If you want a doc to advise web admins, do so.
> That's *exactly* what Daniel started to do when you told him he shouldn't
> do it.

I didn't say a doc to advise web admins wasn't useful. I said it wasn't
an RFC.

It's a web FAQ, a book, etc.


>
>> But most of the items
>> above aren't under admin control; they're buried in app and OS
>> implementations, and most have already evolved do to the right thing.
> That's not true anymore. There was a time where Solaris had one of the
> most configurable TCP stack, you could do ndd /dev/tcp with a lot of
> things. Nowadays all modern operating systems let you fiddle with a ton
> of stuff. And even for the missing parts you can very easily find patches
> all over the web because opensource has changed this considerably.
There are no kernel configs to tell apps to open more than one
connection at a time. You don't need a kernel config to tell Firefox to
disable Nagle, use a reasonable socket size, etc - yes, there are OS
defaults, but most web servers and clients build in the correct
overrides already.

>
>> I agree that a summary of a *focused set* of these might be useful *as a
>> review* (note that review discussions usually include lots of refs).
> The purpose as I understood it was precisely to gather knowledge from
> people here operating such systems and adding some elements found from
> various sources. I also expressed my interest in sharing such experience
> which some will find valid and others not, which is perfect because it
> means there's a choice to make depending on a use case.
>
>> The key question is "what is the focus":
>>
>>     - HTTP/TCP interactions
>>     - server administration advice
>>     - ?
> Those two go hand-in-hand nowadays. You probably know that the difficulty
> for HTTP implementors to find a properly tuned TCP stack is what makes them
> consider UDP-based alternatives,
They want something different for a variety of reasons - the same kind
of airtight logic by which TBL developed HTTP instead of using FTP (he
said that you'd only typically need one file from a location, so why
open 2 connections? now we're stuck trying to mux control and data
rather than having a proper solution that already existed at the time -
it took nearly a decade for HTTP servers to catch up to the performance
of FTP).

> so that they can totally control the
> behaviour from user-space and provide quick updates for their protocol.
> Want an example ?
>
>     https://www.chromium.org/quic

There are many thousands of monkeys typing everywhere - look at the
Linux source if you want even better examples.

> If it was possible to always have a perfectly tuned TCP stack I'm pretty
> sure we wouldn't even hear about such initiatives. And this is not something
> new, I happened to discuss about this subject with some people at IETF83 in
> 2012 already. By then I thought "they have no idea what they're trying to
> reinvent" and now I'm thinking "well, they have their reasons and they
> might be right after all given all the resistance they may be facing on
> the TCP side to get some default timers changed".
>
>> IMO, RFCs should focus on issues specific to the protocols and their
>> deployment - not general knowledge that exists in courses and textbooks.
> Courses and textbooks are totally outdated when they're not simply wrong.
I'm now confused.

You don't want sysadmins to read books or take courses because they're
outdated and thus wrong, but you want to issue an immutable RFC (which
will likely be outdated by the time it's issued)?
> I've first been tought that it was normal to fork a process after performing
> an accept(), resulting in my very first proxy working this way. Then I've
> been taught that it was mandatory to send an empty ACK after a SYN-ACK to
> validate a connection (which is totally wrong otherwise it would not allow
> this ACK to be lost). I've been taught that in order to accept an incoming
> connection you had to have your socket in listen mode exclusively. This is
> false as well, two clients can connect together during their connect()
> phase, it even has some security implications that are often overlooked.
I'm the first to admit there are bad courses, certainly.

> Like it or not, the HTTP protocol has brought TCP to an area it was not
> initially intended for
HTTP makes mistakes that people blame on TCP (like HOL blocking), and
TCP is based on assumptions that are no longer true (not just for HTTP,
but for many other app protocols, e.g., the issue of burst after idle is
based on the outdated assumption that most transfers are roughly symmetric).

> and I find it fantastic to see that this protocol
> still scales so well.
> But we need to consider modern usages of this protocol
> for the web, and not just academic research and e-mail.
You might consider that TCPM and TSVWG don't exist for just "academic
research and e-mail". What do you think we've been doing for the past 40
years?

>
>>>> I.e., at the most this is a man page (specific to an OS). At the least,
>>>> this isn't useful at all.
>>> As you can see above, nothing I cited was OS-specific but only workload
>>> specific. That's why I think that an informational RFC is much more suited
>>> to this than an OS-specific man page. The OS man page may rely on the RFC
>>> to propose various tuning profiles for different workloads however.
>> You have a good point that this is general info, but OS issues are not
>> in the scope of the IETF and there are courses and books that already
>> provide good advice on efficiently running web (and other) servers.
> Well, you want to run a quick test ? Google "tcp tuning for http server".
> Skip the first two responses which are Daniel's document, take the next one :
>
>     https://gist.github.com/kgriffs/4027835

Like I said about Linux... ;-)

> Read a bit... Not bad, could have been much worse. OK found : 5 seconds
> time_wait timeout for conntrack. Dig a little bit more... suggests 2M
> time_wait sockets. At 5s per socket, that's an expected 400k connections
> per second limit. This with buffers that can be as large as 16MB on both
> sides (one could ask why to put 16MB buffers on the Rx path for a web
> server). So basically the guy probably thinks that there's a need to
> fill up to 6.4 TB/s and despite this the max-orphans was not set according
> to the time-wait value, so users of his doc will start to cause some data
> truncation before they run into connection failures, thus only their visitors
> will know there are problems.
>
> And this is the highest ranked doc on google after Daniel's. And overall
> it's really not bad, much better than what we easily find everywhere.
>
> The fact that Daniel's doc found before is a good hope that this sad state
> of affairs can reach an end. That's why I think we should encourage him to
> continue and give him all the reference he needs to have an undiscutably
> good reference doc.

You might try some other terms in your searches. Like just "TCP tuning",
or "TCP HTTP interactions".

Joe

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Adrien de Croy


------ Original Message ------
From: "Joe Touch" <[hidden email]>

>They want something different for a variety of reasons - the same kind
>of airtight logic by which TBL developed HTTP instead of using FTP (he
>said that you'd only typically need one file from a location, so why
>open 2 connections? now we're stuck trying to mux control and data
>rather than having a proper solution that already existed at the time -
>it took nearly a decade for HTTP servers to catch up to the performance
>of FTP).
>
Whilst I've been finding this discussion very informative and
interesting, I have to raise an objection on this point.

FTP was never going to be suitable for the web, and a very simple RTT
analysis shows that.

Apart from initial 3 way TCP handshake and close, which is the same for
both, with http you have a request and a response, whereas FTP requires
you to wait for the server welcome, log in, negotiate another port, set
up a data connection in addition to retrieving the file

So it's at minimum 5 round trips more.

Then try adding all the firewall issues due to transmitting data
connection endpoint information over the control connection and it's no
surprise FTP is not favoured for downloads.
So FTP was never going to be a "proper solution" for the web without a
complete re-architecture.

Adrien


>


Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch
This is a bit of a side track, but...

On 8/17/2016 3:51 PM, Adrien de Croy wrote:

>
>
> ------ Original Message ------
> From: "Joe Touch" <[hidden email]>
>
>> They want something different for a variety of reasons - the same kind
>> of airtight logic by which TBL developed HTTP instead of using FTP (he
>> said that you'd only typically need one file from a location, so why
>> open 2 connections? now we're stuck trying to mux control and data
>> rather than having a proper solution that already existed at the time -
>> it took nearly a decade for HTTP servers to catch up to the performance
>> of FTP).
>>
> Whilst I've been finding this discussion very informative and
> interesting, I have to raise an objection on this point.
>
> FTP was never going to be suitable for the web, and a very simple RTT
> analysis shows that.
>
> Apart from initial 3 way TCP handshake and close, which is the same
> for both, with http you have a request and a response, whereas FTP
> requires you to wait for the server welcome, log in, negotiate another
> port, set up a data connection in addition to retrieving the file

That's only the first time you go somewhere new. You don't need to close
both ports so quickly; the control channel can stay open and you thus
avoid HOL blocking between data and control (and thus the need to
chunk-and-mux within persistent HTTP), which increases other delays for
HTTP.

Neither protocol matches exactly what is really needed for a true
transaction-oriented protocol.

> ...
> Then try adding all the firewall issues due to transmitting data
> connection endpoint information over the control connection and it's
> no surprise FTP is not favoured for downloads.

FTP had a passive mode even back then that avoids this issue. It also
had suspend/resume, compression, and format conversion.

Joe

Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Adrien de Croy


------ Original Message ------
From: "Joe Touch" <[hidden email]>
To: "Adrien de Croy" <[hidden email]>; "Willy Tarreau" <[hidden email]>
Cc: "Mark Nottingham" <[hidden email]>; "[hidden email]" <[hidden email]>;
"HTTP Working Group" <[hidden email]>; "Patrick McManus"
<[hidden email]>; "Daniel Stenberg" <[hidden email]>
Sent: 18/08/2016 11:01:57 AM
Subject: Re: [tcpm] TCP Tuning for HTTP - update

>This is a bit of a side track, but...
>
>On 8/17/2016 3:51 PM, Adrien de Croy wrote:
>>
>>
>>  ------ Original Message ------
>>  From: "Joe Touch" <[hidden email]>
>>
>>>  They want something different for a variety of reasons - the same
>>>kind
>>>  of airtight logic by which TBL developed HTTP instead of using FTP
>>>(he
>>>  said that you'd only typically need one file from a location, so why
>>>  open 2 connections? now we're stuck trying to mux control and data
>>>  rather than having a proper solution that already existed at the
>>>time -
>>>  it took nearly a decade for HTTP servers to catch up to the
>>>performance
>>>  of FTP).
>>>
>>  Whilst I've been finding this discussion very informative and
>>  interesting, I have to raise an objection on this point.
>>
>>  FTP was never going to be suitable for the web, and a very simple RTT
>>  analysis shows that.
>>
>>  Apart from initial 3 way TCP handshake and close, which is the same
>>  for both, with http you have a request and a response, whereas FTP
>>  requires you to wait for the server welcome, log in, negotiate
>>another
>>  port, set up a data connection in addition to retrieving the file
>
>That's only the first time you go somewhere new. You don't need to
>close
>both ports so quickly; the control channel can stay open and you thus
>avoid HOL blocking between data and control (and thus the need to
>chunk-and-mux within persistent HTTP), which increases other delays for
>HTTP.
You still need to send another PORT/PASV and wait for the response
before making another TCP connection for data, since you can't re-use
this one.

So subsequent requests to the same server will be quicker, still at
least 3 round-trips more than a subsequent http request on a persistent
connection.

>
>Neither protocol matches exactly what is really needed for a true
>transaction-oriented protocol.
We are probably in agreement here.

>
>>  ...
>>  Then try adding all the firewall issues due to transmitting data
>>  connection endpoint information over the control connection and it's
>>  no surprise FTP is not favoured for downloads.
>
>FTP had a passive mode even back then that avoids this issue. It also
>had suspend/resume, compression, and format conversion.

Those all take more round trips to negotiate.  As for format
conversion..... bad idea, never should have been the server's job.

We've seen a lot of server-side firewall problems with PASV as well.

Adrien


>
>Joe


Reply | Threaded
Open this post in threaded view
|

Re: [tcpm] TCP Tuning for HTTP - update

Joe Touch



On 8/17/2016 4:17 PM, Adrien de Croy wrote:
FTP had a passive mode even back then that avoids this issue. It also
had suspend/resume, compression, and format conversion.

Those all take more round trips to negotiate.  As for format conversion..... bad idea, never should have been the server's job.
Sure, but they can be negotiated once for a connection.

FTP also supports wildcard retrieval (e.g., "get *"), which can help with anticipation.

My point is that FTP could have been much better place to start towards a transaction protocol.

Joe
123