FtpFindFirstFile and timeout

Discussion:

(too old to reply)

BobM

2006-11-27 11:15:02 UTC

Hi,
I have an application that polls an ftp site for new files, and downloads
them as they appear. I use FtpFindFirstFile/InternetFindNextFile etc. to
enumerate every 60 seconds or so. I keep the ftp connection alive between
calls.
The first time I call FtpFindFirstFile, the response takes around 20
seconds, returning around 7000 - 8000 file names. The second time I call
FtpFindFirstFile 60 seconds later, the call returns 12002 timeout error after
30 seconds. I have tried setting both INTERNET_OPTION_CONNECT_TIMEOUT and
INTERNET_OPTION_RECEIVE_TIMEOUT to 60000ms, but the second call to
FtpFindFirstFile always times out after 30 seconds. The handle returned from
FtpFindFirstFile is closed correctly after each poll.
So, two questions...
1. Why would the second call always timeout?, and
2. How to I set the timeout for FtpFindFirstFile?
Thanks for any help!

Paul Baker [MVP, Windows - Networking]

2006-11-27 14:09:41 UTC

Permalink

Do you believe there really is a network problem causing this?

Or could it simply be that your firewall or AV software/hardware is
interfering?

I wouldn't change the default timeouts if I were you.

Paul

Post by BobM
Hi,
I have an application that polls an ftp site for new files, and downloads
them as they appear. I use FtpFindFirstFile/InternetFindNextFile etc. to
enumerate every 60 seconds or so. I keep the ftp connection alive between
calls.
The first time I call FtpFindFirstFile, the response takes around 20
seconds, returning around 7000 - 8000 file names. The second time I call
FtpFindFirstFile 60 seconds later, the call returns 12002 timeout error after
30 seconds. I have tried setting both INTERNET_OPTION_CONNECT_TIMEOUT and
INTERNET_OPTION_RECEIVE_TIMEOUT to 60000ms, but the second call to
FtpFindFirstFile always times out after 30 seconds. The handle returned from
FtpFindFirstFile is closed correctly after each poll.
So, two questions...
1. Why would the second call always timeout?, and
2. How to I set the timeout for FtpFindFirstFile?
Thanks for any help!

BobM

2006-11-27 21:43:04 UTC

Permalink

Thanks Paul,

OK. I tried the command prompt ftp client and it exhibits the same behaviour
connected to this particular server. I CD to the correct directory, then
first DIR returns the 8000 odd entries (583299kB) in 10 to 15 seconds.
Subsequent DIRs return the same, but it take 50 to 70 seconds. If I close the
connection, first DIR is quick again, subsequent slow.

I disabled my AV, and the network administrator assures me there is nothing
configured in the firewall that could cause this.

As I mentioned I tried to increase the timeout, but I still get a timeout in
30 seconds. How do I set the timeout for FtpFindFirstFile?

Thanks again
Bob

Post by Paul Baker [MVP, Windows - Networking]
Do you believe there really is a network problem causing this?
Or could it simply be that your firewall or AV software/hardware is
interfering?
I wouldn't change the default timeouts if I were you.
Paul

Paul Baker [MVP, Windows - Networking]

2006-11-27 22:40:26 UTC

Permalink

Actually, Internet Explorer will test how FTP works using WinInet while
eliminating your application.

There is an advanced setting in Internet Options that tells you whether it
will do active or passive FTP. Passive is more firewall friendly, though try
it both ways.

The FTP Command Prompt always does active FTP and, as far as I know, does
not use WinInet.

I rarely take the assurances of *my* network administrator at face value. I
would say that the firewall has not been eliminated. What do you think the
cause of the timeout is? Does it happen with just one FTP site or all that
you have tried? Which ones have you tried?

I don't see the point in changing the timeout. I would want to be finding
out its cause, not tweaking how it is handled.

Paul

Post by BobM
Thanks Paul,
OK. I tried the command prompt ftp client and it exhibits the same behaviour
connected to this particular server. I CD to the correct directory, then
first DIR returns the 8000 odd entries (583299kB) in 10 to 15 seconds.
Subsequent DIRs return the same, but it take 50 to 70 seconds. If I close the
connection, first DIR is quick again, subsequent slow.
I disabled my AV, and the network administrator assures me there is nothing
configured in the firewall that could cause this.
As I mentioned I tried to increase the timeout, but I still get a timeout in
30 seconds. How do I set the timeout for FtpFindFirstFile?
Thanks again
Bob

Post by BobM
Hi,
I have an application that polls an ftp site for new files, and downloads
them as they appear. I use FtpFindFirstFile/InternetFindNextFile etc. to
enumerate every 60 seconds or so. I keep the ftp connection alive between
calls.
The first time I call FtpFindFirstFile, the response takes around 20
seconds, returning around 7000 - 8000 file names. The second time I call
FtpFindFirstFile 60 seconds later, the call returns 12002 timeout error after
30 seconds. I have tried setting both INTERNET_OPTION_CONNECT_TIMEOUT and
INTERNET_OPTION_RECEIVE_TIMEOUT to 60000ms, but the second call to
FtpFindFirstFile always times out after 30 seconds. The handle
returned
from
FtpFindFirstFile is closed correctly after each poll.
So, two questions...
1. Why would the second call always timeout?, and
2. How to I set the timeout for FtpFindFirstFile?
Thanks for any help!

BobM

2006-11-27 23:26:01 UTC

Permalink

Thanks Paul,

I suspect that the longer response from the server may be due to bandwith
throttling from the ftp server on a per connection basis. the server is
ProFTPD. I will talk to the ftp administrator later today.

In the meantime, I would still like to know how to increase the timeout for
FtpFindFirstFile if that is possible, as there may be cases where the default
30 seconds is just insufficient.

Thanks
Bob

Post by Paul Baker [MVP, Windows - Networking]
Actually, Internet Explorer will test how FTP works using WinInet while
eliminating your application.
There is an advanced setting in Internet Options that tells you whether it
will do active or passive FTP. Passive is more firewall friendly, though try
it both ways.
The FTP Command Prompt always does active FTP and, as far as I know, does
not use WinInet.
I rarely take the assurances of *my* network administrator at face value. I
would say that the firewall has not been eliminated. What do you think the
cause of the timeout is? Does it happen with just one FTP site or all that
you have tried? Which ones have you tried?
I don't see the point in changing the timeout. I would want to be finding
out its cause, not tweaking how it is handled.
Paul

Post by BobM
Hi,
I have an application that polls an ftp site for new files, and downloads
them as they appear. I use FtpFindFirstFile/InternetFindNextFile etc. to
enumerate every 60 seconds or so. I keep the ftp connection alive between
calls.
The first time I call FtpFindFirstFile, the response takes around 20
seconds, returning around 7000 - 8000 file names. The second time I call
FtpFindFirstFile 60 seconds later, the call returns 12002 timeout error after
30 seconds. I have tried setting both INTERNET_OPTION_CONNECT_TIMEOUT and
INTERNET_OPTION_RECEIVE_TIMEOUT to 60000ms, but the second call to
FtpFindFirstFile always times out after 30 seconds. The handle
returned
from
FtpFindFirstFile is closed correctly after each poll.
So, two questions...
1. Why would the second call always timeout?, and
2. How to I set the timeout for FtpFindFirstFile?
Thanks for any help!

Vladimir Scherbina

2006-11-28 13:47:46 UTC

Permalink

If you're under IE6 then most likely the problem is caused by the bug in
InternetSetOption: http://support.microsoft.com/kb/176420 ;
--
Vladimir (Windows SDK MVP)

Post by BobM
Thanks Paul,
I suspect that the longer response from the server may be due to bandwith
throttling from the ftp server on a per connection basis. the server is
ProFTPD. I will talk to the ftp administrator later today.
In the meantime, I would still like to know how to increase the timeout for
FtpFindFirstFile if that is possible, as there may be cases where the default
30 seconds is just insufficient.
Thanks
Bob

BobM

2006-11-28 14:13:01 UTC

Permalink

Vladimir,

Thanks for the response.

I have IE7 installed, and also tried with IE6 PC.

The referenced KB article http://support.microsoft.com/kb/224318/ in the
link you mentioned is fine if you want to decrease the timeout for a
connection, but will not work if you want to increase the timeout beyond the
default.

So, it appears the SetInternetOption with timeouts is still not fixed in IE7.

Anyway, the cause of the long delay from the ftp server is that proftpd
builds the list in memory before returning anything. With 8000 odd files it
was a bit overloaded. Reducing the files in the directory to around 4000
gives consistent responses in about 8 seconds.

Thanks again Vladimir and Paul for your help.

Bob

Post by Vladimir Scherbina
If you're under IE6 then most likely the problem is caused by the bug in
InternetSetOption: http://support.microsoft.com/kb/176420 ;
--
Vladimir (Windows SDK MVP)

Vladimir Scherbina

2006-11-28 14:38:32 UTC

Permalink

Bob,

Post by BobM
I have IE7 installed, and also tried with IE6 PC.
So, it appears the SetInternetOption with timeouts is still not fixed in IE7.

Probably. If I will have enough time today I will try to look at the (IE7)
InternetSetOption internals.

Post by BobM
Anyway, the cause of the long delay from the ftp server is that proftpd
builds the list in memory before returning anything. With 8000 odd files it
was a bit overloaded. Reducing the files in the directory to around 4000
gives consistent responses in about 8 seconds.

This is not the right solution. The well-designed systems should not depend
linearly upon amount of items to process.

In your particular situation I would choose the asyn. mode of wininet to
control all the things. If it really scares you then you can switch to 3rd
party implementations of FTP protocol, or even write your own wrapper.

--
Vladimir (Windows SDK MVP)