WinInet COM encapsulation slow to unload

[TxITGuy]

Ok, I almost have everything working the way it should here, but I have an
elusive problem in my code somewhere. Everything runs fine when the control
is loaded and when I execute methods / receive events. However, when I
attempt to shut down the component, it takes as long a 30 seconds to unload
and return control to the hosing app.

This problem is most apparent after calling the following method:

STDMETHODIMP CHTTP::GetRequest(BSTR sDomain, BSTR sDocument, BSTR
sQueryString, BSTR sAccept, BOOL bSecure)
{
char *lpszDomain = NULL;
char *lpszDocument = NULL;
char *lpszQueryString = NULL;
char *lpszAccept = NULL;
char **lplpszAccepts = NULL; //variable to hold "Accepts:" string array.
unsigned int uiRows = 0;
BOOL bRequestSent = FALSE;
HINTERNET hSession = NULL;
HINTERNET hRequest = NULL;
DWORD dwErrorNum = NULL;
string sError;
string sResource;

if(m_bOpen)
{
lpszDomain = _com_util::ConvertBSTRToString(sDomain);
lpszDocument = _com_util::ConvertBSTRToString(sDocument);
lpszQueryString = _com_util::ConvertBSTRToString(sQueryString);
sResource.append(lpszDocument); //concatenate document and
sResource.append(lpszQueryString); //querystring for HttpOpenRequest
lpszAccept = _com_util::ConvertBSTRToString(sAccept);

m_bSecure = bSecure;
hSession = GetInetSession(lpszDomain,INTERNET_SERVICE_HTTP,80);

//assemble and save the current URL string.
SetCurrentURL((const char *) lpszDomain,(const char *) lpszDocument,
bSecure);

if(split(lpszAccept,',',&lplpszAccepts,&uiRows))
{
//convert "Accepts:" string array to a constant pointer for API.
hRequest =
HttpOpenRequest(hSession,"GET",sResource.c_str(),"HTTP/1.1","",(const char**)
lplpszAccepts,INTERNET_FLAG_PRAGMA_NOCACHE,(DWORD) this);

delete [] lpszDomain;
delete [] lpszDocument;
delete [] lpszQueryString;
delete [] lpszAccept;
freearray(lplpszAccepts,uiRows);

if(hRequest != NULL)
{
bRequestSent = HttpSendRequest(hRequest,0,0,0,0);

if(bRequestSent) return S_OK;
else sError.append("Unable to send request.\r\n");
}
else sError.append("Unable to open request.\r\n");

dwErrorNum = GetLastError();
sError.append(InetErrorDesc(dwErrorNum));
}
else
{
dwErrorNum = 0xE0000002;
sError.append("Insufficient memory to complete requested operation.");
}
}
else
{
dwErrorNum = 0xE0000003;
sError.append("HTTP class not ready, please call Open method.");
}

return Error(sError.c_str(),GUID_NULL,dwErrorNum);

}

The above method depends on the following functions:

void freearray(char **sArray, unsigned int uiRows)
{
for (unsigned int i = 0; i < uiRows; i++)
free(sArray[i]);

free(sArray);
}

void CHTTP::SetCurrentURL(const char *lpszDomain, const char *lpszResource,
BOOL bSecure)
{
if(bSecure) m_sURL = "https://";
else m_sURL = "http://";

m_sURL += lpszDomain;
m_sURL += lpszResource;
}

This function is overloaded, but the implementation below is the one called
by the above method.

HINTERNET CHTTP::GetInetSession(char* sDomain, DWORD dwService, int iPort)
{
int iCacheIndex = 0;

//do we have a handle cached that will support the requested session?
for(int iSessionIndex=0;iSessionIndex <=9;iSessionIndex++)
{
if(m_isSessions[iSessionIndex].sAddress == sDomain &&
m_isSessions[iSessionIndex].dwService == dwService &&
m_isSessions[iSessionIndex].iPort == iPort)
{ return m_isSessions[iSessionIndex].hSession; }
}

if(m_iCachedSessions == 10)
{
//the array for cached sessions is full, so we need to close a session
handle
InternetCloseHandle(m_isSessions[m_iReplaceIndex].hSession);

//now, 0 out the closed session handle, and store the new server info.
m_isSessions[m_iReplaceIndex].hSession = 0;
m_isSessions[m_iReplaceIndex].sAddress = sDomain;
m_isSessions[m_iReplaceIndex].dwService = dwService;
m_isSessions[m_iReplaceIndex].iPort = iPort;

//save the index of the closed handle
iCacheIndex = m_iReplaceIndex;
//and update the index to indicate the next replacable handle
//(this way, the oldest handle is always the one replaced).
if(++m_iReplaceIndex == 10) m_iReplaceIndex = 0;
}
else
{
iCacheIndex = m_iCachedSessions++;
m_isSessions[m_iReplaceIndex].sAddress = sDomain;
m_isSessions[m_iReplaceIndex].dwService = dwService;
m_isSessions[m_iReplaceIndex].iPort = iPort;
}

switch(dwService)
{
case INTERNET_SERVICE_HTTP:
m_isSessions[iCacheIndex].hSession =
InternetConnect(m_hInternet,sDomain,iPort,"","",dwService,0,(DWORD) this);
break;
case INTERNET_SERVICE_GOPHER:
m_isSessions[iCacheIndex].hSession =
InternetConnect(m_hInternet,sDomain,iPort,NULL,NULL,dwService,0,(DWORD)
this);
break;
case INTERNET_SERVICE_FTP:
m_isSessions[iCacheIndex].hSession =
InternetConnect(m_hInternet,sDomain,iPort,NULL,NULL,dwService,0,(DWORD)
this);
break;
default:
m_isSessions[iCacheIndex].hSession =
InternetConnect(m_hInternet,sDomain,iPort,"","",dwService,0,(DWORD) this);
break;
}

return m_isSessions[iCacheIndex].hSession;
}

The destructor function contains the following code:

~CHTTP()
{
for(int iIndex=0;iIndex <= 0;iIndex++)
{
if(m_isSessions[iIndex].hSession != 0)
{
InternetSetStatusCallback(m_isSessions[iIndex].hSession,NULL);
InternetCloseHandle(m_isSessions[iIndex].hSession);
}
}

if(m_hInternet != 0)
{
InternetSetStatusCallback(m_hInternet,NULL);
InternetCloseHandle(m_hInternet);
}
}

I have stepped through this code until I can almost see it in my sleep, and
can't find what I'm forgetting to to close or deallocate. I'm hoping some
fresh eyes can tell me why the component is taking so horribly long to unload
itself. By the way, it doesn't get bogged down until well after it finishes
executing the destructor code.

Thanks,
Ben

[ismailp]

did you profile your code? frankly, I need to "test" the code in order
to make an accurate comment. however, you can profile your code, and
see where you spend time during destruction of the obejct.

Ismail

[TxITGuy]

Ok, good suggestion ... but the results don't seem to indicate that the delay
is anywhere in my code. Not sure what to make of this.

Program Statistics
------------------
Command line at 2005 Mar 16 11:06: tstcon32
Total time: 108888.565 millisecond
Time outside of functions: 108590.025 millisecond
Call depth: 12
Total functions: 858
Total hits: 4950
Function coverage: 50.3%
Overhead Calculated 7
Overhead Average 7

Module Statistics for httpcomm.dll
----------------------------------
Time in module: 298.541 millisecond
Percent of time in module: 100.0%
Functions in module: 858
Hits in module: 4950
Module function coverage: 50.3%
<snip>

According to this, my functions (collectively) took less than 1/3 of a
second to do their jobs. The host program, however, took 1 minute and 48
seconds to do it's job. Have I been fighting with a Microsoft problem again
these last four days?

The Close method and the destructor function also ran very quickly.

Func Func+Child Hit
Time % Time % Count Function
---------------------------------------------------------
0.002 0.0 0.109 0.0 1 CHTTP::~CHTTP(void)
(httpcomm.obj)
0.001 0.0 12.072 4.0 1 CHTTP::Close(void) (http.obj)

Most of the execution time, by far, in the DLL was spent in a function I
didn't even write:
223.978 75.0 223.994 75.0 1 _$E26 (http.obj)

This execution times are in milliseconds, of course, and so don't even begin
to explain where the delay is occuring.

Any ideas? At this point, I'll package up all the source code in a ZIP file
and send it to anyone who is willing to look at it and thinks they can
identify the problem. Just email me at ***@didasko.net.

Ben

Post by ismailp
did you profile your code? frankly, I need to "test" the code in order
to make an accurate comment. however, you can profile your code, and
see where you spend time during destruction of the obejct.
Ismail

[Stephen Sulzer]

Is there a module in your DLL component named http.cpp/.obj ? (Or does it
belong to the host app?) If it is your module, what are the global
variables in it? Are any of those globals static C++ objects with stuff
that occurs in a destructor?

Also, how do you link to WinInet.dll? Static (wininet.lib) or dynamic
linking (LoadLibrary("wininet.dll"))? Do you use the delay-loading of DLLs
linker option?

Can you unload your HTTP control without shutting down the host application?
If yes, then please try this little experiment: somewhere within your
control add a call to LoadLibrary("wininet.dll") and do not unload that
reference. That is, deliberately keep WinInet.dll loaded in memory beyond
the lifetime of your HTTP control. With this deliberate DLL "leak", does
the delay still occur when your control unloads?


- Stephen

[TxITGuy]

Post by Stephen Sulzer
Also, how do you link to WinInet.dll? Static (wininet.lib) or dynamic
linking (LoadLibrary("wininet.dll"))? Do you use the delay-loading of DLLs
linker option?
Can you unload your HTTP control without shutting down the host application?
If yes, then please try this little experiment: somewhere within your
control add a call to LoadLibrary("wininet.dll") and do not unload that
reference. That is, deliberately keep WinInet.dll loaded in memory beyond
the lifetime of your HTTP control. With this deliberate DLL "leak", does
the delay still occur when your control unloads?

Wow ... that did it! After compiling with LoadLibrary("wininet.dll") in the
constructor, my component unloaded immediately. It can't stay that way for
production, of course ... so what does that mean?

Should I still be looking for a mistake in my use of WinInet, or is there a
problem with the DLL itself that I need to code a workaround for?

Post by Stephen Sulzer
Is there a module in your DLL component named http.cpp/.obj ? (Or does it
belong to the host app?) If it is your module, what are the global
variables in it? Are any of those globals static C++ objects with stuff
that occurs in a destructor?

HTTP.cpp belongs to my component, here is the source from the constructor
and destructor for those functions, along with the variable declarations.
The only global variables with a destructor are those declared as string (4
of them, 1 in a struct), and none of them are declared as static.

//global variables
BOOL m_bOpen;
HINTERNET m_hInternet; //Handle returned by InternetOpenA API function.

struct inet_session //internet session structure, used to cache active
sessions.
{
HINTERNET hSession;
string sAddress;
DWORD dwService;
int iPort;
} m_isSessions [10];

int m_iCachedSessions; //how many open sessions have been cached?
int m_iReplaceIndex; //which session do we replace if the cache is full?
DWORD m_dwLastStatus; //what was the last request status?
string m_sFormFields; //contains the form fields for POST.
// string sReferrer; //the referring URL, if any.
LPSTR m_lpszCookies; //buffer to hold received cookie data

//global variables for public properties.
long m_lBytesSent;
long m_lBytesReceived;
BOOL m_bSecure;
string m_sURL;

CHTTP()
{
LoadLibrary("wininet.dll");

for(int iIndex=0;iIndex <= 9;iIndex++)
{
m_isSessions[iIndex].hSession = 0;
m_isSessions[iIndex].sAddress = "";
m_isSessions[iIndex].dwService = 0;
m_isSessions[iIndex].iPort = 0;
}

m_iCachedSessions = 0;
m_iReplaceIndex = 0;
m_hInternet = 0;
m_bOpen = FALSE;
m_dwLastStatus = 0;
m_lBytesSent = 0;
m_lBytesReceived = 0;
m_bSecure = FALSE;
m_lpszCookies = NULL;
}

~CHTTP()
{
BOOL bSuccess = FALSE;
VOID* pCallback = NULL;

for(int iIndex=0;iIndex <= 9;iIndex++)
{
if(m_isSessions[iIndex].hSession != 0)
{
pCallback = InternetSetStatusCallback(m_isSessions[iIndex].hSession,NULL);
Fire_ClosingInetHandle((long) m_isSessions[iIndex].hSession);
bSuccess = InternetCloseHandle(m_isSessions[iIndex].hSession);
pCallback = NULL;
bSuccess = FALSE;
}
}

pCallback = NULL;
bSuccess = FALSE;

if(m_hInternet != 0)
{
pCallback = InternetSetStatusCallback(m_hInternet,NULL);
Fire_ClosingInetHandle((long) m_hInternet);
bSuccess = InternetCloseHandle(m_hInternet);
}
}

[Stephen Sulzer]

It appears that the delay occurs when wininet.dll is dynamically unloaded
from the process
when your control is unloaded. During a dynamic unload, WinInet will
shutdown any socket connections it has open. My guess is that the delay is
occurring while WinInet closes its open socket connections.

Another way to try to confirm this is the problem is to add a "Connection:
Close" request header to each HTTP request that you send. This will force
WinInet to discard the socket as soon as the HTTP request is finished, so
there will not be a bunch of sockets to close when wininet.dll is unloaded.
Also be sure to remove the LoadLibrary("wininet.dll") call from your CHTTP
constructor.

Can you reproduce the shutdown delay even if your HTTP control only makes
one (or a small few) number of HTTP requests? How many different servers is
it sending requests to? Try to narrow down the scenario further such that
the delay occurs when communicating with only one particular server. Does
the delay occur regardless of what HTTP server your control communicates
with?

What version of Windows (including service pack level) are you testing on?

Also, out of curiosity, why do you create a pool WinInet session handles? A
single WinInet session can be used to send HTTP requests to different
servers. If you are writing a component for a multi-user application, you
cannot use WinInet; instead, you should use WinHTTP.


- Stephen