Discussion:
InternetWriteFile - 12031 - HTTP Post upload
(too old to reply)
Paul Baker [MVP, Windows - SDK]
2004-08-30 20:55:30 UTC
Permalink
Alex,

This error code is ERROR_INTERNET_CONNECTION_RESET. This means that the web
server reset your connection, ie. kicked you off.

HTTP is not intended for large file uploads. FTP is more suitable for this.

Some HTTP (web) servers do not support large file uploads. For example, IIS
has an upper limit of 2GB in some versions and 1GB in others. I think it can
be changed using a registry entry, but there is no point. IIS loads the
entire request into memory before passing it off to the component that will
handle the upload. This is presumably so that all the properties of the
request are available, eg. the Request.TotalBytes property in ASP. Because
it is in memory, IIS tends to be incredibly slow when uploading a file that
is larger than a few hundred MBs.

Paul
I'm sending large files up to our webserver using HTTP file upload, using
InternetWriteFile (and HttpSendRequestEx).
Often users get this error.. I get it repeatedly on my win95 test box in
VirtualPC2004, i can't even complete a 1mb upload over our local 100mbps
lan.
Many users have successfully uploaded 600mb files.
Is there some way I can build in resume functionality? theoriticaly if I
get
this error can I just write the bytes again?
why am I getting this error so consistently on win95?
thx
- Alex
--
Alex Black
Paul Baker [MVP, Windows - SDK]
2004-08-31 15:33:06 UTC
Permalink
Is the web server you are uploading to IIS? If you can be clear on that, it
would help, as we can limit our discussion to that. What version of IIS is
it?

I'm fairly sure the request is held entirely in memory no matter what is
handling it. I think I tested it for ISAPI extensions too. That's why those
third party tools to handle uploads don't help much.

If I were you, I would reconsider your use of HTTP for uploading large
files. I have been down the same path as you, I know it's frustrating, but
it's just not designed for that and it will never be reliable.

Paul
hi Paul, we've successfully uploaded ~1gb files.. we are using currently a
free component from Dundas to do uploading.
- sample code to write an IIS isap extension that can stream uploads to
disk
- I think the dundas controls streams the upload to disk
- I think its only ASP perhaps that keeps it all in memory.
I am going to build an upload resume feature... basically the server will
issue the client an ID. if the client gets kicked off, the client will
ask
where to resume from (by ID) and the server will append.
Post by Paul Baker [MVP, Windows - SDK]
Alex,
This error code is ERROR_INTERNET_CONNECTION_RESET. This means that the web
server reset your connection, ie. kicked you off.
HTTP is not intended for large file uploads. FTP is more suitable for this.
Some HTTP (web) servers do not support large file uploads. For example, IIS
has an upper limit of 2GB in some versions and 1GB in others. I think it can
be changed using a registry entry, but there is no point. IIS loads the
entire request into memory before passing it off to the component that will
handle the upload. This is presumably so that all the properties of the
request are available, eg. the Request.TotalBytes property in ASP. Because
it is in memory, IIS tends to be incredibly slow when uploading a file that
is larger than a few hundred MBs.
Paul
I'm sending large files up to our webserver using HTTP file upload, using
InternetWriteFile (and HttpSendRequestEx).
Often users get this error.. I get it repeatedly on my win95 test box in
VirtualPC2004, i can't even complete a 1mb upload over our local 100mbps
lan.
Many users have successfully uploaded 600mb files.
Is there some way I can build in resume functionality? theoriticaly if I
get
this error can I just write the bytes again?
why am I getting this error so consistently on win95?
thx
- Alex
--
Alex Black
Brian Combs
2004-08-31 16:17:00 UTC
Permalink
Hello
If you need to upload large files with the ability to resume the upload if
it fails you should look at BITS SDK. IIS does not buffer the full
incomming file. However the end application can if it is a POST. In many
cases a PUT will work just fine for uploading a large file.

For more information on BITS SDK
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/b
its_start_page.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/w
hat_s_new.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/i
is_requirements_for_bits_uploads.asp


Thanks
Brian [MSFT]
Microsoft Developer Support
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------
| Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| thread-index: AcSPcXoVQNWtSF82SDytPW2+AvFBJQ==
| X-WBNR-Posting-Host: 209.183.130.140
| From: "=?Utf-8?B?bm9pcnM=?=" <***@discussions.microsoft.com>
| References: <807A87B7-8DA8-481C-A41C-***@microsoft.com>
<***@tk2msftngp13.phx.gbl>
<3248369B-2471-493B-8E38-***@microsoft.com>
<#lNAI$***@tk2msftngp13.phx.gbl>
| Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| Date: Tue, 31 Aug 2004 08:45:01 -0700
| Lines: 103
| Message-ID: <A3318F9D-C766-438E-B655-***@microsoft.com>
| MIME-Version: 1.0
| Content-Type: text/plain;
| charset="Utf-8"
| Content-Transfer-Encoding: 7bit
| X-Newsreader: Microsoft CDO for Windows 2000
| Content-Class: urn:content-classes:message
| Importance: normal
| Priority: normal
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| Newsgroups: microsoft.public.inetsdk.programming.wininet
| NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| Xref: cpmsftngxa10.phx.gbl
microsoft.public.inetsdk.programming.wininet:11633
| X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
|
| Yes, i'm using IIS (5 I believe, on win2k).
|
| I have a hard time believing that IIS stores the whole file in ram, I
will
| try it out :) my machine only has 512mb ram, so uploading a 600mb file
would
| use up all its physical ram and then some.
|
| This company has a version of their upload component which they claim
| supports resume (and I assume streaming), but its v. expensive
|
| http://fileup.softartisans.com/fileup-147.aspx
|
| "Paul Baker [MVP, Windows - SDK]" wrote:
|
| > Is the web server you are uploading to IIS? If you can be clear on
that, it
| > would help, as we can limit our discussion to that. What version of IIS
is
| > it?
| >
| > I'm fairly sure the request is held entirely in memory no matter what is
| > handling it. I think I tested it for ISAPI extensions too. That's why
those
| > third party tools to handle uploads don't help much.
| >
| > If I were you, I would reconsider your use of HTTP for uploading large
| > files. I have been down the same path as you, I know it's frustrating,
but
| > it's just not designed for that and it will never be reliable.
| >
| > Paul
| >
| > "noirs" <***@discussions.microsoft.com> wrote in message
| > news:3248369B-2471-493B-8E38-***@microsoft.com...
| > > hi Paul, we've successfully uploaded ~1gb files.. we are using
currently a
| > > free component from Dundas to do uploading.
| > >
| > > Further investigation i've found:
| > > - sample code to write an IIS isap extension that can stream uploads
to
| > disk
| > > - I think the dundas controls streams the upload to disk
| > > - I think its only ASP perhaps that keeps it all in memory.
| > >
| > > I am going to build an upload resume feature... basically the server
will
| > > issue the client an ID. if the client gets kicked off, the client
will
| > ask
| > > where to resume from (by ID) and the server will append.
| > >
| > > "Paul Baker [MVP, Windows - SDK]" wrote:
| > >
| > > > Alex,
| > > >
| > > > This error code is ERROR_INTERNET_CONNECTION_RESET. This means that
the
| > web
| > > > server reset your connection, ie. kicked you off.
| > > >
| > > > HTTP is not intended for large file uploads. FTP is more suitable
for
| > this.
| > > >
| > > > Some HTTP (web) servers do not support large file uploads. For
example,
| > IIS
| > > > has an upper limit of 2GB in some versions and 1GB in others. I
think it
| > can
| > > > be changed using a registry entry, but there is no point. IIS loads
the
| > > > entire request into memory before passing it off to the component
that
| > will
| > > > handle the upload. This is presumably so that all the properties of
the
| > > > request are available, eg. the Request.TotalBytes property in ASP.
| > Because
| > > > it is in memory, IIS tends to be incredibly slow when uploading a
file
| > that
| > > > is larger than a few hundred MBs.
| > > >
| > > > Paul
| > > >
| > > > "noirs" <***@discussions.microsoft.com> wrote in message
| > > > news:807A87B7-8DA8-481C-A41C-***@microsoft.com...
| > > > > I'm sending large files up to our webserver using HTTP file
upload,
| > using
| > > > > InternetWriteFile (and HttpSendRequestEx).
| > > > >
| > > > > Often users get this error.. I get it repeatedly on my win95 test
box
| > in
| > > > > VirtualPC2004, i can't even complete a 1mb upload over our local
| > 100mbps
| > > > lan.
| > > > >
| > > > > Many users have successfully uploaded 600mb files.
| > > > >
| > > > > Is there some way I can build in resume functionality?
theoriticaly if
| > I
| > > > get
| > > > > this error can I just write the bytes again?
| > > > >
| > > > > why am I getting this error so consistently on win95?
| > > > >
| > > > > thx
| > > > >
| > > > > - Alex
| > > > >
| > > > > --
| > > > > Alex Black
| > > >
| > > >
| > > >
| >
| >
| >
|
Paul Baker [MVP, Windows - SDK]
2004-08-31 18:54:51 UTC
Permalink
I was thinking of the POST verb. I'm sorry I didn't realise that PUT is
different.

Paul
- Thanks for the info Brian that IIS does not buffer the full file in
memory
- BITs looks useful, but doesn't look feasible (requires the client to
have
bits, and appears not to run on win9x)
Post by Brian Combs
Hello
If you need to upload large files with the ability to resume the upload if
it fails you should look at BITS SDK. IIS does not buffer the full
incomming file. However the end application can if it is a POST. In many
cases a PUT will work just fine for uploading a large file.
For more information on BITS SDK
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/b
Post by Brian Combs
its_start_page.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/w
Post by Brian Combs
hat_s_new.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/i
Post by Brian Combs
is_requirements_for_bits_uploads.asp
Thanks
Brian [MSFT]
Microsoft Developer Support
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------
| Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| thread-index: AcSPcXoVQNWtSF82SDytPW2+AvFBJQ==
| X-WBNR-Posting-Host: 209.183.130.140
| Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| Date: Tue, 31 Aug 2004 08:45:01 -0700
| Lines: 103
| MIME-Version: 1.0
| Content-Type: text/plain;
| charset="Utf-8"
| Content-Transfer-Encoding: 7bit
| X-Newsreader: Microsoft CDO for Windows 2000
| Content-Class: urn:content-classes:message
| Importance: normal
| Priority: normal
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| Newsgroups: microsoft.public.inetsdk.programming.wininet
| NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| Xref: cpmsftngxa10.phx.gbl
microsoft.public.inetsdk.programming.wininet:11633
| X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
|
| Yes, i'm using IIS (5 I believe, on win2k).
|
| I have a hard time believing that IIS stores the whole file in ram, I
will
| try it out :) my machine only has 512mb ram, so uploading a 600mb file
would
| use up all its physical ram and then some.
|
| This company has a version of their upload component which they claim
| supports resume (and I assume streaming), but its v. expensive
|
| http://fileup.softartisans.com/fileup-147.aspx
|
|
| > Is the web server you are uploading to IIS? If you can be clear on
that, it
| > would help, as we can limit our discussion to that. What version of IIS
is
| > it?
| >
| > I'm fairly sure the request is held entirely in memory no matter what is
| > handling it. I think I tested it for ISAPI extensions too. That's why
those
| > third party tools to handle uploads don't help much.
| >
| > If I were you, I would reconsider your use of HTTP for uploading large
| > files. I have been down the same path as you, I know it's frustrating,
but
| > it's just not designed for that and it will never be reliable.
| >
| > Paul
| >
| > > hi Paul, we've successfully uploaded ~1gb files.. we are using
currently a
| > > free component from Dundas to do uploading.
| > >
| > > - sample code to write an IIS isap extension that can stream uploads
to
| > disk
| > > - I think the dundas controls streams the upload to disk
| > > - I think its only ASP perhaps that keeps it all in memory.
| > >
| > > I am going to build an upload resume feature... basically the server
will
| > > issue the client an ID. if the client gets kicked off, the client
will
| > ask
| > > where to resume from (by ID) and the server will append.
| > >
| > >
| > > > Alex,
| > > >
| > > > This error code is ERROR_INTERNET_CONNECTION_RESET. This means that
the
| > web
| > > > server reset your connection, ie. kicked you off.
| > > >
| > > > HTTP is not intended for large file uploads. FTP is more suitable
for
| > this.
| > > >
| > > > Some HTTP (web) servers do not support large file uploads. For
example,
| > IIS
| > > > has an upper limit of 2GB in some versions and 1GB in others. I
think it
| > can
| > > > be changed using a registry entry, but there is no point. IIS loads
the
| > > > entire request into memory before passing it off to the component
that
| > will
| > > > handle the upload. This is presumably so that all the properties of
the
| > > > request are available, eg. the Request.TotalBytes property in ASP.
| > Because
| > > > it is in memory, IIS tends to be incredibly slow when uploading a
file
| > that
| > > > is larger than a few hundred MBs.
| > > >
| > > > Paul
| > > >
| > > > > I'm sending large files up to our webserver using HTTP file
upload,
| > using
| > > > > InternetWriteFile (and HttpSendRequestEx).
| > > > >
| > > > > Often users get this error.. I get it repeatedly on my win95 test
box
| > in
| > > > > VirtualPC2004, i can't even complete a 1mb upload over our local
| > 100mbps
| > > > lan.
| > > > >
| > > > > Many users have successfully uploaded 600mb files.
| > > > >
| > > > > Is there some way I can build in resume functionality?
theoriticaly if
| > I
| > > > get
| > > > > this error can I just write the bytes again?
| > > > >
| > > > > why am I getting this error so consistently on win95?
| > > > >
| > > > > thx
| > > > >
| > > > > - Alex
| > > > >
| > > > > --
| > > > > Alex Black
| > > >
| > > >
| > > >
| >
| >
| >
|
Paul Baker [MVP, Windows - SDK]
2004-08-31 21:18:47 UTC
Permalink
Brian said "IIS does not buffer the full incomming file. However the end
application can if it is a POST.".

I am not sure exactly what this means. I *think* I have seen an it load the
entire request into memory before passing control to ASP or an ISAPI
extension (ie. there's nothing the ASP or ISAPI extension itself is doing to
cause it, and nothing they can do about it).

Paul
If i post a huge file to IIS, does it keep it all in memory?
Post by Paul Baker [MVP, Windows - SDK]
I was thinking of the POST verb. I'm sorry I didn't realise that PUT is
different.
Paul
- Thanks for the info Brian that IIS does not buffer the full file in
memory
- BITs looks useful, but doesn't look feasible (requires the client to
have
bits, and appears not to run on win9x)
Post by Brian Combs
Hello
If you need to upload large files with the ability to resume the
upload
Post by Paul Baker [MVP, Windows - SDK]
if
Post by Brian Combs
it fails you should look at BITS SDK. IIS does not buffer the full
incomming file. However the end application can if it is a POST. In many
cases a PUT will work just fine for uploading a large file.
For more information on BITS SDK
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/b
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
its_start_page.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/w
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
hat_s_new.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/i
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
is_requirements_for_bits_uploads.asp
Thanks
Brian [MSFT]
Microsoft Developer Support
This posting is provided "AS IS" with no warranties, and confers no
rights.
Post by Brian Combs
--------------------
| Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| thread-index: AcSPcXoVQNWtSF82SDytPW2+AvFBJQ==
| X-WBNR-Posting-Host: 209.183.130.140
| Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| Date: Tue, 31 Aug 2004 08:45:01 -0700
| Lines: 103
| MIME-Version: 1.0
| Content-Type: text/plain;
| charset="Utf-8"
| Content-Transfer-Encoding: 7bit
| X-Newsreader: Microsoft CDO for Windows 2000
| Content-Class: urn:content-classes:message
| Importance: normal
| Priority: normal
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| Newsgroups: microsoft.public.inetsdk.programming.wininet
| NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| Xref: cpmsftngxa10.phx.gbl
microsoft.public.inetsdk.programming.wininet:11633
| X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
|
| Yes, i'm using IIS (5 I believe, on win2k).
|
| I have a hard time believing that IIS stores the whole file in ram, I
will
| try it out :) my machine only has 512mb ram, so uploading a 600mb
file
Post by Brian Combs
would
| use up all its physical ram and then some.
|
| This company has a version of their upload component which they claim
| supports resume (and I assume streaming), but its v. expensive
|
| http://fileup.softartisans.com/fileup-147.aspx
|
|
| > Is the web server you are uploading to IIS? If you can be clear on
that, it
| > would help, as we can limit our discussion to that. What version
of
Post by Paul Baker [MVP, Windows - SDK]
IIS
Post by Brian Combs
is
| > it?
| >
| > I'm fairly sure the request is held entirely in memory no matter
what is
Post by Brian Combs
| > handling it. I think I tested it for ISAPI extensions too.
That's
Post by Paul Baker [MVP, Windows - SDK]
why
Post by Brian Combs
those
| > third party tools to handle uploads don't help much.
| >
| > If I were you, I would reconsider your use of HTTP for uploading
large
Post by Brian Combs
| > files. I have been down the same path as you, I know it's
frustrating,
Post by Brian Combs
but
| > it's just not designed for that and it will never be reliable.
| >
| > Paul
| >
| > > hi Paul, we've successfully uploaded ~1gb files.. we are using
currently a
| > > free component from Dundas to do uploading.
| > >
| > > - sample code to write an IIS isap extension that can stream
uploads
Post by Brian Combs
to
| > disk
| > > - I think the dundas controls streams the upload to disk
| > > - I think its only ASP perhaps that keeps it all in memory.
| > >
| > > I am going to build an upload resume feature... basically the
server
Post by Brian Combs
will
| > > issue the client an ID. if the client gets kicked off, the client
will
| > ask
| > > where to resume from (by ID) and the server will append.
| > >
| > >
| > > > Alex,
| > > >
| > > > This error code is ERROR_INTERNET_CONNECTION_RESET. This
means
Post by Paul Baker [MVP, Windows - SDK]
that
Post by Brian Combs
the
| > web
| > > > server reset your connection, ie. kicked you off.
| > > >
| > > > HTTP is not intended for large file uploads. FTP is more
suitable
Post by Brian Combs
for
| > this.
| > > >
| > > > Some HTTP (web) servers do not support large file uploads. For
example,
| > IIS
| > > > has an upper limit of 2GB in some versions and 1GB in others. I
think it
| > can
| > > > be changed using a registry entry, but there is no point.
IIS
Post by Paul Baker [MVP, Windows - SDK]
loads
Post by Brian Combs
the
| > > > entire request into memory before passing it off to the
component
Post by Brian Combs
that
| > will
| > > > handle the upload. This is presumably so that all the
properties
Post by Paul Baker [MVP, Windows - SDK]
of
Post by Brian Combs
the
| > > > request are available, eg. the Request.TotalBytes property
in
Post by Paul Baker [MVP, Windows - SDK]
ASP.
Post by Brian Combs
| > Because
| > > > it is in memory, IIS tends to be incredibly slow when
uploading
Post by Paul Baker [MVP, Windows - SDK]
a
Post by Brian Combs
file
| > that
| > > > is larger than a few hundred MBs.
| > > >
| > > > Paul
| > > >
| > > > > I'm sending large files up to our webserver using HTTP file
upload,
| > using
| > > > > InternetWriteFile (and HttpSendRequestEx).
| > > > >
| > > > > Often users get this error.. I get it repeatedly on my
win95
Post by Paul Baker [MVP, Windows - SDK]
test
Post by Brian Combs
box
| > in
| > > > > VirtualPC2004, i can't even complete a 1mb upload over our
local
Post by Brian Combs
| > 100mbps
| > > > lan.
| > > > >
| > > > > Many users have successfully uploaded 600mb files.
| > > > >
| > > > > Is there some way I can build in resume functionality?
theoriticaly if
| > I
| > > > get
| > > > > this error can I just write the bytes again?
| > > > >
| > > > > why am I getting this error so consistently on win95?
| > > > >
| > > > > thx
| > > > >
| > > > > - Alex
| > > > >
| > > > > --
| > > > > Alex Black
| > > >
| > > >
| > > >
| >
| >
| >
|
Paul Baker [MVP, Windows - SDK]
2004-08-31 21:37:23 UTC
Permalink
Yes, it does kind of sound like that. But even if that is so, it is still
unclear. For example, in the case of ASP, what does the application mean?

Does it mean the .asp file itself? If so, then fine. Or does it mean
asp.dll? If so, there is nothing the .asp file itself can do about it.

Yes, we'll see what Brian says :)

Paul
- IIS does not keep the entire post in memory
- an application can keep it in memory if it chose to.
hopefully Brian can clarify for us!
Post by Paul Baker [MVP, Windows - SDK]
Brian said "IIS does not buffer the full incomming file. However the end
application can if it is a POST.".
I am not sure exactly what this means. I *think* I have seen an it load the
entire request into memory before passing control to ASP or an ISAPI
extension (ie. there's nothing the ASP or ISAPI extension itself is doing to
cause it, and nothing they can do about it).
Paul
If i post a huge file to IIS, does it keep it all in memory?
Post by Paul Baker [MVP, Windows - SDK]
I was thinking of the POST verb. I'm sorry I didn't realise that PUT is
different.
Paul
- Thanks for the info Brian that IIS does not buffer the full file in
memory
- BITs looks useful, but doesn't look feasible (requires the client to
have
bits, and appears not to run on win9x)
Post by Brian Combs
Hello
If you need to upload large files with the ability to resume the
upload
Post by Paul Baker [MVP, Windows - SDK]
if
Post by Brian Combs
it fails you should look at BITS SDK. IIS does not buffer the full
incomming file. However the end application can if it is a POST.
In
Post by Paul Baker [MVP, Windows - SDK]
many
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
cases a PUT will work just fine for uploading a large file.
For more information on BITS SDK
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/b
Post by Paul Baker [MVP, Windows - SDK]
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
its_start_page.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/w
Post by Paul Baker [MVP, Windows - SDK]
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
hat_s_new.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/i
Post by Paul Baker [MVP, Windows - SDK]
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
is_requirements_for_bits_uploads.asp
Thanks
Brian [MSFT]
Microsoft Developer Support
This posting is provided "AS IS" with no warranties, and confers no
rights.
Post by Brian Combs
--------------------
| Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| thread-index: AcSPcXoVQNWtSF82SDytPW2+AvFBJQ==
| X-WBNR-Posting-Host: 209.183.130.140
| Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| Date: Tue, 31 Aug 2004 08:45:01 -0700
| Lines: 103
| MIME-Version: 1.0
| Content-Type: text/plain;
| charset="Utf-8"
| Content-Transfer-Encoding: 7bit
| X-Newsreader: Microsoft CDO for Windows 2000
| Content-Class: urn:content-classes:message
| Importance: normal
| Priority: normal
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| Newsgroups: microsoft.public.inetsdk.programming.wininet
| NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| Xref: cpmsftngxa10.phx.gbl
microsoft.public.inetsdk.programming.wininet:11633
| X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
|
| Yes, i'm using IIS (5 I believe, on win2k).
|
| I have a hard time believing that IIS stores the whole file in
ram, I
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
will
| try it out :) my machine only has 512mb ram, so uploading a 600mb
file
Post by Brian Combs
would
| use up all its physical ram and then some.
|
| This company has a version of their upload component which
they
Post by Paul Baker [MVP, Windows - SDK]
claim
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
| supports resume (and I assume streaming), but its v. expensive
|
| http://fileup.softartisans.com/fileup-147.aspx
|
|
| > Is the web server you are uploading to IIS? If you can be
clear
Post by Paul Baker [MVP, Windows - SDK]
on
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
that, it
| > would help, as we can limit our discussion to that. What version
of
Post by Paul Baker [MVP, Windows - SDK]
IIS
Post by Brian Combs
is
| > it?
| >
| > I'm fairly sure the request is held entirely in memory no matter
what is
Post by Brian Combs
| > handling it. I think I tested it for ISAPI extensions too.
That's
Post by Paul Baker [MVP, Windows - SDK]
why
Post by Brian Combs
those
| > third party tools to handle uploads don't help much.
| >
| > If I were you, I would reconsider your use of HTTP for uploading
large
Post by Brian Combs
| > files. I have been down the same path as you, I know it's
frustrating,
Post by Brian Combs
but
| > it's just not designed for that and it will never be reliable.
| >
| > Paul
| >
| > > hi Paul, we've successfully uploaded ~1gb files.. we are using
currently a
| > > free component from Dundas to do uploading.
| > >
| > > - sample code to write an IIS isap extension that can stream
uploads
Post by Brian Combs
to
| > disk
| > > - I think the dundas controls streams the upload to disk
| > > - I think its only ASP perhaps that keeps it all in memory.
| > >
| > > I am going to build an upload resume feature... basically the
server
Post by Brian Combs
will
| > > issue the client an ID. if the client gets kicked off,
the
Post by Paul Baker [MVP, Windows - SDK]
client
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
will
| > ask
| > > where to resume from (by ID) and the server will append.
| > >
| > >
| > > > Alex,
| > > >
| > > > This error code is ERROR_INTERNET_CONNECTION_RESET. This
means
Post by Paul Baker [MVP, Windows - SDK]
that
Post by Brian Combs
the
| > web
| > > > server reset your connection, ie. kicked you off.
| > > >
| > > > HTTP is not intended for large file uploads. FTP is more
suitable
Post by Brian Combs
for
| > this.
| > > >
| > > > Some HTTP (web) servers do not support large file
uploads.
Post by Paul Baker [MVP, Windows - SDK]
For
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
example,
| > IIS
| > > > has an upper limit of 2GB in some versions and 1GB in
others. I
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
think it
| > can
| > > > be changed using a registry entry, but there is no point.
IIS
Post by Paul Baker [MVP, Windows - SDK]
loads
Post by Brian Combs
the
| > > > entire request into memory before passing it off to the
component
Post by Brian Combs
that
| > will
| > > > handle the upload. This is presumably so that all the
properties
Post by Paul Baker [MVP, Windows - SDK]
of
Post by Brian Combs
the
| > > > request are available, eg. the Request.TotalBytes property
in
Post by Paul Baker [MVP, Windows - SDK]
ASP.
Post by Brian Combs
| > Because
| > > > it is in memory, IIS tends to be incredibly slow when
uploading
Post by Paul Baker [MVP, Windows - SDK]
a
Post by Brian Combs
file
| > that
| > > > is larger than a few hundred MBs.
| > > >
| > > > Paul
| > > >
| > > > > I'm sending large files up to our webserver using HTTP
file
Post by Paul Baker [MVP, Windows - SDK]
Post by Brian Combs
upload,
| > using
| > > > > InternetWriteFile (and HttpSendRequestEx).
| > > > >
| > > > > Often users get this error.. I get it repeatedly on my
win95
Post by Paul Baker [MVP, Windows - SDK]
test
Post by Brian Combs
box
| > in
| > > > > VirtualPC2004, i can't even complete a 1mb upload over our
local
Post by Brian Combs
| > 100mbps
| > > > lan.
| > > > >
| > > > > Many users have successfully uploaded 600mb files.
| > > > >
| > > > > Is there some way I can build in resume functionality?
theoriticaly if
| > I
| > > > get
| > > > > this error can I just write the bytes again?
| > > > >
| > > > > why am I getting this error so consistently on win95?
| > > > >
| > > > > thx
| > > > >
| > > > > - Alex
| > > > >
| > > > > --
| > > > > Alex Black
| > > >
| > > >
| > > >
| >
| >
| >
|
Brian Combs
2004-09-03 15:07:06 UTC
Permalink
Hello
I will try and clarify. There are only two types of applications you can
write on IIS.
1. ISAPI
2. CGI
ASP and ASP.Net are both ISAPI. Perl, Cold Fusion, etc. maybe ISAPI or CGI
depending on what runtime you are using. Remember IIS does not call your
.ASP page directly, it calls ASP.DLL and this is in charge of loading and
parsing your .asp file. The same is true for anything else that has an
application script map.

IIS takes the incoming data and just passes it on to the application, that
is receiving the request, as the data is received in IIS. IIS will do a
small amount of buffering for data that is sent in a post, but it will not
buffer the full amount of data. However the underlying ISAPI or CGI can
chose to buffer the full amount of data for use later on. For example
ASP.Net does this, see the more info section of the following article.
323246 How To Upload a File to a Web Server in ASP.NET by Using Visual C#
.NET
http://support.microsoft.com/?id=323246

If you are unable to use BITS you my want to think about using PUT to
upload files.
184352 How To Upload Files to the Internet Information Server
http://support.microsoft.com/?id=184352

You may also want to consider FTP.

Hope this helps.

Thanks
Brian [MSFT]
Microsoft Developer Support
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------
| Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| thread-index: AcSPoiLVaCpizciPSHa5KPYYMbSFYw==
| X-WBNR-Posting-Host: 209.183.130.140
| From: "=?Utf-8?B?bm9pcnM=?=" <***@discussions.microsoft.com>
| References: <807A87B7-8DA8-481C-A41C-***@microsoft.com>
<***@tk2msftngp13.phx.gbl>
<3248369B-2471-493B-8E38-***@microsoft.com>
<#lNAI$***@tk2msftngp13.phx.gbl>
<A3318F9D-C766-438E-B655-***@microsoft.com>
<***@cpmsftngxa10.phx.gbl>
<FB025E6A-8B5B-4D60-BC85-***@microsoft.com>
<***@TK2MSFTNGP15.phx.gbl>
<B8C32228-9653-448D-876C-***@microsoft.com>
<***@TK2MSFTNGP12.phx.gbl>
| Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| Date: Tue, 31 Aug 2004 14:33:20 -0700
| Lines: 264
| Message-ID: <7161AFD8-A533-421B-9B4B-***@microsoft.com>
| MIME-Version: 1.0
| Content-Type: text/plain;
| charset="Utf-8"
| Content-Transfer-Encoding: 7bit
| X-Newsreader: Microsoft CDO for Windows 2000
| Content-Class: urn:content-classes:message
| Importance: normal
| Priority: normal
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| Newsgroups: microsoft.public.inetsdk.programming.wininet
| NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| Xref: cpmsftngxa10.phx.gbl
microsoft.public.inetsdk.programming.wininet:11641
| X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
|
| My interpretation of Brian's comment was that:
| - IIS does not keep the entire post in memory
| - an application can keep it in memory if it chose to.
|
| hopefully Brian can clarify for us!
|
| "Paul Baker [MVP, Windows - SDK]" wrote:
|
| > Brian said "IIS does not buffer the full incomming file. However the end
| > application can if it is a POST.".
| >
| > I am not sure exactly what this means. I *think* I have seen an it load
the
| > entire request into memory before passing control to ASP or an ISAPI
| > extension (ie. there's nothing the ASP or ISAPI extension itself is
doing to
| > cause it, and nothing they can do about it).
| >
| > Paul
| >
| > "noirs" <***@discussions.microsoft.com> wrote in message
| > news:B8C32228-9653-448D-876C-***@microsoft.com...
| > > Brian, can you confirm:
| > >
| > > If i post a huge file to IIS, does it keep it all in memory?
| > >
| > > "Paul Baker [MVP, Windows - SDK]" wrote:
| > >
| > > > I was thinking of the POST verb. I'm sorry I didn't realise that
PUT is
| > > > different.
| > > >
| > > > Paul
| > > >
| > > > "noirs" <***@discussions.microsoft.com> wrote in message
| > > > news:FB025E6A-8B5B-4D60-BC85-***@microsoft.com...
| > > > > - Thanks for the info Brian that IIS does not buffer the full
file in
| > > > memory
| > > > > - BITs looks useful, but doesn't look feasible (requires the
client to
| > > > have
| > > > > bits, and appears not to run on win9x)
| > > > >
| > > > > "Brian Combs" wrote:
| > > > >
| > > > > > Hello
| > > > > > If you need to upload large files with the ability to resume the
| > upload
| > > > if
| > > > > > it fails you should look at BITS SDK. IIS does not buffer the
full
| > > > > > incomming file. However the end application can if it is a
POST. In
| > many
| > > > > > cases a PUT will work just fine for uploading a large file.
| > > > > >
| > > > > > For more information on BITS SDK
| > > > > >
| > > >
| >
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/b
| > > > > > its_start_page.asp
| > > > > >
| > > >
| >
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/w
| > > > > > hat_s_new.asp
| > > > > >
| > > >
| >
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/i
| > > > > > is_requirements_for_bits_uploads.asp
| > > > > >
| > > > > >
| > > > > > Thanks
| > > > > > Brian [MSFT]
| > > > > > Microsoft Developer Support
| > > > > > This posting is provided "AS IS" with no warranties, and
confers no
| > > > rights.
| > > > > > --------------------
| > > > > > | Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| > > > > > | thread-index: AcSPcXoVQNWtSF82SDytPW2+AvFBJQ==
| > > > > > | X-WBNR-Posting-Host: 209.183.130.140
| > > > > > | From: "=?Utf-8?B?bm9pcnM=?=" <***@discussions.microsoft.com>
| > > > > > | References:
<807A87B7-8DA8-481C-A41C-***@microsoft.com>
| > > > > > <***@tk2msftngp13.phx.gbl>
| > > > > > <3248369B-2471-493B-8E38-***@microsoft.com>
| > > > > > <#lNAI$***@tk2msftngp13.phx.gbl>
| > > > > > | Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| > > > > > | Date: Tue, 31 Aug 2004 08:45:01 -0700
| > > > > > | Lines: 103
| > > > > > | Message-ID:
<A3318F9D-C766-438E-B655-***@microsoft.com>
| > > > > > | MIME-Version: 1.0
| > > > > > | Content-Type: text/plain;
| > > > > > | charset="Utf-8"
| > > > > > | Content-Transfer-Encoding: 7bit
| > > > > > | X-Newsreader: Microsoft CDO for Windows 2000
| > > > > > | Content-Class: urn:content-classes:message
| > > > > > | Importance: normal
| > > > > > | Priority: normal
| > > > > > | X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| > > > > > | Newsgroups: microsoft.public.inetsdk.programming.wininet
| > > > > > | NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| > > > > > | Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| > > > > > | Xref: cpmsftngxa10.phx.gbl
| > > > > > microsoft.public.inetsdk.programming.wininet:11633
| > > > > > | X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
| > > > > > |
| > > > > > | Yes, i'm using IIS (5 I believe, on win2k).
| > > > > > |
| > > > > > | I have a hard time believing that IIS stores the whole file in
| > ram, I
| > > > > > will
| > > > > > | try it out :) my machine only has 512mb ram, so uploading a
600mb
| > > > file
| > > > > > would
| > > > > > | use up all its physical ram and then some.
| > > > > > |
| > > > > > | This company has a version of their upload component which
they
| > claim
| > > > > > | supports resume (and I assume streaming), but its v. expensive
| > > > > > |
| > > > > > | http://fileup.softartisans.com/fileup-147.aspx
| > > > > > |
| > > > > > | "Paul Baker [MVP, Windows - SDK]" wrote:
| > > > > > |
| > > > > > | > Is the web server you are uploading to IIS? If you can be
clear
| > on
| > > > > > that, it
| > > > > > | > would help, as we can limit our discussion to that. What
version
| > of
| > > > IIS
| > > > > > is
| > > > > > | > it?
| > > > > > | >
| > > > > > | > I'm fairly sure the request is held entirely in memory no
matter
| > > > what is
| > > > > > | > handling it. I think I tested it for ISAPI extensions too.
| > That's
| > > > why
| > > > > > those
| > > > > > | > third party tools to handle uploads don't help much.
| > > > > > | >
| > > > > > | > If I were you, I would reconsider your use of HTTP for
uploading
| > > > large
| > > > > > | > files. I have been down the same path as you, I know it's
| > > > frustrating,
| > > > > > but
| > > > > > | > it's just not designed for that and it will never be
reliable.
| > > > > > | >
| > > > > > | > Paul
| > > > > > | >
| > > > > > | > "noirs" <***@discussions.microsoft.com> wrote in message
| > > > > > | > news:3248369B-2471-493B-8E38-***@microsoft.com...
| > > > > > | > > hi Paul, we've successfully uploaded ~1gb files.. we are
using
| > > > > > currently a
| > > > > > | > > free component from Dundas to do uploading.
| > > > > > | > >
| > > > > > | > > Further investigation i've found:
| > > > > > | > > - sample code to write an IIS isap extension that can
stream
| > > > uploads
| > > > > > to
| > > > > > | > disk
| > > > > > | > > - I think the dundas controls streams the upload to disk
| > > > > > | > > - I think its only ASP perhaps that keeps it all in
memory.
| > > > > > | > >
| > > > > > | > > I am going to build an upload resume feature... basically
the
| > > > server
| > > > > > will
| > > > > > | > > issue the client an ID. if the client gets kicked off,
the
| > client
| > > > > > will
| > > > > > | > ask
| > > > > > | > > where to resume from (by ID) and the server will append.
| > > > > > | > >
| > > > > > | > > "Paul Baker [MVP, Windows - SDK]" wrote:
| > > > > > | > >
| > > > > > | > > > Alex,
| > > > > > | > > >
| > > > > > | > > > This error code is ERROR_INTERNET_CONNECTION_RESET. This
| > means
| > > > that
| > > > > > the
| > > > > > | > web
| > > > > > | > > > server reset your connection, ie. kicked you off.
| > > > > > | > > >
| > > > > > | > > > HTTP is not intended for large file uploads. FTP is more
| > > > suitable
| > > > > > for
| > > > > > | > this.
| > > > > > | > > >
| > > > > > | > > > Some HTTP (web) servers do not support large file
uploads.
| > For
| > > > > > example,
| > > > > > | > IIS
| > > > > > | > > > has an upper limit of 2GB in some versions and 1GB in
| > others. I
| > > > > > think it
| > > > > > | > can
| > > > > > | > > > be changed using a registry entry, but there is no
point.
| > IIS
| > > > loads
| > > > > > the
| > > > > > | > > > entire request into memory before passing it off to the
| > > > component
| > > > > > that
| > > > > > | > will
| > > > > > | > > > handle the upload. This is presumably so that all the
| > properties
| > > > of
| > > > > > the
| > > > > > | > > > request are available, eg. the Request.TotalBytes
property
| > in
| > > > ASP.
| > > > > > | > Because
| > > > > > | > > > it is in memory, IIS tends to be incredibly slow when
| > uploading
| > > > a
| > > > > > file
| > > > > > | > that
| > > > > > | > > > is larger than a few hundred MBs.
| > > > > > | > > >
| > > > > > | > > > Paul
| > > > > > | > > >
| > > > > > | > > > "noirs" <***@discussions.microsoft.com> wrote in
message
| > > > > > | > > > news:807A87B7-8DA8-481C-A41C-***@microsoft.com...
| > > > > > | > > > > I'm sending large files up to our webserver using HTTP
| > file
| > > > > > upload,
| > > > > > | > using
| > > > > > | > > > > InternetWriteFile (and HttpSendRequestEx).
| > > > > > | > > > >
| > > > > > | > > > > Often users get this error.. I get it repeatedly on my
| > win95
| > > > test
| > > > > > box
| > > > > > | > in
| > > > > > | > > > > VirtualPC2004, i can't even complete a 1mb upload
over our
| > > > local
| > > > > > | > 100mbps
| > > > > > | > > > lan.
| > > > > > | > > > >
| > > > > > | > > > > Many users have successfully uploaded 600mb files.
| > > > > > | > > > >
| > > > > > | > > > > Is there some way I can build in resume functionality?
| > > > > > theoriticaly if
| > > > > > | > I
| > > > > > | > > > get
| > > > > > | > > > > this error can I just write the bytes again?
| > > > > > | > > > >
| > > > > > | > > > > why am I getting this error so consistently on win95?
| > > > > > | > > > >
| > > > > > | > > > > thx
| > > > > > | > > > >
| > > > > > | > > > > - Alex
| > > > > > | > > > >
| > > > > > | > > > > --
| > > > > > | > > > > Alex Black
| > > > > > | > > >
| > > > > > | > > >
| > > > > > | > > >
| > > > > > | >
| > > > > > | >
| > > > > > | >
| > > > > > |
| > > > > >
| > > > > >
| > > >
| > > >
| > > >
| >
| >
| >
|
Paul Baker [MVP, Windows - SDK]
2004-10-25 18:06:03 UTC
Permalink
Brian,

I repeated my earlier tests.

I found that I *could* indeed upload large files to the local host using a
POST request and code similar to this sample.

SAMPLE: Using HttpSendRequestEx for Large POST Requests
http://support.microsoft.com/default.aspx?scid=kb;en-us;177188

On the server side, I have IIS 5.1 on Windows XP Professional. I use an ASP
which examines Request.TotalBytes and calls Request.BinaryRead in a loop. It
did *not* cache the request in memory. It began to write to the file
immediately and only used a few KB of memory.

If the upload took longer than 90 seconds, I received an
ERROR_INTERNET_CONNECTION_RESET (12031), just as Alex did. But this too is
not a problem. It can be dealt with by increasing the value of
Server.ScriptTimeout.

Server.ScriptTimeout
http://msdn.microsoft.com/library/en-us/iissdk/iis/ref_vbom_seropsct.asp

So, I see no problem. But I did my original tests on Windows 2000. Is it
possible that the behaviour of IIS with caching POST requests changed
between IIS 5.0 and IIS 5.1?

Paul
Post by Brian Combs
Hello
I will try and clarify. There are only two types of applications you can
write on IIS.
1. ISAPI
2. CGI
ASP and ASP.Net are both ISAPI. Perl, Cold Fusion, etc. maybe ISAPI or CGI
depending on what runtime you are using. Remember IIS does not call your
ASP page directly, it calls ASP.DLL and this is in charge of loading and
parsing your .asp file. The same is true for anything else that has an
application script map.
IIS takes the incoming data and just passes it on to the application, that
is receiving the request, as the data is received in IIS. IIS will do a
small amount of buffering for data that is sent in a post, but it will not
buffer the full amount of data. However the underlying ISAPI or CGI can
chose to buffer the full amount of data for use later on. For example
ASP.Net does this, see the more info section of the following article.
323246 How To Upload a File to a Web Server in ASP.NET by Using Visual C#
NET
http://support.microsoft.com/?id=323246
If you are unable to use BITS you my want to think about using PUT to
upload files.
184352 How To Upload Files to the Internet Information Server
http://support.microsoft.com/?id=184352
You may also want to consider FTP.
Hope this helps.
Thanks
Brian [MSFT]
Microsoft Developer Support
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------
| Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| thread-index: AcSPoiLVaCpizciPSHa5KPYYMbSFYw==
| X-WBNR-Posting-Host: 209.183.130.140
| Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| Date: Tue, 31 Aug 2004 14:33:20 -0700
| Lines: 264
| MIME-Version: 1.0
| Content-Type: text/plain;
| charset="Utf-8"
| Content-Transfer-Encoding: 7bit
| X-Newsreader: Microsoft CDO for Windows 2000
| Content-Class: urn:content-classes:message
| Importance: normal
| Priority: normal
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| Newsgroups: microsoft.public.inetsdk.programming.wininet
| NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| Xref: cpmsftngxa10.phx.gbl
microsoft.public.inetsdk.programming.wininet:11641
| X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
|
| - IIS does not keep the entire post in memory
| - an application can keep it in memory if it chose to.
|
| hopefully Brian can clarify for us!
|
|
| > Brian said "IIS does not buffer the full incomming file. However the end
| > application can if it is a POST.".
| >
| > I am not sure exactly what this means. I *think* I have seen an it load
the
| > entire request into memory before passing control to ASP or an ISAPI
| > extension (ie. there's nothing the ASP or ISAPI extension itself is
doing to
| > cause it, and nothing they can do about it).
| >
| > Paul
| >
| > >
| > > If i post a huge file to IIS, does it keep it all in memory?
| > >
| > >
| > > > I was thinking of the POST verb. I'm sorry I didn't realise that
PUT is
| > > > different.
| > > >
| > > > Paul
| > > >
| > > > > - Thanks for the info Brian that IIS does not buffer the full
file in
| > > > memory
| > > > > - BITs looks useful, but doesn't look feasible (requires the
client to
| > > > have
| > > > > bits, and appears not to run on win9x)
| > > > >
| > > > >
| > > > > > Hello
| > > > > > If you need to upload large files with the ability to resume the
| > upload
| > > > if
| > > > > > it fails you should look at BITS SDK. IIS does not buffer the
full
| > > > > > incomming file. However the end application can if it is a
POST. In
| > many
| > > > > > cases a PUT will work just fine for uploading a large file.
| > > > > >
| > > > > > For more information on BITS SDK
| > > > > >
| > > >
| >
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/b
Post by Brian Combs
| > > > > > its_start_page.asp
| > > > > >
| > > >
| >
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/w
Post by Brian Combs
| > > > > > hat_s_new.asp
| > > > > >
| > > >
| >
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/i
Post by Brian Combs
| > > > > > is_requirements_for_bits_uploads.asp
| > > > > >
| > > > > >
| > > > > > Thanks
| > > > > > Brian [MSFT]
| > > > > > Microsoft Developer Support
| > > > > > This posting is provided "AS IS" with no warranties, and
confers no
| > > > rights.
| > > > > > --------------------
| > > > > > | Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| > > > > > | thread-index: AcSPcXoVQNWtSF82SDytPW2+AvFBJQ==
| > > > > > | X-WBNR-Posting-Host: 209.183.130.140
| > > > > > | From: "=?Utf-8?B?bm9pcnM=?="
| > > > > > | Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| > > > > > | Date: Tue, 31 Aug 2004 08:45:01 -0700
| > > > > > | Lines: 103
| > > > > > | MIME-Version: 1.0
| > > > > > | Content-Type: text/plain;
| > > > > > | charset="Utf-8"
| > > > > > | Content-Transfer-Encoding: 7bit
| > > > > > | X-Newsreader: Microsoft CDO for Windows 2000
| > > > > > | Content-Class: urn:content-classes:message
| > > > > > | Importance: normal
| > > > > > | Priority: normal
| > > > > > | X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| > > > > > | Newsgroups: microsoft.public.inetsdk.programming.wininet
| > > > > > | NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| > > > > > | Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| > > > > > | Xref: cpmsftngxa10.phx.gbl
| > > > > > microsoft.public.inetsdk.programming.wininet:11633
| > > > > > | X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
| > > > > > |
| > > > > > | Yes, i'm using IIS (5 I believe, on win2k).
| > > > > > |
| > > > > > | I have a hard time believing that IIS stores the whole file in
| > ram, I
| > > > > > will
| > > > > > | try it out :) my machine only has 512mb ram, so uploading a
600mb
| > > > file
| > > > > > would
| > > > > > | use up all its physical ram and then some.
| > > > > > |
| > > > > > | This company has a version of their upload component which
they
| > claim
| > > > > > | supports resume (and I assume streaming), but its v. expensive
| > > > > > |
| > > > > > | http://fileup.softartisans.com/fileup-147.aspx
| > > > > > |
| > > > > > |
| > > > > > | > Is the web server you are uploading to IIS? If you can be
clear
| > on
| > > > > > that, it
| > > > > > | > would help, as we can limit our discussion to that. What
version
| > of
| > > > IIS
| > > > > > is
| > > > > > | > it?
| > > > > > | >
| > > > > > | > I'm fairly sure the request is held entirely in memory no
matter
| > > > what is
| > > > > > | > handling it. I think I tested it for ISAPI extensions too.
| > That's
| > > > why
| > > > > > those
| > > > > > | > third party tools to handle uploads don't help much.
| > > > > > | >
| > > > > > | > If I were you, I would reconsider your use of HTTP for
uploading
| > > > large
| > > > > > | > files. I have been down the same path as you, I know it's
| > > > frustrating,
| > > > > > but
| > > > > > | > it's just not designed for that and it will never be
reliable.
| > > > > > | >
| > > > > > | > Paul
| > > > > > | >
| > > > > > | > > hi Paul, we've successfully uploaded ~1gb files.. we are
using
| > > > > > currently a
| > > > > > | > > free component from Dundas to do uploading.
| > > > > > | > >
| > > > > > | > > - sample code to write an IIS isap extension that can
stream
| > > > uploads
| > > > > > to
| > > > > > | > disk
| > > > > > | > > - I think the dundas controls streams the upload to disk
| > > > > > | > > - I think its only ASP perhaps that keeps it all in
memory.
| > > > > > | > >
| > > > > > | > > I am going to build an upload resume feature... basically
the
| > > > server
| > > > > > will
| > > > > > | > > issue the client an ID. if the client gets kicked off,
the
| > client
| > > > > > will
| > > > > > | > ask
| > > > > > | > > where to resume from (by ID) and the server will append.
| > > > > > | > >
| > > > > > | > >
| > > > > > | > > > Alex,
| > > > > > | > > >
| > > > > > | > > > This error code is ERROR_INTERNET_CONNECTION_RESET. This
| > means
| > > > that
| > > > > > the
| > > > > > | > web
| > > > > > | > > > server reset your connection, ie. kicked you off.
| > > > > > | > > >
| > > > > > | > > > HTTP is not intended for large file uploads. FTP is more
| > > > suitable
| > > > > > for
| > > > > > | > this.
| > > > > > | > > >
| > > > > > | > > > Some HTTP (web) servers do not support large file
uploads.
| > For
| > > > > > example,
| > > > > > | > IIS
| > > > > > | > > > has an upper limit of 2GB in some versions and 1GB in
| > others. I
| > > > > > think it
| > > > > > | > can
| > > > > > | > > > be changed using a registry entry, but there is no
point.
| > IIS
| > > > loads
| > > > > > the
| > > > > > | > > > entire request into memory before passing it off to the
| > > > component
| > > > > > that
| > > > > > | > will
| > > > > > | > > > handle the upload. This is presumably so that all the
| > properties
| > > > of
| > > > > > the
| > > > > > | > > > request are available, eg. the Request.TotalBytes
property
| > in
| > > > ASP.
| > > > > > | > Because
| > > > > > | > > > it is in memory, IIS tends to be incredibly slow when
| > uploading
| > > > a
| > > > > > file
| > > > > > | > that
| > > > > > | > > > is larger than a few hundred MBs.
| > > > > > | > > >
| > > > > > | > > > Paul
| > > > > > | > > >
message
| > > > > > | > > > > I'm sending large files up to our webserver using HTTP
| > file
| > > > > > upload,
| > > > > > | > using
| > > > > > | > > > > InternetWriteFile (and HttpSendRequestEx).
| > > > > > | > > > >
| > > > > > | > > > > Often users get this error.. I get it repeatedly on my
| > win95
| > > > test
| > > > > > box
| > > > > > | > in
| > > > > > | > > > > VirtualPC2004, i can't even complete a 1mb upload
over our
| > > > local
| > > > > > | > 100mbps
| > > > > > | > > > lan.
| > > > > > | > > > >
| > > > > > | > > > > Many users have successfully uploaded 600mb files.
| > > > > > | > > > >
| > > > > > | > > > > Is there some way I can build in resume
functionality?
Post by Brian Combs
| > > > > > theoriticaly if
| > > > > > | > I
| > > > > > | > > > get
| > > > > > | > > > > this error can I just write the bytes again?
| > > > > > | > > > >
| > > > > > | > > > > why am I getting this error so consistently on win95?
| > > > > > | > > > >
| > > > > > | > > > > thx
| > > > > > | > > > >
| > > > > > | > > > > - Alex
| > > > > > | > > > >
| > > > > > | > > > > --
| > > > > > | > > > > Alex Black
| > > > > > | > > >
| > > > > > | > > >
| > > > > > | > > >
| > > > > > | >
| > > > > > | >
| > > > > > | >
| > > > > > |
| > > > > >
| > > > > >
| > > >
| > > >
| > > >
| >
| >
| >
|
Brian Combs
2004-11-02 19:49:52 UTC
Permalink
Hello
I don't know of any changes to IIS 5.1 from 5.0 with how asp handles data.
IIS does not cache POST data. You should run same test that you ran on XP
on 2000. As I suggested try using PUT and not POST to see what happens.

Thanks
Brian [MSFT]
Microsoft Developer Support
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------
| From: "Paul Baker [MVP, Windows - SDK]" <***@online.rochester.rr.com>
| References: <807A87B7-8DA8-481C-A41C-***@microsoft.com>
<***@tk2msftngp13.phx.gbl>
<3248369B-2471-493B-8E38-***@microsoft.com>
<#lNAI$***@tk2msftngp13.phx.gbl>
<A3318F9D-C766-438E-B655-***@microsoft.com>
<***@cpmsftngxa10.phx.gbl>
<FB025E6A-8B5B-4D60-BC85-***@microsoft.com>
<***@TK2MSFTNGP15.phx.gbl>
<B8C32228-9653-448D-876C-***@microsoft.com>
<***@TK2MSFTNGP12.phx.gbl>
<7161AFD8-A533-421B-9B4B-***@microsoft.com>
<***@cpmsftngxa10.phx.gbl>
| Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| Date: Mon, 25 Oct 2004 14:06:03 -0400
| Lines: 417
| X-Priority: 3
| X-MSMail-Priority: Normal
| X-Newsreader: Microsoft Outlook Express 6.00.2800.1409
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
| Message-ID: <***@TK2MSFTNGP15.phx.gbl>
| Newsgroups: microsoft.public.inetsdk.programming.wininet
| NNTP-Posting-Host: 66.192.38.14
| Path:
cpmsftngxa10.phx.gbl!TK2MSFTFEED01.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP15
.phx.gbl
| Xref: cpmsftngxa10.phx.gbl
microsoft.public.inetsdk.programming.wininet:11786
| X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
|
| Brian,
|
| I repeated my earlier tests.
|
| I found that I *could* indeed upload large files to the local host using a
| POST request and code similar to this sample.
|
| SAMPLE: Using HttpSendRequestEx for Large POST Requests
| http://support.microsoft.com/default.aspx?scid=kb;en-us;177188
|
| On the server side, I have IIS 5.1 on Windows XP Professional. I use an
ASP
| which examines Request.TotalBytes and calls Request.BinaryRead in a loop.
It
| did *not* cache the request in memory. It began to write to the file
| immediately and only used a few KB of memory.
|
| If the upload took longer than 90 seconds, I received an
| ERROR_INTERNET_CONNECTION_RESET (12031), just as Alex did. But this too is
| not a problem. It can be dealt with by increasing the value of
| Server.ScriptTimeout.
|
| Server.ScriptTimeout
| http://msdn.microsoft.com/library/en-us/iissdk/iis/ref_vbom_seropsct.asp
|
| So, I see no problem. But I did my original tests on Windows 2000. Is it
| possible that the behaviour of IIS with caching POST requests changed
| between IIS 5.0 and IIS 5.1?
|
| Paul
|
| "Brian Combs" <***@online.microsoft.com> wrote in message
| news:***@cpmsftngxa10.phx.gbl...
| > Hello
| > I will try and clarify. There are only two types of applications you can
| > write on IIS.
| > 1. ISAPI
| > 2. CGI
| > ASP and ASP.Net are both ISAPI. Perl, Cold Fusion, etc. maybe ISAPI or
CGI
| > depending on what runtime you are using. Remember IIS does not call
your
| > ASP page directly, it calls ASP.DLL and this is in charge of loading and
| > parsing your .asp file. The same is true for anything else that has an
| > application script map.
| >
| > IIS takes the incoming data and just passes it on to the application,
that
| > is receiving the request, as the data is received in IIS. IIS will do a
| > small amount of buffering for data that is sent in a post, but it will
not
| > buffer the full amount of data. However the underlying ISAPI or CGI can
| > chose to buffer the full amount of data for use later on. For example
| > ASP.Net does this, see the more info section of the following article.
| > 323246 How To Upload a File to a Web Server in ASP.NET by Using Visual
C#
| > NET
| > http://support.microsoft.com/?id=323246
| >
| > If you are unable to use BITS you my want to think about using PUT to
| > upload files.
| > 184352 How To Upload Files to the Internet Information Server
| > http://support.microsoft.com/?id=184352
| >
| > You may also want to consider FTP.
| >
| > Hope this helps.
| >
| > Thanks
| > Brian [MSFT]
| > Microsoft Developer Support
| > This posting is provided "AS IS" with no warranties, and confers no
| rights.
| > --------------------
| > | Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| > | thread-index: AcSPoiLVaCpizciPSHa5KPYYMbSFYw==
| > | X-WBNR-Posting-Host: 209.183.130.140
| > | From: "=?Utf-8?B?bm9pcnM=?=" <***@discussions.microsoft.com>
| > | References: <807A87B7-8DA8-481C-A41C-***@microsoft.com>
| > <***@tk2msftngp13.phx.gbl>
| > <3248369B-2471-493B-8E38-***@microsoft.com>
| > <#lNAI$***@tk2msftngp13.phx.gbl>
| > <A3318F9D-C766-438E-B655-***@microsoft.com>
| > <***@cpmsftngxa10.phx.gbl>
| > <FB025E6A-8B5B-4D60-BC85-***@microsoft.com>
| > <***@TK2MSFTNGP15.phx.gbl>
| > <B8C32228-9653-448D-876C-***@microsoft.com>
| > <***@TK2MSFTNGP12.phx.gbl>
| > | Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| > | Date: Tue, 31 Aug 2004 14:33:20 -0700
| > | Lines: 264
| > | Message-ID: <7161AFD8-A533-421B-9B4B-***@microsoft.com>
| > | MIME-Version: 1.0
| > | Content-Type: text/plain;
| > | charset="Utf-8"
| > | Content-Transfer-Encoding: 7bit
| > | X-Newsreader: Microsoft CDO for Windows 2000
| > | Content-Class: urn:content-classes:message
| > | Importance: normal
| > | Priority: normal
| > | X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| > | Newsgroups: microsoft.public.inetsdk.programming.wininet
| > | NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| > | Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| > | Xref: cpmsftngxa10.phx.gbl
| > microsoft.public.inetsdk.programming.wininet:11641
| > | X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
| > |
| > | My interpretation of Brian's comment was that:
| > | - IIS does not keep the entire post in memory
| > | - an application can keep it in memory if it chose to.
| > |
| > | hopefully Brian can clarify for us!
| > |
| > | "Paul Baker [MVP, Windows - SDK]" wrote:
| > |
| > | > Brian said "IIS does not buffer the full incomming file. However the
| end
| > | > application can if it is a POST.".
| > | >
| > | > I am not sure exactly what this means. I *think* I have seen an it
| load
| > the
| > | > entire request into memory before passing control to ASP or an ISAPI
| > | > extension (ie. there's nothing the ASP or ISAPI extension itself is
| > doing to
| > | > cause it, and nothing they can do about it).
| > | >
| > | > Paul
| > | >
| > | > "noirs" <***@discussions.microsoft.com> wrote in message
| > | > news:B8C32228-9653-448D-876C-***@microsoft.com...
| > | > > Brian, can you confirm:
| > | > >
| > | > > If i post a huge file to IIS, does it keep it all in memory?
| > | > >
| > | > > "Paul Baker [MVP, Windows - SDK]" wrote:
| > | > >
| > | > > > I was thinking of the POST verb. I'm sorry I didn't realise that
| > PUT is
| > | > > > different.
| > | > > >
| > | > > > Paul
| > | > > >
| > | > > > "noirs" <***@discussions.microsoft.com> wrote in message
| > | > > > news:FB025E6A-8B5B-4D60-BC85-***@microsoft.com...
| > | > > > > - Thanks for the info Brian that IIS does not buffer the full
| > file in
| > | > > > memory
| > | > > > > - BITs looks useful, but doesn't look feasible (requires the
| > client to
| > | > > > have
| > | > > > > bits, and appears not to run on win9x)
| > | > > > >
| > | > > > > "Brian Combs" wrote:
| > | > > > >
| > | > > > > > Hello
| > | > > > > > If you need to upload large files with the ability to resume
| the
| > | > upload
| > | > > > if
| > | > > > > > it fails you should look at BITS SDK. IIS does not buffer
the
| > full
| > | > > > > > incomming file. However the end application can if it is a
| > POST. In
| > | > many
| > | > > > > > cases a PUT will work just fine for uploading a large file.
| > | > > > > >
| > | > > > > > For more information on BITS SDK
| > | > > > > >
| > | > > >
| > | >
| >
|
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/b
| > | > > > > > its_start_page.asp
| > | > > > > >
| > | > > >
| > | >
| >
|
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/w
| > | > > > > > hat_s_new.asp
| > | > > > > >
| > | > > >
| > | >
| >
|
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/i
| > | > > > > > is_requirements_for_bits_uploads.asp
| > | > > > > >
| > | > > > > >
| > | > > > > > Thanks
| > | > > > > > Brian [MSFT]
| > | > > > > > Microsoft Developer Support
| > | > > > > > This posting is provided "AS IS" with no warranties, and
| > confers no
| > | > > > rights.
| > | > > > > > --------------------
| > | > > > > > | Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| > | > > > > > | thread-index: AcSPcXoVQNWtSF82SDytPW2+AvFBJQ==
| > | > > > > > | X-WBNR-Posting-Host: 209.183.130.140
| > | > > > > > | From: "=?Utf-8?B?bm9pcnM=?="
| <***@discussions.microsoft.com>
| > | > > > > > | References:
| > <807A87B7-8DA8-481C-A41C-***@microsoft.com>
| > | > > > > > <***@tk2msftngp13.phx.gbl>
| > | > > > > > <3248369B-2471-493B-8E38-***@microsoft.com>
| > | > > > > > <#lNAI$***@tk2msftngp13.phx.gbl>
| > | > > > > > | Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| > | > > > > > | Date: Tue, 31 Aug 2004 08:45:01 -0700
| > | > > > > > | Lines: 103
| > | > > > > > | Message-ID:
| > <A3318F9D-C766-438E-B655-***@microsoft.com>
| > | > > > > > | MIME-Version: 1.0
| > | > > > > > | Content-Type: text/plain;
| > | > > > > > | charset="Utf-8"
| > | > > > > > | Content-Transfer-Encoding: 7bit
| > | > > > > > | X-Newsreader: Microsoft CDO for Windows 2000
| > | > > > > > | Content-Class: urn:content-classes:message
| > | > > > > > | Importance: normal
| > | > > > > > | Priority: normal
| > | > > > > > | X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| > | > > > > > | Newsgroups: microsoft.public.inetsdk.programming.wininet
| > | > > > > > | NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| > | > > > > > | Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| > | > > > > > | Xref: cpmsftngxa10.phx.gbl
| > | > > > > > microsoft.public.inetsdk.programming.wininet:11633
| > | > > > > > | X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
| > | > > > > > |
| > | > > > > > | Yes, i'm using IIS (5 I believe, on win2k).
| > | > > > > > |
| > | > > > > > | I have a hard time believing that IIS stores the whole
file
| in
| > | > ram, I
| > | > > > > > will
| > | > > > > > | try it out :) my machine only has 512mb ram, so
uploading a
| > 600mb
| > | > > > file
| > | > > > > > would
| > | > > > > > | use up all its physical ram and then some.
| > | > > > > > |
| > | > > > > > | This company has a version of their upload component which
| > they
| > | > claim
| > | > > > > > | supports resume (and I assume streaming), but its v.
| expensive
| > | > > > > > |
| > | > > > > > | http://fileup.softartisans.com/fileup-147.aspx
| > | > > > > > |
| > | > > > > > | "Paul Baker [MVP, Windows - SDK]" wrote:
| > | > > > > > |
| > | > > > > > | > Is the web server you are uploading to IIS? If you can
be
| > clear
| > | > on
| > | > > > > > that, it
| > | > > > > > | > would help, as we can limit our discussion to that. What
| > version
| > | > of
| > | > > > IIS
| > | > > > > > is
| > | > > > > > | > it?
| > | > > > > > | >
| > | > > > > > | > I'm fairly sure the request is held entirely in memory
no
| > matter
| > | > > > what is
| > | > > > > > | > handling it. I think I tested it for ISAPI extensions
too.
| > | > That's
| > | > > > why
| > | > > > > > those
| > | > > > > > | > third party tools to handle uploads don't help much.
| > | > > > > > | >
| > | > > > > > | > If I were you, I would reconsider your use of HTTP for
| > uploading
| > | > > > large
| > | > > > > > | > files. I have been down the same path as you, I know
it's
| > | > > > frustrating,
| > | > > > > > but
| > | > > > > > | > it's just not designed for that and it will never be
| > reliable.
| > | > > > > > | >
| > | > > > > > | > Paul
| > | > > > > > | >
| > | > > > > > | > "noirs" <***@discussions.microsoft.com> wrote in
message
| > | > > > > > | > news:3248369B-2471-493B-8E38-***@microsoft.com...
| > | > > > > > | > > hi Paul, we've successfully uploaded ~1gb files.. we
are
| > using
| > | > > > > > currently a
| > | > > > > > | > > free component from Dundas to do uploading.
| > | > > > > > | > >
| > | > > > > > | > > Further investigation i've found:
| > | > > > > > | > > - sample code to write an IIS isap extension that can
| > stream
| > | > > > uploads
| > | > > > > > to
| > | > > > > > | > disk
| > | > > > > > | > > - I think the dundas controls streams the upload to
disk
| > | > > > > > | > > - I think its only ASP perhaps that keeps it all in
| > memory.
| > | > > > > > | > >
| > | > > > > > | > > I am going to build an upload resume feature...
| basically
| > the
| > | > > > server
| > | > > > > > will
| > | > > > > > | > > issue the client an ID. if the client gets kicked
off,
| > the
| > | > client
| > | > > > > > will
| > | > > > > > | > ask
| > | > > > > > | > > where to resume from (by ID) and the server will
append.
| > | > > > > > | > >
| > | > > > > > | > > "Paul Baker [MVP, Windows - SDK]" wrote:
| > | > > > > > | > >
| > | > > > > > | > > > Alex,
| > | > > > > > | > > >
| > | > > > > > | > > > This error code is ERROR_INTERNET_CONNECTION_RESET.
| This
| > | > means
| > | > > > that
| > | > > > > > the
| > | > > > > > | > web
| > | > > > > > | > > > server reset your connection, ie. kicked you off.
| > | > > > > > | > > >
| > | > > > > > | > > > HTTP is not intended for large file uploads. FTP is
| more
| > | > > > suitable
| > | > > > > > for
| > | > > > > > | > this.
| > | > > > > > | > > >
| > | > > > > > | > > > Some HTTP (web) servers do not support large file
| > uploads.
| > | > For
| > | > > > > > example,
| > | > > > > > | > IIS
| > | > > > > > | > > > has an upper limit of 2GB in some versions and 1GB
in
| > | > others. I
| > | > > > > > think it
| > | > > > > > | > can
| > | > > > > > | > > > be changed using a registry entry, but there is no
| > point.
| > | > IIS
| > | > > > loads
| > | > > > > > the
| > | > > > > > | > > > entire request into memory before passing it off to
| the
| > | > > > component
| > | > > > > > that
| > | > > > > > | > will
| > | > > > > > | > > > handle the upload. This is presumably so that all
the
| > | > properties
| > | > > > of
| > | > > > > > the
| > | > > > > > | > > > request are available, eg. the Request.TotalBytes
| > property
| > | > in
| > | > > > ASP.
| > | > > > > > | > Because
| > | > > > > > | > > > it is in memory, IIS tends to be incredibly slow
when
| > | > uploading
| > | > > > a
| > | > > > > > file
| > | > > > > > | > that
| > | > > > > > | > > > is larger than a few hundred MBs.
| > | > > > > > | > > >
| > | > > > > > | > > > Paul
| > | > > > > > | > > >
| > | > > > > > | > > > "noirs" <***@discussions.microsoft.com> wrote in
| > message
| > | > > > > > | > > >
| > news:807A87B7-8DA8-481C-A41C-***@microsoft.com...
| > | > > > > > | > > > > I'm sending large files up to our webserver using
| HTTP
| > | > file
| > | > > > > > upload,
| > | > > > > > | > using
| > | > > > > > | > > > > InternetWriteFile (and HttpSendRequestEx).
| > | > > > > > | > > > >
| > | > > > > > | > > > > Often users get this error.. I get it repeatedly
on
| my
| > | > win95
| > | > > > test
| > | > > > > > box
| > | > > > > > | > in
| > | > > > > > | > > > > VirtualPC2004, i can't even complete a 1mb upload
| > over our
| > | > > > local
| > | > > > > > | > 100mbps
| > | > > > > > | > > > lan.
| > | > > > > > | > > > >
| > | > > > > > | > > > > Many users have successfully uploaded 600mb files.
| > | > > > > > | > > > >
| > | > > > > > | > > > > Is there some way I can build in resume
| functionality?
| > | > > > > > theoriticaly if
| > | > > > > > | > I
| > | > > > > > | > > > get
| > | > > > > > | > > > > this error can I just write the bytes again?
| > | > > > > > | > > > >
| > | > > > > > | > > > > why am I getting this error so consistently on
| win95?
| > | > > > > > | > > > >
| > | > > > > > | > > > > thx
| > | > > > > > | > > > >
| > | > > > > > | > > > > - Alex
| > | > > > > > | > > > >
| > | > > > > > | > > > > --
| > | > > > > > | > > > > Alex Black
| > | > > > > > | > > >
| > | > > > > > | > > >
| > | > > > > > | > > >
| > | > > > > > | >
| > | > > > > > | >
| > | > > > > > | >
| > | > > > > > |
| > | > > > > >
| > | > > > > >
| > | > > >
| > | > > >
| > | > > >
| > | >
| > | >
| > | >
| > |
| >
|
|
|
Paul Baker [MVP, Windows - SDK]
2004-11-02 22:15:32 UTC
Permalink
Thanks, but I don't want to know badly enough to test IIS 5.0 for you.

Paul
Post by Brian Combs
Hello
I don't know of any changes to IIS 5.1 from 5.0 with how asp handles data.
IIS does not cache POST data. You should run same test that you ran on XP
on 2000. As I suggested try using PUT and not POST to see what happens.
Thanks
Brian [MSFT]
Microsoft Developer Support
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------
| Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| Date: Mon, 25 Oct 2004 14:06:03 -0400
| Lines: 417
| X-Priority: 3
| X-MSMail-Priority: Normal
| X-Newsreader: Microsoft Outlook Express 6.00.2800.1409
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
| Newsgroups: microsoft.public.inetsdk.programming.wininet
| NNTP-Posting-Host: 66.192.38.14
cpmsftngxa10.phx.gbl!TK2MSFTFEED01.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP15
Post by Brian Combs
phx.gbl
| Xref: cpmsftngxa10.phx.gbl
microsoft.public.inetsdk.programming.wininet:11786
| X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
|
| Brian,
|
| I repeated my earlier tests.
|
| I found that I *could* indeed upload large files to the local host using a
| POST request and code similar to this sample.
|
| SAMPLE: Using HttpSendRequestEx for Large POST Requests
| http://support.microsoft.com/default.aspx?scid=kb;en-us;177188
|
| On the server side, I have IIS 5.1 on Windows XP Professional. I use an
ASP
| which examines Request.TotalBytes and calls Request.BinaryRead in a loop.
It
| did *not* cache the request in memory. It began to write to the file
| immediately and only used a few KB of memory.
|
| If the upload took longer than 90 seconds, I received an
| ERROR_INTERNET_CONNECTION_RESET (12031), just as Alex did. But this too is
| not a problem. It can be dealt with by increasing the value of
| Server.ScriptTimeout.
|
| Server.ScriptTimeout
| http://msdn.microsoft.com/library/en-us/iissdk/iis/ref_vbom_seropsct.asp
|
| So, I see no problem. But I did my original tests on Windows 2000. Is it
| possible that the behaviour of IIS with caching POST requests changed
| between IIS 5.0 and IIS 5.1?
|
| Paul
|
| > Hello
| > I will try and clarify. There are only two types of applications you can
| > write on IIS.
| > 1. ISAPI
| > 2. CGI
| > ASP and ASP.Net are both ISAPI. Perl, Cold Fusion, etc. maybe ISAPI or
CGI
| > depending on what runtime you are using. Remember IIS does not call
your
| > ASP page directly, it calls ASP.DLL and this is in charge of loading and
| > parsing your .asp file. The same is true for anything else that has an
| > application script map.
| >
| > IIS takes the incoming data and just passes it on to the application,
that
| > is receiving the request, as the data is received in IIS. IIS will do a
| > small amount of buffering for data that is sent in a post, but it will
not
| > buffer the full amount of data. However the underlying ISAPI or CGI can
| > chose to buffer the full amount of data for use later on. For example
| > ASP.Net does this, see the more info section of the following article.
| > 323246 How To Upload a File to a Web Server in ASP.NET by Using Visual
C#
| > NET
| > http://support.microsoft.com/?id=323246
| >
| > If you are unable to use BITS you my want to think about using PUT to
| > upload files.
| > 184352 How To Upload Files to the Internet Information Server
| > http://support.microsoft.com/?id=184352
| >
| > You may also want to consider FTP.
| >
| > Hope this helps.
| >
| > Thanks
| > Brian [MSFT]
| > Microsoft Developer Support
| > This posting is provided "AS IS" with no warranties, and confers no
| rights.
| > --------------------
| > | Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| > | thread-index: AcSPoiLVaCpizciPSHa5KPYYMbSFYw==
| > | X-WBNR-Posting-Host: 209.183.130.140
| > | Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| > | Date: Tue, 31 Aug 2004 14:33:20 -0700
| > | Lines: 264
| > | MIME-Version: 1.0
| > | Content-Type: text/plain;
| > | charset="Utf-8"
| > | Content-Transfer-Encoding: 7bit
| > | X-Newsreader: Microsoft CDO for Windows 2000
| > | Content-Class: urn:content-classes:message
| > | Importance: normal
| > | Priority: normal
| > | X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| > | Newsgroups: microsoft.public.inetsdk.programming.wininet
| > | NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| > | Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| > | Xref: cpmsftngxa10.phx.gbl
| > microsoft.public.inetsdk.programming.wininet:11641
| > | X-Tomcat-NG: microsoft.public.inetsdk.programming.wininet
| > |
| > | - IIS does not keep the entire post in memory
| > | - an application can keep it in memory if it chose to.
| > |
| > | hopefully Brian can clarify for us!
| > |
| > |
| > | > Brian said "IIS does not buffer the full incomming file. However the
| end
| > | > application can if it is a POST.".
| > | >
| > | > I am not sure exactly what this means. I *think* I have seen an it
| load
| > the
| > | > entire request into memory before passing control to ASP or an ISAPI
| > | > extension (ie. there's nothing the ASP or ISAPI extension itself is
| > doing to
| > | > cause it, and nothing they can do about it).
| > | >
| > | > Paul
| > | >
| > | > >
| > | > > If i post a huge file to IIS, does it keep it all in memory?
| > | > >
| > | > >
| > | > > > I was thinking of the POST verb. I'm sorry I didn't realise that
| > PUT is
| > | > > > different.
| > | > > >
| > | > > > Paul
| > | > > >
| > | > > > > - Thanks for the info Brian that IIS does not buffer the full
| > file in
| > | > > > memory
| > | > > > > - BITs looks useful, but doesn't look feasible (requires the
| > client to
| > | > > > have
| > | > > > > bits, and appears not to run on win9x)
| > | > > > >
| > | > > > >
| > | > > > > > Hello
| > | > > > > > If you need to upload large files with the ability to resume
| the
| > | > upload
| > | > > > if
| > | > > > > > it fails you should look at BITS SDK. IIS does not buffer
the
| > full
| > | > > > > > incomming file. However the end application can if it is a
| > POST. In
| > | > many
| > | > > > > > cases a PUT will work just fine for uploading a large file.
| > | > > > > >
| > | > > > > > For more information on BITS SDK
| > | > > > > >
| > | > > >
| > | >
| >
|
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/b
Post by Brian Combs
| > | > > > > > its_start_page.asp
| > | > > > > >
| > | > > >
| > | >
| >
|
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/w
Post by Brian Combs
| > | > > > > > hat_s_new.asp
| > | > > > > >
| > | > > >
| > | >
| >
|
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/i
Post by Brian Combs
| > | > > > > > is_requirements_for_bits_uploads.asp
| > | > > > > >
| > | > > > > >
| > | > > > > > Thanks
| > | > > > > > Brian [MSFT]
| > | > > > > > Microsoft Developer Support
| > | > > > > > This posting is provided "AS IS" with no warranties, and
| > confers no
| > | > > > rights.
| > | > > > > > --------------------
| > | > > > > > | Thread-Topic: InternetWriteFile - 12031 - HTTP Post upload
| > | > > > > > | thread-index: AcSPcXoVQNWtSF82SDytPW2+AvFBJQ==
| > | > > > > > | X-WBNR-Posting-Host: 209.183.130.140
| > | > > > > > | From: "=?Utf-8?B?bm9pcnM=?="
| > | > > > > > | Subject: Re: InternetWriteFile - 12031 - HTTP Post upload
| > | > > > > > | Date: Tue, 31 Aug 2004 08:45:01 -0700
| > | > > > > > | Lines: 103
| > | > > > > > | MIME-Version: 1.0
| > | > > > > > | Content-Type: text/plain;
| > | > > > > > | charset="Utf-8"
| > | > > > > > | Content-Transfer-Encoding: 7bit
| > | > > > > > | X-Newsreader: Microsoft CDO for Windows 2000
| > | > > > > > | Content-Class: urn:content-classes:message
| > | > > > > > | Importance: normal
| > | > > > > > | Priority: normal
| > | > > > > > | X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
| > | > > > > > | Newsgroups: microsoft.public.inetsdk.programming.wininet
| > | > > > > > | NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.1.29
| > | > > > > > | Path: cpmsftngxa10.phx.gbl!TK2MSFTNGXA03.phx.gbl
| > | > > > > > | Xref: cpmsftngxa10.phx.gbl
| > | > > > > > microsoft.public.inetsdk.programming.wininet:11633
microsoft.public.inetsdk.programming.wininet
Post by Brian Combs
| > | > > > > > |
| > | > > > > > | Yes, i'm using IIS (5 I believe, on win2k).
| > | > > > > > |
| > | > > > > > | I have a hard time believing that IIS stores the whole
file
| in
| > | > ram, I
| > | > > > > > will
| > | > > > > > | try it out :) my machine only has 512mb ram, so
uploading a
| > 600mb
| > | > > > file
| > | > > > > > would
| > | > > > > > | use up all its physical ram and then some.
| > | > > > > > |
| > | > > > > > | This company has a version of their upload component which
| > they
| > | > claim
| > | > > > > > | supports resume (and I assume streaming), but its v.
| expensive
| > | > > > > > |
| > | > > > > > | http://fileup.softartisans.com/fileup-147.aspx
| > | > > > > > |
| > | > > > > > |
| > | > > > > > | > Is the web server you are uploading to IIS? If you can
be
| > clear
| > | > on
| > | > > > > > that, it
| > | > > > > > | > would help, as we can limit our discussion to that. What
| > version
| > | > of
| > | > > > IIS
| > | > > > > > is
| > | > > > > > | > it?
| > | > > > > > | >
| > | > > > > > | > I'm fairly sure the request is held entirely in memory
no
| > matter
| > | > > > what is
| > | > > > > > | > handling it. I think I tested it for ISAPI extensions
too.
| > | > That's
| > | > > > why
| > | > > > > > those
| > | > > > > > | > third party tools to handle uploads don't help much.
| > | > > > > > | >
| > | > > > > > | > If I were you, I would reconsider your use of HTTP for
| > uploading
| > | > > > large
| > | > > > > > | > files. I have been down the same path as you, I know
it's
| > | > > > frustrating,
| > | > > > > > but
| > | > > > > > | > it's just not designed for that and it will never be
| > reliable.
| > | > > > > > | >
| > | > > > > > | > Paul
| > | > > > > > | >
message
| > | > > > > > | > > hi Paul, we've successfully uploaded ~1gb files.. we
are
| > using
| > | > > > > > currently a
| > | > > > > > | > > free component from Dundas to do uploading.
| > | > > > > > | > >
| > | > > > > > | > > - sample code to write an IIS isap extension that can
| > stream
| > | > > > uploads
| > | > > > > > to
| > | > > > > > | > disk
| > | > > > > > | > > - I think the dundas controls streams the upload to
disk
| > | > > > > > | > > - I think its only ASP perhaps that keeps it all in
| > memory.
| > | > > > > > | > >
| > | > > > > > | > > I am going to build an upload resume feature...
| basically
| > the
| > | > > > server
| > | > > > > > will
| > | > > > > > | > > issue the client an ID. if the client gets kicked
off,
| > the
| > | > client
| > | > > > > > will
| > | > > > > > | > ask
| > | > > > > > | > > where to resume from (by ID) and the server will
append.
| > | > > > > > | > >
| > | > > > > > | > >
| > | > > > > > | > > > Alex,
| > | > > > > > | > > >
| > | > > > > > | > > > This error code is
ERROR_INTERNET_CONNECTION_RESET.
Post by Brian Combs
| This
| > | > means
| > | > > > that
| > | > > > > > the
| > | > > > > > | > web
| > | > > > > > | > > > server reset your connection, ie. kicked you off.
| > | > > > > > | > > >
| > | > > > > > | > > > HTTP is not intended for large file uploads. FTP is
| more
| > | > > > suitable
| > | > > > > > for
| > | > > > > > | > this.
| > | > > > > > | > > >
| > | > > > > > | > > > Some HTTP (web) servers do not support large file
| > uploads.
| > | > For
| > | > > > > > example,
| > | > > > > > | > IIS
| > | > > > > > | > > > has an upper limit of 2GB in some versions and 1GB
in
| > | > others. I
| > | > > > > > think it
| > | > > > > > | > can
| > | > > > > > | > > > be changed using a registry entry, but there is no
| > point.
| > | > IIS
| > | > > > loads
| > | > > > > > the
| > | > > > > > | > > > entire request into memory before passing it off to
| the
| > | > > > component
| > | > > > > > that
| > | > > > > > | > will
| > | > > > > > | > > > handle the upload. This is presumably so that all
the
| > | > properties
| > | > > > of
| > | > > > > > the
| > | > > > > > | > > > request are available, eg. the Request.TotalBytes
| > property
| > | > in
| > | > > > ASP.
| > | > > > > > | > Because
| > | > > > > > | > > > it is in memory, IIS tends to be incredibly slow
when
| > | > uploading
| > | > > > a
| > | > > > > > file
| > | > > > > > | > that
| > | > > > > > | > > > is larger than a few hundred MBs.
| > | > > > > > | > > >
| > | > > > > > | > > > Paul
| > | > > > > > | > > >
| > message
| > | > > > > > | > > >
| > | > > > > > | > > > > I'm sending large files up to our webserver using
| HTTP
| > | > file
| > | > > > > > upload,
| > | > > > > > | > using
| > | > > > > > | > > > > InternetWriteFile (and HttpSendRequestEx).
| > | > > > > > | > > > >
| > | > > > > > | > > > > Often users get this error.. I get it repeatedly
on
| my
| > | > win95
| > | > > > test
| > | > > > > > box
| > | > > > > > | > in
| > | > > > > > | > > > > VirtualPC2004, i can't even complete a 1mb upload
| > over our
| > | > > > local
| > | > > > > > | > 100mbps
| > | > > > > > | > > > lan.
| > | > > > > > | > > > >
| > | > > > > > | > > > > Many users have successfully uploaded 600mb files.
| > | > > > > > | > > > >
| > | > > > > > | > > > > Is there some way I can build in resume
| functionality?
| > | > > > > > theoriticaly if
| > | > > > > > | > I
| > | > > > > > | > > > get
| > | > > > > > | > > > > this error can I just write the bytes again?
| > | > > > > > | > > > >
| > | > > > > > | > > > > why am I getting this error so consistently on
| win95?
| > | > > > > > | > > > >
| > | > > > > > | > > > > thx
| > | > > > > > | > > > >
| > | > > > > > | > > > > - Alex
| > | > > > > > | > > > >
| > | > > > > > | > > > > --
| > | > > > > > | > > > > Alex Black
| > | > > > > > | > > >
| > | > > > > > | > > >
| > | > > > > > | > > >
| > | > > > > > | >
| > | > > > > > | >
| > | > > > > > | >
| > | > > > > > |
| > | > > > > >
| > | > > > > >
| > | > > >
| > | > > >
| > | > > >
| > | >
| > | >
| > | >
| > |
| >
|
|
|
Loading...