[C#] soap is slow

[sathenzar]

I was just noticing while transferring my binary files from my database my network utilization was only running about about 5.26%. Is there a reason .NET doesn't use 100% of your bandwidth when you are calling a web soap call? I'm transferring PDF files (which can avg about 6 MB per file) and it is agonizingly slow. The call is super fast for calls that the file is 500 KB or less. I have the feeling if it was really transferring the soap call as quickly as it could the network % would be 100 would it not?

[Kami-]

I was just noticing while transferring my binary files from my database my network utilization was only running about about 5.26%. Is there a reason .NET doesn't use 100% of your bandwidth when you are calling a web soap call? I'm transferring PDF files (which can avg about 6 MB per file) and it is agonizingly slow. The call is super fast for calls that the file is 500 KB or less. I have the feeling if it was really transferring the soap call as quickly as it could the network % would be 100 would it not?

Your assumption sir, is incorrect.

[sathenzar]

ok thanks. Why does it not utilize the entire bandwidth to download the soap message faster?

[Eric Veteran]

SOAP isn't really designed for large files. It's supposed to be for messaging between endpoints. I would recommend using SOAP to expose a URL for the file and using normal HTTP or other methods for transferring the files.

This article may help some: http://msdn.microsoft.com/en-us/library/aa480521.aspx

[IceBreakerG]

I just spent the last couple months implementing a java soap web service in an asp.net application (it took so long because the group that wrote it either didn't care or didn't know that it didn't work in anything except java applications, and they had to fix it). All the service does is return either an image (as a byte array) or a URL to the image. Most of the images are less than 200KB, but there's a few that are over 1MB, and those take noticeably longer to retrieve and this is an internal intranet site. Like GreyWolf said, it'd be better to expose the URL to the file you want instead of actually returning the file, but if this is not an option, then you just have to work with what you have.

[sathenzar]

Well all of the PDFs are stored as a byte array in a database. So I guess in theory I could save a tmp file and then expose the url to that tmp file. I'm guessing then use another method to download that file. The question is would that create more overhead then the way it's already being done?

[IceBreakerG]

Well all of the PDFs are stored as a byte array in a database. So I guess in theory I could save a tmp file and then expose the url to that tmp file. I'm guessing then use another method to download that file. The question is would that create more overhead then the way it's already being done?

That's exactly what I did with the images that were coming back. I saved them on the server, then returned the direct URL to that image. There shouldn't be much (if any) overhead doing this other than the amount of time it'd take to save the file and finish the post back.

[Eric Veteran]

Well all of the PDFs are stored as a byte array in a database. So I guess in theory I could save a tmp file and then expose the url to that tmp file. I'm guessing then use another method to download that file. The question is would that create more overhead then the way it's already being done?

I would say store the PDFs on a server with an accessible URL and store the address in the database. Did you check out the MSDN article I linked earlier? It has a number of suggestions on how to handle exactly what you're trying to do.

This article about SOAP + Attachments is referenced in the MSDN one but the link is old. It may be helpful as well.

[sathenzar]

Sorry I should have mentioned I did look at those articles and they are really interesting. I'm going to go with the WCF MTOM route I think and re-design the core transfer functions. I don't know much about WCF but it looks really similar to regular web service (.asmx) coding. Or I may do the URL part I'm not sure. So many options lol. I really like the idea of the files being stored as a byte array blob in the database but if that just isn't a good practice maybe I should re-consider my methods. Because WCF will be basically the same thing just with a slightly more optimized method of transporting the xml file if I am not mistaken? MTOM just is an optimized method of transferring binary parts via soap message right?

[sathenzar]

can you return sockets from web services to write data to?

[sathenzar]

does anyone know how to enable mtom in the web.config file? I get an error stating the client and the service are mismatched when I enable mtom in the client config. Can't find a spot for mtom other then WSE 3.0 settings (which visual studio 2010 does not seem to have).

[Eric Veteran]

can you return sockets from web services to write data to?

You can, but there can be issues with it. Check the section labeled "Other Hybrid Approaches" in that article. :)

In reference to MTOM: http://msdn.microsof...y/aa528822.aspx

If your client and server are both using .NET you might also look into using remoting.

[sathenzar]

Thanks greywolf! you are the best! :) I will be checking into those. I saw the remoting in the previous article but I did not see the sockets part so I will go back and re-read that info.

[sathenzar]

I think part of the problem with the method I am currently using is that I may have my app.config setup incorrectly. There is a way I've heard to change how much data gets buffered / sent at once. So in theory if my internet can handle 1.2 MB/s then I could buffer 1.2MB then send it off as one chunk right?

[Kami-]

I think part of the problem with the method I am currently using is that I may have my app.config setup incorrectly. There is a way I've heard to change how much data gets buffered / sent at once. So in theory if my internet can handle 1.2 MB/s then I could buffer 1.2MB then send it off as one chunk right?

Theoretically; but I wouldn't advise it