Wednesday, October 10, 2007

There are some questions that seem to be coming back all the time, and one of those is how to download from a web site. Whether it's a file or an image, it's basically the same approach.

I saw a nice article on DevSource name Pulling Data From Internet URLs in C# using the WebClient class which is a good alternative if you are planning on using it in the upcoming Compact Framework 3.5. But if you want a solution now, here's some code. Let's start by downloading a file (references to System.Net and System.IO)...

string url = "http://www.businessanyplace.net/images/ba.gif";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(new Uri(url));
request.Credentials =
new NetworkCredential("UserName", "P@ssW0rd");
request.PreAuthenticate =
true;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream s = response.GetResponseStream();
int size = 2048;
byte[] data = new byte[size];
FileStream fs = File.Create("baOnDisk.gif");
while(true)
{
    size = s.Read(data, 0, data.Length);
    
if(size > 0)
        fs.Write(data, 0, size);
    
else
        break;
}
fs.Close();

...and note that the download is buffered (in 2K blocks). I've also added some credential code to make it more realistic. If you want the contents of a text file in a string, the code can be modified like this (with a reference to System.Text)...

string url = "http://www.businessanyplace.net/sample/test.htm";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(new Uri(url));
request.Credentials =
new NetworkCredential("UserName", "P@ssW0rd");
request.PreAuthenticate =
true;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream s = response.GetResponseStream();
StringBuilder sb = new StringBuilder();
int size = 2048;
byte[] data = new byte[size];
while(true)
{
    size = s.Read(data, 0, data.Length);
    
if(size > 0)
        sb.Append(
Encoding.ASCII.GetString(data, 0, size));
    
else
        break;
}
string strData = sb.ToString();

...and to process that further (parsing the HTML, etc), you can now follow the same approach described in the article mentioned above.