C#: WebClient Usage

Microsoft has provided a great utility since .NET 1.0 for doing quick I/O via websites.  The WebClient class provides basic functionality for downloading from and uploading to webservers.  the WebClient example on MSDN leaves a lot to be desired, especially for beginners.  I’d like to expand on that, as well as provide some example code.

C#: Using WebClient to fetch a page:

// create a new instance of WebClient
        WebClient client = new WebClient();
 
        // set the user agent to IE6
        client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)");
        try
        {
            // actually execute the GET request
            string ret = client.DownloadString("http://www.google.com/");
 
            // ret now contains the contents of the webpage
            Console.WriteLine("First 256 bytes of response: " + ret.Substring(0,265));
        }
        catch (WebException we)
        {
            // WebException.Status holds useful information
            Console.WriteLine(we.Message + "\n" + we.Status.ToString());
        }
        catch (NotSupportedException ne)
        {
            // other errors
            Console.WriteLine(ne.Message);
        }

(This code uses DownloadString, you can also use DownloadData for a more binary-friendly version)

This is great for fetching simple pages that have data encoded in the querystring, but there are some problems with the basic DownloadString method of WebClient.  It’s synchronous, so it will block until the operation completes.  So for slow connections, or large files, this would need to run in another thread.  There is a better way.  But first, here’s another example of another basic, but important, method,DownloadFile.

// create a new instance of WebClient
        WebClient client = new WebClient();
 
        // set the user agent to IE6
        client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)");
        try
        {
            // actually execute the GET request
            client.DownloadFile("http://www.google.com/","google_fetch.txt");
 
            // ret now contains the contents of the webpage
            Console.WriteLine("File Saved.");
        }
        catch (WebException we)
        {
            // WebException.Status holds useful information
            Console.WriteLine(we.Message + "\n" + we.Status.ToString());
        }
        catch (NotSupportedException ne)
        {
            // other errors
            Console.WriteLine(ne.Message);
        }

This example is almost identical to the above, aside from the client.DownloadFile method.  This will save the file to disk, instead of returning a string–often what you would end up doing with the string anyway.

But can I send POST data?

Yes!  Using WebClient.UploadString or WebClient.UploadData you can POST data to the server easily.  I’ll show an example using UploadData, since UploadString is used in the same manner as DownloadString.

byte[] bret = client.UploadData("http://www.website.com/post.php", "POST",
                System.Text.Encoding.ASCII.GetBytes("field1=value1&field2=value2") );
 
            string sret = System.Text.Encoding.ASCII.GetString(bret);

UploadData returns a byte array (byte[]) which contains the contents of the page.  This can easily be converted to a string by using the System.Text.Encoding.ASCII.GetString() method.  You can also use the other encoding types in System.Text.Encoding (such as UTF8) to do the same.

Also note the use of GetBytes(), the UploadData method will only take a byte array as a buffer for the upload.  This works well if you are uploading a binary file.

What about the asynchronous methods of WebClient?

WebClient provides asynchronous methods for fetching webpages as well, they are named similarly to the synchronous methods, DownloadFileAsync, DownloadDataAsync, DownloadStringAsync, UploadFileAsync, UploadDataAsync, UploadStringAsync, and UploadValuesAsync.

In order to execute an Async request, you will need to attach event handlers before the method is executed.  Without these event handlers you will not have any sense of when the operation finishes, or what stage it is at.

C# WebClient Asynchronous call example:

void do_upload() {
   WebClient client = new WebClient();
   // add event handlers for completed and progress changed
   client.UploadProgressChanged += new UploadProgressChangedEventHandler(client_UploadProgressChanged);
 
   client.UploadFileCompleted += new UploadFileCompletedEventHandler(client_UploadFileCompleted);
  // carry out the operation as normal
   client.UploadFileAsync("http://www.daveamenta.com/up.php", @"c:\somefile.bin");
}
 
void client_UploadProgressChanged(object sender, UploadProgressChangedEventArgs e)
{
   Console.WriteLine(e.ProgressPercentage);
}
 
void client_UploadFileCompleted(object sender, UploadFileCompletedEventArgs e)
{
   if(e.Result != null) {
      Console.WriteLine(System.Text.Encoding.ASCII.GetString(e.Result));
   }
}

The call to client.DownloadFileAsync will no longer block the thread, and will execute in the background, periodically calling the event handlers to display progress.  The other methods can be used in this same way.