Skip navigation

Test Run

Write Low-level ASP.NET Web Services Test Automation

asp:feature

LANGUAGES: C#

ASP.NET VERSIONS: 1.1

 

Test Run

Write Low-level ASP.NET Web Services Test Automation

 

By Dr. James McCaffrey

 

The use of ASP.NET Web services is steadily increasing among the companies I work with. Application architectures that use the distributed Web service model are proving to be dramatically easier to design, deploy, and maintain than older architectures. However, I ve discovered that the techniques to write low-level test automation for Web services are not widely known. In this article, I ll show you a technique to write socket-based automation that tests one aspect of Web services security. The best way to explain where I m headed is with two screenshots. Figure 1 shows a simple, but typical, ASP.NET Web application that uses a Web method in a Web service.

 


Figure 1: The BookSearch.aspx Web application.

 

In the example in Figure 1, the BookSearch.aspx Web application uses a Web service method, which searches a SQL database of book information. Searching the database for books with a price of 22.95 returns information about two books, as shown in Figure 1. Now suppose you want to test the application s underlying Web method for its ability to resist an attack by malformed input. There is no good way to send malformed input through the application s UI, or even through the Web service interface. To send malformed input you should send corrupted, raw SOAP messages directly to the Web service method, just as an attacker would. The console application shown in Figure 2 does exactly this.

 


Figure 2: Testing the Web service with malformed input.

 

The test automation starts with a correctly formed raw SOAP message. Then the test program corrupts the message by randomly changing one character in the message. The automation uses a socket to post the malformed SOAP message directly to the Web service method over HTTP and examines the resulting HTTP response for the expected HTTP 500 server error message. If the automation finds HTTP 500 it displays a pass result and continues. If the automation does not detect the expected response, it halts and logs a test case fail message.

 

In the sections that follow I ll show you the ASP.NET Web service method and associated database we are testing, briefly examine the dummy ASP.NET Web application that uses the service, and describe in detail the low-level test automation that exercises the Web service. I ll finish up by discussing how you can adapt and extend the code presented here to meet your own needs in a production environment. If you currently use or are planning to use Web services, I think you ll find the ability to quickly write low-level test automation a valuable addition to your skill set.

 

The Web Service and the Web App

Let s take a look at the underlying Web service we will be testing. Among the most common uses of Web services is exposing centralized data to Web applications. That s what my example Web service under test does, so let s briefly examine its source SQL database. For this article I decided to use the pubs sample database that is installed with SQL Server. The pubs database contains data for a hypothetical publishing company. Figure 3 shows the titles table displayed using the Enterprise Manager program.

 


Figure 3: The underlying Web service data.

 

In a production environment with a real database, you ll want to make sure that you create a dedicated database test bed rather than relying on the development database. I added one row of data to the original pubs.titles table so that I could get more than one row with a price of 22.95 (the book title Test Automation in .NET displayed in the second row in Figure 3). Besides adding a row of data to pubs.titles, I created a short stored procedure named usp_GetTitles to retrieve title data by price:

 

create procedure usp_GetTitles

 @price money

as

 select title_id, title

 from titles

 where price = @price

go

 

Although Web service methods can access SQL data using embedded SQL statements, it s generally a better idea to use stored procedures because they are more secure and have better performance. My stored procedure accepts a book price as an input argument and returns a SQL rowset with matching book IDs and titles. Finally, I augmented the pubs database by creating a SQL login named webServiceLogin that has execute permissions for the usp_GetTitles stored procedure. You can use an existing general purpose SQL login, but creating a dedicated login for Web service permissions is a much better approach because it allows you to fine tune exactly how applications can access your underlying SQL data.

 

Now let s take a quick look at the actual Web service that makes the data in pubs.titles available to applications. If you haven t created an ASP.NET Web service before, you ll be surprised at how easy Visual Studio.NET makes your job. After launching Visual Studio, I created a new C# ASP.NET Web service project and named it TheWebService. Then I added a namespace declaration ( using System.Data.SqlClient ) so I could access the SqlDataAdapter class without having to fully qualify it. Next I added a single Web method, GetTitles, to the PubsService class. The code for GetTitles is shown in Figure 4.

 

[WebMethod]

public DataSet GetTitles(double price)

{

 try

 {

 string connStr = "Server=(local);Database=pubs;

                   UID=webServiceLogin;PWD=secret";

 SqlConnection sc = new SqlConnection(connStr);

 SqlCommand cmd = new SqlCommand("usp_GetTitles", sc);

 cmd.CommandType = CommandType.StoredProcedure;

 cmd.Parameters.Add("@price", SqlDbType.Money);

 cmd.Parameters["@price"].Direction =

   ParameterDirection.Input;

 cmd.Parameters["@price"].Value = price;

 SqlDataAdapter sda = new SqlDataAdapter(cmd);

 DataSet ds = new DataSet();

 sda.Fill(ds);

 sc.Close();

 return ds;

 }

 catch

 {

 return null;

 }

} // GetTitles()

Figure 4: The GetTitles Web method.

 

The GetTitles Web method accepts a book price value, calls the usp_GetTitles stored procedure to retrieve data from the pubs.titles database, and returns that data in a DataSet object. After building my Web service, I manually tested it by hitting [F5] to attach the debugger. Visual Studio.NET launched a Web page, as shown in Figure 5.

 


Figure 5: Manually testing the Web method with Visual Studio.NET.

 

Now that we ve seen the Web service method under test, and its associated database, let s take a brief look at a dummy ASP.NET Web application that uses the Web service. If you refer back to Figure 1, you ll see that the Web application has a TextBox control where the user enters a book price, and a Button control to submit the price to the Web server, which in turn calls the Web service, which in turn accesses the SQL database. The key code in the Web application is listed in Figure 6.

 

private void Button1_Click(object sender,

                          System.EventArgs e)

{

 try

 {

 PubsServiceReference.PubsService ps =

  new PubsServiceReference.PubsService();

 string price = TextBox1.Text.Trim();

 DataSet ds = ps.GetTitles(double.Parse(price));

 DataGrid1.DataSource = ds;

 DataGrid1.DataBind();

 Label3.Text = ds.Tables["Table"].Rows.Count +

  " titles found";

 }

 catch(Exception ex)

 {

 Label3.Text = ex.Message;

 }

} // Button1_Click()

Figure 6: Web application code to call a Web service method.

 

I added a Web reference to the Web application and renamed it from the default localhost to the more descriptive PubsServiceReference . Next, I instantiated a PubsService Web service just as I would an ordinary object. Then I called the GetTitles Web method, just like I would any instance method, and bound the returned DataSet object to a DataGrid control for display.

 

The Test Automation

Now I ll show you the test automation program that generated the output shown in Figure 2. The basic idea is to construct a raw SOAP message, corrupt the message, construct an HTTP header, attach the malformed message to the header, and then post the HTTP request directly to the Web service method under test using a Socket object. In pseudo-code, the plan looks like this:

 

create correct SOAP message

loop

 corrupt the correct SOAP message

 build HTTP headers

 attach corrupted message to headers

 connect to Web service method with socket

 post malformed message to Web method

 if (response is HTTP 500 error)

   log pass message and continue

 else

   log fail message and halt

end loop

 

I implemented this plan using a C# console application, but you can just as easily use Visual Basic.NET or any other .NET language of your choice. I organized the test harness as a Main method, with four helper methods. The test harness is quite short, and is shown in Listing One.

 

I begin by declaring the namespaces that the harness will use so that I won t have to fully qualify each class and object. Notice the reference to the System.Net.Sockets namespace, which will allow me to send information directly to the Web service method under test at a very low level by using sockets. Next I instantiate a Random object:

 

static Random r = new Random(5);

 

I use this object to randomly corrupt my SOAP message. Notice that the Random object is static and declared externally to any method. A common mistake is to instantiate a Random object inside the method that will use the object; if you do this, the pseudorandom number generator is reset at each method call and you end up with the same random number on each call. I passed in 5 to the Random constructor. The 5 is arbitrary and is the so-called seed value for generating the sequence of pseudorandom numbers. By supplying a fixed seed value, my test runs will be reproducible because the sequence of pseudorandom numbers will be the same each time.

 

Inside the Main method, I create a correctly formed SOAP message (see Figure 7). There is a lot of overhead, but in essence I am just sending a price argument of 22.95 to the GetTitles Web method. So where did this message originate? There are several ways to determine a correctly crafted SOAP message for your ASP.NET Web service methods, but by far the easiest is to let Visual Studio.NET do it for you. If you start the Web service debugger, then select one of its Web methods, Visual Studio.NET will generate the appropriate SOAP message template, as shown in Figure 8. Very nice!

 

string soapMessage = "

                    "utf-8\"?>";

soapMessage += "

              "http://www.w3.org/2001/XMLSchema-instance\"";

soapMessage += "xmlns:xsd=\

              "http://www.w3.org/2001/XMLSchema\"";

soapMessage += "xmlns:soap=\

              "http://schemas.xmlsoap.org/soap/envelope/\">";

soapMessage += "";

soapMessage += "";

soapMessage += "22.95";

soapMessage += "";

soapMessage += "";

soapMessage += "";

Figure 7: A correctly formed SOAP message inside the Main method.

 


Figure 8: Visual Studio.NET will generate the appropriate SOAP message template.

 

Notice I used a traditional C/C++ technique to build up the SOAP message string rather than using the more efficient .NET StringBuilder class. I did this to keep my code easier for you to read, but in a production environment you may want to switch to the StringBuilder technique or simply assign the SOAP message as one long string. Next, I set up string variables for the machine that hosts the Web service, the path to the ASP.NET Web service, and the Web method in the Web service I m going to call:

 

string host = "localhost";

string service = "/WebServiceLow/TheWebService/

 PubsService.asmx";

string method = "GetTitles";

 

Now I code the main test loop as shown in Figure 9.

 

for (int i = 1; i <= 5; ++i)

{

 string malformed = randomChange(soapMessage);

 Socket sock = connect(host, 80);

 Console.WriteLine("Sending SOAP message to Web Service

                   method GetTitles");

 string response = sendSOAP(sock, host, service,

                            method, malformed);

 Console.WriteLine("Looking for HTTP 500 Server Error");

 if (isExpected(response))

 {

   Console.WriteLine(i.ToString("0000") + " Pass");

 }

 else

 {

   Console.WriteLine(i.ToString("0000") + " FAIL");

   break;

 }

 Console.WriteLine("===================================");

}

Figure 9: The main test loop.

 

For this article I used a for loop that iterates at most five times. You ll want to either increase this number or re-factor the automation code to use a while loop that iterates forever (or until an error is found). I call a helper method (randomChange) to corrupt the original correct SOAP message. I ll go over randomChange shortly. Next, I call a helper method (connect) that returns a Socket object connected to the test machine. Then I call a helper method (sendSOAP) that posts the corrupted SOAP message to the Web method using the Socket object, and then grab the HTTP response stream. I examine the response to see if it s what I expected using another helper method named isExpected. If isExpected returns true I simply log a pass message to the command shell and continue. However, if I get an unexpected response, I use the break statement to halt the test automation so I can investigate what happened. If the server does not return a 500 error, I may have found an unauthorized way into the system.

 

The randomChange method is listed in Figure 10; it s short, but a bit tricky. The randomChange method picks an almost-random location within a SOAP message and changes the character at that location to an almost-random character. I begin by finding the position within the SOAP message where the element starts. I m really not interested in changing the SOAP envelope or other wrapper information. Next, I generate a random index between the tag and the end of the message. Then I extract two substrings (everything to the left of the selected index and everything to the right). I generate a random character with an ASCII value between 0 and 255, then glue the three substrings back together. I put a Console.WriteLine statement in the method just to help you see what was happening in Figure 2. The result is a SOAP message that has a random character between 0 and 255 replacing a character at a random index between the tag and the end of the message. Of course, you ll have to modify the randomChange method to meet the particular needs of the Web method you are testing. Notice that my crude randomChange method might coincidentally generate a valid SOAP message if it replaces one of the digits in the 22.95 price argument with another valid digit.

 

static string randomChange(string s)

{

   int start = s.IndexOf("");

   int x = r.Next(start, s.Length);

   string left = s.Substring(0, x);

   string right = s.Substring(x+1, s.Length-(x+1));

   string middle = Convert.ToChar(r.Next(0,255)).ToString();

   string result = left + middle + right;

   Console.WriteLine("Changing char at [" + x + "]

                   to " + middle);

   return result;

} // randomChange()

Figure 10: The randomChange method.

 

The connect helper method is also very short:

 

static Socket connect(string host, int port)

{

 IPHostEntry iphe = Dns.Resolve(host);

 IPAddress[] addList = iphe.AddressList;

 EndPoint ep = new IPEndPoint(addList[0], 80);

 Socket sock = new Socket(AddressFamily.InterNetwork,

                  SocketType.Stream, ProtocolType.Tcp);

 sock.Connect(ep);

 return sock;

} // connect()

 

The .NET Framework makes working with sockets much easier than in the pre-.NET days. I use Dns.Resolve to get a list of IP addresses that map to my host name. Normally, there is a one-to-one mapping between host name and IP address, but you can t always count on this. Next, I grab the first IP address returned from Dns.Resolve and pass it to the IPEndPoint constructor, along with a port number. A socket end point is the IP address plus a port number. For example, if your host name is localhost , in most cases your end point will be 127.0.0.1:80 if your system configuration uses default values. Next, I call the Socket constructor and pass in constants to create a normal TCP/IP socket object. Finally, I call the Socket.Connect method to establish a low-level connection from client to server.

 

The bulk of the actual work is performed inside the sendSOAP helper method. The code for sendSOAP is listed in Figure 11.

 

static string sendSOAP(Socket s, string host,

                      string webService,

                      string method, string soapMessage)

{

 string header = "POST " + webService + " HTTP/1.1\r\n";

 header += "Host: " + host + "\r\n";

 header += "Content-Type: text/xml; charset=utf-8\r\n";

 header += "Content-Length: " +

   soapMessage.Length.ToString() + "\r\n";

 header += "Connection: close\r\n";

 header += "SOAPAction: \"http://tempuri.org/" +

   method + "\"\r\n\r\n";

 string requestAsString = header + soapMessage;

 byte[] requestAsBytes =

   Encoding.ASCII.GetBytes(requestAsString);

 int numBytesSent = s.Send(requestAsBytes,

                           requestAsBytes.Length,

                           SocketFlags.None);

 byte[] responseBufferAsBytes = new byte[512];

 string responseAsString = "";

 string entireResponse = "";

 int numBytesReceived = 0;

 while ((numBytesReceived = s.Receive(

   responseBufferAsBytes, 512, SocketFlags.None)) > 0 )

 {

   responseAsString = Encoding.ASCII.GetString(

     responseBufferAsBytes, 0, numBytesReceived);

   entireResponse += responseAsString;

 }

 return entireResponse;

} // sendSOAP()

Figure 11: The sendSOAP helper method.

 

I begin by constructing the HTTP headers. Most of these headers are fairly self-explanatory. Notice that, even though the SOAP message comes after the HTTP headers, you must construct the SOAP message first so you can supply the Content-Length header with the size of the message. I used the optional Connection: close header to explicitly close the HTTP connection after each HTTP request-response exchange. The header that may be unfamiliar to you is the SOAPAction header, which essentially tells the Web server to expect SOAP content. The SOAPAction header has been deprecated in SOAP 1.2 in favor of a new Content-Type: application/soap+xml header, but for now, most servers (including mine) are expecting the SOAPAction header.

 

In the following example, the argument to SOAPAction contains the string http://tempuri.org and the name of the Web method. The tempuri.org is just a dummy namespace supplied by Visual Studio.NET for Web services under development. In a production environment, you ll want to change this to a unique identifier. After setting up the HTTP headers, I prepare to send the HTTP request:

 

string requestAsString = header + soapMessage;

byte[] requestAsBytes = Encoding.ASCII.GetBytes(

 requestAsString);

int numBytesSent = s.Send(requestAsBytes,

                         requestAsBytes.Length,

                         SocketFlags.None);

 

I build the request string, convert it to bytes (all HTTP traffic is in byte form), send the request to the socket, and grab the number of bytes sent. Retrieving the HTTP response through a socket follows a fairly well-known pattern:

 

byte[] responseBufferAsBytes = new byte[512];

string responseAsString = "";

string entireResponse = "";

int numBytesReceived = 0;

while ((numBytesReceived = s.Receive(

      responseBufferAsBytes, 512, SocketFlags.None)) > 0 )

{

 responseAsString = Encoding.ASCII.GetString(

   responseBufferAsBytes, 0, numBytesReceived);

 entireResponse += responseAsString;

}

 

First I create a byte buffer. I used a size of 512 bytes, but this is arbitrary. Socket.Receive will read 512 bytes if they re available, store those bytes into the byte buffer, and return the actual number of bytes read which could be less than 512 if there s not enough response left or 0 if there s no response at all left to read. If I read 0 bytes I know I m done and I exit the loop. Otherwise, I convert the bytes read to a string and append it to a string variable that holds the entire response stream. After reading all the response bytes I return the entire response as a string.

 

The fourth helper method, isExpected, is very short:

 

static bool isExpected(string s)

{

 return s.IndexOf("HTTP/1.1 500") > 0;

}

 

isExpected simply examines the HTTP response stream for an indication that the Web server could not process the SOAP request. Dealing with the response to random test case input can be rather tricky. Here I am expecting a server error. If I don t get a server error, then I need to investigate exactly what is happening. Depending on your particular test scenario, you ll have to code an isExpected method with logic that matches your situation.

 

Conclusion

You can modify and extend in many ways the Web service test automation technique I ve presented here. The test automation harness in this article tests only one tiny aspect of Web service security, but the basic structure will easily adapt to many other testing scenarios. For example, sending randomly malformed data to a Web service as I did in this article is not nearly as likely to find a security hole as sending specifically malformed data. This is not to say that you shouldn t run the random scenario presented in this article with security you have to relentlessly test every interface into the system under test. How you create focused malformed data will depend entirely on the Web service you re testing.

 

I used sockets to programmatically post SOAP input over HTTP to the Web service method under test. This is a very low-level technique. The .NET Framework provides you with higher-level alternatives, such as the HttpWebRequest, WebClient, and TcpClient classes. The advantage of these higher-level objects is that they are significantly easier to use than sockets. However, using sockets gives you maximum control over your test automation. Furthermore, using sockets allows you to test SOAP when not used in conjunction with HTTP. Although SOAP is most often used with HTTP, SOAP is transport protocol independent. Unlike using higher-level classes, the technique I ve presented here can be easily adapted to systems that use alternative or proprietary transport protocols. As a general rule, however, you are better off using one of the higher-level classes I mentioned when testing Web services for basic functionality.

 

Another advantage of using the socket-based, low-level technique I ve presented here is that you can modify it to test Web services for performance testing, load analysis, and functionality verification. The essence of performance testing is to measure how long all the different components of your system take to execute. By placing timers in your test harness you can measure how long different parts of your Web services-based system and their connections require. When testing for performance it is important to work at as low a level as possible to remove spurious affects of the test harness. Similarly, a low-level test harness works very well when you are testing for load characteristics or for low-level functionality analysis. Finally, using a socket-based technique for testing will work with non-.NET-based systems that don t have higher-level classes available.

 

The malformed SOAP test automation technique as presented here simply halts execution if it encounters an unexpected response. When you re running random-input automation, you often want to let it run for weeks, or even months, at a time. So, a good enhancement would be to fire off an e-mail alert on a test case failure (using classes in the System.Web.Mail namespace) instead of just stopping. Along the same lines, you may want to log even successful test case data to external storage (typically SQL or a text file) so that you can easily recreate a failed system state without having to rerun the entire test scenario from the beginning.

 

Software systems have dramatically increased in size and complexity over the past few years and incomplete testing is no longer a viable option. The low-level Web service test automation techniques presented here should play a prominent role in your overall Web services test efforts and be a valuable addition to your developer/tester/manager skill sets.

 

The sample code referenced in this article is available for download.

 

Dr. James McCaffrey works for Volt Information Sciences, Inc., where he manages technical training for software engineers working at Microsoft s Redmond, WA campus. He has worked on several Microsoft products, including Internet Explorer and MSN Search. James can be reached at mailto:[email protected] or mailto:[email protected].

 

Begin Listing One Test Harness Code

using System;

using System.Net.Sockets;

using System.Text;

using System.Net;

using System.Web; // add ref

namespace MalformTest

{

 class Class1

 {

 static Random r = new Random(5);

  [STAThread]

 static void Main(string[] args)

 {

   try

   {

     Console.WriteLine("\nBegin low level Web

                       Service security test\n");

     string soapMessage = "

                          encoding=\"utf-8\"?>";

     soapMessage += "

                    http://www.w3.org/2001/

                    XMLSchema-instance\"";

     soapMessage += "xmlns:xsd=\"http://www.w3.org/

                    2001/XMLSchema\"";

     soapMessage += "xmlns:soap=\"

                    http://schemas.xmlsoap.org/soap/

                    envelope/\">";

     soapMessage += "";

     soapMessage += "

                    http://tempuri.org/\">";

     soapMessage += "22.95";

     soapMessage += "";

     soapMessage += "";

     soapMessage += "";

     Console.WriteLine("Original SOAP message is: \n");

      Console.WriteLine(soapMessage);

     Console.WriteLine(

      "\n===================================");

     string host = "localhost";

     string service = "/WebServiceLow/TheWebService/

                      PubsService.asmx";

     string method = "GetTitles";

     for (int i = 1; i <= 5; ++i)

     {

       string malformed = randomChange(soapMessage);

       Socket sock = connect(host, 80);

       Console.WriteLine("Sending SOAP message to Web

                         Service method GetTitles");

       string response = sendSOAP(sock, host, service,

                                  method, malformed);

       Console.WriteLine("Looking for HTTP 500

                         Server Error");

       if (isExpected(response))

       {

         Console.WriteLine(i.ToString("0000") + " Pass");

       }

       else

       {

         Console.WriteLine(i.ToString("0000") + " FAIL");

         break;

       }

       Console.WriteLine(

        "===================================");

     }

     Console.ReadLine();

   }

   catch(Exception ex)

   {

     Console.WriteLine(ex.Message);

     Console.ReadLine();

   }

 } // Main()

 static Socket connect(string host, int port)

 {

   // return a connected socket

 }

 static string sendSOAP(Socket s, string host,

                        string webService, string method,

                        string soapMessage)

 {

   // post SOAP message, return response

 }

 static string randomChange(string s)

 {

   // change random char of s

 }

 static bool isExpected(string s)

 {

   // does s contain an expected string?

 }

 } // class

} // ns MalformTest

End Listing One

 

 

 

Hide comments

Comments

  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Publish