As a follow-up on my post I Want More Mobile (Web) Services, I will continue with something very common that I want to do with my WM device - look up flight status info. One of the best public services for that is FlightExplorer, and therefore I decided to scrape their site to get the info that I want. Just as I suggested in the previous post, I have now created a ASP.NET Web Service application that takes care of the actual scraping, and the mobile client simply capture the parameter, calls the web service, and present the result. A minimum of information is transferred over the wire, and therefore this solution is very fast and cost effective.
If you want to get right to the code, I have created my second CodePlex project called Windows Mobile Web Services. If you want to be part of the development, please let me know.
Starting with the front end, 
you can see the UX on the right, and the client code looks like this...
WebServices.Service ws = new WebServices.Service();
resultTextBox.Text = ws.FlightLookup(flightTextBox.Text).Replace("\n", "\r\n");
...and on the server side, the code begins like this...
[WebMethod]
public string FlightLookup(string flight)
{
// Get session cookie and viewstate
CookieContainer cookies = new CookieContainer();
string url = "http://travel.flightexplorer.com/TrackFlight.aspx";
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.UserAgent = USER_AGENT;
request.CookieContainer = cookies;
StreamReader responseReader = new StreamReader(request.GetResponse().GetResponseStream());
string responseData = responseReader.ReadToEnd();
responseReader.Close();
// Extract the viewstate value and build POST data
string viewState = extractValue(responseData, "__VIEWSTATE");
string postData = String.Format("__VIEWSTATE={0}&leftcol1...flightNum={1}&FastTrack1...",
viewState, flight);
// Now post to the same page
request = WebRequest.Create(url) as HttpWebRequest;
request.UserAgent = USER_AGENT;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.CookieContainer = cookies;
StreamWriter requestWriter = new StreamWriter(request.GetRequestStream());
requestWriter.Write(postData);
requestWriter.Close();
responseReader = new StreamReader(request.GetResponse().GetResponseStream());
responseData = responseReader.ReadToEnd();
responseReader.Close();
...and this is a typical approach when scraping ASP.NET web applications (note, the above code is simplified to increase readability). First the page is requested without any parameters, just to get the session state cookie and the view state, and then a POST is made to the same page with the cookie and the form data created using the view state. When the resulting page content is captured, it's parsed like this...
// Get flight status
int i = 0;
string status = null;
string result = "Flight not found";
if(responseData.IndexOf(result) < 0)
{
string s = "Status:";
if((i = responseData.IndexOf(">" + s + "<")) > -1)
{
i = responseData.IndexOf("class='ft1'>", i) + 12;
status = responseData.Substring(i, responseData.IndexOf("<", i) - i);
result = string.Format("{0} {1}", s, status);
switch(status)
{
case "Planned":
s = "Departure in:";
i = responseData.IndexOf(">" + s + "<", i);
i = responseData.IndexOf("FTText2' colspan=3>", i) + 19;
result += string.Format("\n{0} {1}", s,
responseData.Substring(i, responseData.IndexOf("<", i) - i));
break;
case "In Flight":
s = "Time remaining:";
i = responseData.IndexOf(">" + s + "<", i);
i = responseData.IndexOf("class='ft1'>", i) + 12;
result += string.Format("\n{0} {1}", s,
responseData.Substring(i, responseData.IndexOf("<", i) - i));
break;
}
}
}
return result;
}
...and here is the helper method for extracting form values (it's very generic, and can be used in most ASP.NET scraping)...
private string extractValue(string s, string nameDelimiter)
{
string valueDelimiter = "value=\"";
int namePosition = s.IndexOf(nameDelimiter);
int valuePosition = s.IndexOf(valueDelimiter, namePosition);
if(namePosition < 0 || valuePosition < 0)
return string.Empty;
int startPosition = valuePosition + valueDelimiter.Length;
int endPosition = s.IndexOf("\"", startPosition);
return HttpUtility.UrlEncode(s.Substring(startPosition, endPosition - startPosition));
}
Finally, I also want to mention a detail in the client implementation. To ease entering the flight id, the input mode changes depending on number of characters entered. For the first three, the input mode is alphanumeric, and otherwise it's numeric. The code looks like this...
private bool inputNumeric = false;
private void flightTextBox_TextChanged(object sender, EventArgs e)
{
if(flightTextBox.Text.Length >= 3 && !inputNumeric)
{
InputModeEditor.SetInputMode(flightTextBox, InputMode.Numeric);
inputNumeric = true;
}
else if(flightTextBox.Text.Length < 3 && inputNumeric)
{
InputModeEditor.SetInputMode(flightTextBox, InputMode.Default);
inputNumeric = false;
}
}
...and the use of the private variable is just to prevent unnecessary input mode changes (the InputModeEditor class is in the Microsoft.WindowsCE.Forms namespace).
I will continue my quest for more, simple, but very usable Web Services, and any suggestions are most welcome. Just point me to a site and how it could be made available in an efficient way from a WM device...
Subscribe