Communicating with FAST ESP from a .NET application through SSL

30. October 2010 Bart X. Tubalinal FAST Search Comments (0)

For a custom application we’re building, we decided to use SSL to communicate between our .NET application and the FAST query server. Setting up SSL to enable secure communication between a .NET application and a FAST query server is fairly simple but the FAST documentation doesn’t exactly explain it a straightforward manner. These were the steps I took to enable and use SSL.

The first step is to request a new certificate.You can use the openssl executable found in c:\esp\bin to request the new certificate. Open up a command prompt and change the working directory to c:\esp\bin. Enter the following command:

openssl req –new –out client.pem –days 365 –config c:\esp\etc\openssl.cnf

The above command creates a new certificate and should be fairly easy to understand. The request file will be saved as client.pem and the certificate generated (later) will be valid for 365 days. The configuration file specified by the –config parameter is the file that specifies the fields that you will be prompted for as identification to be included in the certificate request. When you run the command, you will be asked to enter and verify a passphrase. You will then be asked for the fields specified in the config file (note that you can just leave the ‘extra’ attributes being asked for blank). The command also outputs the private key to a file called privkey.pem. The exact output of the command is below:

C:\Users\btubalinal>openssl req -new -out client.pem -days 365 -config c:\esp\etc\
openssl.cnf
Loading 'screen' into random state - done
Generating a 1024 bit RSA private key
...........++++++
...........++++++
writing new private key to 'privkey.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:
Email Address []:
 
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
 
C:\Users\btubalinal>

The next step is to create a PKCS12 file from the .pem file. Again, you can use the openssl executable to create this:

openssl pkcs12 -export -out client.p12 -in client.pem -inkey privkey.pem

This exports out the PKCS12 file to a file called client.p12. It uses the client.pem file (which is the certificate) and the privkey.pem file (which is the private key) and merges them into the client.p12 file. The client.p12 file needs to be imported into your personal certificate store on whatever machine you’ll use to access the FAST query server. So copy the p12 file to your machine and double click the file to start the certificate import wizard. The wizard will ask you for the password/pass phrase you gave when creating the original client.pem certificate. The rest of the wizard is straightforward.

The last file we need to generate is a .cer file. This file is what is required to be used by your .NET application to issue the queries to the query server. Here is the command to create the .cer file:

openssl x509 -in client.pem -out client.cer -outform DER

This command basically converts the client.pem certificate (which is in PEM format) to a certificate that is in DER format and saves it to the client.cer file. Copy this cer file to a location on your machine where your app has access to.

The following is a code snippet that shows how the .cer file is used:

   1: using System;
   2: using Com.FastSearch.Esp.Search;
   3: using Com.FastSearch.Esp.Search.View;
   4: using Com.FastSearch.Esp.Search.Result;
   5: using Com.FastSearch.Esp.Search.Query;
   6: using Com.FastSearch.Esp.Search.Http;
   7: using System.Collections.Specialized;
   8:  
   9: namespace FASTSampleSSL
  10: {
  11:     class Program
  12:     {
  13:         static void Main(string[] args)
  14:         {
  15:             string fastserver = "myfastserver.deviantpoint.com:443";
  16:             string certfilepath = @"c:\client.cer";
  17:  
  18:             NameValueCollection searchFactoryConfiguration = new NameValueCollection();
  19:             searchFactoryConfiguration.Add("Com.FastSearch.Esp.Search.Http.QRServers", fastserver);
  20:  
  21:             ISearchFactory searchFactory = SearchFactory.NewInstance(searchFactoryConfiguration);
  22:  
  23:             string fastserverurl = "https://" + fastserver;
  24:             HttpSearchEngine searchEngine = searchFactory.GetSearchEngine(new Uri(fastserverurl));
  25:             ISearchView view = searchEngine.GetView("mypublishedview", fastserverurl, false, false, certfilepath);
  26:  
  27:             IQuery query = new Query("string(\"sharepoint\", mode=simpleall)");
  28:             IQueryResult result = view.Search(query);
  29:  
  30:             //PROCESS THE RESULTS
  31:         }
  32:     }
  33: }

Most of the code should be easy enough to understand. The cert file is used in the call to get the view from the instance of the HttpSearchEngine. The FAST ESP API will take care of reading that certificate file and using it to communicate securely to the FAST query server.

Querying an SPFieldMultiLineText field for an exact line match with LINQ

2. October 2010 Bart X. Tubalinal SharePoint Comments (2)

Today, I had to come up with a way to return a value from one column in a SharePoint list based on finding an exact line match of a user’s query terms in an SPFieldMultiLineText field in the same list. To clarify, here’s an example of how the data looked like in the SharePoint list (this is just sample data):

Term Synonyms
SharePoint SharePoint 2010
SP2010
SharePoint 2007
Microsoft Office SharePoint Server
MOSS
Windows SharePoint Services
SharePoint Foundation
SharePoint Server
FAST Search FAST ESP
FAST Search for Internet Sites
FAST Search for SharePoint
FSIS

In the UI provided, if a user looked for the term ‘Microsoft Office SharePoint Server’, I needed to return back the ‘SharePoint’ term (first item, first column in the table).

There were three options I could think of to accomplish this:

  1. CAML query.I ruled this out fairly quickly. Unfortunately, I had to do an exact match on the term that the user entered. So, for instance, if the user entered the term ‘Windows SharePoint Services’, then the first term (SharePoint) would be returned. However, if the user only entered the term ‘Windows’, the ‘SharePoint’ item shouldn’t be returned because it was not an exact match. With CAML, there was really no way for me to specify an exact match per line. I couldn’t use <Eq> because that would necessitate the entire Synonyms field to match what the user entered. I also couldn’t use <Contains> as that would result in false positives. None of the other comparison operators seemed appropriate either.
  2. For-each loop. Of course, i could have iterated through the items one by one and examine the Synonyms field. For each item in the list, I’d have to examine the Synonyms line-by-line to find the exact match. This was originally what I planned on doing, however, it felt inelegant and potentially inefficient, especially because this list is going to contain many items.
  3. LINQ. I didn’t know exactly how I could use LINQ to tackle this but I always felt like I could. So this is the path I attempted. Obviously, it worked or I wouldn’t be writing this blog post :).

Below is the code I wrote to query the field for an exact line match:

 

 1: public string GetTerm(string userSearchTerm)
 2: {
 3:     using (SPSite site = new SPSite("http://mysite"))
 4:     {
 5:         using (SPWeb web = site.OpenWeb("myweb"))
 6:         {
 7:             SPListItemCollection items = web.Lists["mylist"].Items;
 8:  
 9:             string[] delimiters = new string[] { "\r\n" };
 10:  
 11:             var term = (from e in items.OfType<SPListItem>()
 12:                         where ((string)e["Term"]).Equals(userSearchTerm, StringComparison.InvariantCultureIgnoreCase) 
 13:                         || ((string)e["Synonyms"]).ToLower().Split(delimiters, StringSplitOptions.RemoveEmptyEntries).Contains(userSearchTerm.ToLower())
 14:                         select e).FirstOrDefault();
 15:  
 16:             if (term != null)
 17:                 return (string)term["Term"];
 18:             else
 19:                 return null;
 20:         }
 21:     }
 22: }

The code isn’t terribly difficult but a few lines probably warrant some explanation. The first thing to understand is how the values are stored in an SPFieldMultiLineText field. Each line is in the field is split using a carriage return/line-feed combination (\r\n). So if I wanted the value of the second item in the list, the value would be represented as FAST ESP\r\nFAST Search for Internet Sites\r\nFAST Search for SharePoint\r\nFSIS. Line 9 sets up the pattern that I’m going to be looking for to treat each line as a separate item.

Lines 11-14 is the LINQ code. In line 11, I first need to convert the items object into a generic, IEnumerable<T> representation of the collection. Even though the SPListItemCollection class implements IEnumerable and LINQ is supposed to work with IEnumerable objects, LINQ didn’t seem to work directly with the SPListItemCollection. It’s almost like I needed to give LINQ a hint what type of items the collection held.

Line 12 checks for an exact match of the user’s query terms to the term stored in the SharePoint list itself. For example, if the user searches for FAST Search, then the second item in the list better be returned.

Line 13 is where I handle the Synonyms multiline field. The first thing I do is convert the entire string in the field to lowercase, then I split the value of that into an array using the ‘\r\n’ combination as the delimiter. I then just need to call the Contains() method to see if the term the user entered is in the array and return the SharePoint list item if it is or null if it isn’t (Line 14 – call to FirstOrDefault()).

That’s pretty much it. Not hard stuff but I think a pretty useful technique.

Building a Federated Search Experience with FAST Search for Internet Sites (FSIS)

12. September 2010 Bart X. Tubalinal FAST Search Comments (0)

With the vast amounts of information available, it can be a challenge for information workers to find in a timely manner the information they need to perform their daily tasks and make decisions. Information can reside in their company’s intranet portal, it can be in various databases, or it could be completely outside of their organization. The time it takes to search each content source individually is time wasted. With various different search platforms (like FAST ESP and SharePoint 2010), content that resides in these various different data sources can certainly be indexed and made available for searching. You can even create a federated search experience. Federated search is essentially the ability to issue a search query across multiple search engines. For example, I can create a federated search in SharePoint so that whenever a user issues a search, the search will be executed not only with SharePoint’s search engine but Bing Search as well and the search results from each engine would be displayed to the user.

Even though there is the ability to query against different search engines, the results from each engine are not combined into a single result set. On a search results page, each search result set would typically have it’s own area where the results would be displayed. For instance, the search results from SharePoint might be displayed in the center area of the page and the search results from Bing might be displayed on the right-hand side of the page.

But what if you wanted to build a search experience so that all the results were displayed in a single result set (i.e., intermingled with each other)? Along with other capabilities, FAST Search for Internet Sites (FSIS) can be used to build this type of search experience. As an example, I will show how to build a FSIS federated search that will execute the user query against FAST ESP, Bing, and YouTube.

FSIS System Overview

Before building the search experience, it is probably best to get a brief overview of the different components that comprise the FSIS system. FSIS is a platform built on top of FAST ESP (Enterprise Search Platform) for creating search-driven experiences and applications and consists of the following parts:

  • FAST ESP – ESP provides the core capabilities for processing, indexing, and searching both structured and unstructured content. For more information about FAST ESP, download the FAST ESP Product Overview Guide.
  • Content Transformation Services (CTS)– CTS extends ESP’s content processing capabilities and sits between ESP and the content sources. In the typical processing workflow, content will be processed by CTS first before being processed by ESP. It is not necessary for content to be processed by CTS. Content can be fed directly into the ESP content processing pipeline.
  • Interaction Management Services (IMS) – Whereas CTS is an extension of ESP’s content processing, IMS extends ESP’s query and results processing capabilities. IMS sits between ESP and your application. In the typical query flow, the query will be sent from the application to IMS which will then process it and transmit it to the query server. After the query server retrieves the results, IMS can then potentially process the results further before delivering it to the application.
  • FAST Search Designer for Visual Studio – The search designer is an extension for Visual Studio that is used to graphically model the processing jobs (called flows) that CTS and IMS runs.
  • IMS UI Toolkit – The toolkit is a set of UI components that can be used to build the search front-ends.The UI components are wired to work with IMS flows and can be used as-is or can be further extended to meet custom needs. The UI components are built using Microsoft AJAX libraries, although some of the components have been packaged as ASP.NET controls.
  • Search Business Manager – The Search Business Manager is a web-based application that allows you to both manage the IMS and CTS system (e.g., configuring hosts and nodes and managing IMS/CTS flows) and for managing the wiring between the application front-end and the IMS flows in IMS-driven search applications.

image

Figure 1 is an example architecture diagram of a typical FSIS-based search application and shows the different paths content can flow from the source to the application. For example, content that needs to be pre-processed before being sent to FAST ESP (where it would be processed further) would go through the CTS module. Content that doesn’t need any pre-processing can be directly sent to FAST ESP. Likewise, IMS can send queries to FAST ESP or bypass ESP completely and send the query directly to the content source (another search engine).

In the example federated search experience we will build, we will use IMS to go to both FAST ESP and to Bing and YouTube directly.

Building the IMS Flow

In order to build the search experience, the first thing we need to do is build the IMS flow. CTS and IMS flows are essentially workflows that define how content or queries and results will be processed. The content or query/results passes through a set of operators that perform some task on it before handing it over to the next operator where it is processed further. Eventually, the processing of the content or query/results completes and is handed off to the consumer.

Flows are built using the FAST Search Designer for Visual Studio extension. To get started, create a new Visual Studio Class Library project (I called my project MyFlows). In Solution Explorer, right-click on the project and select Add –> Flow. In the Add New Item dialog, choose the Interaction Management Subflow item template (under the FSIS category) and name the flow FederatedFlow.flow. When the new flow is added to the project, the designer canvas should be nothing more than a blank page. We need to drop operators from the Interaction Management section in the Visual Studio Toolbox to our canvas to define our flow.

All IMS flows must start with a Flow Input operator and end with a Flow Output operator. IMS flows all work on what is known as the Context object. This context object is a special object that encapsulates the query and the results and other information about the current context (e.g., user information). The context object passes from the UI layer to IMS where it is processed with an IMS flow and returned back to the UI layer where the various components in the UI toolkit use the information inside the context object. The Flow Input operator specifies where the context object enters the flow and the Flow Output operator specifies where the context object exists the flow, after it’s been processed by the other operators.

Drag a Flow Input operator from the toolbox to the canvas. Double-click the operator to bring up the Properties dialog box. The most important part of this operator is defining the context object. Under the Schema section, you should see an empty table.  Add a line in this table with a name of ContextObject and select the ContextObject type from the dropdown under the Type column. When finished, the Schema should look like this:

image

Next, drag a Flow Output operator to the canvas. There is nothing really to change with the Flow Output operator so there’s no need to bring up its Properties dialog.

The next thing you’ll want to add is a Pre Search Result Mixer operator. This operator defines the type of mixing you want the results to have in a federated search. There are five mixing options:

  • Round Robin – results from each system are mixed in turn.
  • Random – randomizes the results from each system.
  • Relevancy – sorted by the relevancy scores returned from each system.
  • Sort By – sorts by the values in a specified field.
  • Stacker – displays N number of results from each system in a stack.

After you’ve got the Pre Search Result Mixer operator on the canvas, select the Connection item from the toolbox and drag a connection from the Flow Input operator to the mixer operator. Double-click the mixer operator to bring up the operator’s properties. Notice under the Schema section how the ContextObject that was defined in the Flow Input operator is already added. Like I mentioned earlier, the context object is what will be passed throughout the IMS flow’s operators.

For this example, we will use Round Robin mixing so select RoundRobin for the Mixer Type. When using this type of mixing, we need to specify what order we want the results from the various systems to display in. For this example, I want the ordering to be a FAST ESP result, a Bing result, and finally a YouTube result. I specify this in the Source Order property. Note that the names you specify here are arbitrary but will need to match the names you give each source with the Result Source Setter operator (which we will cover later). All other properties for this operator can be left to its default. When completed, the properties should look like this:

image

Next drag an ESP Lookup operator to the canvas and create a connection from the Pre Search Result Mixer to this operator. The ESP lookup operator sends the query to FAST ESP and retrieves the results from ESP. To configure this operator, you need to specify the ESP QR server and the search view that IMS will send the query to. Everything else can remain the same (in fact, don’t change anything else but the QR server and the Search view properties).

Now add a Result Source Setter operator to the flow and connect it to the ESP lookup operator. This operator is used to name the source where the result is coming from. We will need to add one of these operators for each source system we have in our flow (in this example, we have three: ESP, Bing, and YouTube). For this operator instance, specify the source name in the Properties dialog to whatever value you specified in the Source order for the Pre Search Result Mixer. For this example, I named the ESP source as ‘esp1’.

Now we want add operators to handle Bing searching. Add the following three operators to the flow: IMS to Bing Search Adapter, Bing Lookup, Bing to IMS Results Adapter, and another Result Source Setter and connect these operators in that order. You also want to connect from the Pre Search Result Mixer to the IMS to Bing Search Adapter. When all of these operators are added to the flow, this part of your flow should resemble this:

image

The IMS to Bing Search Adapter operator transforms the search request from the internal search query format to a query format that the Bing search engine understands. There is nothing you need to change with this operator.

The Bing Lookup operator actually executes the query and returns the results back to IMS. In order to use the Bing service for your custom applications, you will need to create an App ID using the ‘Create an App ID’ link found here: http://www.bing.com/developers. Once you’ve got a Bing App ID, set the App ID property of the Bing Lookup operator to this ID.

The Bing to IMS Results Adapter operator transforms the results from Bing to the common IMS search results format. Here is where you need to map fields from a Bing search result to the fields in the IMS search results format. By default, the following fields are already mapped for you when you drop this operator on the canvas (and we’re not adding any more mappings in this scenario):

Bing Search Results Field IMS Search Results Field
Title title
Description teaser
Url url
DisplayUrl displayUrl

Finally, set the Result Source Setter for Bing to the same name you gave in the Pre Search Result Mixer (‘bing’, in this example).

To add YouTube federated searching, follow the same pattern that was used for Bing but instead of using the Bing-related operators, use the OpenSearch-related operators. After you drop these operators on the canvas and connected them, this part of the flow should look like this:

image

For the Open Search Lookup operator, configure the URL template as: http://gdata.youtube.com/feeds/api/videos?&q={searchTerms?}&max-results={count?}&start-index={startIndex?}&v=2&alt=rss&fields=item(title,author,atom:link). The URL template specifies the search engine-specific url that is used to execute the search and the query parameter values that are specified with open and closed brackets are dynamically replaced by the IMS engine during query-time. In the template specified, the query parameter values will be replaced at runtime with the following:

URL Parameter Description
{searchTerms?} The keywords entered in by the user.
{count?} The number of results to retrieve from the search engine.
{startIndex?} The starting offset from the entire result set to begin the retrieval from.

For instance, if I have 100 results and I specify the count to be 20 and the start index to be 21, that means I want records 21-40 from the search engine.

 

To explain a little bit more about the URL template specifically for YouTube, I found that I needed to specify the data format I wanted back as RSS (using the alt parameter) and I needed to specify very specific fields to return back (&fields=item(title,author,atom:link). If I just allowed the YouTube data service to return back all of the fields, my flow would always error out because the OpenSearch to IMS Results Adapter couldn’t handle the structure of the XML being returned from the service. Check out the following resources for more information regarding the YouTube data service: http://code.google.com/apis/youtube/2.0/developers_guide_protocol.html.

For the OpenSearch to IMS Results Adapter, I specified the following mappings:

YouTube Results Field IMS Results Field
title title
author teaser
link url

Lastly, you’ll want to set the Result Source Setter to the same name you specified in the Pre Search Result Mixer for YouTube results (set to ‘youtube’ in this example).

Now that we’ve specified the query and results processing for each of the federated search systems, we need to add operators to our flow that will merge the results and mix them together. The two operators we want to add are the Search Result Merger and the Search Result Mixer. Drop one of each to the canvas and connect each of the Result Source Setter operators to the Search Result Merger. Then connect the merger operator to the Search Result Mixer operator and finally, connect the mixer to the Flow Output operator. The good news is that there’s nothing that you need to change with any of these operators’ configurations. So the complete flow should look like:

MyFlows.FederatedFlow

image

After you’ve designed the flow, go ahead and build the project. Building the project will deploy the flow to the FSIS server where it can then be used by your web applications.

Building the Web Application

To build the web application, create a new web application project in Visual Studio. A good starting point to use for any IMS-based application is the sample application that’s part of the IMS UI Toolkit (located at C:\Program Files\FAST IMS UI Toolkit). From this sample application, you’ll want to copy over to your app the web.config, and the Components, images and styles folders. You also want to copy and add references to all of the Microsoft.Ceres.*.dll assemblies from the sample application’s bin folder to your own application’s bin folder.

First, open up the web.config and under the AdminNodes element, make sure you have a reference to your FSIS admin server and port. Everything else can be removed.

Under the flowAliases element, you’ll want to add an alias for the flow we just created. The key is the name I want to use to refer to my flow and the value is the full name of my flow. In the appSettings section, there is a setting for the default flow to use. This specifies the default flow that will be used by the IMS search toolkit if a specific flow isn’t specified via code. You can choose to specify the flow we created as the default but in this example, I will specify the flow to use in code.

The relevant sections of my web.config now look like this:

Web.config

 1:  
 2:   <appSettings>
 3:     <add key="defaultFlow" value="esp" />
 4:     <add key="requireFlowAlias" value="false" />
 5:     <add key="preprocessingFlow" value="Microsoft.DefaultPreprocessing" />
 6:     <add key="debugFlow" value="false"/>
 7:     <!-- add key="defaultNodeSet" value="NodeSet1"/-->
 8:     <!-- add key="prefetchNodeSets" value="NodeSet2;NodeSet3"/-->
 9:   </appSettings>
 10:   <AdminNodes>
 11:     <Node Host="pbfast01" Port="17004" />
 12:   </AdminNodes>
 13:   <flowAliases>
 14:     <add key="esp" value="MyFlows.ESPLookupFlow" />
 15:     <add key="federated" value="MyFlows.FederatedFlow"/>
 16:   </flowAliases>

 

After you’ve modified the web.config, using the Visual Studio Add New Item dialog, add a new Generic Handler item to the website root folder and call it Search.ashx. Delete everything that’s pre-created in this file and instead add the following:

<%@ WebHandler Language="C#" Class="Microsoft.Ceres.InteractionUI.Wdm.SearchHandler" %>

This handler is basically a facade that is used by the different IMS UI components to send the query to the IMS system and work with the results that are returned from IMS.

Next, you’ll want to add a web page to the application. On this web page, we are going to add different IMS components that will comprise our search application. The Components folder that we copied over from the sample application to our new application contains all of the UI components that are part of the IMS UI Toolkit. All of these components are implemented using Microsoft AJAX and ASP.NET client templates. There are many components in the toolkit but for this example, we’ll only be working with the SearchForm, NavigationBar and the HitList components. Note also that for some of these components, an equivalent ASP.NET server control exists but not for all of the components.

Below is what the ASPX page should look like (I will go over the more important pieces of the ASPX page):

Default.aspx

 1: <%@ Page Language="C#" AutoEventWireup="true" CodeFile="Default.aspx.cs" Inherits="_Default" %>
 2:  
 3: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 4: <html xmlns="http://www.w3.org/1999/xhtml">
 5: <head runat="server">
 6:     <title>Sample Search</title>
 7:     <!--#include virtual="/Components/Common/Common.html" -->
 8:     <!-- Core Scripts -->
 9:     <script src="/Components/Ajax/MicrosoftAjax.js" type="text/javascript"></script>
 10:     <script src="/Components/Ajax/MicrosoftAjaxTemplates.js" type="text/javascript"></script>
 11:     <script src="/Components/Common/ImsCore.js" type="text/javascript"></script>
 12:     <!-- Search API scripts -->
 13:     <script src="/Components/Common/json2.js" type="text/javascript"></script>
 14:     <script src="/Components/Search/SearchApi.js" type="text/javascript"></script>
 15:     <!-- Component Scripts -->
 16:     <script src="/Components/SearchForm/searchform.js" type="text/javascript"></script>
 17:     <script src="/Components/NavigationBar/NavigationBar.js" type="text/javascript"></script>
 18:     <script src="/Components/HitList/HitList.js" type="text/javascript"></script>
 19:     <script src="/Components/Trace/Trace.js" type="text/javascript"></script>
 20:     <!-- Style Sheets -->
 21:     <link type="text/css" rel="Stylesheet" href="styles/ims.css"></link>
 22:     <link type="text/css" rel="Stylesheet" href="styles/searchapp.css"></link>
 23: </head>
 24: <body>
 25:     <form id="form1" runat="server">
 26:     <div style="text-align:center">
 27:         <div style="width: 800px;">
 28:             <div id="top">
 29:                 <input id="searchterms" type="text" title="Search Terms" size="40" />
 30:                 <input id="search" type="button" value="Search" class="button" title="Search Button" />
 31:             </div>
 32:             <div id="bottom">
 33:                 <div id="resultsPanel" style="text-align: left">
 34:                     <div id="navbar" class="ims-navbar">
 35:                     </div>
 36:                     <div id="hitlist" class="ims-hitlist">
 37:                     </div>
 38:                 </div>
 39:             </div>
 40:         </div>
 41:     </div>
 42:     </form>
 43:  
 44:     <script type="text/javascript">
 45:  
 46:         function pageLoad() {
 47:             // create a search executor object
 48:             var executor = Ims.Search.SearchExecutorFactory('Search.ashx');
 49:  
 50:             //specify the flow to use
 51:             executor.set_flowName("federated");
 52:  
 53:             // create a search input control attached to a the 'searchterms' textbox
 54:             var textbox = $create(Ims.SearchForm.SearchInput, null, null, null, $get("searchterms"));
 55:  
 56:             // create a search builder to construct the search
 57:             $create(
 58:                 Ims.SearchForm.SearchBuilder,
 59:                 { searchExecutor: executor, searchInputs: [textbox], submitButton: $get('search') },
 60:                 null, null, null
 61:             );
 62:  
 63:             // create a HitList control and attach it to the <div id="hitlist"> element
 64:             var hitListObj = $create(Ims.HitList, { searchExecutor: executor }, null, null, $get("hitlist"));
 65:  
 66:             // create a NavigationBar control and attach it to the 'navbar' element
 67:             $create(Ims.NavigationBar, {
 68:                 searchExecutor: executor,
 69:                 hitList: hitListObj,
 70:                 maxPageCount: 15,
 71:                 templateId: "NumberedPagedNavigationBarTemplateID",
 72:                 numericalPageTemplateId: "NumberedPageTemplateID",
 73:                 noHitsTemplateId: "NoHitsNumberedPagedNavigationBarTemplateID"
 74:             },
 75:             null,
 76:             null,
 77:             $get("navbar"));
 78:  
 79:             // set the focus to the search box
 80:             $get('searchterms') && $get('searchterms').focus();
 81:         };
 82:  
 83:     </script>
 84:  
 85:     <!--#include virtual="/Components/Hitlist/MyHitList.html" -->
 86:     <!--#include virtual="/Components/NavigationBar/MyNumberedPagedNavigationBar.html" -->
 87: </body>
 88: </html>

Like previously mentioned, the UI components use ASP.NET client templating capabilities to render the output. Client  templates are defined using a combination of HTML elements and CSS styles along with placeholders where the data will go when the records are bound to the template. Each component you use should have a client template. It is easiest to make a copy of the out of the box client templates that come with the toolkit and modify them to suit your needs. The templates for this application are defined in the external files ‘/Components/Hitlist/MyHitList.html’ and ‘/Components/NavigationBar/MyNumberedPagedNavigationBar.html’. To use these templates, we use the #include directive on lines 85 and 86.

When the page is first rendered, however, you need to make sure the HTML and CSS that are part of the templates are not displayed since they’ve not been filled with data yet. That is what the Common.html file in line #7 does, it sets the visibility of all elements marked with the class ‘sys-template’ (you must mark your templates with this class) to hidden.

Lines #9-22 add the various scripts and stylesheets needed by the IMS UI components on the page. Lines 9-14 must always be included. Each component in the UI toolkit has its own script file. You only need to add the scripts for the components you’re using on the page. In this case, I’ve added the component scripts for the Search Form, the Navigation Bar, and the Hit List.

Lines #26-41 is where I define the layout of the page. There are different divs on the page that act as placeholders for my various components. Lines #29-30 define my search box and search button, line #34 is for my navigation bar and line #36 is for my hit list where the search results will be placed.

Lines #44-83 is the javascript needed to make the page functional. In the pageLoad() function, I create an instance of an ISearchExecutor object through the SearchExecutorFactory (passing in the name of the handler that we created earlier). I then set the executor to use the federated flow when processing the query (line 51).

In lines #54-61, we are configuring the search textbox and search button as part of the search form. First, at #54, we are defining the text box as a search input (you can have multiple inputs for a search form). Then, in lines #57-61, we create a SearchBuilder object and point its searchInput property to the textbox and its submitButton to the search button. When a search term is entered into the textbox and the search button is pressed, this SearchBuilder will take care building the query based on the inputs and firing off the query using the executor.

Line #64 is where we create an Ims.HitList object and tie it to the hitlist div element on the ASPX page.

Lines #67-77 creates an Ims.NavigationBar object and ties it to the navbar div element on the page. For the NavigationBar, we specify what HitList object it works with, how many pages to show at a time, and the element ids of the elements in the MyNumberedPagedNavigationBar.html template.

Below are the two different templates used by the application:

MyHitList.html

 1: <script type="text/javascript">
 2:     function getLink(originalUrl) {
 3:         var posV = originalUrl.indexOf('v=');
 4:         var posAmp = originalUrl.indexOf('&', posV);
 5:  
 6:         var newLink = 'http://www.youtube.com/v/' + originalUrl.substring(posV + 2, posAmp);
 7:         return newLink;
 8:     }
 9:  
 10:     function createMovie(parent, link) {
 11:         parent.innerHTML = '<object width="400" height="275" type="application/x-shockwave-flash">' +
 12:         '<param name="movie" value="' + link + '"></param>' +
 13:         '<param name="allowFullScreen" value="true"></param>' +
 14:         '<param name="allowscriptaccess" value="always"></param>' +
 15:         '</object>';
 16:     }
 17: </script>
 18:  
 19: <div id="HitListTemplateID" class="sys-template">
 20:     <div>{{title}}</div>
 21:     <!-- these two need to be here, for scrolling to work.  Don't change -->
 22:     <div sys:id="{{outerdivid}}">
 23:         <div sys:id="{{hitlistcontainerid}}">
 24:         </div>
 25:     </div>
 26: </div>
 27: <div id="HitListContainerTemplateID" class="sys-template">
 28:     <div sys:id="{{hitlistitemcontainerid}}" class="sys-template">
 29:         <div class="hitlist-item" sys:id="{{itemid}}">
 30:             <div sys:if="hitSource.toLowerCase().indexOf('bing') != -1 || hitSource.toLowerCase().indexOf('esp') != -1">
 31:                 <div class="ims-hit-list-title" style="margin-bottom: 6px; font-size: 1.2em">
 32:                     <img sys:if="hitSource.toLowerCase().indexOf('bing') != -1" class="ims-hitlist-source"
 33:                         src="http://www.bing.com/siteowner/s/siteowner/icon.png" border="0" />
 34:                     <img sys:if="hitSource.toLowerCase().indexOf('esp') != -1" class="ims-hitlist-source"
 35:                         src="images/esp.gif" border="0" />
 36:                     <a sys:href="{{fields.url}}" class="ims-hit-list-title">{{itemtitle}}</a>
 37:                 </div>
 38:                 <div sys:if="fields.teaser != undefined">
 39:                     <b>Description:</b> {{fields.teaser}}
 40:                 </div>
 41:                 <div style="font-size: .9em; margin-top: 5px; margin-bottom: 5px;">
 42:                     <div style="color: Green;">
 43:                         <span style="font-weight: bold">Source:</span> {{fields.url}}
 44:                     </div>
 45:                     <div sys:if="fields.size != undefined" style="color: Green;">
 46:                         <span style="font-weight: bold">Size:</span> {{fields.size / 1000}}KB</span></div>
 47:                 </div>
 48:             </div>
 49:             <div sys:if="hitSource.toLowerCase().indexOf('youtube') != -1">
 50:                 <div class="ims-hit-list-title" style="margin-bottom: 6px; font-size: 1.2em">
 51:                     <img class="ims-hitlist-source" src="images/youtube.png" border="0" />
 52:                     <a sys:href="{{fields.url}}" class="ims-hit-list-title">{{itemtitle}}</a>
 53:                 </div>
 54:                 <div>
 55:                     <b>by:</b> <a sys:href="{{'http://www.youtube.com/user/' + fields.teaser }}">{{fields.teaser}}</a>
 56:                 </div>
 57:                 <div sys:codeafter="createMovie($element, getLink($dataItem.fields.url));">
 58:                 </div>
 59:                 <div style="font-size: .9em; margin-top: 5px; margin-bottom: 5px;">
 60:                     <div style="color: Green;">
 61:                         <span style="font-weight: bold">Source:</span> {{fields.url}}
 62:                     </div>
 63:                 </div>
 64:             </div>
 65:         </div>
 66:     </div>
 67:     <br />
 68: </div>
 69: <div id="NoHitsTemplateID" class="sys-template">
 70:     There were no matching documents for the current search.</div>

MyNumberedPagedNavigationBar.html

 1: <!-- The default template rendered by the NavigationBar control once a search completes -->
 2: <div id="NumberedPagedNavigationBarTemplateID" class="sys-template">
 3:     <div class="ims-navbar-info">
 4:         Found {{results.get_documentCount()}} results ({{results.get_searchTime()/1000.0}}
 5:         seconds)</div>
 6:     <div class="ims-navbar-navbuttons">
 7:         <img src="images/1px.gif" width="20px" height="0px" alt="Transparent image to provide spacing" />
 8:         <a href="#" class="ims-navbar-navbuttons">
 9:             <img src="/images/first.png" alt="First page of hits" title="First page of hits"
 10:                 sys:id="{{navbar_first_page_id}}" border="0" />
 11:         </a>&nbsp; <a href="#" class="ims-navbar-navbuttons">
 12:             <img src="/images/previous.png" alt="Previous page of hits" title="Previous page of hits"
 13:                 sys:id="{{navbar_previous_page_id}}" border="0" /></a>&nbsp; <span sys:id="{{navbar_num_page_parent_id}}">
 14:                 </span>&nbsp; <a href="#" class="ims-navbar-navbuttons">
 15:                     <img src="/images/next.png" alt="Next page of hits" title="Next page of hits" sys:id="{{navbar_next_page_id}}"
 16:                         border="0" /></a> <a href="#" class="ims-navbar-navbuttons">
 17:                             <img src="/images/last.png" alt="First page of hits" title="First page of hits" sys:id="{{navbar_last_page_id}}"
 18:                                 border="0" /></a>&nbsp;
 19:     </div>
 20: </div>
 21: <div id="NumberedPageTemplateID" class="sys-template">
 22:     <a href="#" sys:id="{{navbar_num_page_id}}" sys:class="{{current_page == num_page ? 'ims-navbar-current_page_number' : 'ims-navbar-numbers'}}">
 23:         {{num_page}}</a>
 24: </div>
 25: <!-- The template rendered by the NavigationBar control when a new search begins (i.e. when no navigation information is available -->
 26: <div id="NoHitsNumberedPagedNavigationBarTemplateID" class="sys-template">
 27:     <div>
 28:         &nbsp;</div>
 29: </div>

There’s really not too much to discuss with these files. With the hitlist template, I check to see what the hitSource is for each item and create the right elements and styles for each type of result (esp, bing, or youtube). Notice that the fields of a result item are available via the fields object (e.g., fields.teaser, fields.url, etc). Also, I created a javascript function that creates the object element necessary to display the videos from YouTube. The reason why you want to build the object element dynamically is because if you just added the object tag directly, like the other elements in the template, then Internet Explorer tries to load it even if there’s no video specified yet for the object (in other words, it will try to load something for the object element when the page is first loaded, before any search takes place).

The Final Product

Once the IMS flow and the application is built, this is the final result (you can also view a video demo of the application here):

Search Results

image

Conclusion

Organizations need to investigate more ways to build effective search applications to provide their users with the information they need to perform their daily tasks and to make critical business decisions. Building a federated search solution is just one of the many things that FAST Search for Internet Sites enables organizations to build compelling, search-centric applications. To learn more about FSIS and other Microsoft Search products, please visit http://sharepoint.microsoft.com/en-us/Pages/Videos.aspx?VideoID=26 and http://sharepoint.microsoft.com/en-us/product/capabilities/search/Pages/default.aspx.

If you are interested in a copy of the source files used for this project, please send me an email at btubalinal@pointbridge.com.

 

Sources