Transfer Query - From Phrase to AND Search

Not sure if anyone will find this helpful, but here goes. A user reported some dissatisfaction with the search engine at CFLib. Specifically a search for FOO GOO would result in an exact phrase match on foo goo. It would not match a UDF named MyFooMilkshakeBetterThanGoo.

This was because my search did the rather simple %SEARCH% style match (ie, match the search term exactly, case insensntive). Here is the original Transfer code I used for the search:

<cfsavecontent variable="sql">
from udf.udf
where (udf.udf.name like :search
or udf.udf.shortdescription like :search
or udf.udf.description like :search)
and udf.udf.released = :active
order by udf.udf.name asc
</cfsavecontent>

<cfset q = variables.transfer.createQuery(sql)>
<cfset q.setParam("search", "%" & arguments.search & "%", "string")>
<cfset q.setParam("active", true, "boolean")>
<cfset q.setCacheEvaluation(true)>

Even if you don't know Transfer, this should be readable to you. I do a match on either name, shortdescription, or description. If you search for FOO, then basically it ends up being a search on %FOO%. Again though this kind of breaks down when you search for multiple words.

I wanted to keep things simple, so I decided that any multiword search would be an AND style search (ie, all the words must match), and I'd split on an empty space. I rewrote the TQL like so:

<cfsavecontent variable="sql">
from udf.udf
where (
   <cfloop index="idx" from="1" to="#arrayLen(words)#">
   <cfoutput>
   (udf.udf.name like :word#idx#
   or udf.udf.shortdescription like :word#idx#
   or udf.udf.description like :word#idx#
   ) <cfif idx lt arrayLen(words)>and</cfif>
   </cfoutput>
   </cfloop>
)
and udf.udf.released = :active
order by udf.udf.name asc
</cfsavecontent>
   
<cfset q = variables.transfer.createQuery(sql)>
<cfloop index="idx" from="1" to="#arrayLen(words)#">
   <cfset q.setParam("word#idx#", "%" & words[idx] & "%", "string")>
</cfloop>
<cfset q.setParam("active", true, "boolean")>
<cfset q.setCacheEvaluation(true)>

First off, the variables words comes from this change in the function header:

<cfset var words = listToArray(arguments.search," ")>

So a search for FOO GOO results in words being equal to ["FOO","GOO"]. Notice then my loop over the array. I didn't use the new array style cfloop as I wanted a counter variable I could check to see if I needed an AND at the end of each block.

The next change was to have a dynamic set of setParams. This will replace word1, word2, etc, with the proper value from the array.

A good test for this is a search for "host url". Before the change it returned nothing. Now it matches getCurrentURL, getHostFromURL, and getHostFromURLJava.

Comments

Timothy Farrar's Gravatar Great feature Ray, I'm not using transfer, but have wished a number of times that this feature existed on cflib.

Any chance of adding this type of functionality to the search on riaForge?
# Posted By Timothy Farrar | 9/23/08 9:12 AM
Raymond Camden's Gravatar Absolutely. I'll be changing from client side ajax search, which worked great when we had < 200 projects, to a server side search (we now have 550+). It's "on the list", along with expanding the capabilities for submanagers. Which I got no feedback on from my users, so I hope someone is using it. ;)
# Posted By Raymond Camden | 9/23/08 9:15 AM
Timothy Farrar's Gravatar John and I are using it... and it works great for what we need so far... thanks :)
# Posted By Timothy Farrar | 9/23/08 9:18 AM
Raymond Camden's Gravatar Heh, glad someone is. ;) Well, as you know, it gives you very little power now. I'd like to finish that area first before going to search, but at the same time, search is a bit more public and would probably be used a lot more.
# Posted By Raymond Camden | 9/23/08 9:21 AM
Nathan D's Gravatar And what happened to author pages and author search?
# Posted By Nathan D | 9/23/08 9:50 AM
Raymond Camden's Gravatar Eh? You mean CFLib or RIAForge? I used to have author search on an 'advanced' search page. I could return that. I don't think I ever had an author 'page' though.
# Posted By Raymond Camden | 9/23/08 9:59 AM
Aegis's Gravatar Probably getting to the point an author search could be really handy :)
# Posted By Aegis | 9/23/08 9:46 PM