coldfusionJedi

<cfsavecontent variable="string">
This is a paragraph with some text in it. Certain words will be repeated, and other words
will not be repeated. The question is though, how much can I write before I begin to sound
like a complete and utter idiot. Let's call that the "Paris Point". At the Paris Point, any
further words sound like gibberish and are completely worthless. 
</cfsavecontent>

<cfset words = reMatch("[[:word:]]+", string)>

<cfset wordCount = structNew()>

<cfloop index="word" array="#words#">
	<cfif structKeyExists(wordCount, word)>
		<cfset wordCount[word]++>
	<cfelse>
		<cfset wordCount[word] = 1>
	</cfif>
</cfloop>

<cfset sorted = structSort(wordCount, "numeric", "desc")>

<table border="1" width="400">
<tr>
	<th width="50%">Word</th>
	<th>Count</th>
</tr>

<cfloop index="word" array="#sorted#">
	<cfoutput>
	<tr>
		<td>#word#</td>
		<td>#wordCount[word]#</td>
	</tr>
	</cfoutput>
</cfloop>

This entry was posted on Thursday, August 2, 2007 at 12:19 PM. It was filed in the following categories: ColdFusion. It has been viewed 4,090 times and has 27 comments.

TweetBacks

Comments

[Add Comment] [Subscribe to Comments]

• Comment 1 written by nick on 2 August 2007, at 12:31 PM

  

  If "Paris Point" becomes part of the daily lexicon, you can officially coin it. Nice code work too.

• Comment 2 written by Ben Nadel on 2 August 2007, at 12:32 PM

  

  REMatch() makes me happy :)

• Comment 3 written by Quan Tran on 2 August 2007, at 12:32 PM

  

  Probably not faster, but you could create a query with a single column and use qoq to get the count with a group by.

• Comment 4 written by Gareth on 2 August 2007, at 12:54 PM

  

  Couldn't you do something like
  #ListLen(string, " #Chr(13)##Chr(10)#")#
  
  (it seems to work with the string variable you posted)

• Comment 5 written by Raymond Camden on 2 August 2007, at 12:58 PM

  

  Gareth, that counts the words. We need a count of the number of each word. Ie, the string has The ten times. Etc.

• Comment 6 written by Gareth on 2 August 2007, at 1:02 PM

  

  Whoops, unique instances...
  
  Let me try that again :)
  
  <cfset new_string = ListSort(REReplaceNoCase(LCase(string), "[^a-z ]", "", "ALL"), "Text", "Asc", " #Chr(13)##Chr(10)#")>
  
  <cfscript>
  // had to use this as CF does not allow lookbehind in regular expressions, but JAVA does
  obj = createobject("java","java.util.regex.Pattern"); // create pattern searching object
  x = obj.compile("(?<=[ ]|^)([^ ]*)([ ]\1)+(?=[ ]|$)"); // compile the regular expression for use
  new_string = x.matcher(new_string).replaceAll("$1"); // remove all duplicates
  </cfscript>
  
  #ListLen(new_string, " ")#

• Comment 7 written by Gareth on 2 August 2007, at 1:05 PM

  

  OK, I'm going to stop now :)
  I got a total of the unique words, but not a count of the number of duplicate words (that's what I get for trying to do write code for one thing while checking out the blogs in another tab :) )

• Comment 8 written by todd sharp on 2 August 2007, at 1:49 PM

  

  Issue: the word "Let's" gets broken into "Let" and "s" because of your RE. Solution? Still working on it... ;)

• Comment 9 written by Ben Nadel on 2 August 2007, at 1:56 PM

  

  @Todd,
  
  I ran into that same problem during one of Ray's Friday Puzzlers... trust me - don't try to figure it out, your brain will only end up hurting. Here's why, these are all single "words":
  
  hatin'
  let's
  sweet-ass
  cf.objective()
  O'connell
  
  ... if you can write an algorithm to use all those "non-word" characters as parts of words, well then, you are the man!

• Comment 10 written by Raymond Camden on 2 August 2007, at 2:09 PM

  

  Although, one could argue that sweet-ass wouldn't be so bad as two words. hatin' is slang, and would become hatin, which is ok.
  
  I think if you could just make get single quotes to work, you would get most "real" words.
  
  I wonder - maybe switch from [[:word:]] to
  
  (any non alpha except single quote)(alpha,1 or more)(optional ' if followed by alpha)(any non alpha except single quote)
  
  Then again - another solution? Remove '. You end up with words like "lets", which could be confused with "Ray lets Paris call him", but it would be better than let and s as words.

• Comment 11 written by noname on 2 August 2007, at 2:26 PM

  

  But, isn't the next word signified by a space? So everything between a space, is a word?

• Comment 12 written by Ben Nadel on 2 August 2007, at 2:27 PM

  

  Yeah, stripping out the single quotes is probably the easiest thing to do. Least amount of damage for the best results.

• Comment 13 written by Ron Alexander on 2 August 2007, at 3:23 PM

  

  So why not just change from "[[:word:]]" to "[a-zA-Z0-9]+'[a-zA-Z0-9]+|[a-zA-Z0-9]+"
  
  That seems to do the trick for let's lets. But it still doesn't make the counting any more elegant.

• Comment 14 written by Raymond Camden on 2 August 2007, at 3:27 PM

  

  Thats pretty cool there Ron.

• Comment 15 written by Jonathon on 2 August 2007, at 3:47 PM

  

  There's a CF IRC channel floating around somewhere? Anyone feel like sharing the info? :)

• Comment 16 written by Raymond Camden on 2 August 2007, at 3:50 PM

  

  The one I use is #coldfusion on Dalnet.

• Comment 17 written by Ron Alexander on 2 August 2007, at 3:59 PM

  

  This is better:
  
  (?:[a-zA-Z0-9\(\)])+(?:\'|-|\.)?(?:[a-zA-Z0-9\(\)])+
  
  It matches method chains like myarray.dedup().sort()

• Comment 18 written by Ron Alexander on 2 August 2007, at 4:02 PM

  

  And just to match Ben's "hatin'" example:
  "(?:[a-zA-Z0-9\(\)])+(?:\'|-|\.)?(?:[a-zA-Z0-9\(\)\-\'])+"
  
  Don't forget (like I did) to throw in the \'\- into the last non-capturing group.
  
  Ben does that meet your needs?

• Comment 19 written by Dustin on 2 August 2007, at 4:07 PM

  

  This is probably how I would do it:
  
  <cfset string = reReplace(string,'(\.|"(?=\w))','','all') />
  <cfset wordAry = listToArray(string,'#chr(10)##chr(13)##chr(32)#') />
  <cfset wordQry = queryNew('word','VarChar') />
  <cfloop from="1" to="#arrayLen(wordAry)#" index="i">
     <cfset queryAddRow(wordQry) />
     <cfset querySetCell(wordQry,'word',reReplace(wordAry[i],'[",]$','')) />
  </cfloop>
  <cfquery dbtype="query" name="uniqueWords">
     SELECT word, count(*) as wordCount FROM wordQry group by word order by wordCount desc
  </cfquery>
  <cfdump var="#uniqueWords#">

• Comment 20 written by db on 3 August 2007, at 8:59 AM

  

  this seems to work for me:
  <cfset words = arrayToList(string.split('\s'))>
  <cfset wordCount = structNew()>
  <cfloop index="word" list="#words#">
  <cfset wordCount[word] = ListValueCountNoCase(words, word)>
  </cfloop>

• Comment 21 written by db on 3 August 2007, at 10:28 AM

  

  no, that's not right - its including punctuation as part of the word. so i tried with the "hatin'" list and got this working:
  <cfset words = arrayToList(string.split("\.?[[^()]\s&&([""()][\s])]"))>

• Comment 22 written by Johnny on 4 August 2007, at 2:31 PM

  

  Wouldn't this work?
  
  <cfset wordcount = structNew()/>
  <cfloop list="#string#" delimiters=' ,"' index="word">
     <cfset word = replaceList(word,"',.","")/>
     <cfif structKeyExists(wordCount, word)>
  <cfset wordCount[word] = wordCount[word] + 1/>
  <cfelse>
  <cfset wordCount[word] = 1/>
  </cfif>
  </cfloop>

• Comment 23 written by Mike Cohen on 25 August 2008, at 11:06 AM

  

  Why not just do:
  
  <cfset myString = "blah blah blah bldfadsff fd ">
  <cfset mCounter = stringToArray(myString,"a")>
  <cfset numberOfAs = arraylen(mcounter)>
  
  ?

• Comment 24 written by todd sharp on 25 August 2008, at 11:25 AM

  

  Unless I'm missing something in an earlier comment, there is no strintToArray() function in ColdFusion...

• Comment 25 written by todd sharp on 25 August 2008, at 11:26 AM

  

  Make that stringToArray()...

• Comment 26 written by D. Davis on 23 February 2009, at 4:54 PM

  

  Nice article, and thanks guys for the different ways of doing this.
  
  Wanted to note: the sort on this ("textnocase") needs to be "numeric","desc" otherwise you're not getting your top numbers right (ie, textnocase sort would look like 4,3,20,17).
  
  Great code on this as a first step to making a word cloud, looping it on DB-pulled text fields.

• Comment 27 written by Raymond Camden on 25 February 2009, at 4:10 PM

  

  Oops. Thanks D. Fixed in the code above.

[Add Comment] [Subscribe to Comments]