Fixing the Facebook Problem, and why one ColdFusion feature needs to die...

I spent some time today working with Anthony Webb and Ben Nadel on a rather interesting problem. Anthony was trying to build a Facebook application. (More info may be found at the cftalk thread.) Facebook's test application sends a form POST to the file you want to respond to it's request. This form POST includes a set of form variables. Here are two of the variables:

FB_SIG FB_SIG_TIME

Among other variables as well. The problem was though that he would get an error whenever the POST was made. Why? Well, there is an ancient ColdFusion feature that lets you do form validation with specifically named form fields:

Validating form fields using hidden form fields

No one uses this feature. Seriously - no one. In fact, the only time you hear it mentioned is when someone accidentally names a form field wrong and trips this validation. When I said in the blog title that this was a feature that needed to die - I was only being partially sarcastic. I hope that ColdFusion 9 will either dump this "feature" or at least let us turn it off with a Application.cfc variable. (I know ColdFusion is all about the backwards compatibility, but come on, it's time to dump this along with parameterExists!)

Ok - so enough ranting. We can't turn it off - so what do we do? Well the first thing I tried was a check in onRequestStart. Nope - even though this is supposedly "before" the request, it isn't before ColdFusion's magical form check. I then tried onRequest. No dice there.

So I knew that there was a cferror tag just for this type of error - so I tried onError. Success! I was able to trap this error and check for a Facebook post and then simply include the file to handle the request:

<cffunction name="onError" returnType="void" output="true">
   <cfargument name="exception" required="true">
   <cfargument name="eventname" type="string" required="true">
   <!--- look for FB post --->
   <cfif findNoCase("Form entries are incomplete or invalid", arguments.exception.message)
      and
    structKeyExists(form, "fb_sig")>
      <cfinclude template="testi.cfm">
      <cfreturn>
   </cfif>            
   <cfdump var="#arguments#">
</cffunction>

This worked fine, but was a bit ugly. Here is where Ben Nadel came up with his solution. Apparently if you muck with the Form scope in the Application.cfc constructor, you can fix the problem. Remember that the constructor is any area outside of the methods. So by just adding this for example:

<cfset structDelete(form, "fb_sig")>

The problem would go away... along with the form field. I liked this solution better than my onError code, so wrote up a quick UDF that could be called from the constructor area. My thinking was that this would keep your constructor area a bit more tidier.

So now you can add:

<cfset fixFacebook()>

And run the method I wrote. But here is where things got wonky. For my solution I decided to simply rename all form keys from FOO to FOO_X. I didn't need to rename them all, but I figured doing them all would make it simpler. My code though continued to throw the error. I dumped the form scope and saw that all my fields were renamed, but I still got NPEs with the form validation. Freaky.

So I then changed to setting the old form fields to an empty string, and that for some silly reason worked. Here is what I ended up with:

<cffunction name="fixFacebook" returnType="void" output="false" hint="Attempt to fix a facebook post.">
   <cfset var f = "">
   <!--- detect if FB is posting to us --->
   <cfif structIsEmpty(form)>
      <cfreturn>
   </cfif>
   <!--- check for one key now- maybe check for more later? --->
   <cfif structKeyExists(form, "FB_SIG")>
      <!--- loop through and rename all to _x --->
      <cfloop item="f" collection="#form#">
         <cfset form[f & "_x"] = form[f]>
         <cfset form[f] = "">
      </cfloop>
   </cfif>
</cffunction>

Maybe overkill a bit - but I'm the kind of guy who would probably write a method to cross the street.

Anthony correctly then made all of this a lot simpler with this code snippet, no UDF, but nice and tight and works fine with Facebook:

<cfif structKeyExists(form, "FB_SIG")>
<cfset form.FB_SIG_FIX = form.FB_SIG>
<cfset form.FB_SIG = ''>
</cfif>

So - anyone writing Facebook applications in ColdFusion yet? I think I may give it a whirl now.

p.s. So I obviously named my title a bit over the top - but this weekend I'll be blogging on something I saw on DZone - specifically - what features should be avoided in ColdFusion. Maybe none should be. Maybe you have you own pet peaves. Think about it and be ready to post.

Comments

Hey Ray, Glad the gladiators are looking into this :). I have been working with Anthony on this also and supplied him my starter template for Facebook, which of course works in CF7:

http://dev.dominicwatson.co.uk/downloads/facebookt...

I don't have a CF8 server to test on as yet but two things:

1. I am quite certain that Anthony's solution that you quoted will not work. I think he was summarising the same thing as you did with the loop on all form fields, i.e.

<cfset form[f & "_x"] = form[f]>
<cfset form[f] = "">

2. If this solution works in CF8 then great! It does not work in CF6.1-7 though. Instead, you must use this (which doesn't work in CF8):

<cfset form[f & "_x"] = form[f]>
<cfset StructDelete(form,f)>

Regards,

Dominic

p.s. I'm just reworking my template now to reflect the CF8 issues, will be publishing it somewhere proper soon.
# Posted By Dominic Watson | 9/21/07 5:27 PM
I must aplogise, Anthony was entirely correct and now I see why ;)

Awesomeness all round!
# Posted By Dominic Watson | 9/21/07 5:30 PM
Ray,

Not sure you if were aware or not but when you use <cfinput type="text" validateAt="onServer" /> ColdFusion generates those exact same tags (hidden fields). So I wouldn't necessarily say nobody is using it, they just may be using it and not know. I know you despise built-in CF validation though. I have yet to have anyone give any valid arguments against it so I would be curious in knowing why it is "evil"
# Posted By TJ Downes | 9/21/07 6:45 PM
Why it is evil, by Dominc Watson.

If an external application sends you a post request containing form fields that match the server-side validator criteria, an error will most likey be thrown. As a 3rd party developer, you may have no control over the names of those post variables and have to hack around it as pointed out in this Facebook example.

If there was a switch to turn it off, or an interface to change the naming rules, I wouldn't have a problem with it.
# Posted By Dominic Watson | 9/21/07 7:09 PM
Yeah, I use validateat some. It has its own bug too. If you run without JS, submit the form, it presents you with a link on the subsequent error page, which is a JS link. It doesn't take you back because you have JS turned off! :)

Will
# Posted By Will Tomlinson | 9/21/07 8:50 PM
Ray, thanks for the info! I'm about to start prototyping a Facebook app this weekend and I just know that would have had me tearing my hair out.
# Posted By Kay Smoljak | 9/21/07 9:59 PM
TJ, in the example you gave, you are specifically asking for server side validation. That is different than someone accidentally naming their fields in such a way as to trigger this behavior.
# Posted By Raymond Camden | 9/21/07 10:01 PM
Instead of pulling your hair out you could have looked around;
http://www.coldfusiondeveloper.com.au/go/blog/2007...

Do check out this in a few days: http://coldface.riaforge.org
I haven't run in to any CF7/8 problems per say. I'm attempting to 'wrap up' the various facebook quirks so we can just get on and build CF apps. Wooo.
# Posted By MrBuzzy | 9/22/07 8:33 PM
Heh, speaking for myself, but I bet Ben N would agree, it was more fun trying to solve the problem myself. ;)
# Posted By Raymond Camden | 9/22/07 9:24 PM
Of course :)
Looking again at that post, you were there anyway ;)
# Posted By MrBuzzy | 9/22/07 9:27 PM
I wouldn't sy that nobody uses the built-in server side validation. It can actually be quite a time saver for folks. hat said, I agree entirely that it'd be nice to be able to turn it off.
# Posted By Simon Horwith | 9/23/07 7:25 AM
I definitely would agree :) Solving problems is much for fun than looking them up. Plus, a problem solved is much better remembered than a solution learned.
# Posted By Ben Nadel | 9/23/07 11:05 AM
For anyone looking at building Facebook apps using FBML, the headache has been borne for you:

http://fbmlstarter.riaforge.org/

I made a final version of this yesterday after fixing the bug mentioned in this article (it was working before with CF7)

Enjoy!
# Posted By Dominic Watson | 9/23/07 3:35 PM
HA! I had this same problem a very long time ago, must be on CF5 as there were no CFC's back then and I solved pretty much the same way but using application.cfm
I also did actually figure out how to disable this validation check on the server, but even if I could remember how I did it wouldn't on current version of CF, so I wont bother trying to remember :-)
# Posted By Russ "Snake" Michaels | 9/27/07 3:59 PM
I've been ranting about this since CF5 and every release this auto validation has crept up and bit me somewhere.
I personally think its rude imposing such a 'feature' on people without asking. Not to mention this without such hacks this renders CF almost unusable for 3rd party integration as you never know what will hit you and when...
I just can't get why nobody ever did anything about it. A simple switch to turn this off from cfadmin (or another bloody hidden form field) is all that's needed!
This makes CF looks so silly...
# Posted By Harel Malka | 11/15/07 10:13 AM
I am still getting a 'Form entries are incomplete or invalid.' response when using Dominic's template application. I've literally changed nothing in his template except for putting in my api key, secret key, and fixing the name in the <cfapplication> tag. I'm using CF 8 - any ideas?
# Posted By Tim | 12/13/07 2:32 PM
Hey Tim, are you using the very latest version downloaded from Riaforge?

If so, you can mail me at watson dot dominic at googlemail dot com. Maybe we can work it out (this beast never seems to die!).

Dominic
# Posted By Dominic Watson | 12/13/07 3:13 PM
Yup, sure am - I'll shoot ya something.
# Posted By Tim | 12/13/07 3:15 PM
Also watch out for FORM.FB_SIG_PROFILE_UPDATE_TIME.
# Posted By MrBuzzy | 12/17/07 2:06 AM
Actually, what is great about Anthony's simple fix is that this newish field is not a problem unless there is also a form field called FB_SIG_UPDATE_PROFILE.

It is down to the way the validation works. i.e. You can submit this form without causing an error:

<h1>Facebook test</h1>
<cfif IsDefined('form.fieldnames')>
   <cfdump var="#form#">
</cfif>
<form action="" method="post">
   <input type="hidden" name="FB_SIG_TIME" value="646574.413">
   <input type="submit" value="Test it!">
</form>

If you do this however, you get the validation error:

<h1>Facebook test</h1>
<cfif IsDefined('form.fieldnames')>
   <cfdump var="#form#">
</cfif>
<form action="" method="post">
   <input type="hidden" name="FB_SIG_TIME" value="646574.413">
<input type="hidden" name="FB_SIG" value="646574.413">
   <input type="submit" value="Test it!">
</form>

This demonstrates that the _TIME fields are not being validated, it is any field with the same name WITHOUT the _TIME that gets validated. If you blank that field in Application.cfm it will pass the validation.

Dominic
# Posted By Dominic Watson | 12/17/07 2:33 AM
I solved the problem with:

<!--- 1. cf6+ tries server-side form validation on various form field names, fix: --->
   <cfset s_trouble = "integer,float,range,date,time,eurodate,key,expires,added,friends,canvas,user,method,fieldnames,fix">
   <cfloop list="#StructKeyList(form)#" index="formField">
      <cfif ListFindNoCase(s_trouble, ListLast(formField,'_'))>
         <cfset StructInsert(form, formField & '_CFFIX', form[formField])>
         <cfset StructDelete(form, formField)>
      </cfif>
   </cfloop>
# Posted By Huge | 1/21/08 4:59 AM
it was an error, i had a cfabort.

Sorry, i continue with it.
# Posted By Huge | 1/21/08 5:15 AM
Now I found a solution:

Maybe some of ours have in Application.cfm(cfc) this code:

<cfif StructKeyExists(form, "fb_sig")>
   <cfinclude template="fb_post.cfm">
<cfelse>
   <cfinclude template="fb_get.cfm">
</cfif>

And when CF procces de page, load de index.cfm page.

It will be the normal processing, but I dont know why, but if you write this:

<cfif StructKeyExists(form, "fb_sig")>
   <cfinclude template="fb_post.cfm">
<cfelse>
   <cfinclude template="fb_get.cfm">
</cfif>

<cfinclude template="index.cfm">
<cfabort>   

The page loads correctly... and why?? I dont know, but it loads....

Urgggg!!!
# Posted By Huge | 1/21/08 8:03 AM
Hey Huge, looks like you're using the Facebook starter kit. You shouldn't need to do anything. As mentioned in this blog, the fix you are using crashes in CF8. The most straightforward solution is to do this in Application.cfm/cfc (works in 6.1 - 8):

<cfif StructKeyExists(form, "FB_SIG")>
<cfset form.FB_SIG_COPY = form.FB_SIG>
<cfset form.FB_SIG = "">
</cfif>

If you download the latest version of the starter kit, this fix is implemented and it should work straightaway for you.

http://fbmlstarter.riaforge.org/

You may need to rename application.cfm to Application.cfm if you are having problems. I am fixing this now.

Dominic
# Posted By Dominic Watson | 1/21/08 8:26 AM
Thanks!!!

I have downloaded the latest version and I amd going to test it.

Sorry from my english, i write from spain ;)
# Posted By Huge | 1/22/08 1:15 AM
How about playing hide and seek with the variables.

Application.cfc
<cfcomponent>
<cfset this.name = "sep09helloworld">
<cfset url.form = structnew()/>
<cfset structappend(url.form,form)/>
<cfset structclear(form)/>
<cffunction name="onRequestStart">
<cfset structappend(form,url.form)/>
<cfset structdelete(url,"form")/>
</cffunction>
</cfcomponent>

This application.cfc will store them in URL as a struct (which is left alone), then retrieve them from URL before processing the request. You won't need to change code on either end of your app.
# Posted By Jason Delmore | 9/30/08 1:37 PM