I had the same problem with my Samsung LED TV. Game mode seemed to fix the input lag for me as well, which is good because it made playing Forza 4 really hard with a wheel. here is a quote from Samsung's Plasma Wiki-thingy
I don't imagine you would want it on for watching movies as it does increase picture quality through magic wand waving. That is unless you notice that your audio and video are not synced!
"Game Mode is available on many current Samsung TVs. When you set a video source (input) to Game mode, your TV electronically bypasses two of the video signal processors in the TV, subsequently cutting down the amount of time the TV needs to process the video input from your game. You may notice a slight difference in the picture, but after you implement Game mode, the audio and video on your game should be synchronized."
Normally the TV takes the signal from the cable, massages it to make it look 'nicer'; applies your current settings for colour temperature, contrast, brightness, etc; resizes it to the actual pixel scale of the screen (if required) with the nicest resize algorithm it can muster; and then outputs it to the screen. And if that all takes 50+ ms then who cares, the only people who will even know are people who run their audio through a separate amp and so (with a zero-lag audio system) get the audio slightly before they see the associated picture (by a few frames). When using the internal audio or passing the audio though the TV then they will usually account for this known lag and delay the audio to sync.
Game mode will do the cheaper resize and less massaging of the data while still applying as many settings as they can (game mode may reduce the number of settings you can edit on the TV with only basics like contrast and brightness left depending on how clever the engineers were designing the low-lag mode and how much the processing chips inside cost them) and then throw it onto the screen. This is designed so hopefully they can get the image up there in under 20ms, ideally we're talking 5ms or less for a really nice game mode.
If you prefer the look of the image (without the added processing it may look better to you, or maybe you can use what settings you can adjust to make game mode look exactly like normal mode) with game mode off then switch it off when watching anything without interaction (if you have a separate sound system then look for a delay setting to sync it up if you notice the audio is slightly off) but, if you don't think it looks worse with game mode on, then enjoy not having to change your settings whenever you put anything interactive on the screen.
Thank you very much. Now its time to try out some 360+ gaming (as Gary would say). There still seems to be more lag than using a monitor but that may just come from the T.V. being an older model. Seems to be fine for single player games but I don't think I would play any competitive multiplayer FPS (BF3) on it. Thanks again.