There should be a plastic air flow baffle that sits over the processors. Make sure that it is in place.
The toughest thing with heat is not that the room itself is cooled but that there are not "hot spots" by the servers. To elaborate on what TheLad has said, if you are not pumping the heat from the servers directly out of the room you need to have extremely good air movement within the room (or a lot of room around the racks) or you will get hot spots. Likewise, if your hot spot is in the path of the air flow it will recirculate back to the front of the rack and get sucked into the server before it has a chance to cool.
If that is the case it may explain what you are experiencing. When the air conditioning is running everything is fine because of the influx of cold air along with the air movement. When the air conditioning is not running you get hot spots that cause the server to "inhale" warm air. Your server alarms, but in the mean time the air conditioning system has kicked back on and cools the server off again.
If you have any remote temperature monitoring sensors or even a small thermometer (we use a digital, wireless one we bought from RadioShack), try mounting it (tape) to the front of the rack by the server. Then you can see how warm the temperature by the server is getting.
Most small data centers, ours included, only have one temperature sensor for the entire room. Depending on the size and layout of everything, it may be a problem.
Good Luck.