What CPU, and what temperature?
I'd certainly not try messing with the voltage levels! By all means check them with a good, accurate voltmeter whilst under load. Then compare with motherboard or CPU manufacturer's figures to see if they're within spec (+/- etc). An old, or cheaply made PSU is more likely to have poorly regulated voltage rails than a quality product.
You could check to see what the airflow is like within your computer case. Airflow for motherboard, RAM, etc., will usually enter at the front of the case, cross over the motherboard, and be exhausted at the rear of the case. Depending upon design, some cases are rather restricted at the front, but often you can assist matters by placing fan(s) for sucking in air, as well as having fan(s) for blowing warm air out out at the back. Some tower cases have an air intake on the side of the case with trunking to channel it directly onto the top of the CPU heatsink fan assembly.
Position ribbon cables as flat as possible to case sides etc. Keep power cables for drives, etc., tied together and route them where they're less likely to restrict airflow.
Remove any unused or unwanted PCI cards and drives, etc. Additional cards and drives use power and generate heat, and can also impede airflow across the motherboard.
If you're going to add fans to the case, the bigger (diameter) the fan, the more air it will shift and it will be marginally quieter than, say two smaller fans.
ROGER - G0AOZ.