Some 4621's have this problem and others don't. You have to see the comcode to tell which ones are affected. The other answer it to swap out the phones with ones that don't have this issue.
SEE PSN PSN003008u
Name of problem Received speech is attenuated when using the 4621 speakerphone
Products affected
4621SW Comcode 700345192
Problem description
The problem occurs on speakerphone, when older, unsupported 4621 telephone hardware (Comcode 700345192) is loaded with newer
R2.9 FW. The newer 4621 with Comcode 700381544 is not affected. What the 4621 speakerphone user will notice is the received
speech on the 4621 Telephone is severely attenuated during speakerphone use. It can occur when there is only a slight noise in the
speakerphone user’s environment (shuffling papers, tapping on the keyboard, etc.). The received speech typically returns to a normal
volume level once all noise is eliminated in the user’s environment.
Resolution
Use only R2.8 based FW on the older 4621 hardware. The last FW release for the older 4621 hardware is R2.8 SP2 (R2.8.87)
Workaround or alternative remediation