I can't remember...if you only set the access class on the VTY lines, does that allow the TCP session to be set up before it denies the connection, or does it drop the initial TCP SYN?
I seem to recall that it won't even allow a TCP connection unless the packet is coming from an allowed address. However, if that isn't the case then you should disallow incoming Telnet on an interface-level access list.
Heck, why not do that? Unless you have a really good reason to allow Telnet connections to your internal network, not just your border router, then I'd drop all incoming Telnet traffic, or any other management traffic like SNMP, SSH, etc.