I don't know how putty handles key management, but using a linux ssh client:
The public key is stored on the remote machine and is *not* encrypted or protected in any way. It doesn't matter who knows it, post it on a website, it doesn't matter.
The private key is stored on any machine you wish to access the remote system from and must be closely guarded, hence the encryption.
To understand how this authentication works you have to understand a fundamental aspect of public key cryptography.
Given a keypair, A1 and A2, if A1 is used to encrypt a message, *only* the key A2 can decrypt it. The key A1 cannot be used to decrypt a message encoded with key A1, you *must* use its complimentary key A2. Conversely A2 encryption must be decrypted with the key A1.
The difference between a public and a private key is simply that one is published and the other is kept secret, other than that, they are interchangable. Also, keypairs can be identified with an ID and that ID does not need to be kept secret.
So, in a simplified form:
When you try to log in to the remote machine, the client tells the server that it would like to try to authenticate using public keys and tells the server the key ID of the key it would like to use. The sshd server process then looks in the authorized keys file to find a public key with that ID, if it doesn't find it, authentication fails.
If it finds a public key with the correct ID, it then generates a random 'challenge' and encrytps it with the public key. The client takes this challenge and uses the private key to decrypt it then re-encrypts it with the private key and sends it back to the server. Remember that the challenge encrypted with the private key looks different than the challenge encrypted with the public key, so you can't just send the same packet back.
The server then uses the public key to decrypt the packet and verifies the contents are indeed the challenge it originally sent and knows that the remote client has the correct private key and allows access, otherwise, authentication fails.