There's more...
SSH is sensitive about the permissions that its files have.
You don't want your private key to be readable by any random user who might be on your system, so as a result, plain SSH won't work if it thinks you have bad permissions set.
Generally, this won't be an issue if you've just generated your keys, but if you later move them between computers, you might find that you've spoiled the permissions slightly.
A good rule of thumb is to assume locked-down settings:
[vagrant@centos1 ~]$ ls -lha .ssh/
total 28K
drwx------. 2 vagrant vagrant 134 Aug 8 14:05 .
drwx------. 3 vagrant vagrant 95 Aug 8 10:29 ..
-rw-------. 1 vagrant vagrant 389 Aug 7 16:40 authorized_keys
-rw-------. 1 vagrant vagrant 464 Aug 8 10:04 id_ed25519
-rw-r--r--. 1 vagrant vagrant 101 Aug 8 10:04 id_ed25519.pub
-rw-------. 1 vagrant vagrant 3.3K Aug 8 11:15 id_rsa
-rw-r--r--. 1 vagrant vagrant 741 Aug 7 16:43 id_rsa.pub
-rw-r--r--. 1 vagrant vagrant 535 Aug 8 11:39 known_hosts
In the above command, we can see that the public and private halves of the keys (id_rsa keys and id_ed25519 keys) have different values.
The public halves of the keys (*.pub) have the value 644 (read/write, read, read):
-rw-r--r--.
The private halved of the keys have the value 600 (read/write, none, none):
-rw-------.