A solution would be to force the key files to be kept permanently, by adding them in your ~/.ssh/config file:

IdentityFile ~/.ssh/gitHubKey
IdentityFile ~/.ssh/id_rsa_buhlServer 

This is documented here:

http://stackoverflow.com/questions/3466626/add-private-key-permanently-with-ssh-add-on-ubuntu

This also works well on macOS.