Using SSH is a great way to remotely manage a server and to securely transfer data to and from it. You basically connect using SSH with your username and password. In that case you authenticate with something you know, which in this case is the password.
But you can also authenticate with something you have, like an SSH key. You can use this to authenticate without a password, and even configure the server to only allow SSH keys to prevent password attacks.
This tutorial shows how to configure the SSH server, generate SSH keys for each user and optionally disable password logins.