Home Fixing slow SSH remote terminal with your Raspberry PI
RPI Computers

Fixing slow SSH remote terminal with your Raspberry PI

fix slow SSH featured image

This article is more a memo for me than a real guide. But I thought that what I learned, even if in form of raw note, can be useful for a lot of people facing my same problem.

I sometimes experienced very slow remote terminal sessions to my Raspberry PI, even if HTOP command showing free resources. So i decided to focus

SSHD (the ssh daemon) has 2 main files useful for my analysis:

  • “/etc/ssh/sshd_config” – which contains ssh service configuration
  • “/var/log/auth.log” – which contains ssh service logs

Looking inside auth.log file, no main problems are visible. In this log I can see correctly starting (“sshd[502]: Server listening on 0.0.0.0 port 22.”) and correctly accepting connections from my PC (“sshd[523]: Accepted password for pi from 192.168.43.182 port 51043 ssh2”).

No visible errors are traced.

So, I decided to increase log verbosity: it may be connected to some minor errors not visible at current log verbosity level. From https://man.openbsd.org/sshd_config I get that I have to edit config file:

sudo nano /etc/ssh/sshd_config

And set following parameter with maximum logging level (DEBUG3):

LogLevel DEBUG3

Once done, a reboot is better to be sure this change has been acquired from system:

sudo reboot now

So, I had only to wait for terminal becoming again slow.

Log Analisys

Once returned back tp check log for error or warning signals, II noted that each time I reconnect, SSH daemon is negotiating client connection with client. This is an expected behaviour for me and appears to be normal. Following lines report initialization log: 

Aug 24 17:04:35 raspberrypi sshd[626]: debug3: receive packet: type 20
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: SSH2_MSG_KEXINIT received
Aug 24 17:04:35 raspberrypi sshd[626]: debug3: send packet: type 20
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: SSH2_MSG_KEXINIT sent
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: local server KEXINIT proposal
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nis$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,s$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-et$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-et$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression ctos: none,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression stoc: none,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages ctos:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages stoc:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: first_kex_follows 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: reserved 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: peer client KEXINIT proposal
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: KEX algorithms: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-ni$
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: SSH2_MSG_KEXINIT sent
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: local server KEXINIT proposal
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nis$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,s$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-et$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-et$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression ctos: none,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression stoc: none,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages ctos:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages stoc:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: first_kex_follows 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: reserved 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: peer client KEXINIT proposal
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: KEX algorithms: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-ni$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: host key algorithms: ssh-ed25519
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers ctos: aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-ctr,a$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers stoc: aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-ctr,a$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs ctos: hmac-sha2-256,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-sha2-256-etm@o$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs stoc: hmac-sha2-256,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-sha2-256-etm@o$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression ctos: none,zlib,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression stoc: none,zlib,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages ctos:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages stoc:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: first_kex_follows 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: reserved 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: kex: algorithm: curve25519-sha256@libssh.org
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: kex: host key algorithm: ssh-ed25519
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: kex: client->server cipher: aes256-ctr MAC: hmac-sha2-256 compression: none
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: kex: server->client cipher: aes256-ctr MAC: hmac-sha2-256 compression: none
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: expecting SSH2_MSG_KEX_ECDH_INIT
......

Trying to avoid to have full log post, scrolling down I saw ssh daemon still continuing in server-client negotiation till ssh console is used. So, no strange behaviours appears to be reported.

First action: PAM

Some row caught my attention: those regarding PAM (Privileged Access Management).

I supposed that PAM times are not logged in auth.log. I also checked from some web articles that PAM debugging is not so simple. Checking PAM configuration:

sudo nano /etc/pam.d/common-session

I discovered that this is a very skinny file. And verified that it includes by default some required (mandatory) modules, plus 2 optional modules. Term “optional” in very low hardware like my Raspberry PI Zero W can be translated in disabled…

The two optional modules are:

  • pam_systemd.so – deputed to register user sessions in the systemd login manager, so appear to be a loggin for my user session. I’m the only one SSH user for my PI, so I can comment out this line.
  • pam_chksshpwd.so – for the very few mentions from web, this appears to be a module deputed to check if your password is still the default one (really guys? a module for this stupid job?)

I decided to comment these 2 optional modules nd use my little PI Zero W for a while.

From this modification, I finally solved my slow SSH sessions problem.

Second Action: DNS

Some users also refer about delays that may be introduced by SSH reverse DNS queries.

SSH daemon controls that client it is communicating with is the same during the entire connection by checking periodically the match with IP address and related hostname. It can add some reverse DNS resolution load to your device, so resulting in slower connection in some configurations (expecially when SSH connection is made from internet. For local network connection it shouldn’t give great impacts.

Anyway, if first action didn’t solved your problem, you can try by disabling DNS check from SSH configuration file:

sudo nano /etc/ssh/sshd_config

And uncommenting related row:

useDNS no

Honestly, I tryed this test alone, but this didn’t gave me same results as first action.

Let me know if these solution solved your slow SSH connection issues and also share your comments!

What’s Next

Interested in more cool ways to use your RPI? Take a look at peppe8o Raspberry PI computer tutorials!

peppe8o

Open source and Raspberry PI lover

View Comments

  • One thing I tried when I was having problems with ssh-ing to my Raspi-4/8gb was to enter "ssh-copy-id my-user-name@raspi-ip-address, This seems to have speed-ed up my work on the Raspi.

  • Worked Great. It was just what I needed.
    I was trying htop. I thought I fried my zero.
    This sped it up.
    Thanks.

  • In my case it was the log size. I'm intensively using hcron (alternative to cron) and it produced almost 170MB of log file which was saved a few times every minute. So it was constantly busy writing, which messed up the executed tasks. I removed the log file, rebooted (which removed the hanging tasks) and from that point it started to work smoothly as before.

  • Thank you. I was having intermittent hangs using my Pi Zero 2, using the latest 64bit OS. I have been using Pis for several years now and have never had this problem before.
    I disabled pam_systemd.so and pam_chksshpwd.so as you describe and the problem seems cured. So far so good.

    • Thank you for your feedback, Ray.
      Don't forget to rate my post and share my blog with your friends ;)

    • Hi Matej.
      I double checked it. The ssh_config file controls the Raspberry PI ssh connection when it is used as client. The sshd_config, on the other side, controls the ssh connection when it is used as server. So, the right place should be sshd_config and I've updated my tutorial accordingly.
      Many thanks for your feedback, Giuseppe

      • Thanks for your help!

        And after looking for solutions i found this command, also add at the end of /etc/ssh/sshd_config:
        IPQoS cs0 cs0
        doing this: OpenSSH sets the TOS (type Of Service) field in the IP datagram as “lowdelay” for interactive sessions and “throughput” for non-interactive sessions. My router doesn’t handle properly those settings, so I changed them in Cs0, Cs0 (aka 0x00, 0x00) (best effort, best effort) and solved the instability/freeze SSH issues.

        ref.: https://discourse.osmc.tv/t/solved-ssh-connection-sometimes-hangs/76504/6

        Thnx and have a nice day! :)

Leave a Comment
Published by
peppe8o
Tags: connectionos literaspberry pislowslow sshsshterminal
2 years ago

Recent Posts

Some links in this post may be affiliate links. We may get paid if you buy something or take an action after clicking one of these, but without addictional costs for you compared to direct buying.

SPI communication between two Arduinos

In this tutorial, we will use two Arduino Uno to demonstrate the Serial Peripheral Interface…

1 week ago

Automatic irrigation system with Arduino and Sensors

In this tutorial, we will be making an automatic irrigation system (AIS) with Arduino and…

2 weeks ago

Beginner’s Guide to Use Python Virtual Environment with Raspberry PI (venv)

This tutorial will show you how to use Python Virtual Environment with Raspberry PI computer…

3 weeks ago

Get Betting Odds with Raspberry PI and Odds-API (free)

This tutorial will show you how to get betting odds with Raspberry PI by using…

1 month ago

Backup Raspberry PI (computer) Data with Rsync to Remote NAS

This tutorial will show you how to perform the backup of Raspberry PI (computer board)…

1 month ago

Honeygain and Raspberry PI: Earn by Sharing Internet Connection

This tutorial will show you how to install Honeygain on a Raspberry PI computer board…

1 month ago