Some links in this post may be affiliate links. We may get paid if you buy something or take an action after clicking one of these, but without addictional costs for you compared to direct buying.
Fixing slow SSH remote terminal with your Raspberry PI
This article is more a memo for me than a real guide. But I thought that what I learned, even if in form of raw note, can be useful for a lot of people facing my same problem.
I sometimes experienced very slow remote terminal sessions to my Raspberry PI, even if HTOP command showing free resources. So i decided to focus
SSHD (the ssh daemon) has 2 main files useful for my analysis:
“/etc/ssh/sshd_config” – which contains ssh service configuration
“/var/log/auth.log” – which contains ssh service logs
Looking inside auth.log file, no main problems are visible. In this log I can see correctly starting (“sshd[502]: Server listening on 0.0.0.0 port 22.”) and correctly accepting connections from my PC (“sshd[523]: Accepted password for pi from 192.168.43.182 port 51043 ssh2”).
No visible errors are traced.
So, I decided to increase log verbosity: it may be connected to some minor errors not visible at current log verbosity level. From https://man.openbsd.org/sshd_config I get that I have to edit config file:
sudo nano /etc/ssh/sshd_config
And set following parameter with maximum logging level (DEBUG3):
LogLevel DEBUG3
Once done, a reboot is better to be sure this change has been acquired from system:
sudo reboot now
So, I had only to wait for terminal becoming again slow.
Log Analisys
Once returned back tp check log for error or warning signals, II noted that each time I reconnect, SSH daemon is negotiating client connection with client. This is an expected behaviour for me and appears to be normal. Following lines report initialization log:
Aug 24 17:04:35 raspberrypi sshd[626]: debug3: receive packet: type 20
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: SSH2_MSG_KEXINIT received
Aug 24 17:04:35 raspberrypi sshd[626]: debug3: send packet: type 20
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: SSH2_MSG_KEXINIT sent
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: local server KEXINIT proposal
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nis$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,s$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-et$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-et$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression ctos: none,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression stoc: none,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages ctos:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages stoc:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: first_kex_follows 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: reserved 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: peer client KEXINIT proposal
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: KEX algorithms: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-ni$
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: SSH2_MSG_KEXINIT sent
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: local server KEXINIT proposal
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nis$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,s$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-et$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-et$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression ctos: none,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression stoc: none,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages ctos:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages stoc:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: first_kex_follows 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: reserved 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: peer client KEXINIT proposal
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: KEX algorithms: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-ni$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: host key algorithms: ssh-ed25519
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers ctos: aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-ctr,a$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: ciphers stoc: aes256-ctr,aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-ctr,a$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs ctos: hmac-sha2-256,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-sha2-256-etm@o$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: MACs stoc: hmac-sha2-256,hmac-sha1,hmac-sha1-96,hmac-md5,hmac-sha2-256-etm@o$
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression ctos: none,zlib,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: compression stoc: none,zlib,zlib@openssh.com
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages ctos:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: languages stoc:
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: first_kex_follows 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug2: reserved 0
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: kex: algorithm: curve25519-sha256@libssh.org
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: kex: host key algorithm: ssh-ed25519
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: kex: client->server cipher: aes256-ctr MAC: hmac-sha2-256 compression: none
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: kex: server->client cipher: aes256-ctr MAC: hmac-sha2-256 compression: none
Aug 24 17:04:35 raspberrypi sshd[626]: debug1: expecting SSH2_MSG_KEX_ECDH_INIT
......
Trying to avoid to have full log post, scrolling down I saw ssh daemon still continuing in server-client negotiation till ssh console is used. So, no strange behaviours appears to be reported.
First action: PAM
Some row caught my attention: those regarding PAM (Privileged Access Management).
I supposed that PAM times are not logged in auth.log. I also checked from some web articles that PAM debugging is not so simple. Checking PAM configuration:
sudo nano /etc/pam.d/common-session
I discovered that this is a very skinny file. And verified that it includes by default some required (mandatory) modules, plus 2 optional modules. Term “optional” in very low hardware like my Raspberry PI Zero W can be translated in disabled…
The two optional modules are:
pam_systemd.so – deputed to register user sessions in the systemd login manager, so appear to be a loggin for my user session. I’m the only one SSH user for my PI, so I can comment out this line.
pam_chksshpwd.so – for the very few mentions from web, this appears to be a module deputed to check if your password is still the default one (really guys? a module for this stupid job?)
I decided to comment these 2 optional modules nd use my little PI Zero W for a while.
From this modification, I finally solved my slow SSH sessions problem.
Second Action: DNS
Some users also refer about delays that may be introduced by SSH reverse DNS queries.
SSH daemon controls that client it is communicating with is the same during the entire connection by checking periodically the match with IP address and related hostname. It can add some reverse DNS resolution load to your device, so resulting in slower connection in some configurations (expecially when SSH connection is made from internet. For local network connection it shouldn’t give great impacts.
Anyway, if first action didn’t solved your problem, you can try by disabling DNS check from SSH configuration file:
sudo nano /etc/ssh/sshd_config
And uncommenting related row:
useDNS no
Honestly, I tryed this test alone, but this didn’t gave me same results as first action.
Let me know if these solution solved your slow SSH connection issues and also share your comments!
13 thoughts on “Fixing slow SSH remote terminal with your Raspberry PI”
One thing I tried when I was having problems with ssh-ing to my Raspi-4/8gb was to enter “ssh-copy-id my-user-name@raspi-ip-address, This seems to have speed-ed up my work on the Raspi.
In my case it was the log size. I’m intensively using hcron (alternative to cron) and it produced almost 170MB of log file which was saved a few times every minute. So it was constantly busy writing, which messed up the executed tasks. I removed the log file, rebooted (which removed the hanging tasks) and from that point it started to work smoothly as before.
Thank you. I was having intermittent hangs using my Pi Zero 2, using the latest 64bit OS. I have been using Pis for several years now and have never had this problem before.
I disabled pam_systemd.so and pam_chksshpwd.so as you describe and the problem seems cured. So far so good.
Hi Matej.
I double checked it. The ssh_config file controls the Raspberry PI ssh connection when it is used as client. The sshd_config, on the other side, controls the ssh connection when it is used as server. So, the right place should be sshd_config and I’ve updated my tutorial accordingly.
Many thanks for your feedback, Giuseppe
And after looking for solutions i found this command, also add at the end of /etc/ssh/sshd_config:
IPQoS cs0 cs0
doing this: OpenSSH sets the TOS (type Of Service) field in the IP datagram as “lowdelay” for interactive sessions and “throughput” for non-interactive sessions. My router doesn’t handle properly those settings, so I changed them in Cs0, Cs0 (aka 0x00, 0x00) (best effort, best effort) and solved the instability/freeze SSH issues.
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Cookie
Duration
Description
AWSELBCORS
session
This cookie is used by Elastic Load Balancing from Amazon Web Services to effectively balance load on the servers.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Cookie
Duration
Description
_ga_*
1 year 1 month 4 days
Google Analytics sets this cookie to store and count page views.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Cookie
Duration
Description
__qca
1 year 26 days
The __qca cookie is associated with Quantcast. This anonymous data helps us to better understand users' needs and customize the website accordingly.
_dlt
1 day
Zendesk set this cookie to create a unique ID for the session. This enables the website to collect data on visitor behaviour for statistical purposes.
mc
1 year 1 month
Quantserve sets the mc cookie to track user behaviour on the website anonymously.
One thing I tried when I was having problems with ssh-ing to my Raspi-4/8gb was to enter “ssh-copy-id my-user-name@raspi-ip-address, This seems to have speed-ed up my work on the Raspi.
Worked Great. It was just what I needed.
I was trying htop. I thought I fried my zero.
This sped it up.
Thanks.
In my case it was the log size. I’m intensively using hcron (alternative to cron) and it produced almost 170MB of log file which was saved a few times every minute. So it was constantly busy writing, which messed up the executed tasks. I removed the log file, rebooted (which removed the hanging tasks) and from that point it started to work smoothly as before.
Thank you for your tip, Marek
Fixed my problem. Thanks.
Thank you for your feedback, Mike.
Please remember to rate my post and share my blog with your friends 😉
Thank you. I was having intermittent hangs using my Pi Zero 2, using the latest 64bit OS. I have been using Pis for several years now and have never had this problem before.
I disabled pam_systemd.so and pam_chksshpwd.so as you describe and the problem seems cured. So far so good.
Thank you for your feedback, Ray.
Don’t forget to rate my post and share my blog with your friends 😉
Only one question:
the command useDNS no is in ssh_config or in sshd_config file?
Hi Matej.
I double checked it. The ssh_config file controls the Raspberry PI ssh connection when it is used as client. The sshd_config, on the other side, controls the ssh connection when it is used as server. So, the right place should be sshd_config and I’ve updated my tutorial accordingly.
Many thanks for your feedback, Giuseppe
Thanks for your help!
And after looking for solutions i found this command, also add at the end of /etc/ssh/sshd_config:
IPQoS cs0 cs0
doing this: OpenSSH sets the TOS (type Of Service) field in the IP datagram as “lowdelay” for interactive sessions and “throughput” for non-interactive sessions. My router doesn’t handle properly those settings, so I changed them in Cs0, Cs0 (aka 0x00, 0x00) (best effort, best effort) and solved the instability/freeze SSH issues.
ref.: https://discourse.osmc.tv/t/solved-ssh-connection-sometimes-hangs/76504/6
Thnx and have a nice day! 🙂
Finally thanks! This was the solution helped me 🙂
Glad it helped you too an thank you for your feedback!