It’s been awhile since I’ve setup my last work system a year back, so this past month I’ve gone through the usual pains of preparing a new machine. One of those pains was realizing some network segments at one of the libraries I am a regular at have port 22 blocked at an internal boundary. Without falling into discussions about how useful such a thing is from a defensive security point of view, I wanted to share a trick not many folks know of.

It turns out that, precisely for those situations, all renowned Git hosting providers offer access to their Git service via an alternative port: 443.

Addendum

git clone Hanging

The initial issue arose when I was working locally on a repository I had cloned already and then needed to clone another personal one, let’s assume for the sake of brevity it was jdsalaro/jdsalaro.com.

$ git clone git@github.com:jdsalaro/jdsalaro.com.git
Cloning into 'jdsalaro.com'...

As I waited for a couple of seconds I observed git just hanged there without doing much; obviously something was amiss. My usual attempt at debugging didn’t yield much that was of interest as can be seen below:

$ GIT_TRACE=2 GIT_CURL_VERBOSE=2 GIT_TRACE_PERFORMANCE=2 GIT_TRACE_PACK_ACCESS=2 \
  GIT_TRACE_PACKET=2 GIT_TRACE_PACKFILE=2 GIT_TRACE_SETUP=2 GIT_TRACE_SHALLOW=2 \
  git clone git@github.com:jdsalaro/jdsalaro.com.git

13:12:27.776079 git.c:465               trace: built-in: git clone git@github.com:jdsalaro/jdsalaro.com.git
Cloning into 'jdsalaro.com'...
13:12:27.781106 run-command.c:657       trace: run_command: unset GIT_DIR; GIT_PROTOCOL=version=2 ssh -o SendEnv=GIT_PROTOCOL git@github.com 'git-upload-pack '\''jdsalaro/jdsalaro.com.git'\'''

Similarly, SSH failed to establish a connection to the server at all for both github.com and gitlab.com; bitbucket.org, cloud.gitea.com and codeberg.org would have failed as well.

$ ssh -T git@github.com
^C

$ ssh -T git@gitlab.com
^C

Nevertheless, SSH’s debug output was much more useful in this case:

$ ssh -T git@github.com -vvv
OpenSSH_9.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/[...]/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/[...]/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/[...]/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: channel_clear_timeouts: clearing
debug1: Connecting to github.com port 22.

It Wasn’t a Me Problem

The connection wasn’t even being established, but internet connectivity wasn’t an issue:

curl example.com | grep title
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1256  100  1256    0     0   6372      0 --:--:-- --:--:-- --:--:--  6408
    <title>Example Domain</title>

And the corresponding providers were available over HTTPS and weren’t reporting an outage:

$ nc github.com 443
AAAA

The network connection looked alright on the wire, should I say waves?, as well:

For port 443 the whole TCP/IP spiel was taking place as expected: SYN → SYNACK → ACK → data transfer. At that point it became obvious that someone had made the decision in the past days to start restricting outbound connections to port 22:

$ nc github.com 22
^C

A quick Wireshark capture showed as much; outbound TCP handshakes to port 22 weren’t taking place.

The Solution

As mentioned in my brief introduction for this post, most Git hosting providers will also offer their service over HTTPS ^{1 2 3}.

Addressing the issue described here is just a matter of configuring the respective hostnames to their respective alternatives:

$ cat ~/.ssh/config

Host github.com
  Hostname ssh.github.com
  Port 443
  User git
  PreferredAuthentications publickey
  IdentityFile ~/.ssh/github

Host gitlab.com
  Hostname altssh.gitlab.com
  Port 443
  User git
  PreferredAuthentications publickey
  IdentityFile ~/.ssh/gitlab

Et voilà, we’re back online:

$ ssh -T git@github.com
Hi jdsalaro! You've successfully authenticated, but GitHub does not provide shell access.

$ ssh -T git@gitlab.com
Welcome to GitLab, @jdsalaro!

Final Words

Hope y’all found this brief debugging session interesting. It’s a bit over-engineered in order to make it interesting and blog-worthy, but is not too far detached from how I’d approach similar problems in reality.

Footnotes