Peter Morgan

Peter Morgan

World ! hello.

The Erlang SSH daemon behind Shrugs

5 minute read

Shrugs is an Apache licensed self hosted git server written in Erlang, with a light sprinkling of Rust implementing the Port talking to git using grimsby. Erlang has built in support for the SSH protocol as part of OTP. This article walks through using ssh:daemon to implement the self hosted git server in shrugs.

Introduction

Shrugs is available as a from scratch Docker container for both amd64 and arm64 architectures built using the BEAM Docker GitHub Release Action. The resulting container weighs in at just 42MB including the BEAM, working well with smaller devices like a Raspberry PI.

The container is hosted on the free GitHub Container Registry (ghcr.io). Assuming that you already have docker installed, you can run shrugs with:

docker run \
  --name shrugs \
  -d \
  -p 22022:22 \
  ghcr.io/shortishly/shrugs

The above command maps the ssh daemon running within the shrugs container to port 22022 on the host in this example.

Either copy your authorized_keys into the container for authentication using docker cp:

docker cp ~/.ssh/authorized_keys shrugs:/users

Or copy the public key of individual users:

docker cp bob.pub shrugs:/users
docker cp alice.pub shrugs:/users

New keys (“*.pub” or “authorized_keys”) in the users directory will be picked up every 5 seconds.

Now you can push a repository into shrugs with:

mkdir demo
cd demo
git init .
echo "world" > hello.txt
git add hello.txt     
git commit --message='demo'
git branch -M main
git remote add origin ssh://localhost:22022/demo
git push --set-upstream origin main

And then clone the demo repository:

git clone ssh://localhost:22022/demo abc

More details about using Shrugs, A self hosted git SSH server, by following the link.

Architecture

Architecture

Writing The Application

The Erlang SSH daemon is an application following the OTP design principles. In practical terms that means we need to add it as a dependency to our project, customising and starting the service to fit our needs.

Dependencies

When using erlang.mk, add a couple of lines your Makefile before running make deps:

LOCAL_DEPS = \
    crypto \
    ssh

Alternatively, with rebar add crypto, ssh to the applications section of your .app.src.

Daemon

With the dependencies added to our project, the SSH daemon can be started. The call to shrugs_config:port(sshd) returns the TCP port on which the daemon should accept connections. The options allow for full customisation of the daemon into a git server:


ssh:daemon(shrugs_config:port(sshd),
           [{inet, inet},
            {subsystems, []},
            {key_cb, {shrugs_key_store, [KeyStore]}},
            {ssh_cli, {shrugs_cli, []}},
            {auth_methods, "publickey"}])

The options are:

In order to use shrugs we need to be authenticated with a public key, so lets look at the key store first.

shrugs_key_store

The responsibilities of the key store implementation are:

  • provide a secure repository of the service (private) host keys; and (public) authorised user keys accessible through the SSH server key API behaviour, implemented using a private ETS table; and
  • the creation any host keys for the service if they are not already present with public key:generate key/1;

For is auth key/3 we can establish whether the supplied PublicUserKey is one authorised with ets:member, where KeyStore is the private ETS key store:


is_auth_key(PublicUserKey, User, _DaemonOptions) ->
    gen_statem:call(
       ?MODULE,
       {?FUNCTION_NAME, PublicUserKey, User}).

...

handle_event({call, From},
             {is_auth_key, Key, _},
             _,
             #{key_store := KeyStore}) ->
    {keep_state_and_data,
     {reply,
      From,
      ets:member(KeyStore, {auth_key, Key})}}.

Similarly for host key/3, an ets:lookup returns the appropriate host key, otherwise an error is returned:


host_key(Algorithm, _DaemonOptions) ->
    gen_statem:call(
       ?MODULE,
       {?FUNCTION_NAME, Algorithm}).


handle_event({call, From},
             {host_key, KeyType},
             _,
             #{key_store := KeyStore}) ->
    {keep_state_and_data,
     {reply,
      From,
      case ets:lookup(
             KeyStore,
             {host_key, key_type(KeyType)}) of

          [] ->
              {error, no_such_key};

          [{_, PrivateKey}] ->
              {ok, PrivateKey}
      end}}.

While ssh_file provides a mechanism to decode the host keys with RFC4716, the functionality to encode keys isn’t present. Shrugs will create a host key if one isn’t already present, persisting it in a term file rather than in the standard RFC4716 format (by default in shrugs_host_keys.terms).

shrugs_users

The role of the shrugs_users module is to watch the /users directory adding new keys (matching “*.pub” or “authorized_keys”) to the key store.

shrugs_cli

ssh server channel is the behaviour for a ssh service to implement. shrugs cli, implements the init/1, handle_msg/2, handle_ssh_msg/2 and terminate/2 functions as part of this behaviour. In addition, shrugs cli also implements callbacks from grimsby to respond to data from git (running as an Erlang Port with grimsby) which are forwarded back over the ssh channel.

A git clone, git pull or git push results in an exec message being received by the ssh connection for the git-receive-pack, git-upload-archive or git-upload-pack executable.


handle_ssh_msg(
  {ssh_cm,
   Connection,
   {exec, Channel, WantReply, Command} = CM},
  #{channels := Channels,
    children := Children,
    envs := Envs} = State) ->

    case string:split(Command, " ") of
        [Executable, QuotedRepo]
          when Executable == "git-receive-pack";
               Executable == "git-upload-archive";
               Executable == "git-upload-pack" ->


            ok = prepare_repo_dir(
               Executable,
               unquote(QuotedRepo)),

            CoCh = {Connection, Channel},

            {ok, Child} = grimsby_port:run(
               #{arg0 => Executable,
                 args => [unquote(QuotedRepo)],
                 executable => filename:join(
                                 shrugs_config:dir(bin),
                                 "git"),
                 cd => shrugs_config:dir(repo),
                 envs => maps:get(CoCh, Envs)}),

            ssh_connection:reply_request(
              Connection,
              WantReply,
              success,
              Channel),

            {ok,
             State#{children := Children#{Child => CoCh},
                    channels := Channels#{CoCh => Child}}};

To handle the operation an Erlang Port is used to spawn git using grimsby_port the internals of which are discussed in more detail here.

The Child process created by grimsby_port is mapped to the SSH Connection and Channel pair (CoCh), and the CoCh is also mapped to the Child, so that data can be forwarded between SSH and the Port and vise versa.

The prepare_repo_dir function calls git init --bare for git-receive-pack where the repository does not already exist in shrugs. This means that you can push new repositories into shrugs without any ceremony.

Data from the SSH channel is forwarded to the git process using grimsby_port:send/2:


handle_ssh_msg({ssh_cm,
                Connection,
                {data, Channel, _Type, Data} = CM},
               #{channels := Channels} = State)
  when is_map_key({Connection, Channel}, Channels) ->
    #{{Connection, Channel} := Child} = Channels,
    grimsby_port:send(Child, Data),
    {ok, State};

Similarly, data from the git process is forwarded to the relevant SSH connection and channel:


handle_call(
  {grimsby_port, {Stream, Child, Data} = Msg},
  _From,
  #{children := Children} = State)
  when Stream == stdout;
       Stream == stderr,
       is_map_key(Child, Children) ->
    #{Child := {Connection, Channel}} = Children,
    case ssh_connection:send(Connection,
                             Channel,
                             data_type(Stream),
                             Data) of
        ok ->
            {reply, ok, State};

        {error, Reason} ->
            {reply, ok, State}
    end;

When end of file is received by the SSH connection it is also forwarded to the port:


handle_ssh_msg({ssh_cm, Connection, {eof, Channel} = CM},
               #{channels := Channels} = State)
  when is_map_key({Connection, Channel}, Channels) ->
    #{{Connection, Channel} := Child} = Channels,
    grimsby_port:close(Child, stdin),
    {ok, State};

That concludes the overview of the code used to implement the SSH interface to shrugs an Apache licensed self hosted git server written in Erlang, with a light sprinkling of Rust implementing the Port talking to git using grimsby.

You May Also Enjoy