Workstation access

Start, stop, connect to, and diagnose a workstation from the command line.

These commands manage a workstation’s lifecycle and connect to a running one. The access commands (shell, tmux, logs, troubleshoot) read the workstation’s address and login user from its status.

When you omit the name, shell and tmux choose a workstation for you if exactly one fits. They look for these, in order:

  • The only Ready workstation.
  • When none is Ready, the only one that is running but not yet Ready. They warn you when they choose it.
  • With --start, the only workstation, even a stopped one.

Otherwise they list the candidates and stop. Name the one you want with -w, or add -A to look in every namespace, not only the current one.

rl shell chooses a workstation only when you give it no command. With a command and no -w, it reads the first word as the workstation’s name. To run a command, name the workstation first or with -w: rl shell -w my-box -- make test.

rl workstation start / stop / restart

Power one or more workstations imperatively: the command equivalent of setting spec.stopped (and, for restart, spec.restartNonce) in a manifest. Target them by name, several names, or a label selector with -l. Alias: ws.

rl workstation start my-box
rl workstation stop my-box other-box
rl ws stop -l tier=dev            # every matching workstation
rl workstation restart my-box
rl ws restart -l tier=dev         # one logical restart across the whole set

A stopped workstation is halted but nothing is lost: its disk, installed tools, and state all persist; it rests in the Stopped phase until you start it again.

Each command returns once the request is accepted, and the machine then changes state on its own. A cloud workstation takes about a minute to start. Add --wait to block until the change is done. For start and restart that means Ready. For stop it means Stopped, which is recorded only once the provider confirms the machine is down.

rl workstation start my-box --wait              # returns when my-box is Ready
rl workstation stop my-box --wait --timeout 2m

restart takes a workstation down and brings it back up. It is a real power cycle, not a reload of its configuration. On a stopped workstation, restart starts it. When it targets several workstations, they all share one restart, and each new run of the command triggers the next one.

FlagDescription
-l, --selector <selector>Label selector (key=value[,key!=value,key in (a,b),key,!key]) instead of name(s).
--waitBlock until each workstation the command moved is Ready (start, restart) or Stopped (stop).
--timeout <duration>With --wait, how long to wait. Default 20m; 0 means no limit.
-n, --namespace <ns>Namespace.
-A, --all-namespacesOperate across all namespaces.
--as <subject>Impersonate a subject.
--home <dir>Data directory.

rl workstation release-finalizer <name>

Close the record of a workstation stuck in Terminating.

Deleting a workstation tears its machine down first, and the control plane will not close the record until it can confirm the machine is gone. When it cannot confirm, because the provider is unreachable, a cloud credential was revoked, or the configuration it needed was already deleted, it holds the record and keeps retrying, so nothing is ever forgotten silently.

This command ends that hold. It closes the record and does not delete the machine. If a machine is still running it keeps running, keeps costing money, and afterwards nothing tracks it, so you are accepting responsibility for finding and deleting it yourself.

rl workstation release-finalizer my-box \
    --acknowledge-orphaned-machine my-box-7f3a91

It is restricted to administrators of the namespace, and refuses unless you name the machine you may be leaving behind. Run it without the flag and the refusal tells you the machine’s name; delete that machine in your provider’s console first if you can reach it at all. This command is for the case where you cannot.

FlagDescription
--acknowledge-orphaned-machine <machine>Required. The machine that may be left running. Shown in the refusal if omitted.
-n, --namespace <ns>Namespace.
--as <subject>Impersonate a subject.
--home <dir>Data directory.

rl workstation get-resolved-configuration <name>

Print the merged effective configuration a workstation resolves to: the answer to “what configuration will my workstation actually get?”.

rl workstation get-resolved-configuration my-box
rl ws get-resolved-configuration my-box -o json

The configuration is resolved on demand from the workstation, every WorkstationConfig layered onto it (explicit spec.configs[] references and selector matches), and any CloudIdentity governing it, by the same resolver the control plane uses to configure the workstation. It is not read from status.

FlagDescription
-o, --output <format>yaml (default) or json.
-n, --namespace <ns>Namespace.
--as <subject>Impersonate a subject.
--home <dir>Data directory.

The document has these top-level keys:

KeyDescription
workstation<namespace>/<name>, so a pasted document says what it is about.
effectiveConfigHashThe configuration fingerprint the control plane wants, read from status.config.effectiveHash. Empty means the workstation has not been set up yet.
appliedConfigsEach config that shaped the workstation, with its name, its source and its priority. source is explicit for a spec.configs[] reference and selector for a label match. A lower priority applies first, so a higher priority wins. A config that targets the workstation but was dropped is listed too, with refused: true and a reason such as author-lacks-reach or configmap-key-not-found.
effectiveThe whole merged spec, including everything status never carried: environment, files, scripts, sources, devtools, toolconfigs, and dotfiles.
configCurrencycurrent when the workstation’s Configured condition was computed from its configuration as it is now. stale when the workstation or one of its configs has changed since.
configCurrencyAtThe workstation’s metadata.resourceVersion that configCurrency was computed against.
cloudIdentityThe CloudIdentity whose selector matches the workstation, or <none matched>. Absent for a local provider.
appliedConfigs:
- name: ai-box
  priority: 200
  source: explicit
- name: base
  priority: 100
  source: selector
configCurrency: current
effective:
  devtools:
  - name: nodejs
  packages:
  - git
  shell: /bin/bash
  uid: 1000
  user: dev
effectiveConfigHash: 8f2c1a9e4d6b...
workstation: local/my-box

effective is the whole spec the workstation is set up to match, so a real document is longer than this: alongside what you declared, it carries the defaults Ringleader resolved for you (the login user and shell, the diagnostics settings, the node key authorized on the workstation, the agent version it should run).

Safe to paste into a ticket

This is the effective spec, not what is finally delivered to the workstation, so ${secret:NAME} references are printed verbatim. The command resolves no secret values and needs no access to any Secret.

A workstation homed at a control plane you are logged into is resolved by that control plane, so the answer is the one its control loop uses on the workstation, not a local approximation.

rl shell [workstation] [command...]

Open a shell on a workstation over SSH using the node key.

rl shell my-box                 # interactive shell
rl shell my-box -- uname -a     # run a remote command
rl shell                        # the sole Ready workstation
FlagDescription
-w, --workstation <name>Target workstation. Defaults to the sole Ready one; required when several are.
-n, --namespace <ns>Namespace.
-A, --all-namespacesLook for the workstation in every namespace you can reach.
--startStart the workstation first if it is stopped, and wait until it is Ready. See below.
--start-timeout <duration>With --start, how long to wait. Default 20m; 0 means no limit.
--ptyGive the command a terminal. See below.
--script <file>Run this local script file on the workstation. See below.
--copy <dir>Send this local directory along with --script.
--exit-code-file <path>Write the remote command’s exit code to this file. See below.
--hostRun a command or --script on the workstation’s VM instead of inside its dev container. It needs a command. For a shell on the VM, use the <user>-host login described in Devcontainers.
--as <subject>Impersonate a subject.
--home <dir>Data directory.

Arguments after -- are the remote command’s argv, delivered word for word; with none, you get an interactive shell.

Starting a stopped workstation

--start brings a stopped workstation up before connecting. It requests a start, waits until the workstation is Ready, then connects. If the workstation is already starting, --start waits for it. If it is already running, rl connects immediately. Everything --start prints goes to stderr, so stdout still carries only the command’s own output.

rl shell --start my-box -- make test     # starts my-box, waits, then runs
rl tmux --start my-box

If a start cannot bring the workstation up, --start stops before connecting and prints one line saying why. That happens in two cases:

  • The workstation is being deleted.
  • The workstation is in the Failed phase and nothing is retrying it. The line then names what will move it, such as rl workstation restart.

A workstation stopped by its maximum run time starts again, for a new run.

Exit codes

The remote command’s exit code passes through unchanged. When Ringleader itself fails, rl shell exits 1 if it could not find or reach the workstation. It exits 2 for a usage error, or when it could not choose one workstation for you. An ssh connection failure exits 255. So 1 and 2 can mean either a remote failure or a Ringleader one.

--exit-code-file tells them apart without reading stderr. The file is written only if the remote command ran, and it holds that command’s exit code. An absent file means Ringleader failed. A file already at that path is removed first.

rl shell my-box --exit-code-file /tmp/ec --script ./ci.sh
[ -f /tmp/ec ] || exit 75          # never ran: retry rather than report a result
exit "$(cat /tmp/ec)"              # the remote command's own exit code

Remote-command semantics

The workstation receives exactly the argv you wrote, one word for one word. A metacharacter inside an argument is data, not syntax, so rl shell my-box -- printf '%s|' a d prints a|d|, and -H "Authorization: Bearer $TOKEN" arrives as a single argument rather than three. It is the contract docker exec and kubectl exec offer over the same ground, and it is what makes rl shell safe to drive from a script.

Asking for a remote shell needs no flag, because sh -c is itself just argv:

rl shell my-box -- sh -c 'cd ~/src && make test 2>&1 | tail -20'

Pipes, &&, redirections, globs and ~ inside that quoted string are the workstation’s own shell to interpret, which is the point of spelling it out.

Two things about the remote command surprise people, and neither is specific to Ringleader:

  • ~, $HOME and globs expand on your machine, not on the workstation. Your shell runs before rl does: by the time rl holds an argv, ~/src has already become /home/you/src, and no implementation can tell the two apart afterwards. So rl shell my-box -- ls ~ asks the workstation for your machine’s home path, and quoting it as rl shell my-box -- ls '~' sends a literal ~, which fails with ls: cannot access '~': No such file or directory. Expand it where you mean it with rl shell my-box -- sh -c 'ls ~', or pass an absolute path. Ansible’s command module documents the same property, and draws the same line with its separate shell module.

  • The command runs in a non-login, non-interactive shell. /etc/profile, /etc/profile.d/* and ~/.profile are not sourced, so anything exported from a profile fragment is missing. That covers a PATH entry added by a devtool and shell integrations such as fzf. Two things reach the command anyway, because neither travels by a profile fragment: environment variables you declare under environment: in a WorkstationConfig, which Ringleader also writes to /etc/environment for PAM to read on every SSH session, and the forwarded SSH agent (see the note below). Wrap the command in a login shell when you need the rest:

    rl shell my-box -- bash -lc 'node --version'

git over SSH in a remote command

Ringleader forwards your local SSH agent into the workstation and makes SSH_AUTH_SOCK ambient at the sshd level, so every session gets it. A plain rl shell my-box -- git -C /home/dev/repo pull authenticates with your own keys, and so does a git run by a script or an on-box agent. No login shell is needed for this, and none of your private keys ever reach the workstation.

Making it ambient is best-effort: it needs the workstation to grant Ringleader’s delivery sudo rule, and an sshd whose configuration Includes /etc/ssh/sshd_config.d/. Both hold on every image Ringleader ships, so this is worth checking only on a custom image or a login identity without that rule.

rl troubleshoot my-box answers it, in two places:

  • An ambient agent line in the controller view, naming which shells lost the agent and which precondition stopped the delivery. It is written only when something is missing, so a workstation that got both files shows no such line, and neither does one your daemon has never connected to, so its absence is not by itself proof that the delivery worked.

  • An ssh-agent-env probe under system probes, which is the half that confirms the working case. It reads SSH_AUTH_SOCK and both files over the same non-login, non-interactive session your commands run in, and reports whether sshd reads /etc/ssh/sshd_config.d/ at all:

      ssh-agent-env :
          SSH_AUTH_SOCK=/home/dev/.ringleader/ssh-auth-sock
          profile.d fragment: present
          sshd drop-in: present
          sshd Include sshd_config.d: yes

rl shell my-box -- printenv SSH_AUTH_SOCK still shows the same variable directly, and is the quicker check when all you want is a yes or no.

Giving the command a terminal

An interactive shell has a terminal. A remote command runs without one, and some programs need one anyway: an editor, a pager, or a command that leaves you at a shell prompt when it finishes.

--pty gives the remote command a terminal:

rl shell --pty my-box -- bash -lc 'cd ~/src/worktree && exec $SHELL -l'

That command moves to a worktree and replaces itself with a login shell, so you land at a prompt already in the right directory.

The cost is that a terminal merges the remote’s stderr into its stdout. Use --pty for a command you are driving by hand, and leave it off for one whose output you are capturing.

--pty does not combine with --script or --copy. A script travels as a file and runs as a file, so there is nothing for a terminal to attach to. Asking for both stops with an error.

Running a local script file

--script takes a script that lives on your machine, copies it to the workstation under its own name, and runs it there. Its exit code becomes the command’s.

rl shell my-box --script ./setup.sh

The script’s bytes are never text a shell parses. They travel as a file, are unpacked as a file and are executed as a file, so a backtick, $(…), |, * or a newline inside the script means exactly what it would mean had you typed the file on the workstation. That is the whole point of the flag, and it is what a heredoc (-- bash -s <<EOF) cannot give you: an unquoted heredoc is evaluated by your local shell before the bytes ever leave, so a backtick in a comment is enough to run something at home.

--copy sends a local directory along with the script, and runs the script from inside the copied directory. Use it for a script that reads its own fixtures or sources a file next to it:

rl shell my-box --copy ./demo --script ./demo/run.sh

The script must be inside the directory you copy. Neither flag combines with a trailing command: a run is either a remote argv or a script file, never both.

rl file cp [-r] <src> <dst>

Copy a file or directory between your machine and a workstation. One side is a local path, the other is workstation:path, in the style of scp:

rl file cp ./local.txt my-box:/remote/path     # upload
rl file cp my-box:/remote/path ./local.txt     # download
rl file cp -r ./dir my-box:/remote/dir         # a whole directory, either way
rl file cp ./notes.md :/tmp/notes.md            # empty name: the same box rl shell would pick
rl file cp ./notes.md my-box:                  # empty path: the login user's home directory

A colon before any slash makes an argument remote, so a local file whose name contains a colon is written ./name.

The destination is the path that will exist afterwards. The one exception is cp(1)’s and applies to a single file only: a destination naming an existing directory, or ending in a separator, receives the file under its own basename. A -r copy never appends a basename, so running it twice does what running it once did, where scp would nest dir/dir on the second run.

FlagDescription
-r, --recursiveCopy a directory and everything under it. Required for a directory, refused for anything else, in both directions.
--hostCopy to and from the workstation’s VM rather than the dev container it hosts.
-n, --namespace <ns>Namespace.
-A, --all-namespacesLook across every namespace.
--as <subject>Impersonate a subject.
--home <dir>Data directory.

What you can rely on:

  • A single file lands by a rename, so a transfer that fails leaves the destination exactly as it was.
  • A -r copy merges into its destination and is not atomic as a tree, though each file within it still is.
  • Symlinks are never created by a download, and a directory copied up follows only links that stay inside it.
  • Any failed transfer exits non-zero and says what failed.

Using ssh directly

rl is a convenience wrapper, not a walled garden. While the daemon is running it maintains a real OpenSSH configuration for every workstation you can reach, and rewrites it in real time as workstations are created, started, stopped, and deleted. Point your own ~/.ssh/config at it, and every OpenSSH-speaking tool reaches a workstation with no rl in the loop.

Add this as the first line of ~/.ssh/config (create the file if it does not exist, with mode 600):

Include ~/.ringleader/ssh/config

That is all. With a running daemon and a Running workstation named my-box in namespace dev, all of these now work:

ssh my-box                                  # or my-box.dev, or my-box.dev.ringleader
ssh my-box uname -a
scp report.csv my-box:/home/dev/
rsync -av ./src/ my-box:/home/dev/src/
sftp my-box
git clone my-box:/home/dev/repo.git

So do the tools that read ~/.ssh/config for you: VS Code and Cursor Remote-SSH, JetBrains Gateway, and Ansible.

Each workstation gets three Host aliases: the bare name (my-box), the namespace-qualified name (my-box.dev), and a fully-qualified form (my-box.dev.ringleader). The .ringleader suffix is a non-resolvable pseudo-TLD, so the qualified alias can never shadow a real hostname.

Put the Include above everything else

OpenSSH uses the first value it obtains for each parameter, so an Include placed after your own Host * block is silently overridden by it. In particular, a Host * block that sets IdentityFile will make ssh offer the wrong key and fail to authenticate. Put the Include line above every Host and Match block in ~/.ssh/config.

For the same reason, note that the bare alias claims the workstation’s name in your SSH namespace: a workstation called prod would shadow a Host prod of your own. Use the <name>.<namespace>.ringleader spelling when in doubt.

What Ringleader manages for you

The Include pulls in ~/.ringleader/ssh/*.conf, one file per workstation. Every file is owned by the daemon and regenerated on change. Never edit them: your edits will be overwritten, and deleting a workstation prunes its file.

Each block carries:

  • the workstation’s current address, port, and login user, refreshed when they change;
  • Ringleader’s node key as the IdentityFile, with IdentitiesOnly yes so none of your own keys are offered;
  • a per-workstation known_hosts file with the host key pinned once the daemon has confirmed it, so your own ~/.ssh/known_hosts is never touched and recreating a workstation under the same name cannot produce a host-key-changed warning;
  • connection multiplexing (ControlMaster), so repeated commands reuse one connection instead of re-handshaking. (Not on Windows: Win32-OpenSSH does not support it.)

The configuration is only as fresh as the daemon: if rl daemon is not running, the files are frozen at their last state, and an address that changed since then will not connect. Check with rl daemon status.

rl code [workstation]

Open a workstation in a desktop editor over Remote-SSH: VS Code, Cursor, or VS Code Insiders. Also available as rl workstation code.

rl code my-box
rl code my-box --editor cursor
rl code my-box --path /home/dev/other-repo
rl code my-box --print            # show the command, launch nothing

The workstation is addressed by its collision-proof <name>.<namespace>.ringleader alias, and the folder opened is the first spec.sources[] path of the resolved configuration; override it with --path. With no sources, a window connects to the host with no folder open. The editor is autodetected in the order code, cursor, code-insiders; pin one with --editor.

Remote-SSH shells out to the system ssh and reads your own ~/.ssh/config, so this needs the Ringleader include wired in (rl ssh-config install; see Using ssh directly). A missing or misplaced include is warned about, not fatal.

The first connect downloads a server

On first connect VS Code downloads its own server (~100 MB) into the workstation from update.code.visualstudio.com. A workstation with restricted outbound access may sit at “Setting up SSH Host” until that fetch can complete.
FlagDescription
-w, --workstation <name>Workstation to open (defaults to the sole Ready one; required when several are).
--editor <name>code, cursor, or code-insiders. Default: autodetect in that order.
--path <dir>Absolute folder to open on the workstation, overriding the resolved source path.
--printPrint the command that would run, one argument per line, and launch nothing.
-n, --namespace <ns>Namespace.
-A, --all-namespacesResolve across all namespaces.
--as <subject>Impersonate a subject.
--home <dir>Data directory.

rl tmux [workstation]

Attach to a persistent tmux session on the workstation (tmux new-session -A -s ringleader). On a transport drop it reconnects automatically after a backoff, ideal for long-running work over a flaky connection.

rl tmux my-box
FlagDescription
-w, --workstation <name>Target workstation. Defaults to the sole Ready one; required when several are.
--startStart the workstation first if it is stopped, and wait until it is Ready. See Starting a stopped workstation.
--start-timeout <duration>With --start, how long to wait. Default 20m; 0 means no limit.
-n, --namespace <ns>Namespace.
-A, --all-namespacesLook for the workstation in every namespace you can reach.
--as <subject>Impersonate a subject.
--home <dir>Data directory.

rl logs <workstation>

Stream the in-VM ringleader-agent logs (or a managed service’s logs), like kubectl logs. It automatically picks the systemd journal or the durable log file on the workstation.

rl logs my-box -f                 # follow
rl logs my-box --tail 200
rl logs my-box --since 30m
rl logs my-box --source service:code-server
FlagDescription
-f, --followStream new lines as they arrive.
--tail <N>Show only the last N lines (0 = all).
--since <duration>Window to show (e.g. 30m, 1h; journal only).
--source <source>agent (default) or service:<name>.
-n, --namespace <ns>Namespace.
--as <subject>Impersonate a subject.
--home <dir>Data directory.

rl troubleshoot <workstation>

Diagnose one workstation. The report has these sections, in this order:

  • The controller view: phase, conditions, last update, and the ambient SSH agent when its delivery fell short.
  • For a workstation with an egress policy, an egress section. It shows the rule set the workstation runs, the destinations it permits, and any it could not enforce. It names the edge VM on your machine that serves the workstation, when there is one. When it can, it also tests from inside the workstation which destinations answer.
  • The port forwarding on your machine: the LocalBindings that forward from the workstation, or a note that none does.
  • Live system probes: OS, docker, disk, user, agent and ssh-agent-env.
  • The in-VM agent’s provisioning timeline.

Run with no workstation to dump local status (daemon, origins/auth, version, host) instead. Either form takes --diagnostics to write a self-contained zip you can send for support, containing no tokens, secrets, or private keys.

For the whole diagnosis workflow, and how to capture and share a bundle from the desktop apps, see the Troubleshooting section and Generate a diagnostics bundle.

rl troubleshoot                          # local status to stdout
rl troubleshoot my-box                   # diagnose one workstation
rl troubleshoot my-box --failed          # only failed steps
rl troubleshoot my-box --kind devtool --since 1h
rl troubleshoot my-box --json            # machine-readable ledger
rl troubleshoot --diagnostics            # local-only zip
rl troubleshoot --all --diagnostics      # zip PLUS every workstation
rl troubleshoot my-box --diagnostics=out.zip
rl troubleshoot my-box --report-only     # only the in-VM diagnostic ledger
rl troubleshoot --diagnostics --log-since=6h   # reach 6 hours back in the daemon log

The bundled daemon log is trimmed to a byte budget, so a bundle captured a while after an incident can carry a window that begins after it. The covered wall-clock range is printed when the zip is written and recorded inside it. Check it against the time you are reporting, and use --log-since to reach further back. (--log-since moves the daemon log; --since filters a workstation’s in-VM timeline.)

FlagDescription
--diagnostics[=PATH]Write a self-contained diagnostics zip (default ./ringleader-diagnostics-<timestamp>.zip).
--allWith --diagnostics: also include every workstation in the zip (an error without --diagnostics).
--failedShow only failed timeline steps.
--kind <kind>Filter the timeline by kind (user, package, devtool, tool-config, script, security-update, self-update).
--target <substring>Filter the timeline by target (substring match).
--since <duration>Steps started within this window (e.g. 30m, 1h).
--reconcile latestShow only the steps from the most recent configuration pass.
--log-since <duration>With --diagnostics: make the bundled daemon log reach at least this far back in wall clock (e.g. 90m, 6h), across rotated segments.
--limit <N>Show only the most recent N steps (0 = remote default).
--report-onlyPrint only the in-VM diagnostic ledger (no controller view or host probes).
--jsonEmit the in-VM diagnostic ledger report as JSON.
-n, --namespace <ns>Namespace.
--as <subject>Impersonate a subject.
--home <dir>Data directory.

Note

rl troubleshoot <workstation> --report-only pulls just the in-VM diagnostic ledger over the daemon’s SSH connection (no controller view or host probes) and renders it with the same formatter the in-VM agent uses.