SEK SystemsDocs Open console

Troubleshooting

Agent problems

Start with the basics on the host:

systemctl status sek-agent
journalctl -u sek-agent -n 100 --no-pager
sudo sec-supervisor status --dir /opt/sec-agent

On Windows: Get-Service SEKSupervisor and C:\ProgramData\SEC\agent.log.

Agent offline

The console marks an agent offline when it has not been seen for longer than your threshold (default 30 minutes).

Check How
Does the service run? systemctl status sek-agent. If it is stopped: sudo systemctl enable --now sek-agent.
Is the host on? Agents of hosts that are shut down are simply offline.
Can the host reach SEK Systems? curl -sS -o /dev/null -w '%{http_code}\n' https://console.sek.systems/repo/tuf/timestamp.json should print 200.
Does a firewall, web filter or proxy block it? See Firewalls and proxies.
Is the host clock right? timedatectl. See Clock.
Was the agent revoked? In the console, the host shows "This agent registration is revoked". A revoked agent can never connect again; enroll the host anew.

After a network interruption the agent reconnects by itself, with a waiting time that grows from 1 second to at most 15 minutes. Restarting the service makes it try again at once: sudo systemctl restart sek-agent.

Expired identity

The agent identity is renewed automatically while the agent is online. A host that was offline for a long time reconnects with a recovery procedure if its identity expired less than 180 days ago. After that, it has to be enrolled again: revoke the old agent and run a new installation line.

Self-update not possible

The console shows Self-update not possible on <host> with a reason:

Reason What to do
installed without supervisor — rerun the installer Run a new installation line on the host. It converts the installation in place, keeps the identity, never downgrades and restores the old installation if the new one does not come up.
no update repository configured — rerun the installer Run a new installation line.
the agent has never reported an update check — rerun the installer Run a new installation line, and check outbound HTTPS.
no update check for more than 24 hours Run sudo sec-agent update-check --update-repo https://console.sek.systems/repo/tuf/: it says per mirror why the check fails (dns, tls, timeout, refused, connect, blocked, http, tuf).

Other reasons in the agent log:

  • policy:self_update: the local policy has self_update = false. That is your choice; set it to true to allow updates.
  • skipped: supervisor fell back from this version: this version was tried and did not come up healthy; it is not tried again. A newer version will be.
  • no trusted TUF root: the trust anchor /etc/sec/tuf-root.json is missing. Run a new installation line.

Firewalls and proxies

The agent needs direct outbound HTTPS (TCP 443) to console.sek.systems and console.sek.network; see Requirements.

  • Allow the names, not IP addresses.
  • HTTP proxies are not supported for the agent connection. If your hosts can only reach the internet through a proxy, ask SEK Systems support about a relay: one host with direct access that forwards the connections of the others (it must allow the role with sec-agent role allow relay).
  • TLS inspection breaks the connection, because the agent only accepts the real certificate. Exclude the names from inspection.

Certificate errors

Symptom Cause
invalid peer certificate: UnknownIssuer (or similar) in the log Something between the host and SEK Systems replaces the certificate: TLS inspection, a captive portal or a transparent proxy. Exclude the names from inspection.
Certificate not yet valid or expired The host clock is wrong. See Clock.
Connection refused after a working period, on your own domain Check the domain under Settings → Domains; a failed DNS check is shown there.

Clock

The agent refuses jobs whose time does not match its own clock by more than 5 minutes (clock_skew), and TLS fails with a clock that is far off. Keep the time synchronised with NTP (timedatectl set-ntp true) or the Windows time service.

Host in the wrong environment

The installation line enrolls the host in the environment where you created it. If a host ended up in the wrong environment:

  1. Revoke the agent in the wrong environment (Agents → host → Revoke agent).

  2. In the right environment, create a new installation line.

  3. On the host, remove the old identity and install again:

    sudo systemctl stop sek-agent
    sudo rm -rf /var/lib/sec-agent
    curl -fsSL https://console.sek.systems/i/<new code> | sudo sh
    

The local policy stays as it was.

Jobs are refused

A refused job shows the reason in the plan; the list of reasons is in Local policy → Refusals. The most common: the action is not allowed by the profile (policy:allowed_actions), the job came outside the local windows (policy:local_windows), or the hourly limit was reached (policy:max_jobs_per_hour).

Enrollment errors

See enroll for the server errors (token_invalid, wrong_endpoint, seat_exhausted) and Install on Linux for the installer messages.

Collecting information for support

sec-agent --version
sudo sec-supervisor status --dir /opt/sec-agent
sudo sec-agent policy
journalctl -u sek-agent --since -2h --no-pager > sek-agent.log

Read the log before you share it, and remove anything you do not want to send.