tvlabs execute

Experimental

tvlabs execute is experimental. Its flags, the TVLABS_* environment variables it sets, and its exit codes may all change without a major version bump, and it is not yet covered by the deprecation policy the rest of the CLI follows.

Pin your CLI version if you depend on it from CI, and tell us what you need at support@tvlabs.ai — feedback now is what settles the interface.

Running your own test harness against a real device

Use tvlabs execute to request a device, connect to it, and run a script of yours against it. Where tvlabs run executes automation workflows you built in the TV Labs web interface, execute runs your harness — Appium, WebDriver, Playwright, a shell script, whatever your suite already uses — and gets out of the way.

tvlabs execute ./run_suite.sh --target 'platform_key:android_tv AND platform_os_variant:fire_tv AND year>=2023'

The command requests a device matching --target, waits for it to warm up, opens the tunnel, then runs your entrypoint with environment variables describing the session. Your entrypoint's exit code becomes the command's exit code. When it finishes, the session is ended and the tunnel torn down.

Everything the command prints itself goes to stderr. Stdout belongs entirely to your entrypoint, so a harness whose output is machine-read — TAP, JUnit on stdout, a $(tvlabs execute ...) substitution — stays clean.

The entrypoint

The entrypoint is the first argument. It runs wherever the CLI runs — your shell, a CI runner, a container — not on the device. It reaches the device over the tunnel.

# Both spellings work; the positional form is shorter.
tvlabs execute ./run_suite.sh --target 'platform_key:roku'
tvlabs execute --entrypoint ./run_suite.sh --target 'platform_key:roku'

It is executed directly, without a shell, so it takes no arguments of its own — put what your harness needs in --env or inside the script. Relative paths resolve against the directory you ran the command from, not against --context.

Environment contract

Two rules hold for every variable below, and both exist so that what is behind these values can change — SSH forwards today, something else later — without your harness changing with it:

• Every value is ready to use. A host:port is complete and a port is one the local end really binds. Never assemble an address from parts, and never assume a default.
• A variable that does not apply to your device's platform is absent, not empty. So ${TVLABS_ADB_ADDR:?} is a working assertion, and a harness can branch on whether a variable is set instead of on the platform key.

Every run

Variable	Description
TVLABS_SESSION_ID	The session's UUID
TVLABS_SESSION_URL	Link to the session in the TV Labs web interface
TVLABS_PLATFORM	Platform key of the matched device, e.g. android_tv. This is the platform, not the OS variant — a Fire TV or Google TV device reports android_tv
TVLABS_DEVICE_NAME	The matched device's name in TV Labs
TVLABS_DEVICE_IP	The device's address on the sidecar's private network. Useful for logs and correlation — not dialable from your harness; use TVLABS_DEVICE_HOST and the ports below
TVLABS_DEVICE_HOST	Host to reach the device's forwarded ports on. Normally the tunnel's bind address (see --bind-address); under --containerized it is host.docker.internal, the container's route back to your machine. Every port below that you dial is reachable there; TVLABS_APP_PORT is the exception
TVLABS_APP_PORT	Port your own web app should listen on for the device to reach it at local.tvlabs.ai. This one is a reverse forward — the CLI connects to your app on loopback whatever --bind-address says — so it is a port you serve, not one you dial
TVLABS_OUTPUT_DIR	Directory the CLI creates for this session's artifacts, at <--context>/.tvlabs/<session id>. Write your reports there and your CI step can collect them from the workspace. Under --containerized this is a /workspace/... path; the files still land under --context on the host

Per platform

Each platform adds the addresses its own tooling needs:

Platform	Adds
android_tv, android_mobile	TVLABS_ADB_ADDR, TVLABS_ADB_SERIAL, TVLABS_INSPECTOR_PORT
tizen	TVLABS_SDB_ADDR, TVLABS_CERTIFICATE_URL, TVLABS_INSPECTOR_PORT
web_os	TVLABS_ARES_DEVICE, TVLABS_SSH_ADDR, TVLABS_INSPECTOR_PORT
roku	TVLABS_ECP_PORT, TVLABS_ROKU_DEV_PORT
vega	TVLABS_VDA_ADDR, TVLABS_INSPECTOR_PORT
tivo	TVLABS_INSPECTOR_PORT
smartcast	TVLABS_CONJURE_PORT, TVLABS_CONJURE_TOKEN
viera	nothing
google_chrome, firefox, safari	nothing — the debugger and WebDriver are forwarded on fixed ports (9222 and 9515) that --inspector-port does not move

Variable	Description
TVLABS_ADB_ADDR	host:port of the adb server the device is attached to. Point a client at it — adb -H "${TVLABS_ADB_ADDR%:*}" -P "${TVLABS_ADB_ADDR##*:}", or ANDROID_ADB_SERVER_ADDRESS / ANDROID_ADB_SERVER_PORT. Nothing to adb connect
TVLABS_ADB_SERIAL	The device's serial as that adb server reports it, so adb -s "$TVLABS_ADB_SERIAL" shell … works without parsing adb devices
TVLABS_SDB_ADDR	host:port of the sdb server the Tizen device is attached to
TVLABS_CERTIFICATE_URL	URL of the signing certificate the Tizen device trusts. A .wgt has to be signed with it before sdb install will accept it. Presigned, so it expires with the session
TVLABS_ARES_DEVICE	Name of the ares device profile registered for this session — always pass it: ares-install -d "$TVLABS_ARES_DEVICE" app.ipk. An ares command without -d targets the LG emulator, not your device
TVLABS_SSH_ADDR	host:port of the webOS device's SSH service. A host run's ares profile dials it directly; a containerized one dials a relay onto it
TVLABS_VDA_ADDR	host:port of the Vega device's VDA debug bridge. VDA is Amazon's rebuild of adbd, so a stock adb drives it: adb connect "$TVLABS_VDA_ADDR". Unlike TVLABS_ADB_ADDR this is the device, not a server — your own adb server attaches to it, and the serial it then answers to is that same address
TVLABS_CONJURE_PORT	Local port forwarded to the Vizio device's Conjure Launcher, the REST API that sends keys, launches apps and reads device state
TVLABS_CONJURE_TOKEN	The Conjure AUTH token for this device. Every Conjure operation worth making needs it: without it the TV answers REQUIRES_PAIRING, and pairing needs a PIN read off the screen. Pass it as an AUTH header
TVLABS_ECP_PORT	Local port forwarded to the Roku device's ECP (External Control Protocol) service
TVLABS_ROKU_DEV_PORT	Local port forwarded to the Roku Development Application Installer — where a sideload is POSTed

TVLABS_INSPECTOR_PORT

Set only on the platforms above, and only because something is listening on it there. It means "a debugger is reachable here" — which debugger, and what is serving it, differs by platform:

Platform	What answers on the port
tizen	The sidecar's debug proxy in front of the device's inspector
web_os	The device's own inspector
vega	The unit's Node inspector
tivo	The unit's debugger
android_tv, android_mobile	The CLI's own DevTools bridge, which speaks CDP to your app's WebView through the device's adb server — this is the address chrome://inspect reaches the device at

On roku, smartcast, viera and the browser platforms the variable is absent: their debuggers listen on fixed ports that --inspector-port does not move (smartcast on 9555, the browsers on 9222 with WebDriver on 9515, Roku's BrightScript console on 8085 and its debug server on 8080), or there is no inspector at all. Earlier CLI versions set the variable on every platform, which on those advertised a port nothing was listening on.

Reserved names, and credentials

Every TVLABS_* name in this contract is reserved on every platform, not just on the one you are targeting: passing one as --env is an error rather than a silent override, so a CI job's flags mean the same thing whichever device it matches. A reserved name your platform does not define is also removed from the inherited environment, so a nested run cannot leak a stale value into an inner one.

Your entrypoint inherits the rest of the CLI's environment, minus TV Labs credentials: TVLABS_API_KEY and any other TVLABS_* variable that names a key, token, secret, password or passphrase is dropped from what it inherits, rather than handed to a test runner that may log its environment. A credential the session itself provides, such as TVLABS_CONJURE_TOKEN, is still set: the rule removes stale inherited ones, it does not withhold the one your device needs. Your own secrets (cloud keys, registry tokens) are untouched. If your harness genuinely needs the API key, pass it explicitly:

tvlabs execute ./run_suite.sh \
  --target 'platform_key:tizen' \
  --env "TVLABS_API_KEY=$TVLABS_API_KEY"

Everything else passes straight through:

tvlabs execute ./run_suite.sh \
  --target 'platform_key:tizen' \
  --env SUITE=smoke \
  --env RETRIES=2

Installing a build

Upload and install a build as part of device warmup with --build:

tvlabs execute ./run_suite.sh --target 'platform_key:tizen' --build ./app.wgt

To reuse a build you already uploaded with tvlabs upload, pass its id instead. The two are mutually exclusive — a mistyped path fails immediately rather than being sent to the server as a build id that doesn't exist.

tvlabs execute ./run_suite.sh --target 'platform_key:tizen' --build-id 408cb137-1b75-4322-b8f6-9eccf5b6b149

Local requirements

Your harness drives the device with the platform's own tools, and those run on this host, not on the device: adb for Android TV, Fire TV and Google TV, sdb for Tizen, the ares CLI for webOS. The tunnel also binds a local port for every service the platform forwards — the session's adb server on 5037 for Android, its sdb server on 26099 for Tizen, and the rest of that platform's ports — and it cannot start if any one of them is already taken, whether by a vendor server of your own or by an unrelated dev server.

When --target names a platform_key, the tool and those ports are checked before a device is requested, so a host that cannot run the platform exits 1 in under a second, with no device allocated and no session created:

port 5037 is already in use. The TV Labs CLI serves the session's adb server on that port, so stop your local one with `adb kill-server` and try again

If the target does not name one platform the CLI recognises, the platform is not known until the server matches a device, and the same checks run then instead — before the tunnel is opened. That case ends the session and still exits 1 rather than 125: the fault is on this host, and no retry can install a missing tool for you.

--webos-tools-dir is part of the answer for webOS: pass it and the check looks for ares-setup-device in that directory instead of on PATH, which is where setup will look for it too. It applies to host runs only: a containerized run has an ares CLI of its own.

--containerized is the other answer — see below.

Containerized runs

--containerized runs your entrypoint in a container on your own machine instead of directly on the host. Two things come with that:

• The image owns the toolchain. Your harness gets what the image provides, not what happens to be installed here, and none of your environment.
• No fixed local ports. The device forwards bind ephemeral ports instead of the platform's documented ones, so a run no longer collides with an adb server, a dev server on 8080, or a second tvlabs run — and two runs can share one machine.

This is not a security boundary. It is your machine, your image and your code; the value is reproducibility and staying out of the way of your own tooling.

tvlabs execute ./ecp_probe.sh \
  --containerized \
  --target 'platform_key:roku' \
  --context .

Read the ports from the contract, never from a literal

This is the one rule the mode depends on. TVLABS_ECP_PORT is 8060 on a host run and something like 49213 in a container — both are correct, because both are what the local end actually bound. A harness that hardcodes 8060 works in one mode and not the other; one that reads the variable works in both, and gains collision immunity for free.

curl -fsS "http://$TVLABS_DEVICE_HOST:$TVLABS_ECP_PORT/query/device-info"

What changes

	Host	Containerized
TVLABS_DEVICE_HOST	--bind-address (default localhost)	host.docker.internal
TVLABS_OUTPUT_DIR	<--context>/.tvlabs/<session>	/workspace/.tvlabs/<session>
Forwarded ports	the platform's documented ports	ephemeral, reported in the contract
Environment	yours, minus TV Labs credentials	only the TVLABS_* contract and your --env

--context is mounted at /workspace and is the only host directory the container can see, so your entrypoint has to live inside it. Artifacts written to TVLABS_OUTPUT_DIR still land under --context on the host, through the mount — the path the harness sees is just spelled differently.

Exit codes, streamed stdout and stderr, and signal handling are unchanged: your harness's exit code is still the command's, and a Ctrl-C still gives it SIGTERM, a grace period, then SIGKILL. The signal reaches your harness's children here too, at the same instant it reaches the harness.

One thing does differ, and it bites a harness that flushes on the way out. Your entrypoint is the container's init child, so when it exits the container goes with it and anything it spawned is killed immediately — a child part-way through writing a report loses the rest of its grace period. If your harness has children whose output you need, wait for them in your SIGTERM handler rather than exiting straight away.

Requirements and limits

• macOS with Docker Desktop. Linux and Windows are not supported yet, and podman is not either.

• roku, android_tv, android_mobile, web_os, vega and tizen only. A --target naming another platform is refused before a device is requested. Run those on the host for now.

• --app-port does not reach into the container. That forward is reversed: the CLI dials your app on the host's loopback, so serve it on the host rather than inside the container.

• --inspector-port stays fixed, because it exists for you to point a browser at. It is the one local port a containerized run still holds.

• What the image carries. Debian with bash, python3 and pip, node and npm, a headless JRE, adb, sdb, the ares-* webOS CLI, curl, jq, unzip and procps. There is no git and no javac.

  The platform's tooling is wired up before your entrypoint starts, so it reaches the device with no flags of yours:

  • Android. ANDROID_ADB_SERVER_ADDRESS, ANDROID_ADB_SERVER_PORT and ANDROID_SERIAL are set, so adb shell … just works.
  • webOS. The device is registered as the ares profile TVLABS_ARES_DEVICE names, so ares-install -d "$TVLABS_ARES_DEVICE" app.ipk just works. Note --webos-tools-dir is a host-run flag and is refused here: the container's own ares CLI is the one your harness reaches.
  • Vega. The container's adb is already attached to the device and ANDROID_SERIAL is set, so adb shell vpm list packages just works. On a host run you attach your own server yourself, with adb connect "$TVLABS_VDA_ADDR".
  • Tizen. sdb is already pointed at this session's device, so sdb shell … works with no flags. On arm64 it is the amd64 binary under qemu-x86_64, because Samsung has never published an arm64 Linux sdb.

  PYTHONUNBUFFERED is set so a Python harness streams its output as it runs.

  To add anything, build on the image and pass --image. Your image keeps all of that wiring, because it comes from the image's entrypoint.

  FROM tvlabsai/execute-runner:0.1.67
  RUN pip install --no-cache-dir pytest

Exit codes

Built so a CI retry policy can tell your failures from ours:

Code	Meaning
entrypoint's own	Your harness ran to completion. Its status is the command's status
1	A local error — bad flags, a --target the server rejects, or a host that cannot run the platform's tools (see Local requirements). Never worth retrying unchanged
124	No device was provisioned and connected within --match-timeout
125	The platform failed before your entrypoint started. Safe to retry
126	The session died under a running entrypoint. Any results it published are partial
130 / 143	Interrupted by SIGINT / SIGTERM

125 and 126 are the two worth wiring into a retry rule: 125 means nothing of yours ran, so a retry costs nothing, while 126 means it ran against a device that disappeared.

Options

Flag	Default	Description
--target	required	KQL device query
--region	—	Teleport region for device selection
--entrypoint	—	Script or binary to run. The positional argument is the shorter spelling
--context	.	Working directory for the entrypoint
--env	—	KEY=VALUE passed to the entrypoint. Repeatable
--build	—	Path to a build file to upload and install during warmup
--build-id	—	Existing build id to install during warmup
--match-timeout	30m	How long to wait for a device. Does not bound your entrypoint's run
--inspector-port, -i	9222	Local port forwarded to the device's web inspector
--app-port, -p	50552	Port of a locally running web application to forward to the device
--bind-address, -b	localhost	Address to bind the tunnel to, and the TVLABS_DEVICE_HOST your entrypoint dials
--webos-tools-dir	—	Directory containing the ares-* CLI tools for webOS, when they are not on PATH. Host runs only
--containerized	false	Run the entrypoint in a container on this machine. See Containerized runs
--image	—	Container image for --containerized. Defaults to tvlabsai/execute-runner:<cli version>

note

--match-timeout bounds provisioning only. Your entrypoint's own run is not bounded here — a long one is bounded by the session's server-side timeout. Name your own timeout inside the harness if you need one.

Signals and cleanup

Your entrypoint runs in its own process group, so a Ctrl-C or a CI job cancellation reaches the whole tree — including anything your harness spawned — rather than orphaning children against a device whose session is about to end. It gets SIGTERM first with a grace period to flush results, then SIGKILL.

Because it is a background process group, an interactive read from a terminal would stop it. Stdin is therefore passed through when it is a pipe or a file — so a CI job feeding its harness on stdin keeps working — and detached when it is a terminal.

GitHub Actions example

name: TV Device Tests
on: [push, pull_request]

jobs:
  device-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install TV Labs CLI
        run: /bin/bash -c "$(curl -fsSL https://tvlabs.ai/install.sh)"

      - name: Run suite against a Fire TV
        env:
          TVLABS_API_KEY: ${{ secrets.TVLABS_API_KEY }}
        run: |
          tvlabs execute ./run_suite.sh \
            --target 'platform_key:android_tv AND platform_os_variant:fire_tv AND year>=2023' \
            --build ./app.apk \
            --env SUITE=smoke