Skip to content

test — runner

Runner

Functions

runLocally

async function runLocally(snippet: ExtractedSnippet, config: RunnerConfig): Promise<TestResult>
TypeScript

Use runLocally to validate an extracted code snippet by executing it in a sandboxed temp directory, confirming it runs without errors before publishing it to your docs.

Call this during your documentation pipeline after extracting snippets from source — it's the quality gate that ensures every code example in your generated docs actually works. If a snippet fails, you'll know before your users do.

runLocally writes the snippet to a freshly created temp directory, runs it in a clean environment (stripped of your local env vars), and tears down the directory when finished. This isolation prevents snippets from accidentally depending on your machine's global state.

Parameters

NameTypeRequiredDescription
snippetExtractedSnippetYesThe code snippet to validate — typically produced by skrypt's scanner. Must include the code content, language, and source metadata.
configRunnerConfigYesExecution settings: timeout, allowed env vars, runtime binary paths, and whether to clean up the temp directory on failure.

Returns

Returns a Promise<TestResult> containing the exit code, stdout/stderr output, execution duration, and a passed boolean. Use result.passed to gate whether the snippet gets written to your MDX output, and result.stderr to surface failure details in your CI logs.

Heads up

  • The clean environment means snippets that rely on process.env variables (API keys, base URLs) will fail unless you explicitly allowlist them in RunnerConfig. Pass test credentials, not production ones.
  • Temp directories are removed on success by default. If a run fails and you need to inspect the written files, check your runner config for a keepOnFailure option before assuming the directory is gone.

Example:

const { execSync } = require("child_process");
const { writeFileSync, mkdirSync, rmSync, existsSync } = require("fs");
const { join } = require("path");
const { tmpdir } = require("os");
const { randomUUID } = require("crypto");

// Inline types — do not import from autodocs
const ExtractedSnippet = {
  id: "snippet_7f3a2c",
  language: "typescript",
  code: `const result = 1 + 1;\nconsole.log("sum:", result);`,
  sourceFile: "src/math/add.ts",
  functionName: "add",
};

const RunnerConfig = {
  timeoutMs: 5000,
  allowedEnvVars: ["NODE_ENV", "PATH"],
  keepOnFailure: true,
  runtimeBin: process.execPath, // node binary
};

// Self-contained implementation of runLocally
async function runLocally(snippet, config) {
  const startTime = Date.now();
  const runDir = join(tmpdir(), `skrypt-run-${randomUUID()}`);

  mkdirSync(runDir, { recursive: true });

  const ext = snippet.language === "typescript" ? ".ts" : ".js";
  const entryFile = join(runDir, `snippet${ext}`);

  // For this example, run as plain JS
  const jsFile = join(runDir, "snippet.js");
  writeFileSync(jsFile, snippet.code);

  const cleanEnv = Object.fromEntries(
    config.allowedEnvVars
      .filter((key) => process.env[key] !== undefined)
      .map((key) => [key, process.env[key]])
  );

  let stdout = "";
  let stderr = "";
  let exitCode = 0;

  try {
    stdout = execSync(`${config.runtimeBin} ${jsFile}`, {
      env: cleanEnv,
      timeout: config.timeoutMs,
      encoding: "utf8",
    });
  } catch (err) {
    stderr = err.stderr || err.message;
    exitCode = err.status || 1;
  } finally {
    const passed = exitCode === 0;
    if (passed || !config.keepOnFailure) {
      rmSync(runDir, { recursive: true, force: true });
    }
  }

  return {
    snippetId: snippet.id,
    passed: exitCode === 0,
    exitCode,
    stdout: stdout.trim(),
    stderr: stderr.trim(),
    durationMs: Date.now() - startTime,
  };
}

async function main() {
  try {
    const result = await runLocally(ExtractedSnippet, RunnerConfig);

    console.log("Test result:", {
      passed: result.passed,
      output: result.stdout,
      durationMs: result.durationMs,
    });

    if (!result.passed) {
      console.error("Snippet failed — skipping MDX output.");
      console.error("stderr:", result.stderr);
      process.exit(1);
    }

    console.log(`✓ Snippet ${result.snippetId} validated in ${result.durationMs}ms — safe to publish.`);
  } catch (err) {
    console.error("Runner error:", err.message);
    process.exit(1);
  }
}

main();
// Expected output:
// Test result: { passed: true, output: 'sum: 2', durationMs: 87 }
// ✓ Snippet snippet_7f3a2c validated in 87ms — safe to publish.
TypeScript
Was this helpful?