INCLUDED WITH SEMANTICOPS PRO

TEST
RUNNER

Guided unit testing for Power BI semantic models.

Build test suites in a desktop UI. Run them against your model. Catch regressions before changes ship.

View SemanticOps Pro How to install

Integrates with Power BI External Tools

Supports Windows and macOS

Built-in test packs

WHAT IT DOES

Build repeatable tests for every model change.

A guided UI for building suites, applying starter packs, running targeted checks, reviewing failures, and exporting results.

Guided test creation

Create semantic model tests through a guided UI instead of hand-authoring MCP payloads.

Starter packs

Start with generated packs, preview what will be created, then apply only the tests you want.

Security validation

Check RLS and OLS behavior from the same workflow used for model validation.

Regression snapshots

Detect drift by comparing current query output to a stored baseline.

Run history

Track recent runs, pass/fail status, degrading tests, and failure details.

Review-ready exports

Export Markdown, JUnit XML, HTML, or JSON for pull requests, tickets, and automation.

APP SURFACES

Inside the app.

Five surfaces - dashboard, runner, wizard, history, settings.

WHAT YOU CAN TEST

Eight supported test types.

Measure Assertion

Assert measure values with tolerance.

DAX Assertion

Assert manual DAX query results.

RLS Validation

Validate row-level security filtering by role or identity.

OLS Validation

Validate table and column access expectations.

Performance Budget

Enforce query duration thresholds.

Regression Snapshot

Compare current query output to a stored baseline.

Referential Integrity

Detect orphan keys in relationships.

Metadata Compliance

Lint descriptions, naming, formatting, folders, and model metadata.

HOW IT WORKS

Power BI launches the app. SemanticOps MCP runs the tests.

Connect to your model

Power BI Desktop on Windows, or Power BI Service from macOS.

Launch Test Runner

From the External Tools ribbon on Windows, or directly from the app on macOS.

Build and run

Apply starter packs, author suites, run targeted checks, review failures.

SemanticOps MCP

Runs the tests and stores definitions, baselines, and history.

INSTALL

Install for your platform.

Windows installers register Test Runner as a Power BI Desktop External Tool. macOS installers connect to Power BI Service.

Download Test Runner

Standard Windows installer

Registers the Power BI Desktop External Tool.

01Download and run the Windows installer.
02Approve the External Tool registration when prompted.
03Restart Power BI Desktop.
04Launch Test Runner from the External Tools ribbon.

Power BI Desktop External Tool is Windows-only. macOS connects to Power BI Service.

FAQ

Practical boundaries.

Is Test Runner included with SemanticOps Pro?

Yes. Every Pro subscription includes the desktop app on the same MCP-backed testing surface.

Where are tests, baselines, and run history stored?

SemanticOps MCP stores tests, baselines, and run history. They persist across sessions, not in the desktop UI.

Is this a standalone CI runner?

No. Test Runner exports JUnit XML, Markdown, HTML, and JSON for automation, but CI execution is a separate workflow.

What platforms are supported?

Windows runs the Power BI Desktop External Tool flow. macOS runs the same app and connects to Power BI Service.

How does masking behave in the desktop app?

Test Runner reads model values directly so test results are accurate. Masking policies in other MCP hosts are unaffected by what you see here.