/testdriver-elements

name: testdriver:elements
description: Locate and interact with UI elements using AI

<!-- Generated from elements.mdx. DO NOT EDIT. -->

Overview

The TestDriver element finding system uses AI. It finds elements on the screen with natural language descriptions. The `find()` method returns an `Element` object. You can interact with the object.

Finding Elements

find()

Find an element on the screen with a natural language description.

const element = await testdriver.find(description)

**Parameters:**

• `description` (string) - A natural language description of the element to find

**Returns:** `Promise<Element>` - The Element instance that TestDriver found

**Example:**

// Find a button
const submitButton = await testdriver.find('the submit button');

// Find an input field with context
const emailField = await testdriver.find('email input field in the login form');

// Find an element by visual characteristics
const redButton = await testdriver.find('red button in the top right corner');

<Tip> Be specific in your descriptions. Include visual details, location context, or nearby text to make the accuracy better. </Tip>

Element Class

The `Element` class represents a located (or to-be-located) UI element. It provides methods for interaction and properties for element information.

Methods

found()

Check if the element was successfully located.

element.found()

**Returns:** `boolean` - True if element coordinates were found

**Example:**

const element = await testdriver.find('login button');
if (element.found()) {
  await element.click();
} else {
  console.log('Element not found');
}

find()

Re-locate the element, optionally with a new description.

await element.find(newDescription)

**Parameters:**

• `newDescription` (string, optional) - New description to search for

**Returns:** `Promise<Element>` - This element instance

**Example:**

// Re-locate if the UI changed
const element = await testdriver.find('submit button');
// ... page updates ...
await element.find(); // Re-locate with same description

// Or update the description
await element.find('blue submit button'); // Now looking for blue button

click()

Click on the element.

await element.click(action)

**Parameters:**

• `action` (string, optional) - Type of click: `'click'` (default), `'double-click'`, `'right-click'`, `'hover'`, `'mouseDown'`, `'mouseUp'`

**Returns:** `Promise<void>`

**Example:**

const button = await testdriver.find('submit button');
await button.click(); // Regular click

const file = await testdriver.find('document.txt');
await file.click('double-click'); // Double-click

const menu = await testdriver.find('settings icon');
await menu.click('right-click'); // Right-click

<Note> The element must be found before clicking. The `find()` method automatically locates the element. </Note>

hover()

Hover over the element without clicking.

await element.hover()

**Returns:** `Promise<void>`

**Example:**

const tooltip = await testdriver.find('info icon');
await tooltip.hover();
// Wait to see tooltip
await new Promise(resolve => setTimeout(resolve, 1000));

doubleClick()

Double-click on the element.

await element.doubleClick()

**Returns:** `Promise<void>`

**Example:**

const file = await testdriver.find('README.txt file icon');
await file.doubleClick();

rightClick()

Right-click on the element to open context menu.

await element.rightClick()

**Returns:** `Promise<void>`

**Example:**

const folder = await testdriver.find('Documents folder');
await folder.rightClick();

mouseDown() / mouseUp()

Press or release mouse button on the element (for drag operations).

await element.mouseDown()
await element.mouseUp()

**Returns:** `Promise<void>`

**Example:**

// Drag and drop
const item = await testdriver.find('draggable item');
await item.mouseDown();

// Move to drop target (using coordinates or another element)
const target = await testdriver.find('drop zone');
await target.hover();
await target.mouseUp();

Properties

Element properties provide additional information about located elements. Properties are available after a successful `find()` call.

coordinates

Get the element's coordinates object containing all position information.

const coords = element.getCoordinates()
// or access directly
element.coordinates

**Returns:** `Object | null` - Coordinate object with `{ x, y, centerX, centerY }`

**Example:**

const button = await testdriver.find('submit button');
const coords = button.coordinates;

if (coords) {
  console.log(`Top-left: (${coords.x}, ${coords.y})`);
  console.log(`Center: (${coords.centerX}, ${coords.centerY})`);
}

x, y, centerX, centerY

Direct access to coordinate values. Always available after successful `find()`.

element.x        // Top-left X coordinate (number)
element.y        // Top-left Y coordinate (number)
element.centerX  // Center X coordinate (number)
element.centerY  // Center Y coordinate (number)

**Example:**

const button = await testdriver.find('submit button');
console.log(`Button at: (${button.x}, ${button.y})`);
console.log(`Button center: (${button.centerX}, ${button.centerY})`);

// Use for custom mouse operations
await testdriver.click(button.centerX, button.centerY);

width, height

Element dimensions in pixels. Available when AI detects element bounds.

element.width   // Width in pixels (number | null)
element.height  // Height in pixels (number | null)

**Example:**

const button = await testdriver.find('submit button');

if (button.width && button.height) {
  console.log(`Button size: