Package Exports

@cheepcode/ask-screen
@cheepcode/ask-screen/dist/index.js

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (@cheepcode/ask-screen) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

ask-screen

TS library to ask AI about what is on the browser screen. Helps agents author end-to-end tests and interact with websites.

Uses OpenAI's API to ask questions and get answers. Requires an OpenAI API key.

Installation

npm install ask-screen

Usage

Get a description of the screen:

import { AskScreen } from "ask-screen";

const askScreen = new AskScreen({
  openaiApiKey: "your-openai-api-key",
});
const description = await askScreen.description();
console.log(description);

Ask a single yes/no question about what is on the screen:

const answer = await askScreen.boolean(
  'Is there a button with text "Click me" on the screen?'
);
console.log(answer);

Ask a numeric question about what is on the screen:

const answer = await askScreen.numeric("How many buttons are on the screen?");
console.log(answer);

Ask a multiple choice question about what is on the screen:

const answer = await askScreen.multipleChoice(
  "Which of the following text elements do you see on the screen?",
  [
    'A button with text "Click me"',
    'A text input with placeholder "Enter your name"',
    'A checkbox with label "I agree to the terms and conditions"',
  ]
);
console.log(answer);

Ask an open-ended question about what is on the screen:

const answer = await askScreen.open(
  "What are the top stories on the homepage?"
);
console.log(answer);

License

This project is licensed under the MIT License. See the LICENSE file for details.