TypeScript SDK for the OAGI API - vision-based task automation.
OAGI is the TypeScript SDK for Lux, the world's most advanced computer-use model from the OpenAGI Foundation.
Computer Use is AI's ability to operate human-facing software — not just through APIs, but by operating computers natively, just as human users do. It's a paradigm shift in what AI can do: not just generating, reasoning, or researching, but actually operating on your computer.
Lux comes in three modes, giving you control over depth, speed, and style of execution:
- Tasker — Strictly follows step-by-step instructions with ultra-stable, controllable execution
- Actor — Ideal for immediate tasks, completing actions at near-instant speed
- Thinker — Understands vague, complex goals, performing hour-long executions
With Lux, possibilities are endless. Here are a few examples:
- Web Scraping & Data Crawl — Navigate websites, sort results, and collect product information autonomously
- Software QA — Automate repetitive testing tasks, navigate applications, perform test actions, and validate expected behaviors
- Financial Data Extraction — Navigate to sites like NASDAQ and extract insider activity data
- Data Entry — Enter accurate data across dashboards and forms
- Workflow Automation — Chain together multi-step tasks across different applications
# If you are using Node.js
npm install @oagi/oagi
yarn add @oagi/oagi
pnpm add @oagi/oagi
# If you are using Deno
deno add npm:@oagi/oagi
# If you are using Bun
bun install @oagi/oagiSet your API credentials:
export OAGI_API_KEY="your-api-key" # get your API key from https://developer.agiopen.org/
# export OAGI_BASE_URL="https://api.agiopen.org/", # optional, defaults to production endpointRun tasks automatically with screenshot capture and action execution:
import { DefaultActionHandler, DefaultAgent, ScreenshotMaker } from '@oagi/oagi';
const agent = new DefaultAgent();
await agent.execute(
'Search weather on Google',
new DefaultActionHandler(),
new ScreenshotMaker(),
);Run agents directly from the terminal:
# Run with actor model
oagi agent run "Go to nasdaq.com, search for AAPL. Under More, go to Insider Activity" --model lux-actor-1
# Run with thinker mode (uses lux-thinker-1 model with more steps)
oagi agent run "Look up the store hours for the nearest Apple Store to zip code 23456 using the Apple Store Locator" --model lux-thinker-1
# Run pre-configured tasker workflows (no instruction needed)
oagi agent run --mode tasker:software_qa
# List all available modes
oagi agent modes
# Check macOS permissions (screen recording & accessibility)
oagi agent permission
# Export execution history
oagi agent run "Complete the form" --export html --export-file report.htmlCLI options:
--mode: Agent mode (default: actor). Useoagi agent modesto list available modes--model: Override the model (default: determined by mode)--max-steps: Maximum steps (default: determined by mode)--temperature: Sampling temperature (default: determined by mode)--step-delay: Delay after each action before next screenshot (default: 0.3s)--export: Export format (markdown, html, json)--export-file: Output file path for export
Process and optimize images before sending to API:
import sharp from 'sharp';
const compressed = await sharp('large_screenshot.png')
.resize(1260, 700, { fit: 'fill' })
.jpeg({ quality: 85 })
.toBuffer();For step-by-step control over task execution:
import { Actor, DefaultActionHandler, ScreenshotMaker } from '@oagi/oagi';
const actor = new Actor();
actor.initTask('Complete the form');
const image_provider = new ScreenshotMaker();
const action_handler = new DefaultActionHandler();
for (let i = 0; i < 10; ++i) {
const image = await image_provider.provide();
const step = await actor.step(image);
if (step.stop) break;
await action_handler.handle(step.actions);
}For full Lux documentation and guides, visit the OAGI Developer Documentation.
MIT