Skyvern (github.com)

Skyvern automates browser-based workflows using large language models and computer vision. It provides a Playwright-compatible SDK that adds AI functionality on top of Playwright, along with a no-code workflow builder, so both technical and non-technical users can automate manual tasks on any website. It replaces brittle automation built on DOM parsing and XPath selectors that break whenever a website's layout changes.

Skyvern uses a swarm of agents, inspired by the Task-Driven autonomous agent design behind BabyAGI and AutoGPT, to comprehend a website and plan and execute its actions through browser automation libraries like Playwright. Rather than relying on pre-determined XPaths, it uses Vision LLMs to map visual elements on a page to the actions needed to complete a workflow, so it can operate on websites it has never seen before and stays resistant to layout changes.

The SDK adds AI-powered page commands directly on the Playwright page object: act performs actions from natural language prompts, extract pulls structured data with an optional JSON schema, validate checks page state and returns a boolean, and prompt sends arbitrary instructions to the LLM. Higher-level agent commands run complete multi-step tasks, log in with stored credentials from Skyvern, Bitwarden or 1Password, download files, and execute pre-built workflows.

Skyvern Cloud is a managed version that runs multiple Skyvern instances in parallel with bundled anti-bot detection, a proxy network and CAPTCHA solvers; sign-up happens at app.skyvern.com. Skyvern can also run locally via pip install or Docker Compose, with a Python SDK (pip install skyvern) and a TypeScript client (npm install @skyvern/client). It is released under the AGPL-3.0 license.

Use cases

• Completing a checkout flow end-to-end from a single natural language command
• Finding information on a website from a single task prompt