ScreenEnv: Deploy your full stack Desktop Agent

TL;DR : ScreenEnv is a powerful Python library that lets you create isolated Ubuntu desktop environments in Docker containers for testing and deploying GUI Agents (aka Computer Use agents). With built-in support for the Model Context Protocol (MCP), it's never been easier to deploy desktop agents that can see, click, and interact with real applications.

Imagine you need to automate desktop tasks, test GUI applications, or build an AI agent that can interact with software. This used to require complex VM setups and brittle automation frameworks.

ScreenEnv changes this by providing a sandboxed desktop environment that runs in a Docker container. Think of it as a complete virtual desktop session that your code can fully control - not just clicking buttons and typing text, but managing the entire desktop experience including launching applications, organizing windows, handling files, executing terminal commands, and recording the entire session.

ScreenEnv provides two complementary ways to integrate with your agents and backend systems, giving you flexibility to choose the approach that best fits your architecture:

This dual approach means ScreenEnv adapts to your existing infrastructure rather than forcing you to change your agent architecture.

Ideal for AI systems that support the Model Context Protocol:

ScreenEnv aims to expand beyond Linux to support Android, macOS, and Windows , unlocking true cross-platform GUI automation. This will enable developers and researchers to build agents that generalize across environments with minimal setup.

Perfect for custom agent frameworks, existing backends, or when you need fine-grained control:

screenenv natively supports smolagents , making it easy to build your own custom Desktop Agent for automation. Here’s how to create your own AI-powered Desktop Agent in just a few steps:

Inherit from DesktopAgentBase and implement the _setup_desktop_tools method to build your own action space!