New Gemini 'Select from Screen' Feature for Chrome; 3.5 Flash Gains Computer Control Abilities

Google is introducing a practical new feature for the Gemini assistant in the Chrome browser—the "Select from Screen" tool. When users click the tool, the current tab is highlighted with the prompt "Select any text or image to ask Gemini." The selected content is automatically added as an image to the conversation input box, making it convenient for users to quickly query information on their screen. This feature is gradually rolling out with Chrome 149. If you don't see it yet, restart your browser to enable it.

Meanwhile, Google announced that the Gemini 3.5 Flash model now includes a built-in "computer use" tool, available for developers via the Gemini API or Gemini Enterprise Agent Platform. This tool allows the model to directly analyze desktop application interfaces and perform actions. For example, in an official demo, 3.5 Flash uses its computer use capability to "analyze the Gemini app and return a categorized list of features." This marks a significant step for AI models—moving from pure text generation to actively interacting with the environment.

Both updates enhance Gemini's practicality on the user and developer sides: the screen selection tool simplifies image input, while computer use opens new possibilities for automation tasks and intelligent agents. As the Chrome ecosystem and AI models continue to converge, this kind of "what you see is what you get" interaction is likely to become the norm for human-machine collaboration.