Gemini Adds 'Select from Screen' to Chrome; 3.5 Flash Now Controls Computers

Google is rolling out a new tool called "Select from screen" for its Gemini assistant in the Chrome browser. This feature lets users quickly add any text or image from the current tab directly into the Gemini query box, without needing to manually take screenshots or copy-paste. When users click the tool, the current tab is highlighted, and they can then select the area they want Gemini to analyze. The feature is being gradually pushed as part of the Chrome 149 update — if you don't see it yet, simply restart the browser to enable it.

At the same time, Google announced that the Gemini 3.5 Flash model now has built-in computer use capabilities. Developers can call this ability through the Gemini API, allowing the model to interact directly with on-screen interfaces for analysis. A demo shows that 3.5 Flash can automatically open the Gemini app, navigate its feature menus, and return a clear, categorized list of functions. This breakthrough means AI assistants can now go beyond understanding text or images — they can actually "operate" software like a human, opening up new possibilities for automated testing, intelligent agents, and RPA scenarios. The model is available now via the Gemini API.

With "Select from screen" lowering the friction of interaction and the 3.5 Flash gaining computer control capabilities, Google is evolving Gemini from a simple Q&A assistant into a more proactive desktop agent. In the future, users may be able to ask Gemini to complete complex cross-app tasks on their behalf — such as organizing files, filling out forms, or analyzing reports. This direction aligns with the "AI agent" concept being pursued by other major players, making it well worth keeping an eye on its real-world performance.