Microsoft wants to bring generative AI to the forefront of Windows and the PCs that run it.
In two keynotes at this week’s annual Build developer conference, the company announced a new line of Windows machines it calls Copilot+ PCs and leverages AI such as Recall to help users find apps, files, and other content. announced the generation function. What I saw in the past. Copilot, Microsoft’s brand of generative AI, will soon be more deeply integrated into the Windows 11 experience. And new Microsoft Surface devices are also coming.
Here are all the major announcements from Monday and Tuesday.
volume measurement app
Microsoft is bringing Windows Volumetric Apps (basically spatially aware, interactive VR apps) to the Meta Quest headset. Microsoft says that through its partnership with Meta, it will bring Windows 365 and local PC connectivity to the Quest headset, allowing developers to extend their apps into 3D space.
During Tuesday’s keynote, Microsoft showed off a digitally exploded 3D view of the Xbox controller from the perspective of the Meta Quest 3 headset, a digital object that the wearer can manipulate with their hands. “We’re deepening our partnership with Meta to make Windows the best-in-class experience on Quest devices,” Pavan Davuluri, Microsoft’s CVP of Windows and Devices, said during the demo. .
What can developers do? Sign up for a preview On Tuesday, you’ll have access to Microsoft’s new volumetric API.
Copilot + PC
Copilot+ PC is the flagship Windows hardware that represents Microsoft’s AI-first vision. All include a specialized chip called an NPU to power AI experiences such as Recall. It also ships with a minimum of 16 GB of RAM and paired with SSD storage.
The first Copilot+ PCs are powered by Qualcomm’s Snapdragon X Elite and Plus chips, which Microsoft claims will deliver up to 15 hours of web browsing and 20 hours of video battery life. Chipmakers Intel and AMD are also partnering with various manufacturers, including Acer, Asus, Dell, HP, Lenovo, and Samsung, to develop processors for Copilot+ devices.
Copilot+ PC prices start at $999, and some are available for pre-order now.
Surface Pro and Surface Laptop
Microsoft’s newly announced Surface devices, Surface Laptop and Surface Pro, focus on performance and battery.
The latest Surface Laptops with 13.8-inch or 15-inch displays have been redesigned with “modern lines” and thinner screen bezels. The company says it lasts up to 22 hours on a single charge and is up to 86% faster than the Surface Laptop 5. It also supports Wi-Fi 7 and has a haptic feedback touchpad.
About the new Surface Pro, Microsoft says it’s up to 90% faster than the previous generation Surface Pro (Surface Pro 9) and features a new OLED with an HDR display, Wi-Fi 7 (and optional 5G). states. Upgraded ultra-wide-angle front camera. Additionally, the removable keyboard is reinforced with additional carbon fiber and features haptic feedback.
recollection
Windows 11’s upcoming recall feature will allow users to “remember” apps and content they accessed on their PC weeks or even months ago, for example when discussing clothes they were considering buying. Helps you find the Discord chats you are looking for. Users can use her Recall timeline to “scroll back” to see recent work or drill down into files such as PowerPoint presentations to see information that may be relevant to their search. You can do that.
Microsoft says Recall can create associations such as colors and images, allowing users to search for virtually anything on their PC in natural language (not unlike technology from startup Rewind). Developers can improve recall by adding contextual information to their apps. And Microsoft claims that all user data associated with Recall is stored privately on the device and, importantly, is not used to train AI models.
Here it is more From Microsoft: “Your snapshots are yours. They remain locally on your PC. You can delete individual snapshots from the system tray icon in the taskbar, or adjust the time range and delete them in Settings. or pause at any time. You can also filter apps and websites from being saved.
Image editing and live translation
Windows has more AI than ever before, and some of it is exclusive to the new Copilot+ PCs.
A new feature called Super Resolution can automatically upscale and restore old photos. Copilot can also now analyze images and provide users with creative composition ideas. A feature called Cocreator allows users to generate images and also ask the AI model to modify or restyle the image according to what it draws.
Live Captioning with Live Translation also translates audio passing through your PC from YouTube or local files into the language of your choice. Live Translate will initially support around 40 languages, including English, Spanish, Mandarin, and Russian.
A separate, related new feature in Microsoft Edge provides real-time video translation on sites like LinkedIn, YouTube, Coursera, Reuters, CNBC, and Bloomberg. Expected to be available in the near future, this feature supports translations from Spanish to English, English to German, Hindi, Italian, Russian, and Spanish, and audio content through both dubbing and subtitles. Translate live.
Team Copilot and Extensions
Team Copilot is the latest extension to Microsoft’s growing suite of Copilot generative AI technologies. It integrates with Teams, the company’s video conferencing app, so you can manage meeting agendas and create notes that all meeting participants can co-author. It also extends to Loop and Planner, Microsoft’s collaboration and planning platforms, where you can create and assign tasks, track deadlines, and notify team members when input is needed.
In somewhat related Copilot news, Microsoft has launched (in private preview) Copilot Extensions, which allows developers to extend GitHub’s code generation tool, GitHub Copilot, with third-party apps and skills. Launch partners include DataStax, Docker, and LambdaTest. Extensions exist on his GitHub Marketplace, but developers can also create their own private extensions to integrate with internal systems and APIs.
Windows CoPilot Runtime
Powering features such as recall and super resolution is Windows Copilot Runtime. It’s a collection of about 40 generative AI models that make up what Microsoft calls a “new layer” of Windows. The Windows Copilot runtime works with Semantic Index, a vector-based system local to each individual Copilot+ PC, to enable generated AI-powered apps, including third-party apps, to run without necessarily requiring an internet connection.
“[The runtime] Studio effects, live caption translation, OCR, recall based on user activity, [more]will be available to developers in June,” Davuluri said on Tuesday.
CapCut, the popular video editor from TikTok owner ByteDance, will use Windows Copilot Runtime and the new Windows Copilot Library, an accompanying set of APIs and AI development tools, to accelerate its AI capabilities, according to Microsoft. And Meta plans to add the aforementioned Studio Effects to WhatsApp, offering features like background blur and eye contact during video calls.
Upgraded bot builder
Azure AI Studio, a toolset within Microsoft’s Azure OpenAI service, allows customers to combine AI models and build apps that “reason” about their data. Developers will soon be able to create apps using pay-as-you-go inference APIs (APIs). This allows developers to access and fine-tune generative AI models hosted on Azure infrastructure. Microsoft calls this the “as-a-service model,” and it’s starting with the Nixtla and Core42 models.
In the adjacent Copilot Studio product suite, Microsoft is launching the Copilot agent. The company describes it as an AI bot that can “independently tailor tasks tailored to specific roles and functions.” (Copilot Studio provides tools to connect Copilot for Microsoft 365, an AI-powered “copilot” for apps like Excel and Word, to third-party data.) Use your knowledge to navigate different types of business workflows. Learn from user feedback and ask for help when you encounter a situation you don’t know how to handle.
Snapdragon development kit
Qualcomm has a new development kit aimed at developers building apps for Copilot+ PCs powered by Arm chips
By the way, the $899.99 Snapdragon development kit for Windows (about the same width, height, and length as Apple’s Mac Mini) includes Qualcomm’s Snapdragon paired with 32GB of RAM, 512GB of storage, and tons of I/O. Equipped with X Elite chip. This development kit supports Wi-Fi 7 and Bluetooth 5.4 and can drive up to three of his 4K monitors simultaneously via various USB-C and HDMI ports.
Phi-3
Microsoft announced Phi-3-vision, a new addition to the Phi family of generative AI models. It can perform common visual analysis and reasoning tasks, such as answering questions about graphs and images. This model can read both text and images and is efficient enough to run on mobile devices.
Phi-3-vision is available in preview, while previously announced text-only models Phi-3-mini, Phi-3-small, and Phi-3-medium are now generally available.
Partnership with Khan Academy
Microsoft is partnering with Khan Academy to donate access to cloud computing infrastructure to enable Khan Academy to provide U.S. educators with free access to Khan Academy’s AI-powered tools. Microsoft announced Tuesday that the two companies will work together to explore opportunities to improve AI apps for math tutoring through generative AI.
Publish an AI newsletter. sign up here It will start arriving in your inbox starting June 5th.