Apple’s push towards Siri improvements continues with a text-powered open-source image editing AI model
There are ongoing rumors that Apple intends to bring some big improvements to Siri later this year. We’ve heard multiple times that Apple is working on new large language models (LLM) that could see its devices gain new AI capabilities the likes of which no Apple platform has been able to boast to date. Apple itself has already confirmed that it’s spending time working on AI initiatives without giving anything away and now it’s released a new open-source AI tool that might not be used by many but does give us a hint at the kinds of things Apple has been focusing on.
Apple has today made a new open-source AI model available that can edit images based on the text instructions provided to it. The model can do a variety of things when performing those edits including various things that some people would normally turn to dedicated apps to do.
Dubbed MGI, or MLLM-Guided Image Editing, the tool uses multimodal LLMs to turn text-based commands into pixel-level edits which in turn spit out an altered image. Examples of what people could do is ask MGIE to change the colors of an image or alter the saturation.
MGIE magic
VentureBeat detailed the new MGIE tool, saying that it can perform many of the tasks that people regularly do with apps like Photoshop. “MGIE can perform common Photoshop-style edits, such as cropping, resizing, rotating, flipping, and adding filters,” the report explains. “The model can also apply more advanced edits, such as changing the background, adding or removing objects, and blending images.”
That’s not all. MGIE is then able to “optimize the overall quality of a photo, such as brightness, contrast, sharpness, and color balance. The model can also apply artistic effects like sketching, painting and cartooning.”
That isn’t all, either. Users can ask the tool to edit specific regions of parts of an object such as a person’s face or their clothes, while “the model can also modify the attributes of these regions or objects, such as shape, size, color, texture, and style.”
The MGIE tool is currently an open-source project available via Github, and there’s a demo that can be used to take the model for a spin. It isn’t perfect, but it’s still impressive even in its current beta form.
As for how this will benefit Apple and Siri users in the future isn’t immediately clear, but it’s an indication of the work that the company is doing. There are possibilities that jump out at us however, not least the ability to hook this kind of AI capability into Shortcuts — potentially allowing text-based inputs to alter images saved in the Photos app. Those who are perhaps overwhelmed by the editing options within the Photos app could also potentially turn to simply telling Siri what they want, with the digital assistant feeding that information into an advanced version of MGIE.
It’s still very early days, of that, there is no doubt. But with Apple potentially making big AI strides with the upcoming iOS 18 and the Apple Vision Pro specifically suited to issuing verbal instructions to something like Siri, there’s hope for big changes to the digital assistant this year.
Apple is expected to preview the iOS 18 software alongside new Mac, iPad, Apple Watch, and Apple TV software updates this June. It’s possible we’ll see visionOS 2.0 as well, with all the new updates likely to be released to the public in the fall.
More from iMore
