Apple released MGIE AI image editing tool capable of detailed editing using text prompts

Apple researchers have released a Artificial intelligence (AI) powered image editing tool called MGIE, capable of editing images using simple text prompts. MGIE, which stands for MLLM Guided Image Editing, is capable of Photoshop-style edits, global corrections, and local edits. The AI ​​tool was released just days after Apple. announced It said in its quarterly earnings that it is investing “tremendous time and effort” into creative AI. The image editing model represents an improvement over existing AI editing tools.

Researchers from apple and from the University of California, Santa Barbara contributed to efforts to develop the instrument. VentureBeat Reports This paper was presented at the International Conference on Learning Representations (ICLR) 2024. Preprint version of the research Paper Also hosted on arXiv.

The AI ​​tool is capable of Photoshop-style editing including cropping, resizing, rotating, adding filters and more. It can also include global corrections where it can change brightness, contrast, sharpness, color balance, and even add creative elements to the image. Additionally, it can perform local edits where it adds, removes or changes a particular object or element in the image.

To edit, users can simply type a simple text prompt such as “make the sky brighter” or “make the house bigger”, which is then interpreted as an image command and used to increase the brightness to a certain percentage. goes Home size by specific metric. Users can also provide more complex and granular adjustments such as “adjust between dark and light areas to bring out details on leaves and tree trunks.” The more detailed a signal is, the closer it is to the desired result.

While AI-based photo-editing tools such as Photoshop’s Generative Fill and Undertesting FireFly, Canva’s Magic Design, and Luminar Neo already exist, they all require the user to interact with the software in order to or So the edit space can be mapped or granular changes can be made. Apple’s MGIE, on the other hand, can be completely self-editing. It uses “instruction-based image editing” or “text-guided image editing”, which is made possible by taking a unique approach to artificial intelligence frameworks.

Instead of relying on a generative adversarial network (GAN) framework, the AI ​​model uses a diffusion model, which is a more advanced architecture for realistic image generation and instruction execution. Next, the researchers moved on to using a multi-modal large language model to ensure that it was able to translate natural language into images and show the desired effect. Further, human evaluators were also used during the process to rate the modifications, and feedback was used to further refine the model.

The tech dev has made MGIE AI image editing tool available. Download As an open source project via GitHub. It is currently unknown whether Apple plans to use this technology for its own devices. However, Apple CEO Tim Cook has said that the company will announce the generative AI features it is working on later this year, while Apple is reportedly working on it. New AI-powered features For the iOS 18 update that is expected to arrive later this year.

---