Apple’s relationship with AI can be summarized in three statements:
A Lucky Coincidence – The initials of “Apple Intelligence” conveniently spell “AI.”
Late to the Party – Despite being the top smartphone seller, Apple was the last to join the AI race. While other brands had been using AI for over six months, Apple only officially announced its AI features at WWDC 2024 in June.
All Talk, No Action – Even after the mid-year announcement and high praise for Apple’s system-level AI integration, Chinese iPhone users still haven’t seen these features in action.
▲ Image source: Google
Add to this the limited updates on the iPhone 16, where the Action Button – meant to be a gateway to visual intelligence – currently serves merely as a camera launcher and is prone to accidental triggers, falling short of the image recognition capabilities promised at the launch event.
However, where there’s a will, there’s a way. If you can’t wait for Apple’s native AI but don’t want your new phone to go to waste, just three steps can give your iPhone the ability to see and understand the world.
Getting Your iPhone to See the World Early
The software that enables AI features on the iPhone 16 is called “YueWen” (跃问), developed by JieYue XingChen, an AI large language model company based in Shanghai founded by Microsoft’s former Global VP Jiang Daxin.
While there are many large language model products with similar features, most have limited interfaces requiring users to open an app for dialogue. YueWen’s home page resembles traditional chat windows, but its core strength lies in visual intelligence, being one of the few products that can integrate with phone hardware buttons.
Setup is straightforward:
- Download the YueWen app
- Go to “Settings – Camera”
- Select “YueWen” under camera controls
After these steps, your iPhone 16 can use AI features similar to those shown at Apple’s launch event.
Note: You’ll need to open the app first and accept the basic settings (privacy policies) before the visual features will work properly – trust me on this one.
YueWen’s “Photo Query” can be activated through both the Action Button and the camera button. To enable it through the Action Button, go to “Settings – Action Button,” find “Control Options,” and search for YueWen to get the same one-touch activation experience.
This means not just iPhone 16 series users, but also iPhone 15 Pro/Pro Max users can access these features.
The Late Bloomer’s Advantage
You might worry: Will a third-party solution match Apple’s native experience?
We had similar concerns initially, viewing it as a temporary solution. However, after a week of use, YueWen proved impressive, particularly with features tailored for Chinese consumers, making it potentially more practical than iPhone’s native capabilities in some ways.
The activation method mirrors Apple’s smart features – press the Action Button, and Photo Query opens readily, helping you find answers about anything you see.
While screen intelligence operates in 2D, Photo Query extends AI capabilities into the 3D real world.
Using the camera as an image recognition tool opens up possibilities limited only by your imagination:
- Identify celebrities in street advertisements when you can’t quite place their name
- Get information about movies from posters or books from covers without unwrapping them
- Identify plants and animals during family outings to botanical gardens or zoos
- Get instant translation of foreign menus and warning signs while traveling
- Calculate calories in meals by simply taking a photo
The app not only recognizes people but has extensive knowledge of films and books. See an interesting poster or an wrapped book with an attractive cover? One press of the camera button reveals all you need to know.
Weekend trips to botanical gardens or zoos become more educational – no more awkward moments when kids ask “What species is that?” Even wildflowers and weeds can be identified.
For students, it’s like having a personal tutor. While it can help with homework, providing solution approaches comparable to standard answers, students should still verify results themselves – teachers might not know who wrote the homework, but they’ll definitely know who’s responsible.
For international travelers, Photo Query eliminates language barriers. No more confusion over foreign menus or warning signs – all text and prices can be clearly translated.
After each photo, the app proactively suggests questions based on the image content. Even if its suggestions aren’t quite what you’re looking for, you can engage in continuous dialogue.
Alongside Photo Query, there’s a practical “Calorie” feature. If you’re trying to lose weight or monitor your dietary intake, this becomes invaluable.
While controlling calorie intake is best done through home cooking, that’s not always practical. When you need to order takeout or dine out, just snap a photo and YueWen calculates the calories and provides a detailed nutritional breakdown.
Beyond basic Photo Query, YueWen offers 14 specialized visual features, which can be categorized into three types:
- Practical Features
- Entertainment Features
- Head-Scratching Features
Practical features include “Stamp Stories,” “Shoe Size Guess,” and “Calorie Counter.” The standout is “Food Reviews” – perfect for restaurants offering perks for positive reviews. Just snap a photo, and it generates a suitable review instantly.
Another valuable feature for learners is “Scene English,” which turns everything you see into high-frequency English vocabulary words – creating an immersive language learning experience.
Entertainment features include “Personality Test,” “CP Compatibility,” “Pet Face Reading,” “Book of Answers,” and “Work Fortune.” While not practical, they’re great conversation starters at gatherings.
For instance, YueWen offers a simplified version of the popular MBTI test, condensing a lengthy questionnaire into an instant photo analysis. Accuracy? Well…
The CP Compatibility feature can either create possibilities or potentially damage relationships. Perhaps it’s best tested on fictional couples first.
Turning the iPhone into an AI Arena
After a week of testing, YueWen offered many pleasant surprises (and a few puzzling moments). Most importantly, it demonstrated the true potential of the iPhone 16’s Action Button.
As a mere camera launcher, the button seems redundant, and its parameter adjustment capabilities are clunky compared to touch controls. Its placement is particularly awkward for landscape photos, where your finger often gets blocked by the phone’s body.
However, as a visual intelligence portal, these issues become unique advantages: quick activation, instant recognition, and streamlined search processes save considerable time.
Apple’s decision to allow third-party apps to utilize the Action Button seems strategic and promising.
Previous AI experiences on Hong Kong iPhone versions were powered by ChatGPT, with Apple displaying the GPT logo on results. Reports suggest Apple might integrate other large language models, letting consumers choose their preferred AI service.
Despite being late to AI, Apple appears to be positioning the iPhone and other devices as platforms where different models can integrate through APIs and hardware buttons – potentially enabling them to overtake competitors through this unique approach.
This transforms the Action Button and iPhone into an arena where AI models can compete based on merit. While YueWen is currently the only player, more Chinese large language models will likely join, turning theoretical benchmarks into practical applications that users can experience firsthand.
Most importantly, this approach puts the power of choice – deciding which AI to use and which performs best – directly in consumers’ hands.