Technical Background

Augmented Reality (AR), also referred to as “Simulated Reality” or “Extended Reality” in relation to Virtual Reality (VR), involves calculating the position and angle of a camera feed and integrating image analysis technology. This allows the virtual world on the screen to interact with the real-world scene. This technology was proposed in 1990. With the increasing computational power of portable electronic devices, the applications of AR are becoming more extensive.

Demo Effect

How to Use

Android: Requires a phone that supports ARCore and has ARCore installed

Check ARCore Support

For information on ARCore support, you can refer to Google’s official documentation on ARCore (https://developers.google.com/ar/devices). If your device is not supported but has the Magisk framework installed, you can download ARCore/Playground Patcher from Magisk to forcibly support it.

For Chinese phones like MIUI, you need to activate Google services in the settings and update ARCore in the Xiaomi App Store. For phones that come with Google services by default, ARCore should already be installed.

Use Google App or Latest Version of Chrome to Open Web Page

Website (Recommended to use VPN):
https://xianfei.github.io/ar2022/

Click to show website QR code

Copy this address into the Google app or Chrome to access. (iOS users can directly visit this link)
The effect is better when opened in the Google app. The demonstration in the video is done through the default browser (which does not support WebXR) and calls the Google APP for 3D AR interaction. Please download the Google app from the Play Store, otherwise, you will see the following prompt:

Alternatively, you can use Chrome or other browsers that support WebXR technology. However, the effect is not as good as the Google app. WebXR is a very cutting-edge web frontend technology that few browsers support.

iOS: Simply visit the website in Safari

https://xianfei.github.io/ar2022/

Click to show website QR code

Click the Cube Icon to Enter AR Mode

Click on the icon at the bottom right of the image to enter AR mode.

If all goes well, you should be able to see Bing Dwen Dwen and Snow Rong Rong.

Coolapk Headlines

I posted it on Coolapk last night before going to sleep. When I woke up, I found that it had made the headlines, and many netizens said they succeeded. Some also offered solutions. Those who didn’t succeed can look for useful information in the comments section of Coolapk.

Here are some success screenshots shared by Coolapk users in the comments:

Technical Solutions

Modeling: Blender Display: model-viewer

For AR display technology, you can refer to Google ARCore WebXR documentation (https://developers.google.com/ar/develop/webxr/model-viewer). It is easy to understand, and those with a slight understanding of frontend development will get it instantly.

Here, all we need to do is export the glb file (if you also want to preview it on iOS, you need to export a usdz file). After completing the 3D modeling in Blender, just export the glb and usdz files (if you don’t know how to model, you can search on Bilibili where many creators have shared model files). Note that materials involving reflection and refraction like glass may not necessarily be realized in AR. It is recommended to use a semi-transparent solid color material to draw Bing Dwen Dwen’s shell.

Project source code:

https://github.com/xianfei/xianfei.github.io/tree/master/static/ar2022