↓ Scroll to continue

0:000:00

← Projects

2023

Non-Site

AR Software Engineer / Technical Artist

The Non-Site project is a material exploration game based on augmented reality utilizing the Niantic Lightship and Google Cloud Vision API. It allows players to use their phones as a tool to explore materials in the real world and transfer them into the virtual world. Three types of materials can be found: Wood, Stone, and Metal. Upon collecting a sufficient amount of materials, players can combine these materials to create objects. The built objects can then be saved and loaded at designated build points, allowing them to remain in the same location even after the game has been closed.

Tools & Software

UnityNiantic LightshipGoogle Cloud Vision APIC#

Explore Mode

In Explore Mode, three distinct material types can be identified. Switch the current material by pressing the Material button and tapping the screen to initiate scanning. Once scanning is complete, collect the material by tapping the material popup.

Face a target

Press to start scanning

Collect popup elements

Compose Mode

Compose Mode can utilize the elements collected in the Explore mode to create the corresponding objects. Each material has three different levels of elements that can be collected.

Gather highlight elements

Not enough elements

Press 'Compose' to create

Map Mode

The Niantic ARDK was used in Map Mode, which offers a powerful Visual Positioning System to locate the virtual world with the corresponding physical world.

List nearby bulild point (VPS)

Press ‘Direction’ to navigate

Within 50m of the build point

Build Mode

In the Build Mode, the composed object can be constructed. The Niantic ARDK provides all the builds that can be saved and loaded in this location. I expanded the stored information by including the generated model types. I loaded them simultaneously with the Waypoint to enable the storage of multiple different models.

Scan VPS to locate

Select an object to build

Save all builds

Technical Detail

Google Cloud Vision API

In collection mode, Google Cloud Vision API's label recognition is the core. Rear camera opens, image is captured and uploaded for recognition, keywords compared, success if they match.

Niantic Lightship VPS

Building points use information from Niantic Lightship VPS, I specially scanned nearby public VPS points to enhance and improve the accuracy of the public VPS.