Non-Site

Non-Site

An AR sandbox between physical and virtual space.

Year

2023

Role

AR Software Engineer /Technical Artist

↓ Scroll to continue
0:000:00

About the project

The Non-Site project is a material exploration game based on augmented reality utilizing the Niantic Lightship and Google Cloud Vision API. It allows players to use their phones as a tool to explore materials in the real world and transfer them into the virtual world. Three types of materials can be found: Wood, Stone, and Metal. Upon collecting a sufficient amount of materials, players can combine these materials to create objects. The built objects can then be saved and loaded at designated build points, allowing them to remain in the same location even after the game has been closed.

Tools & Software

UnityNiantic LightshipGoogle Cloud Vision APIC#

Categories

ARXRInteractiveUnity

Game Play

Explore Mode

In Explore Mode, three distinct material types can be identified. Switch the current material by pressing the Material button and tapping the screen to initiate scanning. Once scanning is complete, collect the material by tapping the material popup.

Wood
Stone
Metal

Face a target → Press to start scanning → Collect popup elements

Compose Mode

Compose Mode can utilize the elements collected in the Explore mode to create the corresponding objects. Each material has three different levels of elements that can be collected.

Browse objects → Gather the highlighted elements → Press ‘Compose’ to create

Map Mode

The Niantic ARDK was used in Map Mode, which offers a powerful Visual Positioning System to locate the virtual world with the corresponding physical world.

Non-Site — Map Mode 1

List nearby bulild point (VPS)

Non-Site — Map Mode 2

Press ‘Direction’ to navigate

Non-Site — Map Mode 3

Within 50m of the build point

Build Mode

In the Build Mode, the composed object can be constructed. The Niantic ARDK provides all the builds that can be saved and loaded in this location. I expanded the stored information by including the generated model types. I loaded them simultaneously with the Waypoint to enable the storage of multiple different models.

Build

Scan VPS to locate → Select an object to build → Save all builds

Load

Scan VPS to relocalize → Load saved builds

Technical Detail

Google Cloud Vision API

In collection mode, Google Cloud Vision API's label recognition is the core. Rear camera opens, image is captured and uploaded for recognition, keywords compared, success if they match.

Non-Site — Google Cloud Vision API 1
Non-Site — Google Cloud Vision API 2

Niantic Lightship VPS

Building points use information from Niantic Lightship VPS, I specially scanned nearby public VPS points to enhance and improve the accuracy of the public VPS.

Non-Site — Niantic Lightship VPS 1
Non-Site — Niantic Lightship VPS 2
Non-Site — Niantic Lightship VPS 3