Scripting with AR Controller

AR Controller beta release 0.7.0 (2022-06-25)

Basic Interaction Components
Specific Types of Manipulable (Objects That Can Be Manipulated)
Specific Types of Manipulator (Components That Manipulate Other Objects)
AR Controller Events
Code Samples
1. AR Controller Touch Code Example (global)
2. Manipulable Touch Code Example (object-specific)

Basic Interaction Components

ARController

The ARController component acts as an interface between AR Controller hardware and Unity. It connects to AR Controller devices and forwards information about them to your Unity apps, via ARController events. Note that the ARController emulator can also produce (fake) AR Controller events in response to your mouse interactions within the Unity Editor’s Game view.

Manipulable

The Manipulable component represents that the Unity GameObject it is attached to can be interacted with by the AR Controller interaction system. It has some basic properties (listed below) that modify how it works, as well as a comprehensive range of events.

Interactable determines whether this Manipulable object is currently responding to Manipulator interactions.
Target determines which object is affected by this Manipulable (Note: By default, the closest ancestor GameObject with a Rigidbody component will be targeted. Additionally, it is completely fine for a Manipulable to target itself).
Grab Mode determines whether an interacting Manipulator grabs the target object or this Manipulable handle object.
Indicator Prefab determines which prefab to display near the object to indicate that this Manipulable is being interacted with.

This component can be used to respond to Manipulator interactions directly, but it is designed to be inherited from. Several AR Controller classes already inherit from Manipulable, and can be used as examples: Positionable, Movable, Scalable, Rotatable, Selectable, Hidable.

Manipulable components may be attached to objects in different configurations and with different targets to achieve different interactions - refer to the AR ControllerShowcase scene for a variety of examples.

Manipulable GameObjects must have at least one Unity Collider component attached to them or attached to one of their child GameObjects in order for a Manipulation to occur. Additionally, the Collider(s) must be assigned to a Unity physics layer that is not ignored by a given Manipulator in order for that Manipulator to interact with the Manipulable.

Manipulator

The Manipulator component represents that the Unity GameObject it is attached to can establish interactions with Manipulable objects by triggering events on them. It has some basic properties (listed below) that modify how it works. The Manipulator class itself is abstract, so cannot be attached to objects directly, but it is inherited by a selection of different implementations, including the Pointer class, which extends its functionality to work using the AR Controller hardware as input.

Strength determines the strength of this Manipulator compared to other Manipulators when performing interactions.
Ignore Layers determines which Unity physics layers should not be considered when determining what this Manipulator is interacting with.
Release Range determines how far a Manipulable may move from this Manipulator’s target grab position before it is released (use -1 to make the range infinite).

Manipulation

The Manipulation component is automatically generated on Manipulables by Manipulators when a hover event begins, and is destroyed when the Manipulator ends its interaction with (i.e. stops hovering and stops grabbing) the Manipulable. It represents the state of the interaction, which is unique to the Manipulable-Manipulator pair.

A reference to this component is passed along with each Manipulable event that is triggered, providing event handlers with information about the manipulation:

Manipulator indicates which Manipulator is involved in this Manipulation.
Manipulable indicates which Manipulable is involved in this Manipulation.
StartTime indicates when this Manipulation started, in Unity time.
GrabPosition indicates the point at which the interaction is taking place (e.g. the handle point by which the Manipulator is holding the Manipulable).
InitialManipulablePosition and InitialManipulableRotation indicate the initial position and rotation of the Manipulable.
IsHovered indicates whether the Manipulable is currently being hovered (highlighted) by the Manipulator.
IsGrabbed indicates whether the Manipulable is currently being grabbed (e.g. for a Pointer, the AR Controller touchpad is pressed) by the Manipulator.
Indicator indicates which GameObject in the scene is showing that this Manipulation is occurring.

Specific Types of `Manipulable` (Objects That Can Be Manipulated)

Positionable

The Positionable component can be attached to a GameObject with a Rigidbody to allow it to be grabbed and repositioned by a Manipulator. Attaching this component to a child of a GameObject with a Rigidbody will make that child a ‘handle’ by which the Rigidbody can be moved. Positionable exposes the following properties in the Unity Inspector (in addition to those of Manipulable):

Recentre Grab Point Time determines how long it should take for the point by which the object is being held to transition to the centre of the object.
Counteract Gravity determines whether this Manipulable should be affected by gravity whilst being manipulated.

Movable

The Movable component extends upon the functionality of Positionable. It can be attached to a GameObject with a Rigidbody to allow it to be grabbed, repositioned and rotated by a Manipulator. Attaching this component to a child of a GameObject with a Rigidbody will make that child a ‘handle’ by which the Rigidbody can be manipulated. Movable exposes the following properties in the Unity Inspector (in addition to those of Manipulable and Positionable):

Rotation Is Controlled determines whether this Manipulable should override control of the object’s rotation.
Neutral Orientation Eulers represents the x,y,z Euler angles the object will have when it is considered to be stood upright (zeros are most common here).
Orders Of Rotational Symmetry represents the number of orientations about each axis that are equivalent to each other (e.g. a cube has 4 equivalent orientations about each axis).

Movable Component

Rotatable

The Rotatable component implements precise angular control of the target object about the given rotation axis - this is controlled by pointing left or right whilst grabbing this Manipulable GameObject (typically a parent GameObject). It can be attached to any GameObject (which will then act as the ‘handle’ of the rotation manipulation), and set to target any GameObject. Rotatable exposes the following properties in the Unity Inspector (in addition to those of Manipulable):

Sensitivity Factor determines how sensitive this Manipulable should be to Manipulator motion.
Rotation Axis determines which axis (local to the target) the rotation should occur around.

Rotatable Component

Scalable

The Scalable component implements precise scale (size) control of the target object - this is controlled by pointing up or down whilst grabbing this Manipulable GameObject. It can be attached to any GameObject (which will then act as the ‘handle’ of the scaling manipulation), and set to target any GameObject (typically a parent GameObject). Scalable exposes the following properties in the Unity Inspector (in addition to those of Manipulable):

Sensitivity Factor determines how sensitive this Manipulable should be to Manipulator motion.
Min Scale Factor represents the smallest scale factor (comparing current scale to initial scale) that the object can be shrunk to.
Max Scale Factor represents the largest scale factor (comparing current scale to initial scale) that the object can be expanded to.

Scalable Component

Selectable

The Selectable component implements basic selection and deselection control. Selectable exposes the following properties in the Unity Inspector (in addition to those of Manipulable):

Mode determines how this Manipulable gets deselected - either when it is released, or every second time it is released (i.e. a toggle mode).

Hidable

The Hidable component can be attached to a GameObject to allow it to toggle the visibility of the target object. Hidable exposes the following properties in the Unity Inspector:

This component can be attached to Manipulable objects (or their child objects) to make the Manipulable (or a chosen target object) disappear and reappear when grabbed by a Manipulator (e.g. a Pointer). It exposes the following properties:

Start Hidden determines the initial state of the target object when this Hidable is created (e.g. at the start of the game).
Animation Period determines how long it should take for the target object to transition from fully shown to fully hidden.

`ManipulatorUnityEventForwarder`

This component can be attached to Manipulable objects in order to expose the Manipulable events as UnityEvents - see more here.

Specific Types of `Manipulator` (Components That Manipulate Other Objects)

Pointer

A Pointer is a subclass of LaserManipulator (a long-range version of Manipulator). This component bridges the gap between the AR Controller and Manipulator classes. A Pointer uses events received from the AR Controller hardware (via the AR Controller component, which must be attached to the same object) to trigger its Manipulator actions.

ItemHolder

An ItemHolder component extends the Manipulator class in order to implement automatic grabbing of Positionable objects that move into range. Note that the Release Range of this component also represents the grab range.

Auto Grab determines whether to automatically grab new Positionables that move into the grab range.
Radius Offset Vector determines the direction in which to offset the position at which an object is held proportionally to the radius of that held object.

ItemSpawner

An ItemSpawner component extends the Manipulator class in order to implement creation of new objects. This creation is triggered by another Manipulator (e.g. a Pointer) grabbing the icon (which is generated by the ItemSpawner) and pulling it beyond the ItemSpawner’s Release Range.

Spawn Prefab determines which prefab to spawn when triggered.
Icon Container Prefab determines which prefab to use as a container for the generated icon (which is a miniature of the Spawn Prefab).
Spawn Parent determines which transform to place newly-spawned objects inside (i.e. as children of).
Spawn Prefab Scale Factor determines an additional scale factor for newly-spawned objects, relative to the Spawn Prefab’s natural localscale and the Spawn Parent’s scale.

ItemDeleter

An ItemDeleter component extends the ItemHolder class in order to implement deletion of Positionable objects. This deletion is triggered by the target Positionable object moving into the Release Range of the ItemDeleter, at which point the ItemDeleter grabs the object and pulls it in the direction of the Deletion Offset Vector whilst shrinking it to delete it.

Deletion Offset Vector determines which direction to move the deleted object in as it is shrunk and deleted - e.g. downwards for a trash can (as the objects falls into the bin).
Deletion Time determines the time period over which an item will be deleted after it is grabbed.
Fast Mode determines whether to speed up deletion - this is enabled automatically when another target comes into range for deletion.

AR Controller Events

If you are unfamiliar with C# events, you can read an overview here and get more detail on using events here.

AR Controller events are processed in two categories: ‘global’ and ‘object-specific’.

Global events occur without any context of where the AR Controller is, what it is doing, or whether it is interacting with an object. Global events are processed and exposed by the AR Controller component, attached to the AR Controller gameobject. They include touch events, which always occur when the AR Controller user interacts with the AR Controller touchpad, without reference to any other objects. Connection status and device information events are also global.

Global events are implemented by the AR Controller script as conventional C# events.
Object-specific events occur on specific Manipulable objects as Manipulators interacts with them. These transmit information about Manipulator interactions to the object that is being interacted with. This information is used by the AR Controller interaction system to implement Manipulations.

Object-specific events are implemented by the Manipulable script as conventional C# events. They can also be exposed as UnityEvents, which work slightly differently, and can be accessed from the Unity Editor Inspector window - see ManipulableUnityEvents.

AR Controller touch events that occur on a Manipulable will first occur as a global event.

Code examples of subscribing to AR Controller and Manipulable events can be found here.

List of Global AR Controller Events

AR Controller Connection Status Events:

OnDeviceFound(string deviceName) is called when an AR Controller device is discovered via Bluetooth.
OnConnected(string deviceName) is called when an AR Controller device is connected to via Bluetooth.
OnConnectionFailed(string deviceName) is called when an attempt is made to connect to an AR Controller device via Bluetooth, but it fails.
OnDisconnected(string deviceName) is called when an AR Controller device is disconnected.

AR Controller Device Info Events:

OnBatteryLevelReceived(int batteryLevel) is called when the connected AR Controller device reports its battery level (which occurs on connection and periodically whilst connected).
OnFirmwareVersionReceived(string deviceInfo) is called when the connected AR Controller device reports its firmware version (which occurs on connection and when requested via the AR Controller -> Get Info menu option in the Unity Editor.
OnHardwareVersionReceived(string deviceInfo) is called when the connected AR Controller device reports its hardware version (which occurs on connection and when requested via the AR Controller -> Get Info menu option in the Unity Editor).
OnModelNumberReceived(string deviceInfo) is called when the connected AR Controller device reports its model number (which occurs on connection and when requested via the AR Controller -> Get Info menu option in the Unity Editor).

AR Controller Touch Events:

See notes on position versus worldPosition.

OnTouchStart(Vector2 position, Vector2 worldPosition) is called when the thumb first makes contact with the touchpad (of the connected AR Controller device).
OnTouchHold(Vector2 position, Vector2 worldPosition) is called for every frame between the start and end of a touch (on the connected AR Controller device), providing updated touch position information.
OnTouchEnd(Vector2 position, Vector2 worldPosition) is called when the thumb loses contact with the touchpad (of the connected AR Controller device).
OnTap(Vector2 position, Vector2 worldPosition) is called when the thumb loses contact with the trackpad (of the connected AR Controller device) a fraction of a second after it initially makes contact.
OnTouchLongHold(Vector2 position, Vector2 worldPosition) is called once per frame after the thumb remains on the trackpad and does not move within a fraction of a second of making contact; after that fraction of a second, the thumb may move around whilst still producing the ‘long hold’ event.

List of Manipulable Events

OnManipulatorEnter(Manipulation manipulation) is called when a Manipulator starts hovering.
OnManipulatorStay(Manipulation manipulation) is called once per frame for each Manipulator that is hovering this object.
OnManipulatorExit(Manipulation manipulation) is called when a Manipulator is no longer hovering this object, and is also no longer grabbing this object.
OnManipulatorGrab(Manipulation manipulation) is called when a Manipulator is hovering this object and that Manipulator’s ‘grab’ action is triggered (e.g. for a Pointer, when ARController.OnTouchDown occurs)
OnManipulatorHold(Manipulation manipulation) is called once per frame between OnManipulatorGrab and OnManipulatorRelease, for each Manipulator that is interacting.
OnManipulatorRelease(Manipulation manipulation) is called if this object is being interacted with by a Manipulator, when that Manipulator’s ‘release’ action is triggered (e.g. for a Pointer, when ARController.OnTouchUp occurs).
OnManipulatorTap(Manipulation manipulation) is called when a Manipulator is hovering this object and that Manipulator’s ‘tap’ action is triggered (e.g. for a Pointer, when ARController.OnTap is invoked - see here).
OnManipulatorLongHold(Manipulation manipulation) is called when a Manipulator is hovering this object and that Manipulator’s ‘long hold’ action is triggered (e.g. for a Pointer, when ARController.OnTouchAndHold is invoked - see here).

Touch Event Notes

AR Controller touch events provide two parameter values for touch positions. It is important to use the correct value in order to provide the same experience to left-handed and right-handed users. In the future these positions might remap the touch coordinates to account for different AR Controller grips.

position - represents the position of the touch relative to the hand (i.e. swiping towards the palm will produce the same sequence of values for both left-handed and right-handed users). Use this value for interactions that are not spatially-aligned with the world, such as scaling objects.
worldPosition - represents the position of the touch relative to the world (i.e. swiping from right to left will produce the same sequence of values for both left-handed and right-handed users). Use this value for interactions that are spatially-aligned with the world, such as scrolling a menu left or right.

Code Samples

AR Controller Touch Code Example (global)

When the script below is attached to a GameObject in your scene, touches on the AR Controller touchpad will be printed to the Unity Console window (whilst in Play mode, when your AR Controller hardware is connected).

using UnityEngine;
// Tell your script that you are going to use AR Controller features
using ARController;

// Define your own class as a subclass of MonoBehaviour, so that it can be attached to GameObjects
public class MyARControllerEventScript : MonoBehaviour
{
    private void Start()
    {
        // At a relevant point in the code, check that there is an AR Controller script in the scene
        if (ARController.Instance != null)
        {
            // If there is an AR Controller script, tell this script to call the HandleAR ControllerTouchStart() function at the moment the touchpad on the currently connected AR Controller is touched
            ARController.Instance.OnTouchStart += HandleARControllerTouchStart;
        }
    }

    // Define the HandleAR ControllerTouchStart() function with specific touch event parameters
    private void HandleARControllerTouchStart(Vector2 position, Vector2 worldPosition)
    {
        // Print a message to the Unity Console, noting the touch positions
        Debug.LogFormat("{0}: Touch started at position ({1}, {2})", this, worldPosition.x, worldPosition.y);
    }
}

Manipulable Touch Code Example (object-specific)

When the script below is attached to a GameObject in your scene, it will insist on also having a Manipulable component attached to it, and will then print a message in the Unity console every time a Manipulator (e.g. a Pointer) stops hovering over it.

using UnityEngine;
// Tell your script that you are going to use AR Controller features
using ARController;

// Specify that this component must also have a Manipulable component attached with it
[RequireComponent(typeof(Manipulable))]
// Define your own class as a subclass of MonoBehaviour, so that it can be attached to GameObjects
public class MyManipulableEventScript : MonoBehaviour
{
    private Manipulable _myManipulable;

    private void Start()
    {
        // Get the reference to the Manipulable attached next to this script
        _myManipulable = GetComponent<Manipulable>();

        // Tell this script to call the HandleManipulatorExit() function when this object stops being hovered: 
        _myManipulable.OnManipulatorExit += HandleManipulatorExit;
    }

    // Define the HandleManipulatorExit() function with specific manipulation event parameters
    private void HandleManipulatorExit(Manipulation manipulation)
    {
        // Print a message to the Unity Console, noting which Manipulator stopped hovering this object
        Debug.LogFormat("{0} stopped hovering over {1}", manipulation.Manipulator, name);
    }
}