
One of the major differences between WebVR and WebXR was the support for different types of controllers. While WebVR supported the non-standard extended gamepad API and a few selected controllers, WebXR already supports a lot of different types of inputs, including touch screens, motion controllers and hands.

## Some terms and classes to clear things up

An XR controller comprises a lot of components that we at babylon sometimes name differently. It is also important to know the terms themselves to be able to use what you actually need. I would recommend reading the [XR Input section](https://www.w3.org/TR/webxr/#input) of the WebXR proposal draft.

An XR Session controls the input source of the current session. Every new input source connected to this session will be registered in the **inputSources** array of the native XR Session and will also trigger the `inputsourceschange` event with the new input source.

An Input source has one of three target Ray modes - `tracked-pointer` for gamepad-like controllers, `screen` for touch-screen oriented inputs, and `gaze` for gaze-based inputs (input sources like google cardboard that has no proper way for user input).

The [WebXRInput](/typedoc/classes/babylon.webxrinput) class is responsible of coordinating the addition and removal of input sources. It creates **WebXR Input Sources** classes and disposes them automatically for you.

Babylon's [WebXR Input Source](/typedoc/classes/babylon.webxrinputsource) class is the container for all user-input related objects. It is created automatically for you by the WebXR Input class for every controller.it is in charge of attaching the motion controller, which, in turn, is in charge of attaching the components and load the model.

An input source has two important reference spaces - one it the **`targetRay`** which represents the pointer's position and direction. Think - the edge of your finger.and the **`gripSpace`** which is the base of a handheld device connected to this input source. Think - the base of your hand. The `gripSpace` is optional and is only available when a motion controller is connected. And, althou sometimes the same, the grip and target ray space can have different transformation altogether.

A controller has an attached **motion controller** if the input source is a gamepad-like device (the oculus touch or the windows motion controllers are a good example for that). In turn, each controller has different **components** (buttons, triggers, thumbpads, touchpads) which have their state updated on each frame.

Note that input sources like of type `screen` are constantly created and removed when touch starts and end. This is called a transient input in the documentation.

## WebXRInput

The WebXR Input, initialized usually by the WebXR Experience Helper has little to no public members and is, in general, a standalone class that you won't interact with. However, it does have two observables that can be helpful for your experience:

### onControllerAddedObservable

This observable will be triggered when a new input source was detected and its corresponding WebXRInputSource class was created.

### onControllerRemovedObservable

Will be triggered when a controller left the experience, right before it is disposed.

## WebXRInputSource

The input source, created by the WebXR input, is your central container for a single input source (or controller). It has a few important members that will allow you to controller your input source.

## Public members

As mentioned before the Input source has two reference spaces - `grip` and `target` , which we call `pointer` . Both of those spaces are represented by a mesh, positioned in the orientation and position that is provided by the XRFrame (and the XR session). Which means that if you want to query the current transformation of the user's hand or the direction the user is pointing at.

To do that you can use the `getWorldPointerRayToRef` function:

``` javascript
const resultRay = new BABYLON.Ray();

// get the pointer direction
xrInputSource.getWorldPointerRayToRef(resultRay);
// get the grip direction, if available. If not, the pointer:
xrInputSource.getWorldPointerRayToRef(resultRay, true);
```

Usually, you would need to pointer direction ray.

### Input source observables

#### onMotionControllerInitObservable

This will be triggered when a motion controller, if available was initialized and its profile loaded.

#### onMeshLoadedObservable

This is a helper observable and is the same as `xrController.motionController.onModelLoadedObservable` . But since the motion controller is created async, using the motionController observable will only be available after `onMotionControllerInitObservable` was triggered:

``` javascript
// async, async, async
xrInput.onControllerAddedObservable.add((inputSource) => {
    inputSource.onMotionControllerInitObservable.add((motionController) => {
        motionController.onModelLoadedObservable.add((model) => {

        });
    });
});

// a little cleaner
xrInput.onControllerAddedObservable.add((inputSource) => {
    inputSource.onModelLoadedObservable.add((model) => {});
});
```

#### onDisposeObservable

Will be triggered right at the end of the `dispose()` function of the input source.

## Motion controllers

In most cases when starting a VR session, the user will have handheld devices, called here motion controllers. A motion controller will be automatically loaded, if available. A motion controller has a profile, containing its different components and their positions in the buttons and axes array, but babylon takes care of this for you so you don't have to know this to interact with the motion controller. You can see the different profiles in the [WebXR Input Profiles repository](https://github.com/immersive-web/webxr-input-profiles)

### Controller components

Each motion controller has different components, that are described in its profile. Each component has one of the types:

* Button
* Trigger
* Squeeze
* Thumbstick
* Touchpad

It also has a **unique** component id, which correlates to the actual component. For example, the **A button** on the oculus touch has the **type** `button` , and the **id** `a-button` .

#### Get the components available

To get a list of the components available, use the `getComponentIds` function. This will return an array of strings containing the IDs of the different components in this motion controller:

``` javascript
const ids = motionController.getComponentIds();
// ids = ["a-button", "b-button", "xr-standard-trigger", .....]
```

You can also get all of the available components using the `components` public member of the Motion Controller class. It is a map of all `WebXRControllerComponent` elements in this motion controller.

#### Get components

To get a component you need to either know its ID or type. When more than one of that type exists, the id would be better.
The IDs can be found in the profile.

To get a component according to its ID:

``` javascript
const triggerComponent = motionController.getComponent("xr-standard-trigger");
if (triggerComponent) {
    // found, do something with it.
}
```

To get a component of a specific type:

``` javascript
const squeezeComponent = motionController.getComponentOfType("squeeze");

// get the first registered button component
const buttonComponent = motionController.getComponentOfType("button");
```

to get all components of a specific type:

``` javascript
// get all button components
const buttonComponents = motionController.getAllComponentsOfType("button");
if (buttonComponents.length) {
    // some were found
}
```

#### The main component

Each controller has a main component, defined by the vendor. In most cases it is the trigger component type that is the main component. To get the main component (as defined in the profile):

``` javascript
const mainComponent = motionController.getMainComponent();
// mainComponent always exists!
```

#### Events and changes of a controller component

A component is updated on each frame with values provided by the gamepad object of the session's input source. Each button has 2 states - `touched` and `pressed` , and a value from 0 to 1 (0 being not pressed at all, 1 being fully pressed). Some components can only have the values 0 and 1 (like the button component).

Some types of components also have axes values (like a thumbstick or touchpad). The axes have values from -1 to 1.
1 in the X-axis means right, and 1 in the Y-Axis means down (towards the user).

To know what the components supports:

``` javascript
if (component.isButton()) {
    // we have a value
}
if (component.isAxes()) {
    // we have axes data
}
```

To get the component values at the current frame:

``` javascript
let value = component.value;
if (value > 0.8) {
    // do something nice with this value
}
if (component.pressed) {
    // the component is pressed, meaning value === 1
}

if (component.touched) {
    // fingers are on the component, might be half-pressed or moved
}
```

To access the axes data:

``` javascript
let axes = component.axes;
if (axes.x > 0.8) {
    // do something nice with the x-axis value
}
```

The component can also return the changes compared to the last frame. Changes are only populated when they exist, otherwise the changes map will be empty:

``` javascript
// maybe nothing happened between this and last frame
if (!component.hasChanges) {
    return;
}
let changes = component.changes;
if (changes.pressed) {
    // pressed state changed
    const isPressedNow = changes.pressed.current;
    const wasPressedInLastFrame = changes.pressed.previous;
}
if (changes.value) {
    // value changed! let's get the delta
    const delta - changes.value.current - changes.value.previous;
}
```

The components have two observables that can be used to fetch changes (and avoid checking value changes on each frame):

``` javascript
component.onButtonStateChangedObservable.add((component) => {
    // something changed, check the changes object
});

component.onAxisValueChangedObservable.add((values) => {
    console.log(values.x, values.y);
})
```

### How to get a model

#### The input-profile online repository

As part of the (successful!) attempt at forcing standards to WebXR, the guys at the [WebXR Input Profiles](https://github.com/immersive-web/webxr-input-profiles) github repository created an online repository that holds models and definition of visual reference definitions for most (if not all) motion controllers available today.

Babylon.js natively supports this repository and currently uses it as the default model delivery method for XR controllers.

There is little to no action required on your end - this is automated when not changing the configuration.

#### Babylon local controller definitions

Before the input-profile repository was published, Babylon had support for different types of controllers - Occulus Touch (1 and 2), Vive, Windows Motion Controllers and more. Since we still wanted to offer local support for those devices, we have decided to not only relay on the online profile repository, but to also deliver babylon-based controller classes for those that are already developed.

Babylon offers local definitions for the following:

* Windows Motion Controllers
* Oculus touch 1 and 2
* Vive
* Generic-Button controller

To use them, import them to your project, while not forgetting to prioritize them or disable the online repository:

``` javascript
// import the ones you want to use
import {
    WebXRMicrosoftMixedRealityController
} from '@babylonjs/core/XR/motionController/webXRMicrosoftMixedRealityController';

// prioritize the local classes (but use online if controller not found)
WebXRMotionControllerManager.PrioritizeOnlineRepository = false;
// or disable the online repository
WebXRMotionControllerManager.UseOnlineRepository = false;

// now, if loading a microsoft motion controller it will use the local class
```


## Why is it needed

The Features manager, our XR plugin system, was born out of a simple need - stay backwards compatible, but still deliver cutting edge APIs in a production system.

Since APIs like [hit test](https://github.com/immersive-web/hit-test) and [anchors](https://github.com/immersive-web/anchors) are constantly changing, and currently still have different support in different browser versions, there was a need for "versioning" of the current development to keep up with API modifications over time.

If you are a bit impatient you can check the [selected WebXR features](/divingDeeper/webXR/WebXRSelectedFeatures) section, but it is recommended to at least read this section quickly.

## How to use

### Construct a new features manager

If you are using the [base WebXR experience helper](/divingDeeper/webXR/webXRExperienceHelpers) a features manager will be created for you and will be available in `xrBaseHelper.featuresManager` . If not, you only need to provide an XR session manager object to initialize a new one:

``` javascript
const fm = new FeaturesManager(xrSessionManager);
```

Note that even before creating the features manager you could call its static methods (check availability, register a new feature, etc').

### What is available

To check what feature is available, use the features manager's static `GetAvailableFeatures` function, which will return an array of strings corresponding to specific features:

``` javascript
const availableFeatures = featuresManager.GetAvailableFeatures();

// availableFeatures = ["xr-hit-test", "xr-pointer-selection", ...]
```

To find if a specific feature is available use this code:

``` javascript
const availableFeatures = featuresManager.GetAvailableFeatures();
// using indexOf, but you can use any other method - find, findIndex, in, etc'
if (availableFeatures.indexOf(WebXRFeatureName.POINTER_SELECTION)) {
    // Pointer selection is available
}
```

`WebXRFeatureName` will always contain the list of possible features.

When using ES6, make sure to import the feature you want to use, otherwise it won't be registered:

``` javascript
import {
    WebXRControllerPhysics
} from '@babylonjs/core/XR/features/WebXRControllerPhysics';

// The physics controller support will now be available and can be enabled:

fm.enableFeature(WebXRControllerPhysics.Name, "latest");
```

### Versioning

Just like any plugin system, the features are versioned numerically. The version is a number. Higher the number, the newer the version (pretty simple).

Two extra definitions available - `stable` for the latest stable version of this feature, and `latest` which is always updated with the plugin with the highest version number. Not that while `latest` will always be available, `stable` is not a must.

To get the available versions use the `GetAvailableVersions` static method. It will return an array of available versions, for example:

``` javascript
const availableVersions = featuresManager.GetAvailableVersions(WebXRFeatureName.POINTER_SELECTION);

// availableVersions = ["latest", "stable", "1", "2"];
```

This means that you can ask for version 1, but also for the stable version which will be one of the 2 - "1" or "2", depends on our definition. The "latest" version will load "2".

### Enable and disable a feature

Enabling a feature means that the feature is ready to be used in a working session. When a feature is enabled, it is ready to be attached, which technically means - influence the scene actively.

As an example, **enabling** the teleportation feature will register the "on controller added/removed" observer.**Attaching** it will register all observers required for each controller to work properly.**Detaching** it will remove this observers, and **disabling** it will make sure the meshes are invisible and the onControllerAdded (and Removed) observer will be removed.**Disposing** the feature will release everything, including all associated meshes.

To enable the feature use a constructed features manager's `enableFeature` function with the name and version you wish to load:

``` javascript
// get the features manager
const fm = xr.baseExperience.featuresManager;

// disable an already-enabled feature
fm.disableFeature(WebXRFeatureName.POINTER_SELECTION);

// enable latest hit-test
const xrHitTestLatest = fm.enableFeature(WebXRFeatureName.HIT_TEST, "latest");

// enable specific version of hit-test. This will disable an older implementation and enable the new one
const xrHitTest1 = fm.enableFeature(WebXRFeatureName.HIT_TES, 1);
```

### Configuring the feature

Every feature (as of now) has a different configuration options object that can be provided when enabling the feature. Each feature has an options interface. For example, Version 1 of Hit-Test has the following options configuration interface:

``` javascript
interface IWebXRHitTestOptions {
    /**
     * Only test when user interacted with the scene. Default - hit test every frame
     */
    testOnPointerDownOnly ? : boolean;
    /**
     * The node to use to transform the local results to world coordinates
     */
    worldParentNode ? : TransformNode;
}
```

To use it:

``` javascript
// get the features manager
const fm = xr.baseExperience.featuresManager;

// enable latest hit-test with a configuration object. Old enabled version will be disposed
const xrHitTestLatest = fm.enableFeature(WebXRFeatureName.HIT_TEST, "latest", {
    testOnPointerDownOnly: true
});

// enable specific version of hit-test with default configuration. Old config will be lost
const xrHitTest1 = fm.enableFeature(WebXRFeatureName.HIT_TES, 1);
```

### Attach and detach a feature

Once you have a feature enabled, you can use its synchronous attach and detach methods to attach (or detach) it to the scene:

``` javascript
// enable version 1 of hit-test
// Feature will be auto-attached once the session starts
const xrHitTest1 = fm.enableFeature(BABYLON.WebXRFeatureName.HIT_TEST, 1);

// detach the feature, but keep it enabled (for reattachment). This also disables auto-attach, if not already attached
xrHitTest1.detach();

// re-attach
xrHitTest1.attach();
```

Enabling a feature will attach it automatically when an XR session starts. If you want to only enable but not attach a feature (so you have full control of when it starts working), set the `attachIfPossible` variable to false (defaults to true):

``` javascript
// get the features manager
const fm = xr.baseExperience.featuresManager;
const xrHitTestLatest = fm.enableFeature(WebXRFeatureName.HIT_TEST, "latest", {
    testOnPointerDownOnly: true
}, false /* prevent attaching automatically */ );
```

## ES6 passive loader

When using the ES6 module loader you will notice that no features apart from transportation and pointer selection are available. That is because the features and their version must be imported by you. It is done so that unneeded code will not be included in the build and will be loaded only when your experience needs it. Wonderful for tree shaking!

So this won't work in ES6

``` javascript
const featuresManager = giveMeMyFeaturesManagerSomehow();
// No hit test is available is not available
const xrHitTest1 = featuresManager.enableFeature("xr-hit-test", 1);
// Error thrown, "feature not found"
```

Adding an import will make it work:

``` javascript
// other imports...
import {
    WebXRHitTestLegacy
} from "@babylonjs/core/Cameras/xr/features/WebXRHitTestLegacy";

const featuresManager = giveMeMyFeaturesManagerSomehow();
// feature will now be enabled
const xrHitTest1 = featuresManager.enableFeature(WebXRFeatureName.HIT_TEST /* Same as "xr-hit-test" */ , 1);
```

## Writing a new feature

If you want to add your own feature here are a few guidelines:

### IWebXRFeature

The feature should implement the `IWebXRFeature` interface, which looks like this:

``` javascript
interface IWebXRFeature extends IDisposable {
    /**
     * Is this feature attached
     */
    attached: boolean;
    /**
     * Should auto-attach be disabled?
     */
    disableAutoAttach: boolean;
    /**
     * Attach the feature to the session
     * Will usually be called by the features manager
     *
     * @param force should attachment be forced (even when already attached)
     * @returns true if successful.
     */
    attach(force ? : boolean): boolean;
    /**
     * Detach the feature from the session
     * Will usually be called by the features manager
     *
     * @returns true if successful.
     */
    detach(): boolean;
}
```

To ease the process, you can, instead, extend the `WebXRAbstractFeature` , which has a few extra help-functions and implements most functions needed for a working feature.

After creating the feature, you will need to register it with the features manager, so it can be enabled.

To register it, use the static `AddWebXRFeature` function:

``` javascript
const nameOfFeature = "awesome-name";
const version = 1;
const isTheFeatureStable = true
WebXRFeaturesManager.AddWebXRFeature(nameOfFeature, (xrSessionManager, options) => {
    return () => new MyNewFeature(xrSessionManager, options);
}, version, isTheFeatureStable);
```

This way you can enable the feature using its name whenever you wish to use it.

WebXR Demos and Examples

Basic scene

Basic Scene with XR Support

Adding teleportation

Adding a color picker to the basic scene

Other demos

Babylon.js scenes with XR support