Multimodal interfaces have become the dominant interface and platform that are used to interact with machines. Analytic techniques are needed that match these data sources in order to evaluate human behaviours during interactions. Our research aims to explore features from various modalities that are signicant to human behaviours. Innovative approaches are investigated to fuse various modalities for effective human-machine interactions. Multimodal behaviour analytics is used to understanding human behaviours in various situations such as cognitive load measurement, trust measurement, human-AI interaction.
The modalities we are using include (but not limited to):
- Eye activities
- Galvanic Skin Responses (GSR)
- Blood Vessel Pulse (BVP)
- Mouse movement
- Keyboard activities
- Writing behaviour
