MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversation

This repository contains various models targetting multimodal representation learning, multimodal fusion for downstream tasks such as multimodal sentiment analysis.

Platform for Situated Intelligence

Multimodal Sarcasm Detection Dataset

A comprehensive reading list for Emotion Recognition in Conversations

Context-Dependent Sentiment Analysis in User-Generated Videos

Mobile application for exploring fitness data using both speech and touch interaction.

Multimodal sentiment analysis using hierarchical fusion with context modeling

Unsupervised Multimodal Clustering for Semantics Discovery in Multimodal Utterances (ACL 2024)

Companion Repo for the Vision Language Modelling YouTube series - https://bit.ly/3PsbsC2 - by Prithivi Da. Open to PRs and collaborations

Using voice and pen to draw diagrams quickly with automatically suggested icons and texts by AI in talking.

A multimodal face liveness detection module that can be used in the context of face anti-spoofing

Multimodal AI Assistant with Google Gemini-1.5-pro, gTTS, PIL, and SpeechRecognition Technologies!

Code for ICMI2020 and ICMI2021 papers: "Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle" and "ML-PersRef: A Machine Learning-based Personalized Multimodal Fusion Approach for Referencing Outside Objects From a Moving Vehicle"

Technical Draft: A platform to augment web applications with multimodal interactions

Project for Multimodal Interaction course (A.Y. 2019/2020), GesturePad

A thumb-worn multimodal smart ring, callled "mRing", is designed to support mid-air interactions with Computers/TV/Phone/AI/Robots, leveraging micro touch gestures between thumb and other 4 fingers

Repository to contain MMI development during UTA CSE REU 2019. Makes use of Open-Myo, a module to get data from a Myo armband using a generic BLE interface.

Systems concept and feasibility study exploring Meta Quest cockpit simulation with OpenXR, voice, supported gaze input, HOTAS and haptics. The complete integrated system remains unvalidated.

Challenge of gesture recognition for the course : "Multimodal Processing Recognition and Interaction" of the HES-SO university (Switzerland)