Understanding Multimedia and Multimodal Data

The data ecosystems around us contain many different media types ranging from audio, image and video to time series, 3D data and textual (meta-)data available especially on (social) media platforms. Multimedia AI deals with the detection, retrieval, analysis, and recommendation of media data in unimodal or multimodal (when combining two or more different media data types) settings.

Research focus

We carry out award-winning multimedia AI research in computer vision (CV), natural language processing (NLP), multimodal retrieval (MMR), and time series analysis & pattern analysis, which we apply inter alia in social media analysis, property market analysis, human gait analysis, digital heritage and digital humanities.

We support real estate companies with the satellite image-based assessment of land plot location quality and the age prediction of buildings based on real estate ad images. In cooperation with paleographers, digitised medieval manuscripts serve as the basis for the retrieval and identification of writers. To support archaeologists, we use 3D scans of rock surfaces as a basis for segmenting human-carved prehistoric figures out of the rocks.

We make use of NLP methods to analyse social media for tasks such as fake news detection or sexism detection (10). We employ multiple modalities for MMR in order to extract information from social media or to allow a more precise assessment of real estate. To support physiotherapists in the diagnosis of human gait deficits, we analyse time-based gait measurements in 2D and 3D to detect and classify characteristic patterns. 

 

Projekte

ImmBild - Location Assessment by Computer Vision

Location is key – especially when it comes to real estate value. “ImmBild” aims at developing a new method for estimating property value using computer vision of satellite data.

InfraBase - Automatic Building Footprint Segmentation

The project deals with the fully automatic analysis of satellite imagery. The goal is to extract a pixel-accurate segmentation of building roofs to generate a rich meta-data layer.

ImmoAge - Visual Age Prediction of Real Estate

Year of construction, architectural period and architectural style have a significant impact on property prices. New methods of classifying and evaluating real estate have been developed on the basis ...

Pitoti 3D

Italy’s petroglyphs were carved into rock faces by prehistoric cultures. This project sets out to investigate and document the 3D nature of these petroglyphs for the first time.

IMREA - Intelligent Multimodal Real Estate Assessment

Multimodal information extraction and machine learning techniques for the extraction of real estate related attributes and parameters from heterogeneous input data

You want to know more? Feel free to ask!

Academic Director Creative Computing (BA)
Department of Media and Digital Technologies
Location: A - Campus-Platz 1
M: +43/676/847 228 245