Audio-visual processing and immersive multimedia

Heads: Achille Peternier, Tiziano Leidi

Today’s advanced multimedia technologies leverage high-definition audio-visual content, while solutions for extended reality allow users to experience new artificial dimensions that coexist with the physical world. The research area for Audio-Visual Processing and Immersive Multimedia covers the full spectrum: development of applications for digital signal processors and systems on chip, digital signal processing techniques, GPU computing, audio/video streaming and processing, computer vision, software infrastructures for scientific simulation, extended reality (XR), 3D audio systems, digital gaming and gamification.

Specialty sectors:

  • Audio-Video Processing
  • Video processing and computer vision
  • Computer graphicvs and XR
  • Digital gaming and gamification

Audio Processing

Contact person: Thierry Heeb, Loris Grossi

The Audio Processing research sector focuses on novel real-time solutions covering the topics of sound processing, immersive audio rendering, and soundfield control. The team has extensive experience in real-world audio digital signal processing and collaborates with both global companies and innovative start-ups through innovation projects.

  • Development of digital signal processing algorithms for audio applications
  • Design and implementation of real-time audio-processing solutions
  • 3D audio based on crosstalk cancellation and binaural technologies
  • Soundfield control and room acoustics correction
  • Cloud-based software infrastructures for audio streaming and processing

Video processing and computer vision

Contact person: Andrea Quattrini

The Video Processing and Computer Vision research sector focuses on leveraging the use of 2D and 3D cameras to bring automation and innovation into technological processes. The unit has many years of experience in the use of hardware for the generation of images and point clouds for various applications and is particularly active in 3D measurement and reconstruction and in the application of deep learning techniques on these types of data. The unit collaborates on several fronts both with innovative startups and with companies of international importance.

  • 2D and 3D vision systems
  • 3D reconstruction systems based on laser triangulation, stereoscopy, structured light, time of flight, and photogrammetry.
  • Analysis and processing of images and point clouds using both standard processing techniques and deep learning, including classification, object detection and tracking, semantic and instance segmentation
  • Autonomous driving systems, both in terms of motion planning (collision avoidance) and automatic obstacle position measurement
  • Real-time video-processing for computer vision (including on embedded system-on-chips)

Computer graphics and extended reality

Contact person: Achille Peternier

The specialty sector on Computer Graphics and Extended Reality covers all the aspects that go from the research and development of graphics systems up to their exploitation within XR contexts. This sector focuses both on the creation of basic, highly customized software instruments that take advantage of modern GPUs to dealing with performance, to the usage of higher-level standards and products (such as Unity 3D and Unreal Engine) for creating immersive applications in fields such as (but not limited to) education, simulation, manufacturing, cultural heritage, and entertainment.

  • Design of customized real-time 3D visualization software on immersive devices
  • New human-computer interaction paradigms based on gesture recognition and body tracking
  • Creation and navigation in interactive virtual environments
  • Virtual, augmented and mixed reality applications

Digital gaming and gamification

Contact person: Masiar Babazadeh

The Digital Gaming and Gamification sector has expertise in research projects on educational (game-based learning) and entertaining contexts, in game design, game development, and in the application of ludic elements in non-ludic contexts. This sector is involved in the organization of the yearly Global Game Jam competition and in the organization of several eSports events on the SUPSI campus.

  • Game Design and Game Development (using Unity 3D, Unreal Engine, …)
  • Gamification
  • Game-based learning
  • Design and implementation of entertainment and serious videogames by using game engines
  • Development of cloud-based server-side infrastructures for multiplayer video games
  • E-Sports