Switzerland

Swiss and S.Korean partners sought for a bilateral project submission. South Korea iconSwitzerland icon

Partner search:
Title: Partner for a project in developing Intelligent Content Creation Tools for Augmented and Virtual Reality (AR & VR)

Summary:
The Media Technology Center (MTC) at ETH Zurich is looking for a Swiss implementation partner to develop content creation tools that allow everyone to create 3D content for AR and VR experiences. The partner should be able to support the project with hardware or software infrastructure (e.g. an existing distribution platform for AR content).

Description:
With the proliferation of social media apps that include image and video editing capabilities, today, everyone is constantly creating and distributing image and video content. While mobile devices and internet infrastructure (5G, 6G) to handle dense, spatial 3D content in real-time will become readily available over the next few years, we are still lacking content creation tools that allow everyone to create 3D content for AR and VR experiences.
In this project, we propose to develop novel tools and methods to let everyday users create VR and AR content using novel authoring interfaces and AI-assisted content generation and placement. Such tools need a semantic understanding of the context such as the real world or the 360° scene of where content is placed. We envision a set of tools that enables both everyday, novice users to create VR and AR content using their smartphone or headsets that can be integrated into future virtual worlds.

Research Idea
Based on what story users want to tell, they should be able to easily decide on what format is best suited. 360 degree video, VR and AR content, and the real world can be blended into rich and immersive experiences. Therefore, this project brings together experts from VR and AR research to build tools that support the creation of such immersive experiences at ease. To achieve this, we believe two core components are required:

Cloud-based Authoring and Delivery Platform
First, we propose a cloud-based authoring and delivery platform, which stores all content, performs data processing tasks, delivers content to consumer devices, such as smartphones and head-mounted displays, and provides the connection to various (collaborative) content creation tools. The platform assists the users in authoring content with novel AI-based methods, such as semantic location-aware content placement. Additionally, it supports multiple users and multiple display platforms (different VR or AR devices). The platform acts as a backend cloud service that is implemented in a modular, extensible way, deployable globally.

Advanced In-situ Content Creation Tools
Second, we propose to create a collection of in-situ content creation tools that plug in to the authoring and delivery platform. The collection offers tools for creating and handling compelling 360 degree video, VR, and AR experiences in-situ, that is, the authoring user is able to create the experience at the same place as the user who will be experiencing the content. Key components include:

• Automatic lighting of virtual objects based on the 360 degree video or the real world conditions,
• Creation happens with AR goggles or smartphone at the location,
• User-adaptive interaction,
• Collaborative authoring with more than 2 users,
• Story-based content creation,
• Voice and gesture recognition,
• Easy generation of animations through puppeteering, e.g. giving the path for an agent
• Offers a collaborative environment between VR and AR users that allows for many different creation scenarios, e.g. authoring for AR users (e.g. educator), experience in 360 VR (students).

Type and role of partner sought:
The specific area of activity of the partner and the tasks to be performed by the partner sought:


1. In Switzerland:
For our implementation partner we are looking for a software, hardware, engineering company in the area of augmented reality or mixed reality. The partner should be able to support the project with hardware or software infrastructure (e.g. an existing distribution platform for AR content) or similar


2. In South Korea:
We already found an implementation partner.

Contact person in Switzerland:
Inès Rossetti This email address is being protected from spambots. You need JavaScript enabled to view it.
+41 58 468 70 38