NYU is developing 3D streaming video tech with the help of its dance department

By admin

NYU
ORG

is launching a project to spur the development of immersive 3D video for dance education — and perhaps other areas. Boosted by a

$1.2 million
MONEY


four-year
DATE

grant from

the National Science Foundation
ORG

, the undertaking will try to make

Point-Cloud Video
PRODUCT

(PCV) tech viable for streaming.

A point cloud is a set of data points in a 3D space representing the surface of a subject or environment.

NYU
ORG

says

Point-Cloud Video
PRODUCT

, which strings together point-cloud frames into a moving scene, has been under development for

the last decade
DATE

. However, it’s typically too data-intensive for practical purposes, requiring bandwidth far beyond the capabilities of

today
DATE

’s connected devices.

The researchers plan to address those obstacles by “reducing bandwidth consumption and delivery latency, and increasing power consumption efficiency so that PCVs can be streamed far more easily,” according to an

NYU
ORG

Engineering blog post published

Monday
DATE

.

Project
ORG

leader

Yong Liu
PERSON

, an

NYU
ORG

electrical and computer engineering professor, believes modern breakthroughs make that possible. “With recent advances in the key enabling technologies, we are now at the verge of completing the puzzle of teleporting holograms of real-world humans, creatures and objects through the global Internet,”

Liu
PERSON

wrote on

Monday
DATE

.

ChatGPT maker

OpenAI
ORG

launched a model

last year
DATE

that can create 3D point clouds from text prompts. Engadget reached out to the project leader to clarify whether it or other generative AI tools are part of the process, and we’ll update this article if we hear back.

The team will test the technology with

the NYU Tisch School of the Arts
ORG

and

the Mark Morris Dance Group’s
ORG


Dance Center
ORG

. Dancers from both organizations will perform on a volumetric capture stage. The team will stream their movements live and on-demand, offering educational content for aspiring dancers looking to study from high-level performers — and allowing engineers to test and tweak their

PCV
ORG

technology.

The researchers envision the work opening doors to more advanced

VR
ORG

and mixed reality streaming content. “The success of the proposed research will contribute towards wide deployment of high quality and robust PCV streaming systems that facilitate immersive augmented, virtual and mixed reality experience and create new opportunities in many domains, including education, business, healthcare and entertainment,”

Liu
PERSON

said.



Point-Cloud Video
PRODUCT

holds tremendous potential to transform a range of industries, and I’m excited that the research team at

NYU Tandon
ORG

prioritized dance education to reap those benefits early,” said

Jelena Kovačević
PERSON

,

NYU
ORG


Tandon Dean
PERSON

.