In red education, intelligent headsets bring history to life; In medical training, the virtual real fusion system allows doctors to repeatedly hone their surgical skills in a safe environment; In consumer scenarios, creativity and information are instantly overlaid on the real world, and smart glasses become the "spatial canvas" for designers. The metaverse technology in Shanghai is moving from concept to industry, creating cutting-edge innovative applications and accelerating the formation of a metaverse ecosystem that integrates virtual and real elements.

Original and authentic preservation of educational content

Wearing a smart headset, the historical scene of the Shanghai Workers' Armed Uprising unfolds before our eyes. Lighting the spark, sniping and shooting, splicing the railway tracks, and starting the train, thanks to natural gesture recognition and keen eye tracking technology, participants immerse themselves in that passionate era from a first person perspective, achieving immersive red education in a real and tangible interactive experience.

In June 2024, Shanghai Business School and the Memorial Hall of the First National Congress of the Communist Party of China jointly established the New Media and Metaverse Laboratory to conduct research on XR core technology and empower red cultural resources with digital technology. Song Xiaoguang, Secretary of the Party Branch of the School of Business Information at Shanghai Business School, introduced that the laboratory has independently developed the MR large space scene "Red Dawn" with the background of the third armed uprising of the Shanghai working class. It replaces the traditional LBE (location-based entertainment) that requires landmark placement with spatial computing, making the interactive experience more natural and smooth.


The MR large space scene 'Red Dawn'.

In the future, the main research goal of the laboratory is to focus on the application of metaverse in various teaching fields in universities, deeply integrate generative artificial intelligence and metaverse, and enable students to quickly build metaverse scenarios. The ultimate goal is for various disciplines to develop their own metaverse applications.

The reason for setting such a goal is that in education, the key bottleneck faced by traditional VR content development is that teachers need to translate instructional designs to technical personnel, which involves "translation loss". Technicians have difficulty fully understanding the essence of teaching, resulting in the final product often being unable to accurately reproduce the teacher's teaching intentions and core knowledge points.

Song Xiaoguang stated that the intervention of AI has changed the paradigm of metaverse development. It can accurately understand teaching points, transform them into VR interactive logic, maintain the integrity and accuracy of teaching design, and enable technology to truly serve teaching itself.

Empowering Medical Training with the Integration of Reality and Reality

Founded in 2016, Kongdao Information Technology (Shanghai) Co., Ltd. develops medical education and training that combines virtual and real elements. Founder Min Shihao introduced that the team has built seven specialized systems that combine virtual and real elements, covering cardiovascular and cerebrovascular intervention, ophthalmic cataract removal, difficult airway management in anesthesia, ultrasound, physician assessment, emergency treatment, etc., to achieve clinical simulation of medical specialties and clinical application of AI technology. With the support of metaverse clinical and AI scientific research, the company provides clinical surgical planning and specialized clinical teaching systems for 30 key tertiary hospitals nationwide, improving medical training efficiency and reducing training costs.

In the field of surgical training, Kongdao Information Technology has built VR+AI surgical simulation training and intelligent analysis and evaluation for clinical skills training. Through the combination of virtual and real modes, immersive surgical simulation is constructed. At the same time, AI technology is used to deeply recognize and analyze operation videos, output objective quantitative scores, and break through the subjective limitations of traditional manual evaluation. This technology has been applied in endoscopic surgery training, where students can upload real endoscopic surgical videos. The system automatically recognizes surgical instruments and anatomical points, achieving automatic disassembly of operation videos and intelligent workflow recognition, assisting surgeons in efficiently analyzing surgical performance.


Ophthalmic cataract removal simulation teaching system.

We will optimize the equipment according to clinical needs and characteristics, and work together with doctors to polish it, "said Min Shihao. The team has developed a visual training system for difficult airways, which faces 10 complex and tricky airway scenarios such as mouth opening difficulties and esophageal reflux in clinical practice. By constructing highly realistic virtual airway scenes and combining them with physical simulation equipment, students can deeply practice the operational details of difficult airway management and repeatedly practice 12 commonly used airway management skills.

Whether it's AI, VR, or future digital patients, they are essentially tools, and the key is to solve real-world problems and help clinical diagnosis and treatment. ”Looking back at the development process of the medical training system that combines virtual and real elements, Min Shihao introduced that it initially started with animations that could only be demonstrated. In order to achieve interaction, virtual simulation and VR were introduced. The lack of tactile sensation and muscle memory required the use of force feedback technology, which was a demand driven gradual process. In the future, the development direction of the industry is to build highly personalized digital twin human bodies from micro cells to macro structures, making clinical training infinitely closer to reality. Massive digital twin human bodies are expected to assist in the training of medical robots and cope with complex clinical situations.

"Tiktok" in the age of smart glasses

Shanghai Ningfeng Intelligent Technology Co., Ltd. exhibited a variety of multi-functional consumer grade smart glasses products in its exhibition hall. Ningfeng Intelligence has deep cooperation with more than ten domestic smart glasses companies, and its low threshold and fast creation intelligent agent platform "Mixed Space Studio Agent" is dedicated to producing metaverse content. Even users who do not know how to program can "drag and drop" to build metaverse scenes, and after completion, distribute them to different intelligent terminals with just one click to achieve a metaverse experience. In the words of He Feixiang, the founder and CEO of Ningfeng Smart, it is to create the "Tiktok" of the era of smart glasses.


Quickly build a metaverse scene.

Currently, commerce and cultural tourism are important application scenarios in the metaverse. Ningfeng Intelligence integrates and innovates cultural, business, and travel resources in the form of a metaverse, transforming static landscapes into dynamic spaces that are "playable, explorable, and shareable". It is compatible with mainstream smart glasses devices at home and abroad, promoting the upgrading of the entertainment industry through immersive spatiotemporal experiences, and extending tourists' stay time.

In museums and art curation settings, visitors can rent AR glasses after entering the exhibition hall to experience the interactive experience of metaverse cultural tourism guidance or virtual real combination. In Penglai Immortal Cave Scenic Spot, Ningfeng Intelligent has developed two core interactive scenes, "AR ringing the clock of heaven and earth" and "flying the virtual Kongming lamp", relying on the positioning and recognition technology of the metauniverse and AR large space, breaking through the single mode of traditional cave tour. During the Spring Festival, Ningfeng Intelligent will also launch the "Luzhou Lantern Festival" metaverse experience in Hefei.

In the past few years, a scenario with virtual real integration and spatial interaction was often directly referred to as the metaverse. However, at that time, the underlying technology and industrial chain processes were not mature, resulting in a lack of scalable application scenarios, making it difficult to popularize the metaverse. He Feixiang said that AI has solved one of the pain points of the metaverse landing. The deep integration of AI and AR has revolutionized production efficiency, significantly reduced production costs, and enabled the development of metaverse applications even with zero foundation. We have reduced the price from the previous custom development costs of 100000 or 200000 yuan to within a thousand yuan

He Feixiang predicts that with the advancement of full-color display and spatial computing technology, as well as the continuous improvement of computing power and endurance, more interesting applications will be developed in the future. Smart glasses will present more exciting images, stimulate consumer usage frequency, and drive sales of smart glasses. In the future trend of scientific and technological development, the only smart product that can replace smartphones as the next universal computing entry point is undoubtedly smart glasses. With full-color display and the integration of AI, the name glasses no longer distinguishes any technological form such as AI, XR, etc

In red education, intelligent headsets bring history to life; In medical training, the virtual real fusion system allows doctors to repeatedly hone their surgical skills in a safe environment; In consumer scenarios, creativity and information are instantly overlaid on the real world, and smart glasses become the "spatial canvas" for designers. The metaverse technology in Shanghai is moving from concept to industry, creating cutting-edge innovative applications and accelerating the formation of a metaverse ecosystem that integrates virtual and real elements.

Original and authentic preservation of educational content

Wearing a smart headset, the historical scene of the Shanghai Workers' Armed Uprising unfolds before our eyes. Lighting the spark, sniping and shooting, splicing the railway tracks, and starting the train, thanks to natural gesture recognition and keen eye tracking technology, participants immerse themselves in that passionate era from a first person perspective, achieving immersive red education in a real and tangible interactive experience.

In June 2024, Shanghai Business School and the Memorial Hall of the First National Congress of the Communist Party of China jointly established the New Media and Metaverse Laboratory to conduct research on XR core technology and empower red cultural resources with digital technology. Song Xiaoguang, Secretary of the Party Branch of the School of Business Information at Shanghai Business School, introduced that the laboratory has independently developed the MR large space scene "Red Dawn" with the background of the third armed uprising of the Shanghai working class. It replaces the traditional LBE (location-based entertainment) that requires landmark placement with spatial computing, making the interactive experience more natural and smooth.


The MR large space scene 'Red Dawn'.

In the future, the main research goal of the laboratory is to focus on the application of metaverse in various teaching fields in universities, deeply integrate generative artificial intelligence and metaverse, and enable students to quickly build metaverse scenarios. The ultimate goal is for various disciplines to develop their own metaverse applications.

The reason for setting such a goal is that in education, the key bottleneck faced by traditional VR content development is that teachers need to translate instructional designs to technical personnel, which involves "translation loss". Technicians have difficulty fully understanding the essence of teaching, resulting in the final product often being unable to accurately reproduce the teacher's teaching intentions and core knowledge points.

Song Xiaoguang stated that the intervention of AI has changed the paradigm of metaverse development. It can accurately understand teaching points, transform them into VR interactive logic, maintain the integrity and accuracy of teaching design, and enable technology to truly serve teaching itself.

Empowering Medical Training with the Integration of Reality and Reality

Founded in 2016, Kongdao Information Technology (Shanghai) Co., Ltd. develops medical education and training that combines virtual and real elements. Founder Min Shihao introduced that the team has built seven specialized systems that combine virtual and real elements, covering cardiovascular and cerebrovascular intervention, ophthalmic cataract removal, difficult airway management in anesthesia, ultrasound, physician assessment, emergency treatment, etc., to achieve clinical simulation of medical specialties and clinical application of AI technology. With the support of metaverse clinical and AI scientific research, the company provides clinical surgical planning and specialized clinical teaching systems for 30 key tertiary hospitals nationwide, improving medical training efficiency and reducing training costs.

In the field of surgical training, Kongdao Information Technology has built VR+AI surgical simulation training and intelligent analysis and evaluation for clinical skills training. Through the combination of virtual and real modes, immersive surgical simulation is constructed. At the same time, AI technology is used to deeply recognize and analyze operation videos, output objective quantitative scores, and break through the subjective limitations of traditional manual evaluation. This technology has been applied in endoscopic surgery training, where students can upload real endoscopic surgical videos. The system automatically recognizes surgical instruments and anatomical points, achieving automatic disassembly of operation videos and intelligent workflow recognition, assisting surgeons in efficiently analyzing surgical performance.


Ophthalmic cataract removal simulation teaching system.

We will optimize the equipment according to clinical needs and characteristics, and work together with doctors to polish it, "said Min Shihao. The team has developed a visual training system for difficult airways, which faces 10 complex and tricky airway scenarios such as mouth opening difficulties and esophageal reflux in clinical practice. By constructing highly realistic virtual airway scenes and combining them with physical simulation equipment, students can deeply practice the operational details of difficult airway management and repeatedly practice 12 commonly used airway management skills.

Whether it's AI, VR, or future digital patients, they are essentially tools, and the key is to solve real-world problems and help clinical diagnosis and treatment. ”Looking back at the development process of the medical training system that combines virtual and real elements, Min Shihao introduced that it initially started with animations that could only be demonstrated. In order to achieve interaction, virtual simulation and VR were introduced. The lack of tactile sensation and muscle memory required the use of force feedback technology, which was a demand driven gradual process. In the future, the development direction of the industry is to build highly personalized digital twin human bodies from micro cells to macro structures, making clinical training infinitely closer to reality. Massive digital twin human bodies are expected to assist in the training of medical robots and cope with complex clinical situations.

"Tiktok" in the age of smart glasses

Shanghai Ningfeng Intelligent Technology Co., Ltd. exhibited a variety of multi-functional consumer grade smart glasses products in its exhibition hall. Ningfeng Intelligence has deep cooperation with more than ten domestic smart glasses companies, and its low threshold and fast creation intelligent agent platform "Mixed Space Studio Agent" is dedicated to producing metaverse content. Even users who do not know how to program can "drag and drop" to build metaverse scenes, and after completion, distribute them to different intelligent terminals with just one click to achieve a metaverse experience. In the words of He Feixiang, the founder and CEO of Ningfeng Smart, it is to create the "Tiktok" of the era of smart glasses.


Quickly build a metaverse scene.

Currently, commerce and cultural tourism are important application scenarios in the metaverse. Ningfeng Intelligence integrates and innovates cultural, business, and travel resources in the form of a metaverse, transforming static landscapes into dynamic spaces that are "playable, explorable, and shareable". It is compatible with mainstream smart glasses devices at home and abroad, promoting the upgrading of the entertainment industry through immersive spatiotemporal experiences, and extending tourists' stay time.

In museums and art curation settings, visitors can rent AR glasses after entering the exhibition hall to experience the interactive experience of metaverse cultural tourism guidance or virtual real combination. In Penglai Immortal Cave Scenic Spot, Ningfeng Intelligent has developed two core interactive scenes, "AR ringing the clock of heaven and earth" and "flying the virtual Kongming lamp", relying on the positioning and recognition technology of the metauniverse and AR large space, breaking through the single mode of traditional cave tour. During the Spring Festival, Ningfeng Intelligent will also launch the "Luzhou Lantern Festival" metaverse experience in Hefei.

In the past few years, a scenario with virtual real integration and spatial interaction was often directly referred to as the metaverse. However, at that time, the underlying technology and industrial chain processes were not mature, resulting in a lack of scalable application scenarios, making it difficult to popularize the metaverse. He Feixiang said that AI has solved one of the pain points of the metaverse landing. The deep integration of AI and AR has revolutionized production efficiency, significantly reduced production costs, and enabled the development of metaverse applications even with zero foundation. We have reduced the price from the previous custom development costs of 100000 or 200000 yuan to within a thousand yuan

He Feixiang predicts that with the advancement of full-color display and spatial computing technology, as well as the continuous improvement of computing power and endurance, more interesting applications will be developed in the future. Smart glasses will present more exciting images, stimulate consumer usage frequency, and drive sales of smart glasses. In the future trend of scientific and technological development, the only smart product that can replace smartphones as the next universal computing entry point is undoubtedly smart glasses. With full-color display and the integration of AI, the name glasses no longer distinguishes any technological form such as AI, XR, etc