Case Studies: Arque Inc.

Real-time AR Information Projects
the Thrill of Sport Climbing

"Camera-tracking information generated with FreeD for consistently high-quality AR content throughout many hours of competition"

― Mr. Masato Tsutsui, President of Arque Inc.

Challenge

Both the rules and flow of climbing competitions can be hard to understand, making it difficult to convey the appeal of this unique sport.

Solutions

Panasonic PTZ Cameras compatible with the FreeD* protocol were used to construct an AR system capable of real-time synthesizing competition information as live video.

* FreeD, a widely used protocol for transmitting camera-tracking information to AR/VR systems, is used in the AW-UE150W/K cameras to transmit pan, tilt, zoom and focus information for AR/VR compositing.

Background

AR content enriches streaming of live sports

Japan’s largest Sport Climbing competition, BOULDERING JAPAN CUP 2022 sponsored by the Japan Mountaineering & Sport Climbing Association, was held on February 5–6, 2022 to crown Japan’s best climbers and choose members for the national team. To capture all the action, two Panasonic 4K Integrated Cameras (AW-UE150W/K) were used to gather AR/VR video content to virtually show climbing routes and wall-surface angles on the climbing wall in real time for powerful viewing experiences.

Why choose Panasonic products?

Simple configuration and superior scalability

The video production system at BOULDERING JAPAN CUP 2022 was managed by Mr. Masato Tsutsui of Arque Inc., developer of the OnlineObservation^® AR/VR video system for Sport Climbing. Asked why his company chose the Panasonic solution, Mr. Tsutsui explained: “Our system was designed for temporary use during the competition, so we wanted cameras that would allow us to build a compact system that could be installed and operated by a small number of people. In addition, we were interested in future scalability, including functionality for other diverse applications, and capabilities to operate in combination with our own software. We chose Panasonic’s 4K Integrated Camera (AW-UE150W/K) for such reasons, including its flexible software and advanced shooting functions.”

Benefits of this system

AR Production Supported with FreeD

For this year's competition, two 4K Integrated Cameras (AW-UE150W/K) were used, one installed on the second floor and the other mounted on a motorized dolly near the climbing wall. Using camera-tracking information transmitted via FreeD, wireframes showing climbing routes and wall-surface angles were inserted into live video in real time. “Camera-tracking information, such as pan, tilt, and zoom, was automatically output via FreeD for consistently high-quality AR content production throughout the long hours of competition. This year, we pushed the limits by incorporating mixed-reality (MR) transition combining AR images from two AW-UE150s with VR technology,” said Tsutsui.

During competition, video is composited with AR to virtually show climbing routes and wall-surface angles.

High-performance Videography

An AW-UE150W positioned on the second floor was used mainly for viewing the entire venue, including the competition wall and audience. The camera’s 75.1° horizontal wide-angle easily framed the competition wall in a landscape view. It also captured the competition area with excellent image quality from a distance of some 60m (about 200 feet). Upfront, an AW-UE150K mounted on a motorized dolly produced smooth, vibration-free video thanks to its optical image stabilization (OIS) system and the dolly's anti-vibration design. “The OIS’s robust correction capability allowed us to create highly natural video blending live images and AR. The OIS feature is great for AR shooting with a motorized dolly," said Tsutsui.

On the second floor, the AW-UE150W camera’s excellent pan & tilt simplified control when videoing different competitors at varying distances.

The AW-UE150K on a motorized dolly was connected to the relay booth via a 50m fiber-optic cable for a simple, labor-saving setup.

In the Customer’s Own Words

Video that Conveys the Appeal of Bouldering

According to Ryusuke Fujieda, a Sport Climbing official, “We have been looking for ways to better communicate the appeal of competitive climbing through live streaming, which is why we adopted AR in 2019. Our broadcasts have been well received because we have been able to integrate information and video to help people better understand the rules and flow of competition without disrupting the viewing experience. Also, we anticipated people not attending the event this time due to the pandemic, so we could stage the event with special consideration for online viewers. Going forward, we hope to improve the quality of not only our live streams but also our events in ways that encourage more viewers to attend in person.”

Capturing dynamic action without interfering with competitors or spectators

Streamed video also displayed on the venue's LED big board

System configuration

Enriching On-site Commentary

Mr. Tsutsui also noted: “In this year's competition we achieved a number of firsts, including using two 4K Integrated Cameras to deliver superb video quality. Looking ahead, in addition to enhancing our AR technology, we also want to enhance our delivery of on-site commentary for the enjoyment of live spectators, specifically by overcoming the restrictions on broadcasting live commentary via PA systems.”

Mr. Ryusuke Fujieda (right)
Permanent Representative Competition Committee,
Japan Mountaineering & Sport Climbing Association

Mr. Masato Tsutsui (left)
President, Arque Inc.

Note: Current job titles at time of interview

OnlineObservation^® Supports Live Competition Streaming

Arque Inc. is a creative studio that specializes in developing interactive video systems for advertising, events and other fields. Its OnlineObservation^® Sport Climbing video system, which integrates VR for 3D modeling of climbing routes and AR for displaying competition information, has been used to live-stream the Bouldering Japan Cup since 2018.