By Jay Kakade 30 Nov, 2024

Collected at: https://www.techexplorist.com/virtual-reality-open-new-window-into-animal-behaviour/93757/

Virtual and Augmented Reality have come a long way, and they are now used in various fields, including gaming and movies. Now, researchers have found an intriguing way to gain better insights into animal behaviors like course stabilization and obstacle avoidance during navigation.

To view new perspectives on the aerodynamic powers of flying insects, researchers at Flinders University have developed a computer program that sets a virtual reality experience to move through. Since animals are tethered and not moving physically, studying their movements and comprehending responses becomes easier.

Fascinatingly, since the virtual world is computer-generated, it can swiftly be altered, allowing better control over the space and facilitating the identification of visual triggers of behavior.

Using machine learning and computer vision algorithms, we were able to observe the animals and work out what they are doing, whether that is a hoverfly attempting to turn to the left in its flight, or a fiddler crab avoiding a virtual bird flying overhead,” says Dr. Yuri Ogawa.

The software then adapts the visual scenery to match the movements that the animal has made.

One of the most popular ways of developing insect VR is tethered flight arenas. However, these tethered flights don’t include translational motions, crucial to studying flying insects. Insects often perform rotational and translational behaviors, such as forward motion and sideslip.

Therefore, several new VR arenas have been developed to offer unrestricted animal movement, including translational motions. VRs like TrackFly and FreemoVR have been validated in flies and mice, where the visual surrounding is updated based on the animal’s current position.

In this context, Antarium could be considered an important VR framework. In this VR set, both rotational yaw and translational motions are generated in the Unity virtual environment.

fly on VR set
Virtual reality arena overview. (a) The tethered insect (here a female Eristalis tenax hoverfly) placed in the center of three gaming monitors, oriented vertically. The insect is filmed from above using a PS3 camera (Kaushik et al., 2020) with the video fed into DeepLabCut-live (DLC-live) which performs markerless pose estimation in real time (Kane et al., 2020). Unity uses this information to calculate the requisite scene translation and rotation based on the wing beat amplitude and updates the virtual world on monitors with a 165 Hz refresh rate. Hoverfly pictogram by Malin Thyselius. (b) Lateral (left) and dorsal view (right) of the tethered E. tenax illuminated in infrared with two USB lights from the side, and an infrared ring light from above. The PS3 camera is equipped with an infrared pass filter, and a musou black surface is placed below the insect. (c) Example video frame with two points on the anterior edge of each wing stroke and two points along the thorax tracked using DLC-live (Kane et al., 2020). (d) The frequency of the resulting DLC-live data using different spatial (240 × 320 or 480 × 640 pixels) and temporal (10, 60 or 100 Hz) video resolutions. Each semi-transparent data point shows the time between two subsequent frame updates (N = 132, 427, 588, 483, 597).

However, to provide an immersive experience, it is important to reduce the latency. More importantly, there is a lower limit to these delays when using conventional cameras or visual displays.

This has truly been a team effort where every author on the paper has been instrumental in making the VR work. We look forward to using the VR to investigate the mechanisms underlying decision-making in insects,” says Professor Nordström.

Journal Reference

  1. Ogawa, Y., Aoukar, R., Leibbrandt, R., Manger, J. S., Bagheri, Z. M., Turnbull, L., Johnston, C., Kaushik, P. K., Mitchell, J., Hemmi, J. M., & Nordström, K. Combining Unity with machine vision to create low latency, flexible and simple virtual realities. Methods in Ecology and Evolution. DOI: 10.1111/2041-210X.14449

Leave a Reply

Your email address will not be published. Required fields are marked *

0 0 votes
Article Rating
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments