FlyoVeR: 2D virtual reality for head-fixed walking flies

This page provides technical details and additional resources for the 2D virtual reality for head-fixed walking flies described in Haberkern et al. (2018) [1].

Please note that this website is still under construction. We are working on adding more information shortly.

If you have comments or feedback, please contact haberkern[at]janelia.hhmi.org.

[1] https://www.biorxiv.org/content/10.1101/462028v1

Additional information on fixation behavior in VR

In Haberkern et al. (2019) we compare fixation behavior of head-fixed walking flies in 1D and 2D virtual environments. Here we provide additional data on fixation  behavior across different wild type genotypes. Further, we show how experimental conditions and preparation of flies before the experiment can influence the behavioral phenotype. We tested three wild type genotypes:

• DL: Wild type strain from the Dickinson laboratory
• WTB: Isogenic wild type Berlin strain
• WTB hybrid: Flies generated by crossing WTB virgins with males from an enhancerless “empty GAL4” line 

We chose the WTB and DL strains because they have been used in numerous previous publications on various navigational behaviors. WTB hybrid flies were chosen to approximate the genotypes used in optogenetic activation experiments, where an effector line with WTB background is crossed to GAL4 driver lines with variable genetic backgrounds.

Extended data on fixation behavior

Fixation behavior across sexes and genotypes (A-E) Comparison of fixation behavior in WTB, WTB hybrid and DL flies measured at high room temperature (30 ?C) and with wings cut. (A) Fixation plots of unimodal fixation for female flies. These "fixation plots” illustrate the angular location of the fixation peak and the fixation strength based on von Mises fits. The location parameter (µ) is plotted on the circumferential axis and the shape parameter (k) on the radial axis. Markers (empty or filled circle) indicate categorization of trial based on fit. The grey arrow points toward the frontal position in the fly’s field of view. Grey shading marks the part of the field of view that is not covered by the panoramic screen. Visualization as in Fig. 3 F of Haberkern et al (2019). (B) Fixation plots of bimodal for female flies. Note different axis scale in bright and dark scenes. Visualization as in Fig. 3 G of Haberkern et al (2019). (C) Fraction of male and female flies that walked for at least 20 % of the trial time across the four trials. (D, E) Fraction of flies showing unimodal (D) and bimodal (E) fixation. (F-H) Same as (C-E), but for different experimental protocols: Flies were either wing cut and measured at high (30 ?C) or low (25 ?C) room temperature or wing glued and measured at high (30 ?C) room temperature.

AdditionalMaterialText.pdf

Technical details about FlyoVeR

The FlyoVeR application is a modified version of the Jovian/MouseoVeR software [1], hence the name. Like MouseoVeR, FlyoVeR was built from several powerful third-party, free and open source software components. The 3D graphics, in-memory scene model, and callback-oriented rendering loop in FlyoVeR were implemented in the C++ programming language using the cross-platform open source OpenSceneGraph library (version 3.4). The graphical user interface was also implemented using C++ based on the cross-platform open source Qt toolkit (version 4.8). The cross-platform build and packaging system was implemented with CMake (version 3.4).

FlyoVeR differs from the parent Jovian/MouseoVeR software in the following respects:

• It incorporates a palette of location-dependent behavioral reinforcement capabilities. This allows the FlyoVeR application itself to determine the final reinforcement parameters, thus simplifying the responsibilities of the software running on the programmable microcontroller reinforcement apparatus.
• In FlyoVeR, the treadmill motion is directly represented as translational (horizontal) and rotational (heading) motion of the animal in the virtual world (with one exception: the animal motion is not permitted to penetrate vertical obstacles in the virtual scene). By contrast, in the parent Jovian/MouseoVeR software system, the treadmill motion is incorporated into a full physical mechanistic dynamic simulation. The simpler motion system used in FlyoVeR was easier to debug and sufficient for our needs.
• It includes an option to display a radially symmetric visual fog effect, as opposed to an axis-aligned fog effect. With axis-aligned fog, equidistant objects can be hidden or visible through the fog depending on their position within the FOV.
• It is distributed as a self-installing executable program.
• FlyoVeR uses a human-readable xml configuration file format, enabling easier recording of experimental details, and also enabling off-line adjustment of experimental parameters.
• It records various experimental details in the log file header, to help keep track of the parameters were used for a particular experiment.

References:

[1]  Cohen, J.D., Bolstad, M., and Lee, A.K. (2017). Experience-dependent shaping of hippocampal CA1 intracellular activity in novel and familiar environments. eLife

Virtual world construction

Naming conventions used in FlyoVeR

Custom Collada (version 2.4) format scene files used in our experiments were created with the free 3D modeling program Blender (version 2.73). The Collada file format is a standardized XML format used to describe 3D graphics. Each object within the 3D scene had a unique name and a set of properties. Properties such as the color and texture were specified in Blender as “materials” that were then assigned to the respective object. Three other properties were communicated to FlyoVeR as part of the object’s name string:

• Visibility: An object could be marked as invisible by placing underscores at the beginning and end of an object name, e.g. “_Cone_”.
• Penetrability: An underscore followed by “p” as in “Cone01_p” marked an object as “physical”, i.e. not penetrable by the fly.
• Concavity: An underscore followed by “c” as in “Cone01_c” marked an object as “concave”. This property was only relevant for objects that were hollow and impenetrable, as it defined whether the fly was excluded from the inside or the outside. For example, to prevent a fly from entering the inside of a small hollow cylinder used as a virtual landmark, the landmark model had to be marked as “convex”. In contrast, to prevent a fly from leaving the inside of a large hollow cylinder that marked the arena wall, the arena model had to be marked as “concave”.
• Scene item names containing the letter "z", proceeded by an underscore, and followed by an optional number (e.g. "item_z2"), represent locations where positive or negative reinforcement might be delivered. The detailed effects of each of these reinforcement zones can be specified in the FlyoVeR graphical user interface.

The flags for visibility, penetrability and concavity could be combined in the name of a single object. By default, objects were “visible”, “penetrable” and “convex”. When loading a new scene file, FlyoVeR parsed the names of each object and assigned corresponding properties to the objects composing the virtual scene. Invisible objects could be used to mark special areas within the scene, such as the starting point of each trial, and control the delivery of other stimuli based on the fly’s position inside the VR. Two items, a small sphere and a plane above it, representing the virtual animal size and viewpoint and the initial animal location, are required in every scene and have to be names "_camera_block_pm_" and "_start_", respectively.

Exporting blender scene as collada file

The finished scene can be exported in .dae format. In Blender (Version 2.73) chose File -> Export -> Collada (.dae). A new window opens, where the user can select a file name and directory for the to be exported collada file. If the scene contains materials with mapped image textures, make sure to select "Include UV Textures" and "Include Material Textures" in the "Texture Options" panels on the lower left.

Virtual worlds used in the paper

Under construction

Building FlyoVeR

FlyoVeR can be quickly installed using an installer. [Link to github directory will be posted shortly.] 

How to run FlyoVeR

[Links to FlyoVeR userinterface elements currently not functional. Info on GUI will follow.]

1. Launch RemoteDataServer
2. Launch FlyOver version of your choosing.
3. Click the Initialize button in the "Setup" tab [You should see your VR display turn black now]
4. In the top left corner of the GUI window chose File->Open->[browse to your chosen .dae scene file]
5. Then choose File -> Open -> Settings from XML to load configurations that were previously saved to file.
6. Go through tabs and manually configure various parameters not automatically set by the XML settings file.
   1. In the “Setup” tab:
      • Set “Camera” from 1 to 2. Then, under “Warping*” select then deselect the “Enable” checkbox. 
      • Select “Enable Timed Trial” and set length to time of your choosing.
      • If running a 1D condition, select the “Yaw Only” box.
   2. In the “Configuration” tab:
      • On the right side adjust parameters that control how the virtual scene is displayed. Note that these vary with the geometry of the display on which the image generated by FlyOver is projected.
        • Set “Field of View” to 176.
        • Under “Frame Indicator”, set “Size” to 30, “Screen” to 1, and the horizontal slider bar from 1200 to 0.
        • Note that changing some of these parameters may not change what is displayed on the screen right away, but only after a "refresh" is triggered by e.g. enabling frame packing (see point c).
      • On the left side you can adjust the file export options:
        • Under “Output”, select “File Export?” and “Treadmill output”.  Then set “Style” to “Time Stamp”.
        • Then, next to “File Name” click on the “…” button to chose name and directory where the raw data file will be saved.
   3. In the “Display” tab, select Enable Frame Packing. Note that toggling this button can lead to FlyOver crashing.
7. Go back to the “Setup” tab and click the Connect button to start the trial.