SoniCube
ViewCube has also implemented a sonification, module, named SoniCube, which provides an open, comprehensive, and general-purpose multimodal tool for IFS analysis, supporting the development of future sonification techniques focused on specific potential features within datacubes.
The aim of the SoniCube interface is to offer a diverse “palette of sonifications”, akin to the range of color palettes available for visual 2D maps in ViewCube. This variety of sonification options will allow users to extract or enhance different data characteristics by selecting specific sonification methods, much like how a color palette reveals visual details.
A comprehensive description of the sonification module, along with its evaluation, is available in the following paper:
Adrián García Riber, Rubén García-Benito, Francisco Serradilla. Interactive multimodal integral field spectroscopy, RAS Techniques and Instruments, Volume 3, Issue 1, January 2024, Pages 748-758 https://doi.org/10.1093/rasti/rzae049
Installing Dependencies
Two libraries are required to make the SoniCube module functional.
1. Csound: An open-source sound and music computing system for audio synthesis and signal processing. You can download the version for your operating system from the official webpage:
https://csound.com/download.html
Csound is available for all major operating systems.
Python Libraries: Ensure the following Python libraries are installed (if not already):
hashlib
tensorflow
python-osc
ctcsound
librosa
These libraries can be installed using pip.
Note
Old Autoencoders: If you are using old autoencoders with TensorFlow 2.16 or newer,
you must install tf-keras:
pip install tf-keras
Without this package, loading old autoencoders will fail.
CALIFA sonicubes
In this version, we introduce the first sonification method integrated into SoniCube’s toolset. This initial implementation is intended to replicate the functionality of the visual ViewCube, offering a quick qualitative overview of the datacube.
SoniCube’s sonification utilizes an autoencoder architecture to accurately transform the spectral information of CALIFA survey COMBO datacubes into sound.
Configuration
To access the sonification, users need an original datacube from CALIFA Data Release 3 (in COMBO grating) along with its corresponding autoencoder-generated sonification files.
Download the CALIFA DR3 COMBO datacubes from:
https: //califa.caha.es/FTP-PUB/reduced/COMB/reduced_v2.2/
and the corresponding sonicubes from:
https://zenodo.org/records/10570065
Create a folder (e.g. sonicube) at any location of your choice. Inside this folder,
deploy the contents from the sonicube folder of the
ViewCube GitHub repository.
The directory should contain three subfolders for sound configuration files, along
with a data folder for the CALIFA sonicubes.
For example, if your folder is named sonicube, it should contain the following structure:
autoencoder/ binaural/ csound/ data/
The subfolders autoencoder, binaural, and csound are obtained from
the sonicube directory in the ViewCube GitHub repository. The data folder,
on the other hand, should contain the CALIFA sonicubes, which can be downloaded
from the Zenodo repository.
For instance, your data folder might look like:
...
NGC5732.COMB.rscube.fits/
NGC5784.COMB.rscube.fits/
NGC5794.COMB.rscube.fits/
NGC5797.COMB.rscube.fits/
...
and the complete contents within each directory might be:
...
NGC5732.COMB.rscube.fits:
NGC5732.COMB.rscube.fits_Reference.npy
NGC5732.COMB.rscube.fits_Weights.data-00000-of-00001
NGC5732.COMB.rscube.fits_Weights.index
NGC5732.COMB.rscube.fits_learning_rate.png
checkpoint
NGC5784.COMB.rscube.fits:
NGC5784.COMB.rscube.fits_Reference.npy
NGC5784.COMB.rscube.fits_Weights.data-00000-of-00001
NGC5784.COMB.rscube.fits_Weights.index
NGC5784.COMB.rscube.fits_learning_rate.png
checkpoint
NGC5794.COMB.rscube.fits:
NGC5794.COMB.rscube.fits_Reference.npy
NGC5794.COMB.rscube.fits_Weights.data-00000-of-00001
NGC5794.COMB.rscube.fits_Weights.index
NGC5794.COMB.rscube.fits_learning_rate.png
checkpoint
NGC5797.COMB.rscube.fits:
NGC5797.COMB.rscube.fits_Reference.npy
NGC5797.COMB.rscube.fits_Weights.data-00000-of-00001
NGC5797.COMB.rscube.fits_Weights.index
NGC5797.COMB.rscube.fits_learning_rate.png
checkpoint
...
You do not need to download all of the sonicubes—just those that are relevant to your project.
Now in the ViewCube configuration file .viewcuberc uncomment the keyword dsoni and
write the absolute path of the sonicube directoy:
dsoni : "/my/absolute/path/to/sonicube/"
Sounding [Data/Soni]Cubes
Once you have installed the dependencies and configured ViewCube, you need at least one original CALIFA datacube and its corresponding sonicube. The datacube can be located in any directory, while the sonicube should be placed as explained in the previous configuration section.
Open the datacube in the standard way using ViewCube.
Ensure you are wearing your headphones correctly (right headphone on the right ear and left on the left). Adjust the volume to a lower or medium setting to avoid high volumes at the beginning. You can increase the volume later, once you are familiar with the sound of that particular galaxy.
To activate sonification mode, press the h key. The first time you enable this mode,
it may take 5-8 seconds to load the necessary libraries (tensorflow is known to take
some time to import).
Explore as usual in ViewCube by moving the mouse over the spaxel window. If the mouse moves outside the axis or window, no sound will be produced.
In standard mode, the volume corresponds to the median intensity of the spectrum. Dim regions of the galaxy will have a lower volume, and the sky will be practically silent. In contrast, high-intensity HII regions or the galaxy’s center will produce louder sounds.
If you want to deactivate the volume intensity linkage and have a uniform volume for all
spaxels, press the j key. This is useful if there is an interesting region with lower
flux that you want to listen to carefully but its volume is relatively low.
Press the same key again to restore intensity-sensitive volume.