Projects

Background. Cell lineage is the development of a (plant) embryo from a single ovum cell. Based on a confocal microscopy dataset, biologists traditionally constructed the cell lineage manually, starting from this observation and reasoning backward in time to establish inheritance.

Challenges.

1. Cells in the embryo are densely packed together.
2. The traditional lineage construction process is tedious and time-consuming.

Solutions. We first explored selection interaction techniques in 3D environments. Then we developed a specialized tool for biologists to construct cell lineages. This tool features a hierarchical tree built using both top-down and bottom-up approaches: the top-down approach follows the original temporal development, while the bottom-up approach aligns with the traditional lineage-building method. We also made use of machine learning (ML) models trained on a database of manually established cell lineages to assist biologists in cell assignments.

Most biologists, however, are not familiar with ML, nor is it clear to them which model best predicts the embryo’s development. We thus enhanced the system to support biologists in exploring and comparing ML models, checking model predictions, detecting possible model mistakes, and deciding on the most likely embryo development.

We examine visual representations of data that make use of combinations of both 2D and 3D data mappings. Combining 2D and 3D representations is a common technique that allows viewers to understand multiple facets of the data with which they are interacting. While 3D representations focus on the spatial character of the data or the dedicated 3D data mapping, 2D representations often show abstract data properties and take advantage of the unique benefits of mapping to a plane. Many systems have used unique combinations of both types of data mappings effectively, yet there are no systematic reviews of the methods for linking 2D and 3D representations.

We systematically survey the relationships between 2D and 3D visual representations in major visualization publications—IEEE VIS, IEEE TVCG, and EuroVis—from 2012 to 2022. We closely examined 105 papers where 2D and 3D representations are connected visually, interactively, or through animation. These approaches are designed based on their visual environment, the relationships between their visual representations, and their possible layouts. Through our analysis, we introduce a design space and provide design guidelines for effectively linking 2D and 3D visual representations.

The development of cancer is difficult to express on a simple and intuitive level due to its complexity. Since cancer is so widespread, raising public awareness about its mechanisms can help those affected cope with its realities, as well as inspire others to make lifestyle adjustments and screen for the disease. Unfortunately, studies have shown that cancer literature is too technical for the general public to understand.

We found that musification, the process of turning data into music, remains an unexplored avenue for conveying this information. We explore the pedagogical effectiveness of musification through an algorithm that manipulates a piece of music in a manner analogous to the development of cancer. We conducted two lab studies and found that our approach is marginally more effective at promoting cancer literacy when accompanied by a text-based article than text-based articles alone.

The environmental impact of carbon dioxide emissions arising from travel has emerged as a significant concern. Notably, the recent frequent realignments within the Power Five conferences have led to substantial fluctuations in the annual carbon footprint of football games played between member teams.

In this study, we collected and analysed conference data spanning the previous decade, as well as forthcoming schedules (wherever available), to evaluate shifts in this carbon emissions footprint. Our findings underscore the potential environmental ramifications of the impending realignment, commencing in 2024, revealing the possibility of an almost twofold increase in carbon emissions.

Mr. Noter is a tangible interactive device that produces music based on the items in front of it.

As two major perceptual channels, auditory and visual sensation are similar in many ways and can evoke similar feelings. If information from these two channels can be mapped, synchronous perception of hearing and vision can be realized. Previous music interaction designs combining auditory and visual sensation generally focused on music visualization. From the perspective of interaction, however, making sound from vision can help users understand objects from another view. Moreover, as a kind of natural interaction, tangible interaction lets users express their actions by moving objects. Tactile music interaction combining audition and vision lets users experience the synchronization of two senses while operating the product freely, and enjoy creating music anytime and anywhere.

In this project, I designed a new music interaction device that maps the visual characteristics of existing products—color, shape, size, and position—to notes that evoke the same kind of feeling through timbre, tone, sound length, and playback order. Existing items can thus produce their own sounds without changing their appearance or function. The design can inspire professionals, and also lets children or adults who know nothing about music play music—together, since more than one user can play at once.