Role UX Designer
Team Brandon Badger, Sam Hu, Shiva Sankar Kannan, Pavan Paravasthu
Timeline 4 months
Team Brandon Badger, Sam Hu, Shiva Sankar Kannan, Pavan Paravasthu
Timeline 4 months
CONTEXT
Digital Audio Workstations, the software that is used for song mixing, are very intimidating to newcomers.
In sound mixing processes, the way sound engineers or music producers communicate is through picturing how the music will be in their head, and transfer the thought using tools on DAWs. This process can be very abstract, and also, the interface of DAWs are complicated.
OBJECTIVE
Discovering intuitive interactions in VR that introduce novice musicians to sound mixing tools and concepts.
Our goal is to demystify the craft of sound mixing by avoiding the complex traditional interfaces and prototyping different visualizations and interactions in VR.
In particular, we tackle the sound mixing part of the music production process:
DIVE DEEP
1. How might we build something like a transition/tutorial to DAWs?
Since DAWs work within a stereo sound setup, we also want to present sound the same way as they will transit back to DAWs, meaning using stereo instead of spacial sound.
2. How might we make sound mixing concepts more understandable in VR?
Concepts including: EQ/reverb/volume/panning/automation
3. How might we present the tools without interacting with sliders and knobs on a panel, but a more interesting and intuitive way?
Novice learn the mixing tools and concepts through visual associations, but need to operate on the complex mixing panel.
4. How might we make the space and the interface/interactions accessible for guests? (this came later in the development phase)
Accessibility has been brought up multiple times with ETC advisers and industry professionals. Not only colors and interfaces, but we need to consider the functionality and content as well.
GOALS
DESIGN CONSTRAINTS
1. Balance between labelings and visuals
1. Balance between labelings and visuals
Should not show all parameters and labels, but still leave some trails on the interface or in the interactions to match DAW sound mixing techniques.
2. Customize a design pipeline
2. Customize a design pipeline
We have the resource of novice and industry professionals to gain insights and get the best out of the interviews and testings.
3. Simple interactions
3. Simple interactions
Interactions should be simple, because the goal is to let guests understand sound mixing concepts.
Importance level: visualization > interaction
Importance level: visualization > interaction
Set up a DESIGN PIPELINE !
MY WORK
1. Create UI
1. Create UI
In charge of logo design and poster design. At the same time, set up design pipeline, mood-board for art theme.
Create instruction board for interactions within each prototype.
2. Lead Design Panels
Research about sound mixing concepts and interactions before discussing with teammates. Sketch storyboards on Zoom whiteboards and Procreate/Paper.
Organize design brainstorm ideas in shared document for efficient communication with development team, artist, and producer.
3. Interview & Analyze Playtest Feedback
Construct interview structure to get valuable insights from industry professionals. Lead affinity diagramming and consolidate key takeaways.
Prepare post-surveys for prototype playtests. Clarify goals and pain points with playtesters. Bring up questions aside from survey during and after the playtest. Organize playtest data and takeaways into shared document.4. Documentation
Finalize each prototype development phase by documenting all design reasonings, findings, and future perspectives. Refine project website and media.
THE PROCESS
Our approach to representing sound mixing is two-fold: tools and concepts. After coming up with visualizations and interactions for mixing tools, we’ll engage with more abstract and conceptual guidelines within the craft of audio mixing. The team will build prototypes inspired by insights and feedback from both industry professionals and novice musicians.
Our approach to representing sound mixing is two-fold: tools and concepts. After coming up with visualizations and interactions for mixing tools, we’ll engage with more abstract and conceptual guidelines within the craft of audio mixing. The team will build prototypes inspired by insights and feedback from both industry professionals and novice musicians.
Sound designer pain points analyzed with qualitative research:
[ click the images to zoom in ⇩ ]
Storyboard for prototype interactions
Playtests and workshop
Workshop with Arvi School of Music
Demo during workshop
Playtest with Rebecca Wolfinger
THE DESIGN & FINDINGS HIGHLIGHTS
Reverb
From the interviews with professionals and novice musicians, we understand that real world association bridges the gap between parameters and laymen nouns. Therefore we aim to visualize the laymen nouns such as ‘hallway’ and ‘church’ reverb.
Automation + Volume + Panning
Automating volume and panning together was an innovative experiment spotting the limitations and inconveniences while mixing in DAWs. Instead of horizontal and lifeless automation lines, we leveraged the VR space and matched the project theme by designing the lines into dropping constellations.
EQ
Parameters in EQ layout were carefully chosen to let novice understand clashing concepts along with reifying some commonly used adjectives.
Reverb
From the interviews with professionals and novice musicians, we understand that real world association bridges the gap between parameters and laymen nouns. Therefore we aim to visualize the laymen nouns such as ‘hallway’ and ‘church’ reverb.
Automation + Volume + Panning
Automating volume and panning together was an innovative experiment spotting the limitations and inconveniences while mixing in DAWs. Instead of horizontal and lifeless automation lines, we leveraged the VR space and matched the project theme by designing the lines into dropping constellations.
EQ
Parameters in EQ layout were carefully chosen to let novice understand clashing concepts along with reifying some commonly used adjectives.
Through playtests, we proved the efficiency of interacting with sound tracks and visualizing concepts in VR space. Also foresee the potential of understanding preliminary sound mixing concepts with our prototypes.
Start
Basic functionalities track field
Reverb
EQ in audio cube
Automation
MORE DETAILS
For more in depth development process and documentation, visit Mixthesia website.
Impact
This project exceeded our expectations. The audio community seemed to love our work. We held a workshop with India Arvi School of Music, with 25 novice musicians attended it virtually.
Also the professor at CMU Music School wanted to incorporate our prototypes into his curriculum. I was glad to be a part of this project with my passionate teammates!
*TIPS for VR playtesting remotely
1. Share screen to observe behaviors and resolve confusions.
2. Prepare instructions in virtual space about buttons and interactions.
Instructions for onboarding playtesters
Mixthesia poster