Transcription
Cleverdon:
Hi. This is David from 360 Immersive. Today, we have the pleasure of talking to David Reyes from Immersive Experience Studios.
Now David is all about audio for virtual reality. Now, you know, audio is something that is, sometimes I view it as the poor stepchild. Extremely important, but a lot of producers don’t pay enough attention to that audio experience, especially with VR.
Tell us a little bit about your passions, maybe some of your background, and why that is something we should as producers really pay attention to.
Reyes:
Yep. I’ve been composing, working in music since the mid-90s. I’ve been playing music since I was a kid, worked in a number of technology roles, DTS, JBL, you know, been in and around pro audio for quite a while. I’ve seen it evolve. I think we’re in some really, really exciting times right now with, you know, VR and with the Spatial Audio solutions that we have and the horsepower.
You know, audio is, if it’s 50 percent of the cinematic experience as George Lucas, you know, leads us to believe, then it’s easily 60 to 70 percent of the VR, you know, mobile game experience.
And so for that reason, I believe that audio is something that as producers and content creators, you know, we ought to be thinking about from the very, very beginning.
Cleverdon:
Well, if you think about it, you know, people talk about virtual-reality movies, and yet it’s all one big shot, and the only way that we can help direct or guide the audience, because we don’t have the luxury of that tight shot right here, is using audio, because we’re framed up over here, we’re looking around and all of a sudden we hear something over here, and that directs our view to where the director of the experience wants you to go.
Reyes:
That’s right. Yep. So, whether it’s 360 video, Dave, as you indicated, or a VR game, you know, an AR experience, audio is yet one more tool that content creators can use to queue, to direct, you know, to conform their players, right. The folks who are undergoing this experience, because it is something that is totally 360.
Now remember, your ears are the only truly 360 sense that you have.
Cleverdon:
So, explain that, because I think most people are going to go what? I mean I can look around.
Reyes:
Yep. Yep. So, you’re, you know, there’s – one of the great things about, you know, being in this space right now as I mentioned, you know, Spatial Audio, which is really the culmination of kind of three different areas, right. You know, Ambisonics, psychoacoustics, and a wave field synthesis.
You know, these are tools and techniques that have been with us since the 1950s, the 1970s. There have been a lot of studies done on how we hear and how we perceive sound. It’s actually a very complicated system, but we know some basic rules of thumb, and I share these in my material, in my presentation.
You know, for example, things like sounds above 1500 Hz are far easier for the human auditory system to localize than sounds that are let’s say 200 Hz or 800 Hz. And I think we’ve had that experience, right. You’re sitting in traffic and somebody has, you know, really, really deep base in their car and you’re listening to the boom, boom, boom, but you’re not real sure which car that’s coming from.
Cleverdon:
That’s right.
Reyes:
Right? And the reason is because it’s really hard for your ears to localize low-frequency sounds. And that really just has to do with the distance between our ears and the wavelength of those frequencies. It’s just physics. It’s just geometry.
Cleverdon:
So, let’s take it back to the world of the average – let’s say somebody is producing a game. How can Spatial Audio and Ambisonics audio help them produce a better game experience for the user?
Reyes: Yep. So, that’s a great question. I have some material. I’ll provide the link at the end of this presentation that can really help producers get started.
Cleverdon:
Actually, it’s one of those look down below or look someplace. Find that link.
Reyes:
Right. Yep. And so, really is a matter of a couple of things. You know, it is first of all, what do you as a producer, as a creator, want the audio to do for you? What’s its function? Right? Because it has a role. It has a function in this experience. That’s the first thing I think.
And the second is, you know, what’s your budget. Are you going to, you know, use free samples? Are you going to use samples from the Internet? Are you going to hire a composer like me or a sound designer? You know, how big or how large can you make this? How much do you have to squeeze? You can still create a great product, you know, within the constraints of your budget, right.
And then I think the third one is the solution. You know, what solution do you want to go with, you know, to deliver the value. And you know, all the solutions that I’ve looked at, there were 13 of them in my presentation, there were a couple of others that are less well-known, maybe, don’t have such a great user community. So, I’d say maybe we’re talking about 15 solutions. I cover 13.
You know, they all have their pros and cons. There is no perfect solution. What you need to know as a content developer is, you know, which trade-offs am I willing to live with to get my product out there, to get in front of, you know, the music community, to get in front of players and my audience.
Cleverdon:
So, let’s say somebody is producing something that’s near and dear to our hearts, which is training content. How does Ambisonics or Spatial Audio help us?
Reyes:
Oh. Yes. Great question. So, I think it helps in a few different ways. You know, one is creating a sense of presence, which we hear a lot about in VR videos. And for me, presence from an audio perspective means the ambience in the room, you know, any sound design that queues the listener in terms of, you know, place, setting, situation, you know, what’s going on, right.
And then the other is any audio queues, voiceover, or music that you may want to add to again, create that mood, create that ambience for the viewer, for the listener.
Cleverdon:
Let’s talk about something more. For instance, in a training scenario where we’re definitely talking about situational awareness, and it can be workplace safety such as maybe overhead crane or forklift training or law enforcement where you have an active shooter situation.
Being able to identify audio that, in the real world would pull you over here, your attention, and have that same experience in a training piece. So, that I’m looking here and I’m looking here and even maybe directly behind me, that to us is kind of that icing on the cake when it comes to our real meaningful experience that has value and increases retention. And that’s exactly what you’re talking about?
Reyes:
Yeah. Exactly. And depending upon, again your application, right, the role of audio, what your ultimate desired user experience is, right, in this particular case, you know, is it just a straight, you know, documentary? Is it more of a dramatized piece? You know, if it’s more of a dramatized piece, you can certainly use audio to create that mood, to create that.
And we see that a lot with, you know, reality shows, you know, things like that on TV, right. Those would be 2-D examples. You know, we could certainly extend that to a 3-D, to the VR, AR domain.
Cleverdon:
So, if somebody wanted to step into the world of Spatial Audio or Ambisonic audio, obviously they could contact somebody like yourself. They could also – the latest creative suite from Adobe, they support to a limited extent Ambisonic audio, but how else could they do it? I mean do you have to buy some real specialized tools or how’s it done?
Reyes:
Yep. It depends on how much of the content you want to create yourself and how much you want to download or purchase.
So, for example, if you want to create some recordings, some field recordings, let’s say for training video, or let’s say to create some sort of a mood or an ambience, there are plenty of microphone solutions. I know of about eight. All different price ranges that you could use to create.
So, if you want to do that type of recording work, you need to buy one of those microphones and a laptop. They’re fairly easy to use. You may need some sort of interface just to give you a bit more flexibility in terms of your audio sources going into the laptop, you know.
So, depending upon the cost of the mic, you know, the audio interface is not that expensive. You’ll need some headphones, monitoring. So, that’s if you want to go and do some field recordings.
If you just want to get samples from the Internet, you know, download music, that sort of thing, you know, ultimately I think what really dictates the tool is the application obviously. So, if you’re going for mobile, and let’s say if you’re all about mobile and you really, really want as Spatial Audio solution that’s highly optimized for mobile, you know, Google Residence Audio is a great solution, okay. That would be a headphone only solution.
If your application is let’s say a training video where, you know, you’ve got a room full of people and they’re all experiencing this training situation as a group, as a team, you’ll need speaker output some of the solutions that I’ve reviewed support what are called higher-order Ambisonics all the way up to 24 speakers. But that may not be appropriate for mobile, like. That may be more of a PC-based solution, right.
So, it really depends on I think as he said, you know, the function that you want audio to perform, you know, what is your ultimate rendering situation? Is it over headphones, binaural listening. Is it over speakers, some sort of a, you know, higher order Ambisonics type situation, right.
And then, you know, your budget and your, you know, time I guess investment on, you know, how much of this audio content you want to create yourself. Do you want to hire out, you know, composers, sound designers, or do you want to try to download from the Internet.
You know, one thing our caution, and I see this a lot, you know, producers who maybe don’t have a lot of audio background is trying to marry, you know, prerecorded audio to download from the Internet, you know, sometimes can be a lot of work. You can wind up dicing and slicing to the point where the audio just doesn’t really fit or doesn’t really hang together.
Sometimes it’s just easier to hire someone like me, you know, or a composer or sound designer, you know, who can take your vision, right, and really give you something that’s customized for your application.
Cleverdon:
So, the bottom line is that paying attention to audio, whether it’s a computer-based project like a game or whether it’s a video-based project, your first 360 movie or training or entertainment, is all important.
Reyes: It is. It’s really important to think about this from the very beginning, just like anything else, only because the content drives the audio, the audio drives the content, and it goes around and around and around, you know, until you finally decide that you’re done.
So, it’s really, really important to be thinking about this upfront. It’s also important I believe, whichever technology you believe will work for you, to do some tests, to do some proof of concepts, some prototypes, just to really test the technology before you invest a lot of time and energy.
Again, every one of the solutions is different. They have their pros and cons. No solution is perfect. Some are free, some you have to pay for, but I think you want to do some tests up front to convince yourself that this is really a solution for you before you spend, you know, six weeks, six months, a year on a solution and then realize that you need to back up and rethink the solution and rip out a bunch of code or redo a bunch of content, right. That’s a very, very expensive lesson to learn.
Cleverdon:
Absolutely. So, we will have David’s contact information and some actual information in the links below, and for today, David, thank you.
Reyes:
My pleasure.
Cleverdon:
And remember that virtual reality is something that, it’s not just about the reality of the visuals, but it is also about the reality of the audio.
So, David, we so appreciate you coming in and talking to us today.
This is David from 360 Immersive, and we’ll see you next time.
End of Presentation
