Our panelists:
- Amy LaMeyer - Managing Partner of the WXR Fund
- Christina Heller - CEO of Metastage
- Tom Impallomeni - CEO and Co-Founder of Tribe XR
- Roman Rappak - Co-Founder of Overview Ark
- David Shiyang Liu - Creative Director of AFVR.co
The music industry has already seen incredible growth and transformation driven by advances in technology. With the advent of 5G, we could be looking at VR/AR concerts, with immersive experiences increasingly more realistic and realtime. But even without VR/AR, performance collaborations in realtime could become strikingly like the “real thing”, and that’s only the beginning. Join us as we explore the future of music in a 5G world with Verizon 5G Labs.
Verizon 5G Labs:
Verizon's 5G Labs works with startups, academia and enterprise teams to build a 5G-powered world. We work on 5G trials, hackathons, industry partnerships, prototyping challenges and more.
Transcript:
Miriam Tinberg 0:04
Okay, can everyone hear me? Are we—? I think we are getting ready to begin. Okay. Hi everyone! Welcome to the 5G Music and Emerging Fan Experiences event. My name is Miriam Tinberg, and I am on the Los Angeles— the Verizon LA 5G Labs partnerships team. So before we begin, I just want to give a few introductions. This event is put on in a collaborative effort by Alley, WXR, and Verizon, so I want to give a few short intros. So Alley is a strong partner of Verizon, they have been a partner with us for a few years. They are full of innovators and founders and startups and they create a whole slate of programming, including events like this one. And WXR Fund, we have been putting on a few events with them over the last few months, and, I pulled this from their website, and I really love it. "The WXR Fund invests in two of the greatest opportunities of our time, the next wave of computing and female entrepreneurs." And a little bit about Verizon 5G and the ecosystem partnerships team— we sit under the tech and product development org within Verizon. So we are building co-creation opportunities spanning industries, horizontals, verticals, education, all spheres, and we're showing what's possible with 5G. So we are exploring use cases and experiences that will be supercharged and disrupted by 5G. We have labs all across the country and happy to announce we have a lab that opened pretty recently in London. And I— before we begin, I want to give a quick preview of some of the cool things that the Los Angeles 5G Lab is doing. It's very relevant to today's presentation and it'll be a sneak preview since the lab is not yet open and construction is really just beginning. So I want to share my screen and give you a little preview of that. Okay, I hope everyone can see this. So we're working with renders right now for the most part, but I want to just tell you a couple cool things that the Los Angeles 5G Lab is doing. You'll see here, this is the full space. It's a 28,000 square-foot lab. I believe it's the largest Verizon 5G Lab and we're definitely taking advantage of that. We have a couple cool pillars that I want to mention. I bring this up A) to give you a sneak preview. It's relevant to today's discussion about music but also we are always interested in ecosystem partners and hearing what you're doing, where the intersections are. So if any of this resonates with you, please do reach out to us. If you're in the Los Angeles area, we are always looking to figure out ways to connect. So the most— the biggest pillar that we have at the Verizon Los Angeles 5G Lab is gaming. And this is a big thing that Verizon is getting into now. In— we announced a couple months ago this super cool first of its kind partnership with an Esports team for Verizon. So Dignitas is sitting in our Los Angeles lab, you'll see these are pictures from the— these are not renders, these are pictures from the actual lab. This is their West Coast HQ. So we have three different teams who are practicing in here, obviously, with COVID we've had some slight adjustments, but you'll see here there's this cool tunnel, and this leads into their whole space. So we have two 5 v. 5, (five versus five) stages where, you know, post-COVID, we're planning to get people in and do actual in-person competitions and tournaments. We have a lounge, we're really setting them up for success and figuring out all of the various ways and avenues that 5G can really disrupt the gaming industry and where Dignitas and Verizon can be the pioneers in that regard. And then the next up is music and entertainment. And this is particularly relevant for today. You'll see these are renders but we're planning to have a stage here and we'll have a variety of tech and we're always interested to hear from you all what else do you think we should be adding in here, but we're planning to do live concerts powered by 5G. We're planning to do content production, creation, distribution, editing, the whole slate. You'll see here we have some seats, planning to figure out really cool ways to engage with various 5G devices before, during, and after shows. And so we'll— you'll hear more about that today from the various presentations and discussions that happen today, but very excited to be leading the charge on that. And then finally, and this is obviously the most up in the air is the co-working. This is a key pillar of the Verizon 5G Labs. Frankly, it's unclear what will happen with COVID. But we are hopeful that we will still be able to get partnerships into the lab and really utilizing the latest kind of co-working and collaborative technology to actually create, ideate, tinker, explore in our space. Get back to blue jeans, hold on one second. Okay, now I'll stop sharing. Okay. And so finally, I just want to introduce the event today because I know we're eager to get started. Some of you might have gone to the event a couple weeks ago that was with the same title. And you'll notice that some of the presenters are the same, and the title is the same. We tested it out in Mozilla Hubs, and we had such a great reception for the actual discussion, and a lot of great conversation and insights came out of it, but there were some technical difficulties. So we wanted to make sure we were able to redo that event here today. And we have a few different panelists, so I think the discussion will continue to evolve, and we'll hear a lot of different diverse voices. But I'm very excited for us to get started today. A couple things and notes before we begin, a couple things we're very excited about first, I imagine you've already noticed but we have an ASL interpreter on the call. We are very excited, this is an initiative that Alley, in particular, suggested, and of course, this is something that we are going to be mindful of moving forward. And we're really excited to be able to test this out today. So please let us know before or after what you think about, you know, if there are issues, technological issues with the ASL interpreter, if there are other things like that, that you'd like to see. I'm very excited for that. And then the second note is if you have a question, please put it in the Q&A. We will have a discussion in the second half of the event and I will be monitoring the Q&A to make sure that all of the questions are seen. If there are some that you like, I believe BlueJeans has a cool function where you can like, "like" you do on Instagram, you can like comments and give them— bring them higher up on the list. So if there's some that really resonate with you, please feel free to like those so we can make sure that these— the top questions are getting heard. Okay, so without further ado, I want to bring on Amy LaMeyer who is from WXR, and she's going to introduce the panelists and get the show on the road.
Amy LaMeyer 6:54
Thanks, Miriam. You guys at Verizon 5G Labs and Alley have been such great partners and have done some fantastic programming during this time. But I am really, really, really excited to get to that 5G Labs in-person someday and see a show. Like, that will be— that will be epic. I am counting the days. Great! So as Miriam mentioned, we're having a conversation today about the future of music and 5G and immersive fan experiences. So I'm going to run through our panelists real quick and we've got some videos to show you what they're working on. First, we have Christina Heller. She's the founder and CEO of Metastage. Metastage is the premier volumetric and holographic studio in Los Angeles. So let's take a look at what they're doing there in video if you can cue that up Cynthia.
Christina Heller 7:57
Immersive media takes digital content and puts it in a three-dimensional space. So it's taking what you like about your digital world and breaking it completely free from the frame. It's really exciting. In the past few years, we've spent a lot of time figuring out how to make content instead of just getting to make it. Every time we would try to do something, it was like a science project. And so creators were constantly having to think about the tech instead of the content. That is the mission of Metastage to make immersive content creation easy and reliable. And we're starting this journey with the Microsoft mixed reality capture studios.
Ben Grossmann 8:31
Volumetric video is the idea of capturing a person from an array of cameras around them, and then creating a 3D model that's animated with that character that captures every nuance of their performance, but it can be seen and walked around and moved in on from any different direction.
Skylar Sweetman 8:46
From there, we've processed the data to create a high fidelity, three-dimensional video compressed into a streamable file.
Christina Heller 8:54
The processing is key because getting a clean capture is a feat but a lot of people can do that. Processing those files into a high-quality asset that can give somebody a face-to face-experience, but also use that same asset to stream on a mobile device, that is what makes the Microsoft System the best in the market.
Ben Grossmann 9:11
So when we were creating the Blade Runner 2049 Experience, we knew we had to have legitimate, authentic human performances. At first, we were terrified at the complexity of putting all this into an in-home app. In the end, it turned out to be very simple. The first day that we did the shoot, we got the results back, we looked at them in our environment, the developers, the engineers, everybody, were just kind of blown away by what it was like to actually have a human performance face-to-face for the first time in an immersive media. That human presence brings authenticity to these environments into the stories that we want to tell and the content that we want to create.
Skylar Sweetman 9:46
Any situation where you need to feel the presence of a performance, whether that's an actor, a musician, or even a training instructor. That's where this technology excels.
Christina Heller 9:58
When I tell people that my company makes holograms, I sometimes have to pinch myself because it's just so cool. People are beautiful. It's time you put them in immersive experiences that way.
Amy LaMeyer 10:21
So amazing. I'm excited to see all of the different ways that you're pushing technology boundaries and making it easier to capture beautiful people. We're going to move on to David Liu. David's had over 10 years of experience in immersive music, and technology, and gaming. He was the creative director at Viacom NEXT, and in this video, we're going to take a look at Aeronauts, which— it features Billy Corgan from The Smashing Pumpkins. It won a Cannes Lion award two years ago.
Billy Corgan 11:17
(sung) "I went down the middle,
The world survives
. Look out son the air's alive
. Call it ether elemental eye
Ooh, dark nights
One magic heart deceives
One little mind to ease off
One silver drum that beats, oh
This mountain was tor,n from us
This mountain was torn from us
If I'm leaving you without return a snare
Can a poem fit through time and space
You can call it home if you wish or if it's fair
But it's mine to share and share alike
Won't you mourn with me?
Lover won't you mourn with me?
Lovers won't you mourn with me?
Fall across the middle,
The world survives.
Look out son the air's alive.
Call it ether elemental eye
Ooh, dark nights
One magic heart to fly
One little life to light up
One silver eye to eye, ooh
This mountain was born of us
This mountain was born of us
If I'm leaving you without return
A snare
Can a poem fit through time and space
You can call it home if you wish or if it's fair
But it's mine to share and share alike
Won't you mourn with me
Lover won't you mourn with me?
Lovers won't you mourn with me?
Lovers won't you mourn with me?
Lovers won't you mourn with me?
Lovers won't you mourn with me?
Lovers won't you mourn with me?
Lovers won't you mourn with me
Lovers won't you mourn with me?"
Amy LaMeyer 12:48
So beautiful. Next we've got Roman Rappak. He's the founder of TheBoolean.io, and he's also the lead singer of a band called Miro Shot. They recently debuted an album called “CONTENT”. They started in 2017 as a collective of musicians, and technologists, and designers, coders, filmmakers, and are kicking off our virtual worlds tour. So let's check this out.
Amy LaMeyer 15:22
So exciting. And finally, we have Tom Impallomeni. Tom Impallomeni is CEO and founder of Tribe XR. Tribe is a— helps you learn creative skills, and right now they have the DJ school launched, which you can participate and learn how to DJ in VR and then stream your experiences over to Twitch. So definitely recommend checking that out, and here we will take a look at what it looks like.
Amy LaMeyer 17:21
Super fun. I absolutely recommend checking them out on Twitch and the Quest. So let's jump in. All of you have obviously had amazing and diverse experiences in creating immersive music content as we've just seen. I'd also like to hear your opinion on other content that you've seen that you thought did a good job connecting artists and fans and maybe some of the technical underpinnings underneath that that you thought were interesting. Do you want to kick it off Roman?
Roman Rappak 18:01
Yeah, I guess the first one that springs to mind is one of the— I guess the first success story of this crossover is Wave, (inaudible) Wave VR, and it's a guy called Adam Arrigo, who started a little while ago. I think they've— they're probably the closest to something that is an indication of how this will work, in the sense that it won't really be concerts and it won't really be a game. But it'll be this hybrid of all those forms. And they're doing really interesting stuff. But also they're— they've come up against the same problems that I guess everyone on this panel will have encountered, which is that the whole thing about music is that it's a physical event and that it's something that it's visceral, and it's something that is exciting because it's happening right now. And so when you put someone in a headset, it instantly kills most of the things that make music amazing or concerts amazing. And I think that there is a way that it's going to evolve, and I think everyone on this panel is kind of evolving it in a different way which is really exciting to see. But there's also a lot of the things that make concerts exciting that we're all trying to bring in and I think everyone's got their own answer to that.
Christina Heller 19:32
I want to kind of highlight something one of our partners is doing with Jadu, J-A-D-U created by Asad Malik. And they're essentially taking volumetric holograms and integrating them into social media like Tik Tok and Instagram. It's sort of a new take on more of a music video concept. The performers aren't doing live performances, at least yet, but they are creating, you know, short, fun, holographic content to whatever single they're trying to promote. So picture like 15, 20 seconds of whatever their lead single is at any given point, and basing the creative and the content on something that would be fun for their fans to engage with. So, an example, you know, maybe doing a fun dance or something like that, then going into Jadu, the fans can, you know, make videos with those holograms, you know. For instance, one that was really funny, it was like, somebody put the hologram on top of their dog, and right at the moment where the beat dropped, he managed to make it look like the dog was like waking up at that point, and created like a really fun, cute segment that then they published directly through the app into Tik Tok, and now we're starting to see I think a little bit of what's to come, you know, in terms of, you know, new ways of engaging with your favorite artists and holographic content.
Amy LaMeyer 21:03
Absolutely, absolutely. Yeah, they've had some of their videos go pretty viral.
Christina Heller 21:07
Yes. I mean, I don't have like the latest stats, but I know that with the initial five holograms they launched on the platform, they had over 2 million impressions on Tik Tok with just those first five holograms. So, you know, I think part of the key to Jadu's successes is really the curation. And that's where Asad— I mean, Asad's a genius, like, in a lot of senses, like the design of the app is really smart, you know, the types of artists he's engaging are very young. And so to be honest, like, you know, this is a teen thing, you know, more than it is, you know, something of my generation, but I think that that's really exciting because, you know, that means that we're starting— we're gonna see a lot of interesting engagement from this next generation.
David Liu 21:54
Yeah, I want to actually ride off what Christina's saying because there's a lot— so I think we're sort of entering this phase where, you know, participatory culture, collaborative culture, where fans want to co-create with the artist, is a thing. And that's why Tik Tok's such a big thing. Like it gets to— and especially with Jadu, be with the artist, but also use your music to spin off something else entirely. That's super exciting, right? So one thing we've noticed, especially on our side is— I'm gonna talk about some new forms of music that are out there, actually. Like, so if you look at what Fuser's doing with Harmonix, basically, using stems, let you play with music and like, you play, almost like— (inaudible) game mechanics mixed around. There's a lot of opportunities there. But one thing that came to mind, most of all, for me when I'm exploring music in the spatial space is, when you create— music is a linear piece— is a linear medium. But when you as a player, get to play around with it gets super exciting and (inaudible) Tetris effect, if anyone here has played it, it's essentially a song, but then every movement you make is an instrument. And the song actually waits for you to make several cues before escalating to the next line of music. And that's something that just blows my mind. And I think something that artists are starting to really understand and start playing around with so that their fans can also participate in the music in a way that is more visible than just listening to it.
Amy LaMeyer 23:23
Yeah, it's fantastic. I'm really excited to see all of the things that we haven't imagined yet, right? Tom, did you want to add anything?
Tom Impallomeni 23:37
Yeah, I think it's a really exciting time for artists. COVID, in a weird way, has created this amazingly tricky period for people making money in the industry but has also formed creativity and an acceleration of virtual consoles in a way that I think is unprecedented. And for me, what's really interesting, is both— the virtual consoles ecosystem is sort of split into two ends of the spectrum. One, which is deeper, highly-produced beautiful pieces of content that are, you know, often pushing the boundaries of what can be done using volumetric capture. And then at Tribe, we're more at the other end of the spectrum, which is how do we make everyone be able to perform at home? And we've ebbed towards allowing people to sort of customize their environment, but we focus primarily on interactivity with the audience. And we think that that's kind of critical because, for a lot of like home performance, they want feedback. They want people to interact and to be active and not passive watchers. So that's just really how we think about it for this sort of mass-market home performance, which, hopefully, at Tribe, we're going to be producing some artists who eventually end up with these super highly produced concerts.
Amy LaMeyer 25:02
I'm sure there's a lot of people at home right now that have a little bit of extra time on their hands. And so hopefully they're jumping in. Those people that have always wanted to be a DJ and making their dreams happen, right?
Tom Impallomeni 25:13
Well, we have some— we— our most-cited example is a mom in Ohio who started messing around with our products and streams. And it got picked up by a Facebook group called Music Crowns. And so she ended up performing to sort of— I think they have 3 million people in the group. And so she went completely viral, just playing this beginner set to a concurrent audience of something like 20,000 people. So sometimes some of the things that can happen naturally are pretty wild.
Amy LaMeyer 25:45
Yeah, that's awesome. We talked a little bit earlier about some of the technical issues that we'd had even with the event that we tried to do before which half of you were in, and obviously, there are some things that we need to overcome. So maybe let's talk a little bit about that. If you guys could highlight some of the challenges that you see in terms of scale and growth in this space and just kind of where the technology is compared to what it is you want to do and create?
Christina Heller 26:17
Well, I think if we're talking about headset experiences, the— I'm like a Quest-only user these days, but because it's, you know, just basically a cell phone strapped to my face. I think that the limitations of what you can do on that kind of a chip are probably the biggest, or probably the biggest limit, like things we have to overcome right now. Because, you know, for— if you're in like VR chat, for instance, like I'm a big fan of VR chat, but a lot of the functionality that would make it comfortable and easy for a large group to gather and engage and assemble, I think it's still you know, there's still a lot of technical hurdles that you know, we just haven't been able to address yet. Like in terms of— I think we experienced it a bit on Hubs, like issues with audio, or— are certainly an area where we should see a lot of improvement. But I also want to be considerate to the fact that, you know, we're asking— we're already doing like minor miracles when you think about the— what we're able to do on something like this, right? Obviously, I think, like, you know, this is a 5G panel. So I think that you know, there's a lot of excitement around what 5G is going to enable, and hopefully, it will, you know, solve some of the problems that we currently hit when it comes to, you know, what we're able to do on limited bandwidth.
Amy LaMeyer 27:43
For sure. Because we— remember what it was like to try and do video on these with 4G, right? Like, kind of a little bit that reminds me of what we're trying to do now with immersive tech and 5G, right? I just think it's going to be a true enabler on so many levels.
Christina Heller 28:00
So yeah, like, I think we should just actually admit that this is a minor miracle, you know, or a major miracle and we should be pretty impressed with what it's doing already, but it still doesn't quite compare to, as Tom said, or maybe it was what Roman said, you know, the in-person experience, which is the highest fidelity, and, you know, easiest for consumption.
Amy LaMeyer 28:21
And all the haptics, right? Yeah, I was in a car the other day, the first time in a while, for a long drive and I was listening to some music, and I realized I could feel it again, right? Because it was so— it was in my car and so I could feel the beat and it was the first time in— since I had seen a show on March 6, that I could feel the beat again. And as great as some of these live streams are and as some of the, you know, the Wave performances or you know, Travis Scott in Fortnite or Diplo today, right? Like those are really fun, but it's not that fully immersive experience. That you get when it's live. I'll take them both, I want it all, but, yeah.
Unknown Speaker 29:08
(inaudible)
David Liu 29:11
Sorry.
Roman Rappak 29:12
No, no, go on, David.
David Liu 29:14
I just want to say that just to just deconstruct it a little bit like, you know, you have sort of, you know, the capture side of the artists, right? Like, how do you get what they are making, out? And you have the sort of experience side, like, what is it that we want to experience? Is it just the track? Is it their physical presence? Is it Livestream? And on the other end, you have sort of like the delivery, the actual experience at home, like, you know, right now, there's a lot of dis— like, there's not much sync between these two strategies, and I think every pillar is trying to find its own solution. And I think that's a bit of tension right now because it's like, I'm not sure that— going to your point, Amy, I think the concert's great and that's the simplest one and that's for a specific type of music— a specific way of experiencing music, right? But I think for the at-home experience, I'm not sure it direct one-to-one live concert experience, you know, is the right format, to your point. Because we can't feel that immediacy, I don't think we would be able to exactly replicate it. I think the more important question is how do we evolve that experience to something that's perfect for the at-home experience knowing what we know that works for live concerts, and what we know that works at home. And I think that's where we're at right now balancing it all.
Amy LaMeyer 30:28
And that works right now with the technology that we have right now, right? Because I think we all have visions for things that we would like to happen, but maybe can't fully be supported right now because of headsets or latency, or, you know, whatever the case may be. Audio.
Roman Rappak 30:46
Yeah, I was gonna say something but then David actually expressed it in a much more articulated way that I was going to. So, I'll extend the point I was gonna say which was that—
Amy LaMeyer 30:56
I mean, you've had the most recent experience with technical challenges.
Roman Rappak 30:59
(inaudible) on the sort of depressing end of the conversation. But like, the thing that David's is talking about really makes a lot of sense to me. Because we're coming at this from the music industry point-of-view and the music industry was already in a complete state— after downloads, and after you know everything from Napster to Limewire, before it became monetized, before you had Spotify, and this thing that was initially a threat, the technology suddenly became the main source of income for the entire music industry. So it was an example of the suspicion of technology. And again, while it isn't really like going and queuing up and buying a physical record— just suddenly being like, well, there are artists whose entire career are thanks to this. You would never have heard of them before that. And I think when you're— when David used the word format, that's a really interesting way of looking at this because you— and I have to deal with a lot of people in the music industry talk about you know, the album format, which is this thing that's existed for 60 years and is this like time-honored thing. But that's just a piece of technology that someone was able to record something on a piece of shellac or whatever. And I think that like, as the same— that same thing with all the people on this panel, like I think Tom is on the kind of music creation side of it. That's the working with, the idea of you interacting with music. And David and Christina are working on the idea of how you capture these moments. And we're on the side of how do we perform these moments. And I think that it's a new format, which we don't actually know, like, if you remember that, when someone recorded the first record, it was years and years before you had radio plugins, and an album cycle, and a campaign strategy, and a festival circuit, and all those things. But I think these little movements towards this, is basically us finding out what that format is going to be you know, what will be the album format of 2026 or whatever.
Amy LaMeyer 31:18
Absolutely.
Christina Heller 31:19
One thing that I think might also build off of what David and Roman are saying is, is that you have— you can't get it all right now. Like, you have to decide what's most important to you. And so let's— I want to use like volumetric video and the wave as two examples of both awesome executions for a musician that's not able to get out there and perform at the moment, but talk about what's— the benefits and disadvantages to both. So, you know, if you're capturing at Meta Stage, you know, we're using 106 cameras, as you saw in the video, to capture a real performance that is as true to the live performance as possible. That being said, there are limitations around what we can do with instruments like, that's a whole other conversation. So let's just assume we're talking about like a singer performing a live song. So what's great about that is it looks— it will look like the real live performance in that it will carry all the micro-expressions, the nuances, like the way that the singer moves their body and their actual voice, it will look like them in the face, all of that. That being said it will be pre-rendered. So even if— so then when we distribute it to the audience, it will be new to them. They won't have seen it before so it will carry with it some of the excitement of a live performance and that you know you're seeing the real performer do something that no one else has seen yet before, but it won't be live. Volumetric live is a new medium that's evolving but for the quality that like Meta Stage like stands behind, I wouldn't say that live volumetric is really like at that level yet. So again, pre-rendered, but authentic and looks like the real performer that you know and love. The Wave is doing a very interesting kind of different approach where they're having— they have an animated avatar, that the performer is now moving in real time live. So like the Lindsey Stirling concert, which, you know, was, Lindsey Stirling, like with a mo-cap suit on actually performing her violin live, and you could tune in either on YouTube or in headset, and therefore you were getting the magic of watching your performer do their work in real time, and also responding to some audience feedback. But again, like you didn't— you don't get to see like Lindsey smile, or like get to see the way that, you know, she would look or be in real life. So again, like if the live part's really, really important than something like the Wave might be a great execution. But if you want to make it as close to the real thing as possible, volumetric might be a better solution. And you know, that's kind of where we're at. You can't quite get it all just yet.
Roman Rappak 35:23
Can I just say one thing as well, which is kind of a really specific point for us as— well, as a band we've been going out, we've been touring these shows, and we're playing VR shows. And the thing we come up against all the time, which is, people go "Yeah, but the thing is, you're doing— it's VR thing, it's a tech thing. It's never gonna be like, like the real show." And the— I think the important point is it isn't the real show, it's a different thing. If I record a drum kit, you mic up a drum kit in a specific way and you have the studio sound of a drum kit, it's nothing like what being in front of a live band is like, it's a new format, and it's a new idea. And I think it's like what Christina said, there'll be— and there will be artists that will use this format and use these tools and go to— like David's space in New York and this incredible green screen, multi-camera volumetric capture. And I kind of really, really noticed that this is going to be completely different art form. So there aren't— it's not about trying to recreate these moments, it's about saying, "well, this is a new version in the same way that when you listen to the single, it feels like this, when you watch the video, it feels like this, and when you see live, it's like this, and when you experience Christina or David's volumetric capture, it's a completely different format.
Amy LaMeyer 36:32
Absolutely. I'm going to add to that, too. But I'm also going to remind people that we're going to do questions in about 10 minutes. So the question is the Q&A button on the right, and then also there are some polls, so if the participants could fill out the polls, they're fun, and we'd be interested to hear what your experiences have been like. To build on you, Roman, what you were saying, Roman, I like— a couple different experiences I thought were really great, as much as I love concerts, again, like hundreds a year usually. Not this year, but normally. But Imogen Heap in the way it was really interesting because again you could— not only were you interacting with her, but you were interacting with the visuals that were happening and the visuals themselves were mimicking the song. So when she was talking about wind, there was a wind visual. Like that gave me goosebumps. And the other one is the Secret Rose that Rebecca Barkin from Magic Leap helped to produce that and just like literally touching and interacting with the sound and the images. Like those things you can't do at a concert, you cannot do that. And those are the ways that I think I'm excited about seeing things grow.
Roman Rappak 37:45
It's a new art form. And there's a new art form that didn't exist 20 years ago, which is game design. And it's the biggest art form there is. Twice as big as the music industry is the gaming industry. So people who know how to manipulate those well, who know how to use interactivity and user experience that's— they'll be the Hendrixes or the Joy Divisions or the Wu Tang Clan of the next decade.
Amy LaMeyer 38:10
Right. When Travis Scott's Fortnite jumped— oh.
David Liu 38:14
Sorry, go ahead.
Amy LaMeyer 38:15
Go ahead, David. I was just gonna say, when Travis Knight in Fortnite jumped, right? The beat dropped and then like our avatars blew up in the air. Come on, that's amazing! Amazing! Again, things that can't be done live. Go ahead, David.
David Liu 38:30
No, I was just gonna say I'm just gonna thumbs up that. There's a lot of convergence right now, right, with the different disciplines, especially if it's— we're trying to bridge interactive digital space with music, there's going to be some element of game design, level design, spatial design that's got to be thought about, and I think people are getting that now. Like, you know, like, I'm so glad to hear you say that, Roman, because it's like, it's true. It's something we need to start thinking about. And I still think there's still some purism in an industry where like "no! That's— games are for kids!" You know, and it's just like not something that, you know, that the industry wants to touch. So I think the more open-minded we are as we approach to sort of— all these mediums, I think the better off we'll all be. Both for artists, both for labels, and the consumer. I just feel like that conversation needs to happen now, so thank you for saying that.
Roman Rappak 39:17
It is, but it also— the conversation you have with like— we have to have them with the music agent, we have to like speak to a label, we have to speak to a booking agent. And there is a huge amount of cynicism and suspicion, whether it's about "Oh, it's just a game, etc." Then you're— you come from a quite unique point, David, because you're working on this stuff now. But then you also worked on some of the biggest games of the past like 10-15 years and IPs that were— like cultural kind of moments. And I think that that's the thing that is needed. And playing kind of devil's advocate, instead of it just being like, pro-tech and pro-interactive, is that the music industry has a bit of a point because those guys went in into the MelodyVR experience, they went into the Travis Scott thing, well, now what? Okay, great, like you threw an emoji at a 90-foot motion-captured Travis Scott that was recorded three weeks ago. Like how is that in any way as good as the Glastonbury Festival or, you know, Lollapalooza or whatever. And I think gradually, we'll start, like you said, merging these art forms.
Christina Heller 40:20
Well, I don't think anybody— we want to go to Glastonbury. I mean, that's the thing. It's not like we're just saying that, you know, screw live concerts, let's only do you know, at-home virtual experiences. You know, obviously, I think as a society, we're all just kind of reacting to this new normal that seems like it's going to be pervasive for at least, you know, the next year or so. So, you know, I have been hearing that music labels are more open-minded than ever at the moment about things like immersive technology and alternative mediums because, like, just— doesn't seem like we're going to be able to safely go to a live concert for the next year or so.
Roman Rappak 41:01
Yeah. But also, don't forget that they have a hit of like millions and millions of dollars in the industry that they used to make a lot of money out of now there's no constant. So suddenly, people like you guys are like the only hope they have.
David Liu 41:19
I want to add to what you just said there, Roman. I feel like right now we're at the part— we're at the beginning of a new media, right? And the earliest forms of that new medium is always a projection of the previous medium, it won't be that good. Which is why I like early film, you know, it's like, it's there are no shots, there isn't— editing was not a thing. It was just like— play. With just a super wide shot, and it's not a perfect version of film. It took a lot of innovation before it got to being film, which separated it from theater. And I think we're at a point right now where you know, like, the first few things we're making, they're not gonna be that good. Let's just face it. We'll be trying to bring digital concert— real concerts into the digital space and I don't think it's a one-for-one analogy. I think there's got to be some iteration of that form, and we just don't know what it is yet. That's exciting to me.
Tom Impallomeni 42:04
We've seen some of that happening particularly in the DJ space at the beginning of quarantine. And suddenly all the DJs around the world weren't making any money because all of their club bookings were canceled and there were these online festivals which featured— they did like this front room and Steve Aoki in his front room. And like as a spectacle, it was kind of— it was great, but it was also a little bit sad to see them sort of confined to their living rooms. And then we've seen this like— it's this endless integration. So suddenly, now there's sort of this like projection mapping and green screening and mixed reality DJ sets and virtual worlds. So we've seen this kind of like experimentation. One thing that we've noticed that's really interesting is that there's been a— you know, there's obviously these high-production retro concerts like Wave and you know, the John Legend thing that's coming up and so on that are almost like a 30-minute recorded video sort of Michael Jackson "Thriller"-type spectacle with fantastic, beautiful visuals. And then you have these like homemade, sort of Diplo at home with a green screen. But then you've got this third category, which is like ways that artists are now interacting with fans. But like Mike Shinoda from Linkin Park, sitting there and doing like music production, and just sitting with like his screen share for Ableton for like two hours and just having fan interaction. And that's like the most basic level, but super interesting, and it gets massively high engagement. So I think there's like a definitely like a different strata of ways that fans are interacting with the artists.
Amy LaMeyer 43:43
Great. All right, I'm gonna ask one more question. And that was a perfect lead-in. We've talked a lot about the experiences that we can as users interact with, but what advice or tips or suggestions would you give artists and musicians if they're starting to explore this space and create in this space. And after this question, we'll go to Fan Q&A— or into Participant Q&A.
Christina Heller 44:14
Well, I think for volumetric capture, you know, obviously I'm super bullish on it because of, you know, it's like the next evolution of photo and video, it's— in that it's really just documentation of a real person doing a real thing. So I just wanted to add that in that, like, I think that there's so much that hasn't been done that, like all it would take would be the right performer and the right buy-in from talent to do something incredibly cool. And one thing I would say, though, is that like I said, we have some limitations on what we can capture in terms of instruments and doing live performances. So because with the cameras, you need to get as much data as possible you need to be able to see as much of the subject as possible. So if you throw a piano in there, for instance, it's blocking a lot of the cameras and you're going to get up a lot of artifacting and if that's okay, like for instance. one of the things that kind of makes Uzzit an interesting partner is that he's really okay with the captures not looking perfect as long as it captures the essence of what the artist is trying to do. So he doesn't really freak out if like there's something weird on the hair, he'd rather the artists just kind of do what they will with their hair and move forward. But that being said, like if you're looking for the cleanest and most authentic capture then we have to figure out how to limit as much prop and instrument use as possible. So live singing is easy and so just keeping that in mind if there's something you're thinking about doing live that doesn't require a ton of props or a large volume, you know, happy to talk and see what we can make happen.
Amy LaMeyer 45:52
Great. Any others?
Roman Rappak 45:55
Yeah, I've got a couple of points of what— if there's anyone listening who is either an artist or who was thinking of how you adapt to what you're doing when you're a DJ, or a rapper, or a band, or composer, or whatever, which is to kind of remember that all of the history of music— of recorded music has been history of technology. From the fact that we get the first Edison record, means we can record stuff, and then the Stratocaster gives us Hendrix and the hit 60s counterculture, and the synthesizer gives us electronic music and audio. It's always been a progression of technology. And that if you're using a new tool, then you can express something that hasn't been expressed before. So all these things that David was saying about game design, or the stuff that Christina's doing with volumetric capture, or that Tom's doing with even this idea of the role of the person who's listening to music, who's a fan of music, suddenly is interacting with it. I think that's the best approach you can take is that the story of music has been about people who were— you're watching an orchestra, and then you have blues, and then you can interact with three chords, and then you only need three chords, because punk happens, then you don't even need a guitar because DJ culture happens. And then I think this with the idea of interacting as the audience is the most important part of it. And we're just building worlds around.
Amy LaMeyer 47:23
That's great.
David Liu 47:24
Wholehearted agreement. And I'll just riff off that very quickly as well. Like, I would urge all musicians to sort of consider what their form is now, right? Like to rethink— I play music, too and the reason why— this comes from a really personal space like I play music because it communicates a certain language with people, it makes you feel a certain way. And I feel that at this point in time, where we are in the world right now, like there's never been a better opportunity for you to have a direct one-to-one connection with the fans, first of all. And second, for them to participate in whatever form you make. Like, it used to be just listening to your music, now they react with you, remix it, they take it apart, and deconstruct it. And I think the more open you are to what that means in whatever form or what technology you use, I think the more exciting the next wave of music is going to be. So please explore and experiment like I would be so happy to see how you take it away.
Amy LaMeyer 48:26
Yeah, I'm also looking forward to seeing the reach that it has as well, right? Because given technology, we can just— we can go global and have this more global experience and music is a unifier. And so let's hope that it helps the current state that we have. Great, I'm gonna switch it over to Miriam who has some questions, a lot of good feedback on the Q&A channel.
Miriam Tinberg 48:55
Yeah, there's a lot of activity here. So I wanted to start with posing a generic question so anyone who has experience or thoughts please answer. But Peter is curious about what kind of work you all are doing in the 5G space if you have— if you aren't actually working on 5G yet, where do you see that actually affecting the experience, the product, the building? General question out to everyone.
Christina Heller 49:19
At Meta Stage we're doing a lot of projects for mobile 5G right now you know, I think, you know, 5G is about pushing like way more data through the pipes and with volumetric data— volumetric video, we're now talking about 3D performances, you know, captured with, like I said, 106 cameras, it's— even though we can compress those files to be pretty small, they're still you know, larger than most photos and, you know, like an mp4— short mp4 video. So, I do think that one of the promises of 5G is that you'll be able to engage with your favorite artists in full three-dimensions in your living room. And we're doing a lot of I would say, really— I won't even call them proof of concepts like episodic AR, mobile AR experiences that are going to be unveiled with the next generation of (inaudible).
Roman Rappak 50:14
You know, from our point of view, we've been performing these shows, which initially was just, if you imagine 30 people in VR headsets that are switching between the live show in front of them, and a VR landscape built by game designers and motion graphics artists. And 5G for us is the first moment where we can send that exact same experience to headsets around the world. So you can be at the physical space— and I think with the sort of COVID and safe distancing, if you look at what the plan is for how concerts are going to work for the next 12 months, they're going to be a completely different form for a long time. And so the way that 5G for us is looking like a really attractive prospect in the development of the new version of our app, is this idea that yes, you have this distance-safe concept that is happening in the real world. But you can also stream that in an immersive experience, high-resolution to headsets around the world as well and they appear as avatars within the live concert.
Amy LaMeyer 51:15
I can't wait.
David Liu 51:20
So very quickly I—
Miriam Tinberg 51:21
Okay, let's keep move—
David Liu 51:22
Oh, never mind, it's okay. Oh very very quickly—
Miriam Tinberg 51:26
Take it away.
David Liu 51:27
It's just some folks I'm working with were extremely excited about 5G Edge Compute, right? So can we— and this is a couple years early, but, you know, basically without giving too much away like, I can't wait for the day where we can just live without mobile phones and be in a highly, highly, highly, reactive, real-time, 3D-rendered world that is that is easy to access anywhere you are. And I think we're again early days but really excited for what edge computing can do. Just want to throw it out there, everyone working on 5G edge compute please, we're waiting for you to be done. Yeah.
Miriam Tinberg 52:05
You're leaving us with a little bit of mystery? I like it. Suspense. So moving on to the next question that Marlin Fuentes asks and this is super interesting. I don't actually know if we've talked much about this in the discussion today. We've been talking a lot about the actual in-concert, in-venue experience, but what about after the fact? How are we preserving this? How are we curating this? And he asks, what are some opportunities for immersive tech in music museums now that doors are closed? So things like Hall of Fames, Grammy museums, do you all see sort of the curation of music and the preservation of music changing?
Christina Heller 52:41
Well, jumping in yes, I mean, everything— so when we capture a performer doing, let's say, one of their, you know, lead singles, we are actually capturing that person at this particular moment in time, at this age. And so I believe that any— like, we try— like we didn't actually have these capabilities when we first opened, but since we are now fully on the Azure pipeline with Microsoft, we put anybody of significance that we capture right into cold storage. And we try to save as much of the source material as we can, too because, you know, as processing may increasingly develop over the next, you know, 10 years or so, it would be amazing to see if we could get even more fidelity, you know, five years from now with some of the stuff we're capturing. So I love to imagine that the— like that maybe, for instance, all of the captures we're doing now with Jadu, you know, in 10 years, could be a museum exhibit. Were now maybe using the latest in holographic display tech, you no longer even need glasses, and you can just watch— walk around the museum and see these holograms of these iconic performers performing their works at that singular moment in time. So yes, I think that everybody who's doing this kind of work, if we can just put it on some kind of a safe device and lock it away, I think interactive museums, both I guess in real life and maybe even just virtual museums are definitely something that we'll be seeing in the future.
Roman Rappak 54:15
I've got this idea that you know, the stuff that you're capturing, Christina and David, though that is the highest, the highest-resolution technology that exists to mankind. And so musicians that are captured like the Billy Corgan thing that you did, David, those are going to be the things that are— that you can access later on. Like we— all our memory of music, when you think about what our memory of all these important moments in history that's like— it's Michael Jackson at the Grammys, it's that camera filming that, it's Hendrix of Woodstock, it's this— these grainy videos of things and vinyl recordings of Marvin Gaye, that it's always been about capturing it. And so if people who are at the forefront of capturing it are doing that now, then that's what later generations will be watching.
Amy LaMeyer 55:03
Yeah, I was just I was gonna say it just literally crushes me that we weren't able— that Christina you weren't able to capture Prince before he left us.
Roman Rappak 55:11
Oh yeah, that would have been great.
Amy LaMeyer 55:12
Would that have been epic? I'm from Minnesota like I— part of my soul, and that would have just been so fantastic. I hope all musicians think about getting—
Christina Heller 55:25
All the musicians— come to Meta Stage.
Amy LaMeyer 55:28
Yes, everyone go to Meta Stage.
Christina Heller 55:29
Just all of them.
Amy LaMeyer 55:31
Miriam, Did you have one more question for us?
Miriam Tinberg 55:35
Yeah, let's do one more. I think this is a good way to end it. A few people asked this. Are these types of volumetric capture, you know, the variety of tech that we've been talking about, is it only approachable for these artists with big budgets and for creators and builders with huge stages and you know, pieces of hardware and software and expertise, or is there an opportunity for smaller artists and smaller creators to engage in this?
David Liu 55:58
Yes. So I'll just jump in very quickly for this one. Like, I think there are many, many, many ways in which you can sort of start engaging with your fans, right? Like, definitely, volumetric is a really, really strong way to do it. And it's definitely the— it's got the most fidelity, you know, you're capturing micro-expressions and everything, like Christina, said. But like, there are a lot of really interesting mediums that are available out there right now. And we talked about the Wave, you know, I think the Wave's just still sort of like, curating the artists. But if you explore things like Mozilla Hubs, I'm really fond of what they did, which makes it really easy for you to start streaming a virtual world that you can build very easily, you don't need to know any coding at all, you just spin it up and you send the invite link out, and people can check it out. You know, it's in its early days, but it works really well for out-space VR, does that pretty well as well. And this is just the XR space. You can— you know, I'll even urge you to look beyond XR as well, you know, as the XR world is sort of building upon itself. TikTok, honestly, it's so— it's such an immediate format right now. I think that that is something that is worth exploring as well.
Unknown Speaker 57:00
(inaudible)
Christina Heller 57:00
I also want to shout-out to Depth Kit. If you're looking to do volumetric at more of an independent like in your apartment or just like it, you know, indie film level, Scatter and Depth Kit is a great solution. And I love the style of the way that that capture comes out. It's very ethereal and very cool.
Tom Impallomeni 57:19
Sorry, just to add to like, some of the highest performing pieces of musical content online are sort of home-produced at low cost. So there's— there are lots of examples of artists getting really creative, as David mentioned, with Tik Tok. With VR, you know, we certainly with Tribe, we have people performing into out space, to Twitch, to YouTube, to Facebook, you know, seamlessly. You know, high-end volumetric capture is still expensive, the cost is going to come down. There's sort of like home versions that you can use, but I would say yeah, just look at YouTube, some of the most amazing pieces of content are just high made and focused on the artist's abilities. So, yeah, it's really a great opportunity for artists to get creative.
Roman Rappak 58:11
Yeah, I think that it kind of comes back to this thing that really, again, go look at the history of music, like indie always wins, like the independents always come up with something more interesting. The form that was so expensive that you have to be a massive artist is something that everyone's like, Oh, that's impressive, I'm really glad YouTube did a 360 video of that, but I'm not particularly interested in watching The Edge play another solo. DJ Marshmello playing in Fortnite was amazing, but I've probably had enough cheesy EDM to last a century. But then going into that world and finding an artist who no one knows, that I have an emotional reaction to, that I feel like I discover, that's gonna be the next thing. That's when this really goes well.
Christina Heller 58:52
I want to bring—
David Liu 58:53
Can I just say— oh, sorry.
Amy LaMeyer 58:54
Yeah, go ahead.
David Liu 58:56
As an indie artist, you have the opportunity to have that small fan base right now, it's really a one-on-one direct connection with them. So like, if you were to play like even like a small Twitch concert and react to them and solicit feedback to them and have this loop where they're actually engaging with you, they're gonna love you far more than if you honestly, you know, put up something super glitzy in Fortnite, where you're just— you know what, you're great and big, but you're not exactly forming that close personal connection, which I think is much harder for someone like Travis Scott to do at this point. So don't let this opportunity of being indie slip you at all.
Roman Rappak 59:29
You know, there's a business structure in here which is obviously these are companies— are massive tech companies. They're not gonna say, oh, we'll just bring this unheard artist or this person who's a guitarist, or a rapper no one's heard of in. They're gonna want to bring in the big tentpole artists because they're going to bring in numbers, but I think after that, then you're gonna have people exactly what happened with SoundCloud or with MySpace, where there's going to be artists who use that platform and no one's ever heard of them and they explode. It's happening on Tik Tok. And that's— it's their platform. It's their new media.
Miriam Tinberg 59:58
Well, I want to say that obviously— there is a curation process. So us— it has to basically approve you. But for the artists that are on Jadu, they're not paying for their holograms, and neither are their labels. So that is just— you know if you're an artist that has an ardent following, but you're independent, you know, I would recommend checking it out. And again, like I said, you have to get approval from the Jadu folks, but we're not charging the artists or the labels.
Amy LaMeyer 1:00:28
Fantastic. I want to thank all of you for what you're doing, what you're creating, for the boundaries that you're pushing, for the music that you're sharing. This was a really fun and great conversation, and I look forward to seeing what you all make over the next few years. And thank you again to Verizon and Alley for hosting this. Really appreciate that. Take care, everyone.
David Liu 1:00:52
Thanks for having us. Thank you. Thank you, Verizon.
Amy LaMeyer 1:00:55
Bye.
Roman Rappak 1:00:55
See you guys, bye.
.png)
