Transcript 0:00 We need more sophisticated instruments than just word input. But words being the only input doesn't work with a team of world-class artists. 0:09 That's Edward Dawson-Taylor, who runs CGPro, one of the top Unreal Engine and VFX educational platforms. After training VFX professionals for years, he's now teaching them how AI changes everything. 0:19 AI is something that touches essentially every industry and is now injecting into most of them. The world I'm used to of polygonal 3D has 30, 40-plus years of development. Gaussian splats have not. 0:32 With generative AI and sort of the applications in virtual production, where do you see that going? Uh, crystal ball. Hang on, let me... [upbeat music] From AI on the Lot and VP Land, this is Inside the AI Studio. 0:47 Ed, good to see you. Joey. Yeah, you too. Uh, so tell me a bit about CGPro and also what you were presenting here at AI on the Lot. Sure. 0:55 Um, so CGPro started its life as a training company, um, about five years ago, beginning of COVID, beginning of the kind of, um, the virtual production revolution that kind of grew through COVID and as a response to people needing training in it. 1:12 I'd just come off the back of working on some movies like Jungle Book and Lion King that were helped kind of figure out what virtual production was beyond Avatar and Lord of the Rings and some of the precursors to that. 1:24 Um, we started training people in virtual production, which, a lot of which was being based in Unreal Engine, so it was lots of Unreal Engine du- to do with that. And we became a Unreal Authorized Training Center. 1:38 We became a, what was a Premier, it is now a Gold Unreal Authorized Training Center about a couple of years ago. We were the only one in North America. We might still be, I'm not sure. But, uh, 1:49 that, that was the start of CGPro. We've been all over the world training everybody from LMFAO to NASA [laughs] And everything in between. And, you know, been 2:02 watching this next disruption happen through several others since then of... It was kinda COVID, film strikes, AI, you know, all the world economic and governmental changes, and all these kind of, uh, moments. 2:16 But now, now the kind of the edge, the diff that everybody wants to pick up now is the AI component. So, what difference does that make to everybody and, and not just in the production industry? 2:28 So we're now doing a lot of that as well, and how it kind of injects into traditional things like visual effects, which is where I come from, working in visual effects for the last 20 or so years. 2:38 And now we are, we are here at AI on the Lot. We're doing trainings in AI. We're also, as a company, CGPro, have, have diversified beyond the media and entertainment industry of, um, serving other industries. 2:52 NASA is not in the media and entertainment industry primarily. 2:55 So, but also other technologies other than virtual production related ones, and AI is something that touches essentially every industry and is now injecting into most of them in some way. 3:07 So we're doing trainings in that. But we've also, as a company, extended beyond training, and we've, we've been making things for people for quite a while. 3:16 We actually just became a Epic service partner as well as a training provider. We're an authorized service provider for them, with them, uh, in partnership with them. 3:27 So still a lot of what we do is involved in, in real time, and real time is, um, growing in a sense in that lots of other people wanting real-time graphics, but also real-time workflows of all different natures. 3:42 Um, so it's not all just about the graphics, it's not all just about pixels, um, but it's also about real-time iteration and experimentation, and even through things like admin and project management and all kinds of things. 3:54 So we're doing a lot of consulting and work for hire, and we're developing some of our own technology in the production space and the edtech space. 4:03 And we just rebuilt our platform for our learning materials, and we have an app, and we have all, all sorts of things going on. So I don't know. There's more. The platform. [laughs] I'll leave it there. 4:15 And what were you talking about here? It was, like, uh, focused on AI and virtual production. Yeah. 4:19 So, uh, we were asked to do something relating to Unreal Engine, being that that is still the predominant kind of real-time visual development environment of virtual production. 4:31 There are other things that are now entering the market that also will do, um, real-time 3D graphics and also some of the photographic backdrops and, and things like video and, um, 3D volumetrics. 4:45 And we did something that was Unreal Engine and AI, which could have been all sorts of things, but we decided to focus it around, um, Gaussian splats, which are made with AI. 4:56 They don't necessarily use AI in the visualization or the rendering- Mm-hmm... as such, but they... 5:02 AI machine learning is used in the creation of a Gaussian splat, and then it becomes a f- sort of photographic, volumetric 3D photographic background, which you can use on a LED wall or a green screen or just in a, in anything. 5:17 We wanted to talk about the implications, what AI is, because I think it, it's become, like virtual production was, a kind of banner or umbrella term that catches an awful lot of things and is thrown around a lot these days to... 5:33 You know, e- everybody making anything these days has AI in their marketing plan. [laughs] If you make toasters or socks, you have AI socks. Also some of the applications of it and what it can mean, that- Yeah... 5:43 like, we need more, like, better language of defining- Right... what the use is. Yeah. So we sort of tried to s- And, and I actually studied it in 1998. 5:50 I did a computer science degree and studied artificial intelligence. I wrote... My, my textbooks were called The Turing Omnibus. Turing is famous for having the Turing test named after him or, or coming up with it. 6:02 That, that was in the f- 1940s. Uh, here we are, you know, some, some years later- [laughs]... uh, talking about AI still. 6:11 And, um, what we're now describing as AI is a, an increase in the capacity of what, what the average person can access in terms of computing power and an extension of what was possible, but also a kind of, another overused term, democratizing, uh, access to computing power. 6:29 That for a long time it's been very hard to talk to computers. 6:33 Um, it's in- initially involved-Electrical wiring and mechanical operations of, uh, only a few people in the world could do, and then it fast-forward to code, and now at this point where we can talk to them with language. 6:46 Um, so it's really giving people access to what used to be just geniuses. Now it's more laypeople who can do some of the things that only those people could do before. 6:55 And machine learning has graduated to the point where we can now use it. It does more things than it used to be able to, and more people can access it. 7:05 So I tried to start off by defining what AI means and what it is, 'cause some of it is just computer programming, and some of it is the language model stuff is being accessed by the logic that wraps it. 7:17 So we tried to start by saying that, you know, to help people understand that it's not scary. It's, um, it's quite accessible, and what i- what is it? What does it mean? What does AI mean? 7:28 'Cause we're all throwing it around a lot at the moment. And then show something, and not just talk, but show, uh, the production of a Gaussian splat. 7:36 And we then put it into Unreal Engine so that we could have it in an environment that is a thing you can move around in and, and use as... More people can use that than can use some of the traditional 3D tools. 7:48 Cinematographers are learning it. They can learn Unreal Engine enough to be able to use it for their own purposes without a team of experts. Bring in a Gaussian splat and navigate around it. Yeah. Yeah. 8:00 So we wanted to show how with my phone, which is an Android phone, and a laptop and, um, Unreal Engine and a Gaussian splat that you could do VCam on a photographic background and infer what you could have-- how much further you could go with that. 8:18 But it was presented on LED screen, so you could kinda see how if you've got those, that almost rudimentary equipment, um, you can do something that is virtual production that's extremely accessible to people financially and skills-wise. 8:34 Yeah, we talked a lot about Gaussian splats, like around NAB time, and that was a bit buzzy. But like- Yeah... are you finding that you're able to do final pixel shooting with Gaussian splats? 8:43 What are the limitations with Gaussian splats versus traditional, like polygons and- There's pros and cons to everything. 8:49 So you know, what I think a lot of people in virtual production are finding is that in order to put extremely good-looking backgrounds in, you have to have extremely good-looking material. Mm. 8:59 And lots of people talked about the gamey look of the background and saying that it's 'cause it's a game engine that's rendering it. It is not anything to do with that. 9:07 It's because people are putting game assets into an engine that look like a game, 'cause they're made for games. They're not made for production. 9:16 If you put production material into anything, it should look photorealistic. There's no reason why you can't put photorealistic things into Unreal and expect it to look photorealistic. 9:26 That said, to make those photorealistic assets is... It requires skill, it requires time, it requires access to the things. If you're scanning the real world and putting it in, it requires skill and access. 9:39 If you're making it from an artist, it requires a lot of skill, 'cause it's much harder to make things manually look photorealistic. And then with, with Gaussian splats, the, the pros are that they're quite easy to make. 9:51 They are, uh, very ph- they're photographic by nature. They are-- They give you not, not only... When you change the camera position, not only do you get parallax, but you also get lighting shift. 10:04 So you move, move your camera position, and you, your... Because of the, each point has an angle basis for its color. 10:12 So if you move, you see the reflection or the lighting, which all, all lighting hitting a surface is reflection. But you see-- you'll see if you're on this table, I can see a reflection of that light there. 10:23 So if I move in a ph- uh, photogrammetric scanned environment, that doesn't change. 10:29 If I'm in a Gaussian splat, I move the camera, the lighting shifts, so you get reflections, you get parallax, and they're quite easy to make. They're hard to edit. 10:38 They are not really relightable typically, although there are some hacks to being able to do it. So there's, you know, there's pros and cons to it. 10:46 With some of the traditional 3D approach, you can do those things more easily, but you don't get the lighting change, and they're harder to make. So I think Gaussian splats, yes, definitely people will be using them. 10:59 They're also really good for things like trip, fiddly bits of 3D object like plant behind you or a tree or s- those kinds of things which photogrammetry will turn into big nasty blobs. Very messy. Yeah. 11:09 [chuckles] And you know, that will look much closer to something that's more realistic because of the way they, they kind of train and learn what, what shapes each splat is and how they join together and the angles. 11:22 It's, it's easier to reconstruct more to- more photographically. So yeah, it... There's pros and cons to them, you know. They're not been around as long, so there's not as much- Yeah... edit capability. 11:31 It's harder to, like- Is that something that's gonna change and get-... animate one... developed even more as time goes on? Oh, for sure. I'm sure. Yeah, I mean, you can already see it happening. 11:39 Actually, I had a slide in, in a talk where I showed a splat being put into Houdini and animated, and now you can have four 4D splats, which is the time... The, the fourth dimension being time, you can animate splats. 11:54 It's still more practical to do it the traditional way, but you can make them move, you can deform them. 12:00 I had a silly Grogu Gaussian splat, uh, drop onto the floor, and its ears were wiggling, just to show that it was put into Houdini. It was simulated, and it was still a Gaussian splat. 12:12 It wasn't a polygonal object, which is what people actually thought it was. Yeah, I mean, it's... A place like Houdini is a very good place to work on that kind of material 'cause it's just a point cloud ultimately. 12:24 But there are m- where... The, the world of, the world I'm used to of polygonal 3D or volumetric 3D has 12:32 30, 40-plus years of development and tools and, you know, people spending time on, on making that aPlace that's workable. Gorgeous bless have not. 12:42 So I'm sure that it will continue to get-- become more malleable, more directable. But, uh, did that answer the question? Yeah. Yeah. 12:51 With generative AI and sort of the applications in virtual production, where do you see that going? Uh, crystal ball. Hang on. Let me [laughs] Or what have you seen even in, uh... 13:01 'cause I'm thinking also of just, like, hybrid approaches. 13:03 Like, I know I've heard of productions experimenting with gen backgrounds, real actors, or possibly kind of layering it and then running that through another process. Um, but what are you seeing? 13:12 I mean, it's a massive topic, so I, I'll, I'll do my best with it. AI has been in visual effects for quite a long time. It's just been only a few people in the world have been capable of using it, doing anything with it. 13:24 Now it's been being done for a lot longer than the hype has been describing it. Now we have, like, the things like face replacement and de-aging are becoming more accessible to people. 13:34 The-- But there's all the generative, like pure generative from words to a final picture or video or a sequence or movie, and then we're starting to see people experimenting with that and trying to make entire movies just purely from, from prompts. 13:49 And th- there's some coming out. Even this week, there was the plastic bottle video that went viral, the plastic bottle baby that was extraordinary. 13:59 It would've taken a huge amount of effort to do that in visual effects traditionally. 14:03 And, you know, that was made on a weekend by talented storyteller artist, but with a tool that allowed them to be able to do it very quickly and without needing a visual effects team. They had humans in it. 14:13 They were all doing human-like things and talking at the same time, and it was using Veo 3, the new Google video model. 14:20 And it, it looked amazing and it-- everyone would see it and go, "Wow, you know, this is how everyone's gonna make film from now on." 14:27 In the world of movie production, the two things we're trying to get a grip on is the two challenges it has is control and copyright. 14:35 Um, the copyright thing, you know, is, is an uncontrollable thing, but, uh, is starting to be figured out at least whether we're going to care about it or not or to what degree. Uh, that's a work in progress. 14:48 Control, these things are extremely sophisticated in w- in terms of what they can produce. 14:54 They're, they're meant to take a tiny amount of input and produce this really sophisticated looking output, and that, that is like a, you know, a few levers at the beginning to making something that would've taken a million levers traditionally. 15:09 You know, teams and teams of people, tons of time to create that output. Now one person, a few levers, and very sophisticated output. So it's possible. 15:20 In the world of movie making, usually in the world I've come from of seeing in dailies on Jurassic World with a VFX supervisor there giving you-- 15:28 zooming in on, on the eight pixels through broken glass and fire of a helicopter with a pilot of a shoulder and giving me notes about the wrinkles on their shirt that I can't see. 15:40 You know, that, that level of scrutiny, the words he used to describe that was like a prompt. You know, he was talking to me with words. 15:47 It's like a prompt to an AI, but it's a prompt to a human who's gonna go off and fix it. And I know because I know what to do to go and correct that little thing. 15:57 And if you're talking to a machine and you're talking about it, AI doesn't really know anything. It looks like it knows things, but extremely sophisticated. 16:06 It's very hard to just talk even in that situation with a V-VFX sup on a big movie and a team of artists who are all at the top of their game. They can still get that really wrong at just using words. 16:18 And he didn't do it with words. He zoomed in, isolated an area, drew a mark on it. Then someone typed that up, send it to me with all the visuals with it for like, uh, that much of a picture. 16:29 So [laughs] and there was a lot of the rest of the picture that was having notes. 16:32 So to cut a long story short there, the scrutiny with which that world goes through, we're either gonna keep being that scrutinous, in which case we need more sophisticated instruments than just word input. 16:44 Like words being the only input doesn't work with a team of world-class artists, and we use, we use imagery to communicate, and, uh, that's emerging in the AI world as well. 16:56 We're using imagery to communicate with the AI. We're feeding in reference and concept and notes and paint over and prompts and all of those things kind of in a similar kinda way to how we work with humans. 17:08 I think where it's going is [laughs] we're trying to get more and more inputs for the AI so we can explain ourselves better and we can ask it for what we want more specifically. 17:18 It's still a little bit like dice rolly and a, and a movie, like a film that you wanna make with it is kinda like, at this point it can make something really cool. Is it exactly what you want? No. 17:28 Can we accept that it's not? In some cases, yes, and in some cases, no. No. High-end movie making is still using the traditional method. 17:36 All those companies are rampantly researching how AI works, how it can inject into the traditional process and empower it or make it more efficient, maybe make some jobs go away. And also can we skip to the end? 17:47 And we're seeing companies emerging, um, calling themselves AI production companies. I-- Which is interesting to me and, and fine. Uh, I don't think the goal is to try to make everything with AI. 18:00 For me, the goal is to make stuff that's good, whether we need to... Like using AI, nothing, you know, I neither condone nor condemn it. 18:09 You know, I, I'm actually quite excited about it 'cause I've studied it for my entire life and I've used it a lot over the years, and I'm really actively pursuing it right now. 18:18 I don't think the goal is-- and people are obsessed with making a whole feature film out of AI. I'm like, why is that the goal? Why isn't it to make a good movie? And I get it. It's 'cause it's a race right now. 18:28 We're trying to be the first person to say, "This was made with AI." People have already done that. It's just not very good. 18:34 Um, [laughs] we're trying to make the first one where people actually pay for it, and they will say, "I enjoyed it, and it was all made with AI," which I still think is like a moment in time kinda thing where we can say, "We did it. 18:45 Okay, great." Do we need to make it with all of AI?For me, no. But again, I'm not black and white about it. I think it's go- it's already been empowering visual effects for a long time. 18:55 It start-- is going to, I think, change that industry, my industry completely, and I think we, we all have to learn how it works. 19:03 And I think there will be a lot of content that people make where it is that skip to the end, don't bother about the whole in-between part. 19:11 Let go of some of the scrutiny and control and just make cool stuff, which is fine. 19:15 But I think the movie makers who really, really, really want to make stuff and aren't making it because it's a hype train or because they wanna be cool or stand out or be this kind of first to the gate or whatever, who actually have something to say, wanna express, want to tell a story, I see it in doing all of the things. 19:34 It's gonna make tons of stuff that's really impressive, but nobody to sit through an hour and a half of it. 19:40 Or people in the professional world that I'm used to will make it, wield it to make their processes faster, or maybe make some things that they couldn't even make before as they try and figure out the legality and all that kind of stuff. 19:54 I, I think it's just gonna do all of it, and, you know, the-- ultimately, the consumer is the person who will decide which was good. You know, how do, how do we keep consuming this stuff? Is it a good story? 20:05 Is it emotionally engaging? Yeah. There's a- Does it connect with me? You gotta invest in it, right? 20:09 Like I have so little time right now with two small children and a business, and I have so little time to watch anything. Uh, the risk for me is not just financial. If I was to spend... In fact, that's the least risk. 20:20 The risk is time. I have to invest my time into watching something. 20:24 I don't-- A lot of people talk about, "Oh, you're gonna have generative movies that you ask it what you want to see, and then you watch a movie and maybe you're even in it." 20:33 I'm like, that's-- for me, that's the last thing I wanna do. When I sit down to watch something, I don't wanna have any agency in it at all. 20:39 I don't-- I would definitely not wanna be in it, and I don't wanna have anything to do with it being made to-- at this point in my life. That might change if I get more time in the future. 20:49 But I think those things will happen too. I don't-- and I'm not here to put that down either. I, I just personally don't want it. But maybe I'll be wrong when I get the ability to do it and I'll see it. 20:58 I think people need to want to be educated, informed, entertained, and all, all of these things touched by art, and that's why we make art. 21:09 But we've got into this place as a society where there's all the people who wanna make art, and then there's a whole bunch of business people that wanna make money and are trying to commodify this thing, as we've always done as a species, and they're not coming at it from quite the same angle. 21:26 And we- we're at this weird crossroads as a, as a, uh, as a species where we, we're trying to figure out, how do we continue with this? 21:33 And I think what AI does do, and what I'm seeing through our school, is as-- kind of as real-time did in, in some other ways, is empower people to make their own stuff more and not need this big kind of system that traditionally was required to make anything of, of length, money and some companies and distribution and all these kinds of things. 21:57 And we're seeing people come through the school even before the advent of AI with virtual production, make their own feature films. 22:04 Some of them even got into the theater, and that is a, an incred-- Like people making their own films, sure, been around for a while. 22:10 People making their own visual effects films is very new, and I think that's only gonna increase with this. I think there'll be a lot of stuff that gets put out that's experimental and it's cool. 22:22 I think there'll be, uh, all the traditional companies will try and get their head around it. 22:26 I feel like the whole entire model is going through a disruption that's nothing to do with AI, that's like everything to do with how do we make and distribute this anyway, uh, even before the advent of this technology. 22:37 And I think that they're, they're all struggling to, uh, to figure out what can they sell or make money out of. And the consumer, again, it's, it's always the consumer. 22:46 They, they're, they're gonna pay for things that they love, what they care about, what they need and want. They don't care about what corporations need and want. They care about what they wanna see. 22:55 And they don't necessarily think they wanna see something that they've had a ton of choice in, although maybe a, a niche or maybe a big niche. But I do think that it- it's gonna empower creatives to individuals. 23:07 I had an alumni event, and I'll stop in a second, but I had an alu- alumni event a couple of days ago. All these people came back. They were all, you know, into their careers, mainly. 23:17 They had a fifteen-year-old there as well. But there was, there was a lot of talk. 23:22 I was expecting because AI has come through with a lot of fear and, um, uh, con- condemnation, and I understand that and I sympathize with it. I was expecting that kind of talk 'cause we're embracing it. 23:34 We're, we're trying to help people figure it out and have been very cautious about it because of the polarity in the conversation. 23:40 But this whole group were all saying about how it enabled them to do things that they couldn't have done before. 23:45 They couldn't-- They could not have done it without funding, without a ton of people, without corporate injection or a massive kickstart or-- Now people are, you know, they're more empowered. 23:56 They're, they're exploring it. They're very curious. I'm cautiously optimistic about it. Uh, I think there's a lot of things that need to get figured out, and I think it will be all of the things. 24:05 I don't think it's gonna be... And I, and I have no idea where it's going. I will say that. [laughs] I'm like, I don't know. I've got certain things- Or how fast it's gonna get here. 24:12 [laughs] No, I mean, it's kinda here already. I, I think it's like, just what do people have the appetite for? 24:19 And do the people who are jumping on it and making things, will they keep doing that beyond the sort of novelty moment that we're in right now, where it's like, it's cool to see something? Wow. 24:29 People saw that baby bottle movie, and I did too, and I thought that was really cool. And people are making all sorts of really cool clips of things and shorts and starting to experiment with longer form. 24:41 I don't think the goal is to make a whole movie out of AI. The goal is to make good things that people want and that people care about and that, you know, people should do it from that perspective. 24:49 People will make everything, though. People will make horrible things and wonderful things and... 24:53 But I, I-- my hope is with it that it allows all of my community, the visual effects community, has made things for other people their whole lives, and now I'm seeing them making their own stuff kind of for the first time in mass. 25:05 Uh, and that gives me a lot of hope for it, that maybe they'll make things that people want, and they'll be the solution to the kind of the over-- I've made a bunch of remakes of things that were made thirty years ago in my career. 25:17 So I'm the last person that should criticize a industry that's making lots of remakes. But people are getting tired of that, and maybe this is the solution that, that it's gonna allow originality through again. 25:29 And I really will stop. [laughs] No, I appreciate it, Ed. Uh, where can people find out more about you and, uh, CGPro? Website, becomecgpro.com. Look me up, Edward Dawson-Taylor, on LinkedIn or wherever else. 25:41 I'm not very active on Instagram. Um, I'm on Facebook, although a bit less 'cause it's a bit weird these days. Um, LinkedIn is a good place to connect with me. 25:49 Becomecgpro.com is a great place to come join one of our open days and come have a chat. Yeah. Cool. Thank you so much. You bet. Thank you.