The Next Leap: How A.I. will change the 3D industry - Andrew Price

so show a hand, who here makes money somehow within the 3D

industry

, they got a job in a studio, yeah almost everyone who doesn't, what are they doing?, so I love this job, it's really fun and so childish. It's a wonder, yes, to bring something from my head to life, still, I really enjoy it, but I want to keep doing this, you know, because it's a fun job, but if you've been paying attention to the news and you've certainly I've seen reports about automation and artificial intelligence, the apocalyptic theories that we'll all be out of work, all sitting on the street, with nothing to do, and I always thought so, but that doesn't apply to 3D because we like what we're doing.

It's art and computers could never replicate art, but then you see a couple of stories like how machine learning was used to create high fidelity animations on Thanos' face, okay, and then you see an algorithm that

will

take the styles of famous paintings. in the story and I apply it to a photo and then I sit back and pay attention because I'm okay, I didn't think a computer could do that and then I wondered what are we looking at here. It made me wonder if these are the good old days when Disney's 2D animators were replaced by 3D animators.

Will we be replaced by racks of servers running AI and machine learning to automatically generate movies, so that's what I do? I am going to discuss in this presentation how I and automation could

change

the 3D

industry

. Quick note. I'm going to use terms like AI and machine learning and some people say no, that's not AI, you have to call it that and for all intents and purposes, it doesn't really matter what I'm talking about in the end, it's still the same result, right, It's software that does something that an artist does. Anyway, there is one guy who is very good at making predictions for the future. but yeah, Masanga, Jeff Bezos, who you know is in trouble right now for his treatment of staff, he wasn't very good at predicting, but for the last 20 years he's basically been ahead of the industry and that's why he said that with They often ask me what is going to happen.

change

in the

next

10 years I'm rarely asked what's not going to change in the

next

10 years and that's actually the more important of the two and he went on to explain that you know an Amazon customer of the future

will

never say "wish." the

price

s were higher or I love Amazon but I wish my packages took longer to arrive correctly so those two variables he knows for sure will be desirable in the future so really what's not going to change is what you should concentrate.

I don't worry about hypotheses so I thought that's cool, it's a great exercise and when applying it to 3D I think that any technology that improves things faster or cheaper will eventually become standard, it's inevitable that things will be implemented once that the studio executives realize that this is To save them money, they put it into practice. So what costs money right now, games cost money. So Raph Koster plotted some games from 1985 to today and found a worrying trend that every ten years the cost of games increases tenfold. so this is a logarithmic scale, it seems misleading as it's not that big of an increase, but each of those horizontal lines represents a 10x increase from the previous one, so basically each year it increases by 25% by 2020 , the triple average. a game can cost 200 million dollars so they are already more expensive than movies and it's getting hard to manage because everyone has to spend more to try to outdo each other and mobile games, yes they started out cheap, but that is also happening because it is now a saturated space, so the costs are too high and a large part of these costs need to be reduced, the assets are excessively expensive, so let's say, for example, that you are making a video game that It involves the street you have to go to. you have characters running down the street for whatever reason so you have to model a building okay and it's a pretty detailed apartment so you work pretty hard you might be able to do it in 12 hours so you have a texture to texture frequently .

It takes as much time as modeling, but let's say you do it in 10 hours and then in the middle of the work week you do it in 22 hours, which, by the way, is not reasonable because that assumes that you are productive 100% of the time. time. Usually about 50% of the time, but let's say you're 100% productive, so if you're in the studio, there's also multiple revision two to four times, you know, narrative changes, maybe they want it to have place in Paris. New Yorkers may want to have a tunnel go through the building for whatever reason, so changes have to happen repeatedly and they have to go back and review old assets, so at a cost of $60 an hour, the salary average of this building ends up being about three thousand nine hundred dollars, so when you look at a game like division, you put

price

tags on everything and it's very easy to see why games cost hundreds of millions of dollars, and this is a bit silly, because as we work, virtually none of these things exist in the real world, so we should be able to address this from the beginning because I think a lot of this cost is due to the fact that this is a flow of static job you are getting one. input to output ratio for one, if you want to make another building, you often have to repeat the work and sure you can start from the base of the other one, but often you have to do a lot of details in changing topology and then you have to read texture to still get a pretty similar result, so I think the first

leap

that will happen in the 3D industry in the next five to ten years is that procedural workflow is becoming common or the standard to say to address the problem of modeling a building to do it procedurally would look like this instead of modeling instead of building it yourself, you would set the parameters of what a building should look like within certain ranges, which is why our building should be as tall for this. height, it could have five to ten windows, two to three doors, it could have so many floors within reach and then you basically let that self-generate and that's how you reduce those costs, so this was actually a Given course. by Anastasia opera I think it's actually Dutch, I could be wrong, but she made a procedural village on the lake using Houdini and went on to explain that she thought the procedural ISM would really take away the creative aspect of it, but it actually forced her to understand what something does looks good because often you don't know, as an artist you just like to do things without thinking about it, but when you had to put it in a program you had to explicitly define what it looks like and that was a good exercise and then also , what you got from the machine on the other end were often ideas you hadn't thought of before, so it can actually be a huge advantage.

This is procedural modeling, obviously you need to texture it, in this example it is textured. I'm not sure how he made the texture of a button, so I run Polygon, who has heard of Polygon? Well, my advertising worked, so when we started Polygon two years ago, every texture on the website was captured with a camera and we would find a floor like that and then we said great, we need another floor, we would have to find another wooden floor, capture it, make it seamless, process it, so it was that one-to-one input to output, then we figured out the substance designer through which you would spend a lot more time generating. a node setup that does something 100% procedural, meaning there's no camera involved, you'd do it digitally, but once you've done it, it's very easy to create variations from that and we immediately realized the benefit of This, not only could we capture things that are very difficult to capture like marble wood, you have to go to people's houses and capture their floors, but it was also a huge cost savings, so basically that's all we we do now, many of our materials are now made with substance.

The designer says that for photo scanning, which is still best captured in four terrains, but yes, we have doubled the size of our team and the game studios have realized this too, which is why you look at the job offers and everyone hires substance designer artists, so of course, I have to apply this to a model, okay, mortal, you'll know the folds and crevices and all kinds of occlusions that you have to take into account, so the algorithmic was very good at have its brother substance painter software let you take which will bake all the maps for you and then apply the smart materials you got from the substance designer smart skins and things to add in grunge style and all that, so it's the number one texturing software in the world right now.

This very reason is saving the studio hundreds of thousands of dollars because it's a procedure when you go back and change something with the model, if you have the right pipeline it should automatically update with the texture, it's very very cool and finally I have the design of levels so I put the assets in it so I had the D as the start and the New York City example but let's say you're making a forest it's okay so far cry 5 they've made a lot of FARC rise up , five of them. actually, but previously, you know, you would make your forest, usually you would have particle systems and you would place things by hand, but the problem was that as the story changed over the months, the landscapes would move as they should. do it.

They have paths that go through different places and every time they did it they had to update in the same place and remove all the trees, this time they did it in a different way, they created an ecosystem, so they established rules to define certain trees in the West and plants would live in such a way that large trees would grow towards the center of the forests, small trees towards the edges under the trees, there would be ecosystems of smaller plants growing in the shadows, it was near a lake, there would be different plants there , it would update automatically. according to the altitude, so if they get tired, there will be smaller trees and then they can incorporate useful tools to quickly build roads and buildings and by the way, the whole talk is definitely worth watching if you get the chance.

So I think this will be the future, this workflow, procedural modeling, materials, texturing and world building. All the grading and texturing materials are pretty well integrated into the industry now, it's pretty standard, but I think shaft modeling and construction will take the next spot. At the moment, Houdini seems to be very good at this, but I really hope that the jocks wherever he is, yeah, tease him all over the place, because I really hope this can blend in, so that was number one. It was machine

leap

number two, okay, so this is where it starts to get interesting, so traditional software generally works like this: you have an input, you have a photograph, you take it into Photoshop, and then you want to apply a filter to it. you apply the action to the photo and then you get a result, it's very predictable, you know what you're going to get and obviously it also requires a lot of manual work to adjust things as you need machine learning, it's different, you start with an input that could evaluate it , you can apply an appropriate action and then most importantly compare it with others in your training data set and then be able to judge whether it is good or bad, sometimes a human has to take that step if they don't try again and this using the method is a gross oversimplification of machine learning.

I'm not a computer scientist. I didn't go to college, but this is what helped me as an artist understand this process and basically the result you get from it is usually the same. much better than traditional software, so the key point here is that it has the ability to learn and improve over time. Now what you need to make that happen are huge data sets and fast hardware, so I think five years ago you started. I know I heard the news train about machine learning taking away all of our jobs and everyone was excited and scared and then nothing really happened and I think a big part of that is because there isn't enough data and there isn't fast enough hardware either. .

I think we're getting to the tipping point now and you're actually starting to see some consumer software that's already using machine learning, which I'll show you in a second, so one thing is that machine learning is very cool. for Dien noise, okay? Everyone is familiar with noise: you render something that has a grainy look and D noises usually smooth it out. A machine learning denoiser will do this very very well, so this one is proprietary to nvidia. I assume it is the same one usedfor the RTF. allowing them to perform ray tracing in real time. By the way, the way your r-tx can do it is to render a sample on each frame of the game and then apply additional noise on the same frame, so is it doing this in real time and how the heck is it able to take that square blue, that noisy blue square and reading anything from it, is a testament to how machine learning works.

It's absolutely crazy that it's doing that, which is why blenders run denoise cycles so far. As I know, it has nothing to do with machine learning or AI, so it would obviously fail in this situation, so I really think Blender needs to get into machine learning and AI. Disney and Pixar also own one and are trying to solve the frame flicker issue because it's a common problem with D noises, but also creating artist tools to do it. Nvidia owns most of the papers on this and I think it's because they realize that there is a lot of money in it because not only will every renderer need to use their denoiser but also every camera manufacturer because if you have low noise sensitivity the light and there's all this grain well they can solve it with D noises, so one thing is good at another thing is evaluate, so you have a small, small, JPEG image, you're fine, you like the art, but I need it to have the twice that size, this is a lift, so as a test.

I took this image that I'm making at the moment of a kitchen, rendered it at 50% and then using AI gigapixel from Topes Labs, raised it by 200% so that it now looks like it was rendered at 100% and then compared it to a real 100% render and if you look at it there isn't much difference and obviously there is a little bit more detail where it was actually rendered at 100% but not a lot and certainly not enough to qualify as a four times render so This really caught my attention. I thought, wow, and this is at the consumer level.

You can actually buy it and get the trial version, or it's $99 or something like that to buy the full thing. This is now available. This is a start, I think where the industry is going and there are also other uses like for example motion capture, so I think very soon we will eliminate mocap suits and expensive studios because now you can capture just from the raw format. video, so there are no censors, this is not a special camera, it's just a simple video and the algorithm, whatever it is, I don't know what I'm talking about, was able to figure out where the bodies are.

The most impressive thing is that he's able to guess the occluded part like the one on the other side of that arm because he can't see it, but he's guessing and he's doing an amazing job, so I think in the future they won't have mocap suits. film the actors doing whatever and that'll be it and then there's this other one which, to be honest, I don't know how it works, but they filmed a dog in a mocap suit for an hour running and jumping and then using neural networks it to be. that is, they translated it to player movement in a game and the most impressive thing is his transition from one behavior to the next, from running to walking and jumping, it is incredibly fluid, there is also very little foot slippage and there is also another example with a human being and adapts to the terrain and jumps on things.

I think this is inevitable. I think you'll start to see it in games, so basically it's a lying machine. We'll start to see it in our software in the future. You'll start now. I know you'll have an action in Photoshop and you'll say, "Oh, that looks really cool," you find out that it was machine learning built into Photoshop Premiere, most Autodesk products, I think I really think Blender needs to really get involved in that, believe. It's the future and I think all the Silicon Valley companies have realized it and yes, there is a quote from the guys who made the famous facial animation.

If you're not using machine learning in your software, you're doing it wrong. I think it is very true. So elite number three is machine-assisted creativity. This is really fun because it's something I always thought computers could never do. Creativity is something very human and it is true that intention is very difficult for a computer. but you'd be surprised what it can actually do to help you, so as an example, as I mentioned, I'm working on this kitchen scene and I wish I could have gotten the tutorial app before coming to Amsterdam, but I ran out. of time so it's going to take another month but I'm working on this scene and you know every time I do something I get to a certain stage like this where it's okay but I know it could be better if I tried more ideas. so you just stop throwing things at the scene and see if it looks better than before, so you can change the lighting, add some little blinds and there might be a marble on the left, you know, and then try it, like adding food. to the table there is no food adding some boxes, there are no boxes in the refrigerator and you just have to keep doing this and I have realized that like my workflow, this actually consumes 50 to 70 percent of the time of production because there are a lot that I haven't shown you another 20 renders where you just have to test these things and it takes a long time, sometimes you have to model something like what will look good with a bottle of wine.

I have to model a bottle of wine or you buy it or whatever, then you place it and then you have to render it and that's a really long process, so if there was software that could do this for you without you having to do all the effort to add it in there in order for you to make the decision of whether this is good or not, that would be incredibly powerful, so hey, this is a really impressive document, it gives you an outline of an object, the photo in which the schema was based and then it will be generated.

Lots of ideas for you and it works very, very, very well, so it really helps to see it while it is generated, so this is the facade of a building, the only thing that was given was the top row and then these are the shoes and if you paused it. at any point in this, any of these boxes here could have a unique design, so I think this is kind of the future. I think what we're going to see is that you'll create a character, for example, and then you'll put it into the software and try out a bunch of ideas.

Different clothes. Different types. This is a use for an environment. I posted it and I think this will definitely be part of many creative meetings in the future. You can actually try it online if you go to that web address. It is based on a different paper technology, but it is very similar, so I drew. This cat, very simple, looks like a cat and then I press a button, it reminds me of those things where it's like the dad finishes the kids' drawings and he's a really good artist and he just makes it look like reality is obviously not .

It doesn't look very good because my drawing is shit, but obviously you can see that everything will be a finished product, but it would definitely be a starting point for many concept artists, so I think it's a sign of the times we're going to start see this talking about does anyone recognize these celebrities? Does anyone know who they are? You don't want to tell me a name, whatever name you think it is, they look like you, they look familiar, right? like music, it's like a soccer ball, so I've seen it in something, whether you're a musician or an actor, well, I see your imaginary, they don't exist, they were generated.

This is a separate document where you are fed with a bunch of images. gives a really impressive result, so obviously these are realistic examples that you can get. You know if you like a game where you need to have 20 different NPCs and you don't know, you just take pictures of people that match the region. the one you're making the game for, I need 20 Mongolian NPCs, you just downloaded a set of Mongolian faces and then you got unique faces, there's no similarity infringement or whatever, you just have uniqueness, but it also worked for other things that don't.

I know how he did this because these photos have perspective, but this was also generated in these bedrooms, which are bananas, that's exactly what I mean with my kitchen, if you could put it in and then generate a bunch of things, so I think that That's what's going to happen. It will have your mood board with all your ideas for your environments and generate ideas. Oh, and this is where it gets really weird, really spooky for Halloween. What if you didn't even have to provide it with images? And if? you just wrote your idea like this bird is red and brown with a stubby beak what's real that's not it's not pulled from a file library the way it was generated so amazingly the way this What works is that it was trained to recognize what the features are and what something redbird looks like, okay, that's what a redbird looks like, then when it realizes, when you write that down, it creates the blob the way it should look. and then there is a second pass where he adds the There are details about that and it is crazy, it is the closest thing to witchcraft.

I don't understand it, in fact, like many articles I've seen. I am very skeptical. I think they are cheating. It's something that I'm not saying because this can't be true, but it is, if it's as good as it is and it's like today. I think this was actually from 2016. I think it's inevitable. You'll have a creative meeting with a director who like, okay, let's create some ideas, ancient city roads through the Middle Canyon in the background, they spit out like twenty different ideas and then one of those ideas you give to your one conceptual artist to I developed it.

I don't know, I think it's inevitable, right? and then another really interesting one is the style transfers. Anyone has seen them. By the way, you transfer a position. Basically, you give it a bunch of images that you like. I want this Claude Monet style, whatever, and then you put an entry on this is my photo that I took. on my vacation and then he spits it out, which is crazy and it also works for video, which is really cool if you look at the newspaper that they filmed like in I don't know somewhere in Germany, people walking down the street and it's a painting but every frame moves and is very strange and the craziest thing of all is that it actually fooled 39% of art historians into thinking they were looking at something that was actually painted and not generated.

I can't think of anyone with more desire. the machine failed and an art historian, but it worked well, it worked, so my prediction is that artists will use machine learning to explore new ideas. I think that's going to be the future, that's going to be part of our process, first of all. Let's start going to the studios and it'll be in the consumer content and we'll all be like, Yeah, how did I make this scene. First, I generated a ton of ideas. I think this is how it will work, so these are the expected changes. I predict that will happen within the next five years.

I think procedural workflows will become standard. I mean, textures definitely already exist with substance designer and substance painter. I think it will happen with that level design and with the modeling. Machine learning will creep slowly. It works its way into all the software we use and will do a lot of really tedious technical things and I think we'll also get creative assistance from the machines, so I can feel the vibe in the room, it's a bit like oh no what. Are we being replaced? So people had this thought actually in 1997, when Kasparov was defeated by the IBM Deep machine, that's what it was called, it was the first time a computer had been a human being and people thought that was it. chess and it could also be the end of humanity, it was depressing and they felt that chess was dead and then Kasparov realized that, as you know, if the computer has access to all the known potential moves of all the hundreds of thousands of previous games and is able to work from that, it should be fair if the human also has access to that, so he created something called advanced chess whereby the human player has access to the same information that the computer has and will listen to the computer and will occasionally override.

It's like you would do with GPS navigation when you know something is wrong based on what the computer is giving you and today the best player is a human to machine counterpart, a purely AI chess will win in a 142 game tournament where as human and machine 152 so there is a purpose for your life, it's not just all machines and what's more, since then the number of grandmasters in the world has doubled, so it wasn't the end of chess, it was good for chess, so I think it's exactly the same that will apply to our industry. I don't think these new technologies won't make artists superfluous because you know you give this to a monkey and he won't be able to do it.

To do something good, you will press, press some buttons and yes, it could generate something like an image, but what it does is the human being puts intention into it, so I feel that based on these, these coatings are what are at greater risk. Jobs are those that are labor-intensive, low-skilled, and repetitive. The great jobs like mocap cleaner, rotoscoping,cleaning retopo screens, they make hundreds of these jobs basically the kind of thing that is usually outsourced to a labor floor in India, where they have I've been told to rotoscope these shots from the next Hollywood movie in the one who simply tries to do manual work.

I think that's something that will definitely be replaced. Secure jobs are those that involve critical thinking and tasks with broad skills where you know how to use different software and different solutions in the right way and specific tasks, so they are 2 or they are too small to bother automating, so A good generalist art direction project management program is obviously to oversee a lot of the scripts and programs that people and freelancers use because there will always be a market for people who don't want to use any software and instead want to pay you, so really the average jobs are undesirable drudgery, they are the things that no one wants to do except us.

I have to do it because it is part of the job, the real art, I think it is safe and, failing that, if that in itself does not It was enough to make you feel better, the forecasts for the 3D industry are crazy, it's exploding for almost every single one of them. These sectors are expected to grow in the next five to ten years and that is not true, even taking into account virtual reality. One report says 3D rendering and visualization is expected to see growth of twenty-five point five percent per year through 2025 compound annual growth or basically doubling by 2022 and quadrupling by 2025, so the number of people at this conference within four years it could double by 2025, quadruple that I think the 3D industry is about to explode, so yes, maybe each In the studio there would be a reduction in the drudgery of the young people they are hiring to do the things they no one enjoys it anyway, but instead there could have been five studios around it, so that was anticlimactic, wasn't it? thanks guys