How AI is reshaping filmmaking?

Over the last 100 years, the film industry has come a long way. From recording the Roundhay Garden Scene in 1888 to filming Avengers Endgame, we see the positive impact of technology on this industry.

And with the recent advancements in artificial intelligence, it is needless to say that AI also made its debut in the glamorous film industry. Leading production houses and filmmakers have integrated AI into different aspects of filmmaking, like writing scripts, VFX, animations, graphics, and predicting the success of a movie. However, humans still dominate the production side, including cinematography, art direction, set design, lighting, sound design, costume design, and makeup, among other areas.

History of AI in Filmmaking

AI has been associated with the film industry for a long time now. Of course, a few years back, we didn't have the AI we know today.

For example, when Peter Jackson was involved in the pre-production of the Lord Of The Rings trilogy, he was looking for software that could make a huge battle scene possible where tens of thousands will fight in a battleground. That's when Weta FX (previously Weta Digital), a New Zealand-based VFX company founded by Peter Jackson, Richard Taylor, and Jamie Selkirk, worked with Stephen Regelous to create software named Massive.

Massive software is powered by computer animation and AI that helps in creating computer-generated virtual crowds or groups, primarily for battle scenes. The software was extensively used in the scenes of the Battle of Helm's Deep of the second film of the trilogy, The Lord of the Rings: The Two Towers (2002).

Our recent advancements in generative AI have taken things way further. Anybody with no filmmaking knowledge or experience can also create a short film using new AI tools.

How is AI Revolutionizing Filmmaking?

There are various aspects of filmmaking where AI is making its contribution, and some of those key areas are:

Visual effects

We all know Thanos, right? But did you ever wonder how did they create it in the first place? The answer is simple- they leveraged AI to turn Josh Brolin into a purple monster.

AI can now automate and improve various processes of producing visual effects while saving time and money at the same time. Here's how it is making a difference:

• It can automate tasks that require a huge amount of labor like match-moving, tracking, rotoscoping, compositing, etc.
• It can generate and add photorealistic objects to a scene, no matter if the camera or the character is moving.
• AI is making the expensive process of motion capture affordable by removing the need for physical activities which included actors wearing suits and trackers in a studio, hundreds of experts, and a bunch load of hardware.
• It is also revolutionizing digital human synthesis where you can easily generate digital humans, avatars, and other real-time animation.

Screenwriting

Needless to say that AI has done wonders when it comes to generating creative content. From creating plot ideas and developing characters to creating narrative arcs and writing dialogues, AI is helping storytellers explore millions of creative possibilities and outcomes within a matter of a few seconds.

Apart from this, AI can also read a script, recognize plot structures, and story patterns, and make suggestions on how to refine the whole narrative and writing process. Some AI tools have been in the limelight, like Sudowrite, AI Screenwriter, Jasper, Shortly, Writesonic, and more.

Pre-production

Yes, pre-production is the most crucial part of any filmmaking project. It is basically the initial phase where everything is meticulously planned, including creating a screenplay, and storyboard, doing recce, deciding on budgets, casting, blocking scenes, designing sets, getting permits, scheduling shoots, renting equipment, and a lot more.

While the 1st assistant director plays a key role in this pre-production phase but with the advent of AI, things are changing around the set. AI is contributing at different steps of this phase. It is helping in predicting costs, reducing unnecessary overheads, scouting locations based on the descriptions in the story, writing and visualizing the story, and scheduling shoots as per the actor's availability.

Predictive forecasting

AI is influencing how production houses, studios, and distributors make strategic decisions. Since AI is fed with a large amount of data, it can analyze numerous aspects like script, storyline, historical data on box office performances, demographics, psychographics, trending genres, popular actors, and social media sentiments. These are used to forecast the commercial success of a film. These predictive models can project how much revenue is the film capable of generating.

Apart from that, based on these predictive analytics, AI also helps production houses and studios make informed strategic decisions like finalizing a release date, crafting marketing & PR strategies, and choosing a key demographic to market the film. AI provides numerous other valuable insights that help studios and filmmakers refine their strategies and reduce their investment risks. One of the prime examples of a predictive forecasting tool is Cinelytics which is trusted by some of the Hollywood giants like Warner Bros, Sony Pictures, and Lionsgate.

Casting

AI is unlocking a new level of efficiency in the casting process. Certain AI algorithms are analyzing actor profiles, their previous performances, and how well were they received by the audience to gain insights that can help during casting.

AI is also offering recommendations for specific roles based on acting skills, physical appearances, and on-screen chemistry with co-actors. It is also helping casting directors figure out how an actor will look in a certain role.

Here's a video created by AI on the Lord Of The Rings if it was directed by Wes Anderson. This video not only recreates LOTR in Anderson's visual style but also visualizes how Anderson's preferred casts will look in these legendary roles.

Editing

Post-production workflows are witnessing a revolution in the wake of AI. Film editing has become more efficient due to the new AI-powered editing tools that help editors boost the whole process as they can analyze large chunks of footage, identify key moments, and facial expressions, and select shots accordingly to build an intense visual narrative.

These new AI tools are also proficient at automating repetitive tasks like syncing audio, color grading, etc. They can also suggest alternative editing ideas based on the story narrative and this feature opens up brand new possibilities of how will your story look if it was edited in a different way. Even one of our favorite editing tools Adobe Premiere Pro is now powered by AI and it comes with numerous new features:

• Color match
• Auto reframe
• Music and dialogue blending
• Morph Cut
• Scene edit detection
• Removing objects from the video
• Remixing music to match video length

Creating music

The whole idea of experimenting with new sounds, composing and producing music is slowly changing since AI entered the game. Artificial Intelligence, in itself, is capable of creating new music, melodies, soundtracks, rhythms, and more.

Apart from that, AI is also playing a wonderful role in automating tasks like audio mixing, mastering, sound engineering, and more. Collaborations of AI and humans can really push all the creative boundaries known to mankind and create something unique.

My two cents

I feel there are two perspectives of looking at how AI is changing the film industry. The first one is where AI helps us unlock new creative possibilities and make the whole process more calculated and efficient. And the second one is where AI is basically like Skynet from Terminator who are taking over human jobs and slowly taking control of this planet.

I personally lean towards the first perspective- AI collaborating with creative individuals can take storytelling and the whole entertainment industry to another level. It can unlock efficient and time and cost-saving ways of making films and help humans create more content to meet the growing needs of better content consumption.