Article begins
In spring 2021, I took a role with a large media company I will call Streamer to work on a domain they called “content intelligence.” The goal was to help the technology (software and machine learning) teams improve and expand the role of data supporting creative executives—the people who decide what content to make or buy for distribution on their digital platform. Part of the challenge was that the creative executives were not always enthusiastic about the idea of data-driven decision-making in relation to their jobs, which they viewed as an alchemy of instinct, talent, personal networks, vision, experience, and skill.
For creative executives, while there was value in tracking projects and a constant appetite for “information,” efforts to use data in evaluating potential projects represented a “decision by computer” that was overly prescriptive, rooted in the past, and potentially damaging to their relationships with creative partners. The tension between statements about the lack of value in data alongside the desire for information represented an intriguing and potentially valuable signal: What makes data recognizable as information? How can or should data operate in the context of creative processes?
In addition to widespread skepticism about the capacity for data to usefully inform content decisions, creative executives were frustrated by the ineffectiveness of our tracking systems. These were something they had initiated, requested, and even written up requirements and visions for. And yet it still wasn’t really what they wanted it to be. They explained to me that they needed a good tracking system because Streamer received so many pitches from so many entry points that it was important to know if, say, this idea had been submitted before to someone else or if another team was considering a similar project. It really shouldn’t be hard to do this one simple thing, and yet from the perspective of the creative executives, it just wasn’t working as well as it should.
As creative executives failed to see data as information when presented as a decision-making tool, data tooling also grappled with accurately capturing the nuances of Hollywood production at scale. Even the seemingly simple objective of capturing and tracking what projects were and were not in flight could be less obvious than it might seem and bring into relief the ambiguity of human processes as data.
What’s in a decision?
When Streamer first expanded its focus from distribution to production, about a decade ago, Hollywood and creators in other venues were reluctant to work with the company. Both the business model and the company were eyed with suspicion. Streamer overcame these reservations in part by offering higher-than-usual industry payments (sometimes described to me as the “Streamer Tax”) and by offering deals that were direct-to-production. Streamer would commit to producing an entire project or full season of content based on a pitch or initial idea. This was distinct from network and cable television, where most deals commit a studio to a pilot (or single episode), after which a decision is made on whether to proceed with more. By 2021, when I joined the company, Streamer had transformed itself into a powerhouse in Hollywood and a major player in content production around the globe.
By this time, other producers had entered the fray, some of them with equally deep pockets and several that held name recognition and large backlogs of popular content. The competition was intensifying. Largely in response to this threat, Streamer had become increasingly concerned with maintaining and improving the quality of their content. The riskiness of committing to large, expensive projects on the basis of early-stage ideas was seen as a barrier to this objective and as no longer necessary: they were no longer the new kid on the block, no longer willing to pay the Streamer Tax. While Streamer still preferred to make full seasons rather than pilots, they began avoiding direct-to-production deals for early-stage ideas in favor of development deals in which creative partners receive money to evolve and refine an idea. This enabled creative executives to work with the (potential) creative partners, developing and assessing the relationship (which they saw as critical to successful projects) and the idea before making a full commitment.
Creative executives emphasized over and over that a pitch is vastly different from a finished film or series.
While the creative executives I spoke to liked the new strategy, the software and technology systems in place for centrally tracking projects across the company had not kept pace. Designed to track proposals that were either accepted or rejected, they had no way of capturing deals that were partial or contingent. This meant that data in the tracking system could be inaccurate, which in turn made it hard for other parts of the organization to know where things stood, accurately account for spending, or anticipate resources needed in planning.
Pinning down development
Talking to the software and technology teams tasked with building these tracking systems, we realized that while we knew development was a thing we needed to solve for, we didn’t know what the actual problem was and for whom. We didn’t know how development worked, when it began and ended, or who needed to know what about it, when, and why.
With an eye toward variation, we conducted a series of case studies on shows that were either in active production or had recently been completed: a large-budget full-length feature film in the United States, an animated series produced in Asia, and a serial drama created in the Middle East. For each of these, we established a simple timeline using data we had, marking down the period between when a pitch was first recorded and where it was clear that the project was greenlit (approved) for active production. We then asked employees who had signed off on that approval to tell us about what had happened in between. We supplemented these case studies with conversational interviews with creative executives and other people involved in the process and who depended on the information.
Several things quickly became clear. First, while some elements of development, such as script writing, were relatively formulaic (the process, not the writing), development could also include a wide range of other activities that in turn depended on a range of variables, including the location of the production, the structure of the industry in that region, the resources available to Streamer in that region, the genre of the project, the availability of the talent attached to the project, the storyline or subject matter, and even the portfolio of other potential projects in the pipeline. Second, although a development phase tended to have a clear beginning, the transition from development to something else was harder to pinpoint and comprised different definitions and activities for different workstreams. Third, the point at which a decision was made about whether or not to pursue a project operated independently of any specific activity and was surprisingly difficult to pin down.
A tipping point of confidence
The activities that took place in these development phases were shaped by the purpose of having a development phase to begin with. From the perspective of the creative executive teams, development deals as a strategy were entirely about increasing confidence and decreasing risk.
Although projects do not, despite what creative executives like to say, “come from anywhere,” they do come in a wide range of formats, configurations, and degrees of progress. A solid majority come as pitches—early-stage ideas that may include an outline of the story or storylines, a cast of characters, and a sense of the visual style. Some of them come with talent (actors, writers, musicians, producers) attached, but many do not or have gaps. Often, they are little more than concepts.
Creative executives evaluate project pitches along several vectors. These include aspects intrinsic to the idea: Is it a compelling story? Does it have interesting characters and plotlines? They also include aspects that are extrinsic to the idea: Does it fit with current objectives (perhaps we want to make more content with strong female leads or more content focused on sports or are looking for something to replace a show that is ending and will appeal to that particular audience)? Is it too close to other content we have in the pipeline or on the platform? Does it come from a creative team that we already have a deal with or are interested in cultivating? Do we have the relevant resources available at the right time?
Development, it turns out, is different things to different people or from different perspectives.
Creative executives emphasized over and over that a pitch is vastly different from a finished film or series. Given the costs and complexity of getting from pitch to finished product, creative executives stress the importance of the people making the project: how much faith do we have that this particular creative partner can execute the vision? Development deals offer production studios the opportunity to work with a creative team, testing that relationship, validating the feasibility of the vision, and understanding its likely costs. Except in the case of unscripted content like documentaries or reality shows, such deals nearly always involve a writing component and there is a somewhat standardized contract set by the Writers Guild for what development looks like for writers. Other aspects of development depend entirely on whatever open questions the team has in relation to any of the vectors described above, and are thus highly variable and often idiosyncratic.
The drama series we looked at, for example, included a storyline about a disabled character. Creative executives evaluating the project were concerned about inadvertently perpetuating stereotypes or otherwise poorly handling issues around disability. As part of the development process, a consultant was hired to ensure that the storyline avoided misrepresentations or ableist tropes in the writing of the character and their dialogue. The project was also in a region where Streamer does not own its own facilities but partners with local production companies. To answer some key questions about the physical feasibility and the visual look and feel of the show, a production partner was paid to explore locations and to develop a preliminary production budget.
Because feature films tend to be more actor-name driven than television series, the team evaluating the feature film project, a romantic comedy, wanted to ensure they had the right people in the lead roles. Given the celebrity status of the actors and their busy schedules, this was a complicated Rubik’s Cube of lining up the actors, timelines, and an extended wrangling of budgets and locations. Once they had identified one lead character, they had to find a second lead who was not only a good fit for the first but also available at the same time. There were some concerns about location availability and cost, so as part of the development phase the production team spent time (and money) evaluating alternate locations.
Animation is highly stylized and as much about look and feel as about dialogue or casting. For the animated series in our case studies, development focused on the audiovisual style of the characters and the action. The process included the composition of music, sound effects, and artwork, and even the production of short animated sequences to explore and demonstrate these choices.
In all these contexts, the objective from the perspective of the creative executive teams at Streamer was to move toward a tipping point of confidence. This point of confidence was not dependent on the delivery of all this work, but rather on making sufficient progress to the point where the team felt secure in making a decision to greenlight a project or to let it go.
Projects did not move beyond this grab bag of activities into active production without a positive greenlight from the creative executive. Decisions to pass on a project were generally entered into the central tracking system. In theory, decisions to greenlight a project were captured in a memo compiling the assessment of the project from the creative executive team as well as production, legal, and business analysts. These memos, which we used in selecting our case studies, were accessible across the company but not all positive decisions were captured in them, and creative executives sometimes delayed the memo well beyond the decision point. Where stakes were high, and sometimes at the behest of the external creative partners, creative teams sought to keep the project under wraps for as long as possible. Finance, programming, and other partnering organizations might only discover a project was a go when it moved into a new production phase.
A prelude to production
Development is also used to refer to a specific stage in the production process. This phase can and often does extend well beyond the decision to greenlight a project. Production teams told us that development ends when preproduction (or sometimes production) activity begins. Generally, this is at the point where above-the-line talent (such as actors, directors, producers, and writers) has been settled and is marked by the staffing up of the below-the-line crew and the logistics of initial shooting, including contractual arrangements.
In truth, writing and revisions—activities considered development from a production perspective—often continue well into active production and beyond. This is particularly characteristic of scripted series, where there are multiple episodes. Some episodes may be in active production while later episodes are still in development. It is also the case that preproduction, which can involve significant financial commitments, generally does not move forward until a project has been greenlit by the creative executive team. And yet, as with the Middle Eastern drama series, aspects of preproduction could be wrapped into the evaluation phase, while other aspects of development could continue well after a decision to commit to a project.
Members of the production team that participated in our project didn’t particularly care whether development activities took place before or after the decision to greenlight a project; they cared about knowing which projects were likely to go into full production so that they could plan their resources. Because a go decision could have immediate implications for their own workloads and facilities, these teams really wanted an assessment of the confidence level of the creative executive team, even before the decision to greenlight or pass. By staying aware of the probability of a particular show being approved, they could apply these assumptions to better estimate what resources they might have for other projects, which in turn could affect budgets, which in their own turn might affect not only project timelines but whether other projects could be made without stretching capacity.
A financial headache
From the perspective of finance, the frustrations and definitions were slightly different again. Film and media production deals include detailed budgets worked out as part of the dealmaking, and changes to these become part of the work of creative executives whose role it is to ensure projects stay on budget or have clear rationales for changes. Because development deals are exploratory and may include whatever the decision-makers need, and because new questions may arise in the process of development that then require additional work, development takes place as an open spend with no set budget and no specific limit. As one might imagine this is not a happy place for people who work in finance.
For the three projects we looked at, all of which were ultimately approved for production, money spent on development was later wrapped into the overall budget approved as part of the production deal. In these production budgets, expenses are linked to production phases such that some of those expenses are accounted for as development, but others (such as work paid to the partner production company in the case of the drama series) are considered preproduction costs and retroactively applied to that part of the accounting. Had the same project been dropped, those costs would have been counted as development. In this way, what is and isn’t development from a finance perspective depends in part on whether or not the project moves out of the development phase and into preproduction.
The lack of predetermined limits alongside the retroactive allocation of spend created a clear challenge, exacerbated by the lack of accurate tracking in the centralized software systems and the lack of visibility and consistency in the unofficial tracking systems. We discovered that the finance teams had developed their own parallel tracking system that they populated by capturing information from informal communications with creative executive teams or other means of sleuthing in their efforts to more effectively track, gauge, guess, and plan.
Developmental differences
Development, it turns out, is different things to different people or from different perspectives. For the creative executives tasked with making go/no-go decisions, development is a period of early exploratory work that allows them to make progress on a project until they can make a confident decision. It ends when confidence one way or another reaches a tipping point and a decision is made to fully commit or let a project go.
For the production wing, development is a phase of work that focuses on evolving the vision for the project that comes before preproduction, when concrete plans are laid. But the two can and often do overlap. What is and is not development is a question of the activity: script writing is development, contracting a location is preproduction.
For the legal department, development characterizes certain kinds of legal agreements (generally with writers) that have a prescribed set of deliverables. Development ends when the obligations captured in the contract have been fulfilled. From their perspective, the greenlight decision is irrelevant to the development agreement.
For finance, development is an open-ended, uncontrolled spend that can only be fully accounted for after the fact, with specific line items belonging to different categories depending on the go/no-go outcome.
Headaches aside, most of the people we spoke to did not see development as especially unclear. The challenge for most was that because it wasn’t being tracked in our systems, they found it hard to plan ahead. They didn’t perceive the problem as definitional and had no issue with understanding what development was from their point of view and how it might be different from another. For them, the issue was procedural.
Data thinks differently
From a data perspective, the basic problem was not procedural but definitional. With an objective of tracking for everyone, the different ways that development worked and was defined meant that there was no obvious or simple way to just track it, because “it” was variable, unstable, contingent, and contextual. In data systems, the mechanisms for tracking are reasonably straightforward and can be accomplished in a variety of ways, from human data entry to system-generated data attached to some other action in the system. Unlike humans, however, data does not do multivalent contextual thinking. Data, to be effective, requires consistency and stability. A thing is either a cat or not a cat, human or not human, go or no-go, development or not development.
Arguably, today’s large language models appear to do a lot of complex reasoning and can produce conversational responses that look and feel very human. But most data contexts nonetheless rely on some variation of is and is not. Our more complex data models can determine the likelihood of an object being it or not it based on a percentage (e.g., a specific image is x percent likely a human, the most appropriate next word in a sentence is y percent this and z percent that), but underlying such percentages remains a presumption of binaries: yes or no, one or zero.
What began as a seemingly simple question of better understanding development so that we could insert it into our tracking system turned out to be much more complicated. Should we create different terms for the different aspects of development? This could account for the different processes, but also required introducing new terms that could lack coherence for human users. Should we force fit a single definition, and if so, which one? How should we balance the need for clear indications of status, spend, and resources needed to facilitate effective planning from a business perspective with the variability of actual development activities? How should a data tracking system track something as squishy as “confidence” in ways that are meaningful and useful? How could we convince creative executives to shift from a system as elastic and familiar as a handwritten list in a notebook to something that finance, legal, and production teams could use to track and anticipate needs? How could we give those teams the data they needed while also allowing for the confidentiality demands of high-profile celebrity partners?
It now seems entirely possible to imagine data-driven AI engines writing, if not masterpieces, then solidly entertaining B movies.
The case studies did not resolve these questions, but in uncovering the shape of the problem we enabled the technology teams to see the challenge more clearly, to begin imagining and discussing potential solutions, and to evaluate the implications and complications of any given approach.
Unraveling the nature of development as it took place at Streamer also highlights the kinds of decisions that data and technology teams make in creating systems to capture and track these processes. Far from being obvious, automatic, or neutral, data systems require the transformation of squishy multivalent human activities into concrete, nonoverlapping values.
So should data be writing movies?
The development project was a side project to my primary focus of content intelligence—exploring ways to use data more effectively in making decisions about what projects to make at Streamer. It sheds a different angle of light on the tension between creative executive thirst for more information, their frustration with the tracking system, and their strong aversion to data-produced recommendations. It reframes those tensions as rooted in recognition and translation and the respective structures that shape how humans and data systems work in the world.
Creative executives and other participants had trouble understanding why it should be hard to track projects because for them it is not hard to understand that development can mean a few different things. Organizational partners were adept at translating the nuances of a project that was in development as a production phase as differentiated from development as a financial and legal framework representing a commitment to explore an idea before making a further decision on whether or not to finalize a deal and make the project into a show. Data tracking systems largely cannot accommodate much less interpret contextual meanings that depend on who is speaking and who is asking. What is obvious to a human speaking to another human is complicated to capture from a data point of view.
At the same time, the reason that the organization wanted to track these things in the first place is because humans have a hard time keeping large numbers of disparate facts—such as the status of hundreds of different projects and hundreds more potential projects—in mind at any given time. Humans tend to digest overwhelming amounts of information and experience into a more manageable sense of trends or even instincts—the gut feeling that helps creative executives sift promising from lackluster projects. Data systems by way of contrast can accommodate immense amounts of data that can be used to identify patterns and predict outcomes that are hard for humans to do with the same precision. The shortcomings of data modeling, and especially their tendency to replicate the biases, injustices, and inequalities of the data they are trained on as well as those of the technologists and organizations that create them are well documented if not sufficiently accounted for. What I am interested in here are the structures and perceptions that make it difficult for data to account for humans and for humans to recognize themselves in data—why is this so hard?
At issue is not simply that data requires precision and humans do not, or even the idea that technological structures structure capabilities and possibilities, but that the ability to make use of a thing requires a certain amount of recognition of what that thing is and how it works, and also a degree of translation from what that thing can do to what that thing can do for me or for us, and what it can’t. This is more than data literacy as the ability to read and understand data visualizations. It demands a deeper analysis of the ways that data structures intersect and interact with human processes in a particular context.
Today, while I, along with most of the media world, talk about “data” as if it were all one thing, data technologies are changing. New ways of ingesting, organizing, analyzing, and training data are creating extraordinary breakthroughs that are at once exhilarating and unnerving. It now seems entirely possible to imagine data-driven AI engines writing, if not masterpieces, then solidly entertaining B movies. But it is also true that we still struggle to resolve seemingly simple questions of human processes into terms a data system would find recognizable and vice versa.
To use data better, not only in creative endeavors like filmmaking but in human projects more broadly, we need a stronger focus on understanding how humans think, how data “thinks,” and how we translate between the two. In the months since we did the development study, I have increasingly found it useful to think of my work as data ethnography. Admittedly, this is a bit of a fiction in the sense that data systems are produced by humans, but different data structures process and organize information in different ways that in turn shape the meaning that they are able to make of the world they encounter through interactions with other data systems and humans. A bit like cultures, they shape how we perceive and make sense of the world.
Illustrator bio: An Pan is a multimedia designer, illustrator, and culture lover. He is currently a designer-accessory to Chinese consumerism but works with a big dream of decolonizing design. He enjoys traveling and doll collecting.