Malcolm Gladwell had an interesting piece recently in The New Yorker called The Formula, about companies which attempt to use neural networks to predict the commercial success of songs and movies based on an anaysis of their component parts:
In a small New York loft, just below Union Square, for example, there is a tech startup called Platinum Blue that consults for companies in the music business. [...] It has a proprietary computer program that uses “spectral deconvolution software” to measure the mathematical relationships among all of a song’s structural components: melody, harmony, beat, tempo, rhythm, octave, pitch, chord progression, cadence, sonic brilliance, frequency, and so on. On the basis of that analysis, the firm believes it can predict whether a song is likely to become a hit with eighty-per-cent accuracy. [...] The head of Platinum Blue is a man named Mike McCready, and the service he is providing for the music business is an exact model of what Dick Copaken would like to do for the movie business.
[...] [O]ne morning recently McCready sat down and opened his laptop to demonstrate the Platinum Blue technology. On his screen was a cluster of thousands of white dots, resembling a cloud. This was a “map” of the songs his group had run through its software: each dot represented a single song, and each song was positioned in the cloud according to its particular mathematical signature. “You could have one piano sonata by Beethoven at this end and another one here,” McCready said, pointing at the opposite end, “as long as they have completely different chord progressions and completely different melodic structures.”
McCready then hit a button on his computer, which had the effect of eliminating all the songs that had not made the Billboard Top 30 in the past five years. The screen went from an undifferentiated cloud to sixty discrete clusters. This is what the universe of hit songs from the past five years looks like structurally; hits come out of a small, predictable, and highly conserved set of mathematical patterns. “We take a new CD far in advance of its release date,” McCready said. “We analyze all twelve tracks. Then we overlay them on top of the already existing hit clusters, and what we can tell a record company is which of those songs conform to the mathematical pattern of past hits. Now, that doesn’t mean that they will be hits. But what we are saying is that, almost certainly, songs that fall outside these clusters will not be hits—regardless of how much they sound and feel like hit songs, and regardless of how positive your call-out research or focus-group research is.”
Epagogix is the name of the company that's developed a similar formula for determining hit movies:
If you were developing a $75-million buddy picture for Bruce Willis and Colin Farrell, Epagogix says, it can tell you, based on past experience, what that script’s particular combination of narrative elements can be expected to make at the box office. If the formula says it’s a $50-million script, you pull the plug.
[...]
[A Hollywood studio gave Epagogix] nine unreleased movies to analyze. [They] worked only from the script—without reference to the stars or the director or the marketing budget or the producer. On three of the films—two of which were low-budget—the Epagogix estimates were way off. On the remaining six—including two of the studio’s biggest-budget productions—they correctly identified whether the film would make or lose money. On one film, the studio thought it had a picture that would make a good deal more than $100 million. Epagogix said $49 million. The movie made less than $40 million. On another, a big-budget picture, the team’s estimate came within $1.2 million of the final gross. On a number of films, they were surprisingly close. “They were basically within a few million,” a senior executive at the studio said. “It was shocking. It was kind of weird.” Had the studio used Epagogix on those nine scripts before filming started, it could have saved tens of millions of dollars. “I was impressed by a couple of things,” another executive at the same studio said. “I was impressed by the things they thought mattered to a movie. They weren’t the things that we typically give credit to. They cared about the venue, and whether it was a love story, and very specific things about the plot that they were convinced determined the outcome more than anything else. It felt very objective. And they could care less about whether the lead was Tom Cruise or Tom Jones.”
[...]
The team once gave a studio a script analysis in which almost everything they suggested was, in Hollywood terms, small. They wanted the lead to jump off the page a little more. They wanted the lead to have a young sidekick—a relatively minor character—to connect with a younger demographic, and they wanted the city where the film was set to be much more of a presence. The neural network put the potential value of better characterization at an extra $2.46 million in U.S. box-office revenue; the value of locale adjustment at $4.92 million; the value of a sidekick at $12.3 million—and the value of all three together (given the resulting synergies) at $24.6 million. That’s another $25 million for a few weeks of rewrites and maybe a day or two of extra filming. [They] ran the numbers and concluded that the script would make $47 million if the suggested changes were not made. The changes were not made. The movie made $50 million.
[...]
That was the thing about the formula: it didn’t make the task of filmmaking easier. It made it harder. So long as nobody knows anything, you’ve got license to do whatever you want. You can start a movie in Africa. You can have male and female leads not go off together—all in the name of making something new. Once you came to think that you knew something, though, you had to decide just how much money you were willing to risk for your vision. Did the Epagogix team know what the answer to that question was? Of course not. That question required imagination, and they weren’t in the imagination business. They were technicians with tools: computer programs and analytical systems and proprietary software that calculated mathematical relationships among a laundry list of structural variables. At Platinum Blue, Mike McCready could tell you that the bass line was pushing your song out of the center of hit cluster 31. But he couldn’t tell you exactly how to fix the bass line, and he couldn’t guarantee that the redone version would still sound like a hit, and you didn’t see him releasing his own album of computer-validated pop music.
The danger of these systems (if they are as accurate as Gladwell portrays, which I don't have any reason to doubt beyond the normal exaggerations of entrepreneurs selling themselves and their goods) is not, in my opinion, that the people who develop them aren't artists -- in any collaborative entertainment or art enterprise there are artists and there are technicians and there are money-people and they are all a necessary part of the project. In one sense these predicive systems simply provide another tool to be used, although this tool is probably more useful to the business side of show business than to the creative side.
No, what concerns me are those 60 clusters of hit songs that Platinum Blue found, and their movies equivalents, and I want to know more about them -- have there always been the same clusters?, do they wax and wane over time?, are there other locii that are not ascendant now but used to be, and may be again in the future?, what is the lifespan and lifecycle of a cluster?, do the clusters change shape and position over time? -- stuff like that. I suspect, but I don't know (of course) that given the constraints of human psychology and physiology there will be a limited number of potential hit song or hit movie clusters, but that the shape, center and intensity of those clusters will change over time as tastes change under the influence of things like fashion, technology, and the demand for (apparent) novelty working against the comfort of the recognizable. That's been the way things seem to be falling out in the sociobiological sciences, that we are simultaneously constrained by our heredity and free within those limits -- the clusters exist, but they can mutate or float somewhat as the zeitgeist changes.
Except, of course, if you reinforce the current state of the clusters by making songs and movies which land with some precision in their center, thus maximizing the chance for profit, but, at the same time, inhibiting the ability of the clusters to flex and change. The effect of this could be to fix the clusters in their current state, or, at least, to prevent them from evolving or changing over time. The general acceptance of the predictive systems in the arts and entertainment would make it even harder to find financing to make movies or songs which push the envelope and influence the shape and position of the clusters by their (surprise) success. Art, it seems to me, lives somewhere in the interstices between the clusters, and the continued existence of the entertainment industries depends on art or art-like projects which ignore (some of) the rules and yet turn out to be well-liked or widely influential.
It would be an interesting project, for instance, for Epagogix to remove from its database every film made before 1977, and then put the script for Star Wars through its analysis. Would it predict the great success of that film and the existence (and money-making ability) of its progeny, grandchildren, cousins, nieces and nephews? Or would the system, as I suspect, merely encourage the making of the sort of films that were successful up to then, before Star War existed?
absolutist
aggresive
anti-Constitutional
anti-intellectual
arrogant
authoritarian
blame-placers
blameworthy
blinkered
buckpassers
calculating
class warriors
clueless
compassionless
con artists
conniving
conscienceless
conspiratorial
corrupt
craven
criminal
crooked
culpable
damaging
dangerous
deadly
debased
deceitful
delusional
despotic
destructive
devious
disconnected
dishonorable
dishonest
disingenuous
disrespectful
dogmatic
doomed
fanatical
fantasists
felonious
hateful
heinous
hostile to science
hypocritical
ideologues
ignorant
immoral
incompetent
indifferent
inflexible
insensitive
insincere
irrational
isolated
kleptocratic
lacking in empathy
lacking in public spirit
liars
mendacious
misleading
mistrustful
non-rational
not candid
not "reality-based"
not trustworthy
oblivious
oligarchic
opportunistic
out of control
pernicious
perverse
philistine
plutocratic
prevaricating
propagandists
rapacious
relentless
reprehensible
rigid
scandalous
schemers
selfish
secretive
shameless
sleazy
tricky
unAmerican
uncaring
uncivil
uncompromising
unconstitutional
undemocratic
unethical
unpopular
unprincipled
unrealistic
unreliable
unrepresentative
unscientific
unscrupulous
unsympathetic
venal
vile
virtueless
warmongers
wicked
without integrity
wrong-headed
Thanks to: Breeze, Chuck, Ivan Raikov, Kaiju, Kathy, Roger, Shirley, S.M. Dixon
recently seen
i've got a little list...
Elliott Abrams
Steven Abrams (Kansas BofE)
David Addington
Howard Fieldstead Ahmanson
Roger Ailes (FNC)
John Ashcroft
Bob Bennett
William Bennett
Joe Biden
John Bolton
Alan Bonsell (Dover BofE)
Pat Buchanan
Bill Buckingham (Dover BofE)
George W. Bush
Saxby Chambliss
Bruce Chapman (DI)
Dick Cheney
Lynne Cheney
Richard Cohen
The Coors Family
Ann Coulter
Michael Crichton
Lanny Davis
Tom DeLay
William A. Dembski
James Dobson
Leonard Downie (WaPo)
Dinesh D’Souza
Gregg Easterbrook
Jerry Falwell
Douglas Feith
Arthur Finkelstein
Bill Frist
George Gilder
Newt Gingrich
John Gibson (FNC)
Alberto Gonzalez
Rudolph Giuliani
Sean Hannity
Katherine Harris
Fred Hiatt (WaPo)
Christopher Hitchens
David Horowitz
Don Imus
James F. Inhofe
Jesse Jackson
Philip E. Johnson
Daryn Kagan
Joe Klein
Phil Kline
Ron Klink
William Kristol
Ken Lay
Joe Lieberman
Rush Limbaugh
Trent Lott
Frank Luntz
"American Fundamentalists"
by Joel Pelletier
(click on image for more info)
Chris Matthews
Mitch McConnell
Stephen C. Meyer (DI)
Judith Miller (ex-NYT)
Zell Miller
Tom Monaghan
Sun Myung Moon
Roy Moore
Dick Morris
Rupert Murdoch
Ralph Nader
John Negroponte
Grover Norquist
Robert Novak
Ted Olson
Elspeth Reeve (TNR)
Bill O'Reilly
Martin Peretz (TNR)
Richard Perle
Ramesh Ponnuru
Ralph Reed
Pat Robertson
Karl Rove
Tim Russert
Rick Santorum
Richard Mellon Scaife
Antonin Scalia
Joe Scarborough
Susan Schmidt (WaPo)
Bill Schneider
Al Sharpton
Ron Silver
John Solomon (WaPo)
Margaret Spellings
Kenneth Starr
Randall Terry
Clarence Thomas
Richard Thompson (TMLC)
Donald Trump
Richard Viguere
Donald Wildmon
Paul Wolfowitz
Bob Woodward (WaPo)
John Yoo
guest-blogging
All the fine sites I've
guest-blogged for:
Be sure to visit them all!!
recent listening
influences
John Adams
Laurie Anderson
Aphex Twin
Isaac Asimov
Fred Astaire
J.G. Ballard
The Beatles
Busby Berkeley
John Cage
"Catch-22"
Raymond Chandler
Arthur C. Clarke
Elvis Costello
Richard Dawkins
Daniel C. Dennett
Philip K. Dick
Kevin Drum
Brian Eno
Fela
Firesign Theatre
Eliot Gelwan
William Gibson
Philip Glass
David Gordon
Stephen Jay Gould
Dashiell Hammett
"The Harder They Come"
Robert Heinlein
Joseph Heller
Frank Herbert
Douglas Hofstadter
Bill James
Gene Kelly
Stanley Kubrick
Jefferson Airplane
Ursula K. LeGuin
The Marx Brothers
John McPhee
Harry Partch
Michael C. Penta
Monty Python
Orbital
Michael Powell & Emeric Pressburger
"The Prisoner"
"The Red Shoes"
Steve Reich
Terry Riley
Oliver Sacks
Erik Satie
"Singin' in the Rain"
Stephen Sondheim
The Specials
Morton Subotnick
Talking Heads/David Byrne
Tangerine Dream
Hunter S. Thompson
J.R.R. Tolkien
"2001: A Space Odyssey"
Kurt Vonnegut
Yes
Bullshit, trolling, unthinking knee-jerk dogmatism and the drivel of idiots will be ruthlessly deleted and the posters banned.
Entertaining, interesting, intelligent, informed and informative comments will always be welcome, even when I disagree with them.
I am the sole judge of which of these qualities pertains.
E-mail
All e-mail received is subject to being published on unfutz without identifying names or addresses.
Corrections
I correct typos and other simple errors of grammar, syntax, style and presentation in my posts after the fact without necessarily posting notification of the change.
Substantive textual changes, especially reversals or major corrections, will be noted in an "Update" or a footnote.
Also, illustrations may be added to entries after their initial publication.
the story so far
unfutz: toiling in almost complete obscurity for almost 1500 days
If you read unfutz at least once a week, without fail, your teeth will be whiter and your love life more satisfying.
If you read it daily, I will come to your house, kiss you on the forehead, bathe your feet, and cook pancakes for you, with yummy syrup and everything.
(You might want to keep a watch on me, though, just to avoid the syrup ending up on your feet and the pancakes on your forehead.)
Finally, on a more mundane level, since I don't believe that anyone actually reads this stuff, I make this offer: I'll give five bucks to the first person who contacts me and asks for it -- and, believe me, right now five bucks might as well be five hundred, so this is no trivial offer.