Category Archives: Blog

Blog

How can a computer win at Jeopardy? Elementary, my dear Watson

This is not only an interesting story, but an engaging piece of journalism, and I heartily recommend you go read it: it’s an NYT magazine piece about Watson, an IBM artificial intelligence project headed by one David Ferucci that does something that artificial intelligences have heretofore been unable to do: beat human players at Jeopardy! [found in a tweet by @noahtron, which was retweeted by someone I follow who, regrettably, has slipped both my memory and my notetaking process – apologies for incomplete attribution]

I’ll pick out a few highlights for the short-on-time, but bookmark it for reading later anyway. We’ll start off with the methodology:

The great shift in artificial intelligence began in the last 10 years, when computer scientists began using statistics to analyze huge piles of documents, like books and news stories. They wrote algorithms that could take any subject and automatically learn what types of words are, statistically speaking, most (and least) associated with it. Using this method, you could put hundreds of articles and books and movie reviews discussing Sherlock Holmes into the computer, and it would calculate that the words “deerstalker hat” and “Professor Moriarty” and “opium” are frequently correlated with one another, but not with, say, the Super Bowl. So at that point you could present the computer with a question that didn’t mention Sherlock Holmes by name, but if the machine detected certain associated words, it could conclude that Holmes was the probable subject — and it could also identify hundreds of other concepts and words that weren’t present but that were likely to be related to Holmes, like “Baker Street” and “chemistry.”

In theory, this sort of statistical computation has been possible for decades, but it was impractical. Computers weren’t fast enough, memory wasn’t expansive enough and in any case there was no easy way to put millions of documents into a computer.

Those are no longer obstacles, of course, or at least not obstacles on the same scale. So, add multiple parallel algorithms, shake vigorously, and…

Watson’s speed allows it to try thousands of ways of simultaneously tackling a “Jeopardy!” clue. Most question-answering systems rely on a handful of algorithms, but Ferrucci decided this was why those systems do not work very well: no single algorithm can simulate the human ability to parse language and facts. Instead, Watson uses more than a hundred algorithms at the same time to analyze a question in different ways, generating hundreds of possible solutions. Another set of algorithms ranks these answers according to plausibility; for example, if dozens of algorithms working in different directions all arrive at the same answer, it’s more likely to be the right one. In essence, Watson thinks in probabilities. It produces not one single “right” answer, but an enormous number of possibilities, then ranks them by assessing how likely each one is to answer the question.

The result? Watson actually competes pretty well against players in the “winner cloud” of Jeopardy! performance, though it’s by no means cock of the rock. Not yet, anyway.

What made the article itself so enjoyable for me was the human story behind it – Ferucci comes across as a real Driven Man, striving to come first in a fiercely competitive and high-stakes scientific race:

Ferrucci refused to talk on the record about Watson’s blind spots. He’s aware of them; indeed, his team does “error analysis” after each game, tracing how and why Watson messed up. But he is terrified that if competitors knew what types of questions Watson was bad at, they could prepare by boning up in specific areas. I.B.M. required all its sparring-match contestants to sign nondisclosure agreements prohibiting them from discussing their own observations on what, precisely, Watson was good and bad at. I signed no such agreement, so I was free to describe what I saw; but Ferrucci wasn’t about to make it easier for me by cataloguing Watson’s vulnerabilities.

As with most AI projects, however, Watson only does one thing, though it (he?) does it pretty well. It’s a function with potential commercial uses (which is why IBM is still throwing money at Ferucci and team), but a general artificial intelligence needs to be able to do more than win at a certain quizshow format. The difficulties of producing a natural-language question-answering intelligence on a par with human learning were pretty neatly showcased by Wolfram|Alpha last year (which, despite being disappointing to the public, is a pretty impressive piece of work in its own right):

This, Wolfram says, is the deep challenge of artificial intelligence: a lot of human knowledge isn’t represented in words alone, and a computer won’t learn that stuff just by encoding English language texts, as Watson does. The only way to program a computer to do this type of mathematical reasoning might be to do precisely what Ferrucci doesn’t want to do — sit down and slowly teach it about the world, one fact at a time. […] Watson can answer only questions asking for an objectively knowable fact. It cannot produce an answer that requires judgment. It cannot offer a new, unique answer to questions like “What’s the best high-tech company to invest in?” or “When will there be peace in the Middle East?” All it will do is look for source material in its database that appears to have addressed those issues and then collate and compose a string of text that seems to be a statistically likely answer. Neither Watson nor Wolfram Alpha, in other words, comes close to replicating human wisdom.

So don’t go announcing the Singularity just yet, eh? Even so, it’s a pretty big leap that Ferucci and friends have made, and the practical applications should hopefully pay the way for more research. Weird times ahead… though Ferucci’s suggestion that Watson could replace call centre drones has a certain appeal.

Blog

John “Global Guerrillas” Robb interviewed

2 Comments

Regular readers will know I follow John Robb’s Global Guerrillas blog quite closely; Robb cropped up yesterday as an interviewee on Boing Boing, restating his case for turning our backs on our governments (who have, in many ways, turned their backs on us) and building grass-roots “resilient communities”:

BB: Do you see a diminishing role for the state in large-scale governance? Does this compel communities to do it for themselves?

JR: Yes, large scale governance is on the way out. Not only are nearly all governments financially insolvent, they can’t protect citizens from a global system that is running amok. As services and security begin to fade, local sources of order will emerge to fill the void. Hopefully, most people will opt to take control of this process by joining together with others to build resilient communities that can offer the independence, security, and prosperity that isn’t offered by the nation-state anymore. However, this is something you will have to build for yourself. Nobody is going to help you build it.

Robb’s is a potentially grim vision (and he appears to rather revel in that grimness from time to time, like any good gadfly); some commenters have pointed out to me that a pinch of salt added to Robb’s posts is a sensible precaution, and I’d agree, but I still think there’s a lot of useful stuff in what he has to say. That said, it’s good to question received wisdom, especially when it confirms what you already believe to be true… so via Technoccult, here’s a critique of Robb’s last book at Reason:

… Robb claims global guerrillas can successfully wage strategic war on nation-states. But a successful strategic war is one in which a guerrilla group attains its strategic goals. If global guerrillas really just want failed states, the world has no shortage, and Robb is correct. If they want the things guerrilla groups have always wanted—regional autonomy, a greater share of the economic pie, dominion over ethnic or sectarian rivals, an end to foreign occupation, social revolution, national control—it’s much harder to say that any global guerrilla group has yet been “successful.”

[…]

What most of the global guerrilla groups have managed so far is to not lose. It’s a truism of counterinsurgency that “guerrillas win by not losing,” but successful guerrilla movements eventually win by winning. It’s much harder for global guerrillas to “win” than Robb thinks, because most of these groups have larger goals than he acknowledges.

These peer-to-peer networks of resistance would be pretty easy to hijack, I suppose; we’re rather attached to hierarchies as a species, though whether that’s a predisposition or a psychological artefact is beyond my knowledge. So, what starts as a scattering of people who think of themselves as freedom fighters can be corralled together and steered by another group with a wider agenda and more resources… or maybe just a bigger axe to grind. But perhaps I’m naively assuming that most small insurgencies start as a valiant resistance to some sort of oppression. More research needed (my hourly mantra).

Still, Robb’s points about having to look out for ourselves as nation-states decline and stability decreases ring pretty true, even if they have a Mad Max-esque vibe of dramatic overstatement to them. Security can be offered to you (in exchange for taxes, or whatever else, and not necessarily delivered on when it comes to the crunch), but resilience you must make for yourself. Resilience can fail as well, of course, but then you can blame no one but yourself… perhaps that’s why we’re all so resistant to the idea?

Blog

Go read (or listen to) Brenda Cooper’s story at Clarkesworld

1 Comment

Hey, it’s hump day – you should probably reward yourself for surviving to the half-way point of the week. So why not celebrate with some new fiction to read?

Brenda Cooper, who writes the Today’s Tomorrows column here at Futurismic, has a story in the latest issue of the excellent Clarkesworld online zine; it’s called “My father’s Singularity”, and you should go and read it. If you’re too busy (yeah, right), there’s an audio option as well, so no excuses.

Blog

Ken MacLeod goes head to head with Annalee Newitz

No, not some sort of sci-fi celebrity blogosphere death-match scenario (though that might be kind of cool – we could clone Andy Remic and see how soon he could dismember himself with oddly-named axes!): BloggingHeads.tv publish video-conference interviews between notable figures in certain spheres of interest to the intertubes, and io9 head honcho Annalee got to have a good long chat with Ken MacLeod, Scots science fiction author extraordinaire (and, I might add, thoroughly nice bloke).

To quote Ken himself, topics covered include “politics, Craig Ventner’s synthetic organism, Scotland, The Night Sessions and The Restoration Game, near-future and far-future SF, and galactic princesses.” That’s my lunchtime entertainment sorted, then.

Blog

Computerising the music critics

2 Comments

Keeping with today’s vague (and completely unplanned) theme of critical assessments of cultural product, here’s a piece at New Scientist that looks at attempts to create a kind of expert system for music criticism and taxonomy. Well, OK – they’re actually trying to build recommendation engines, but in The Future that’s all a meatbag music critic/curator will really be, AMIRITE*?

So, there’s the melody analysis approach:

Barrington is building software that can analyse a piece of music and distil information about it that may be useful for software trying to compile a playlist. With this information, the software can assign the music a genre or even give it descriptions which may appear more subjective, such as whether or not a track is “funky”, he says.

Before any software can recommend music in this way, it needs to be capable of understanding what distinguishes one genre of music from another. Early approaches to this problem used tricks employed in speech recognition technology. One of these is the so-called mel-frequency cepstral coefficients (MFCC) approach, which breaks down audio into short chunks, then uses an algorithm known as a fast Fourier transform to represent each chunk as a sum of sine waves of different frequency and amplitude.

And then the rhythm analysis approach (which, not entirely surprisingly, comes from a Brazilian university):

Unlike melody, rhythm is potentially a useful way for computers to find a song’s genre, da F. Costa says, because it is simple to extract and is independent of instruments or vocals. Previous efforts to analyse rhythm tended to focus on the duration of notes, such as quarter or eighth-notes (crotchets or quavers), and would look for groups and patterns that were characteristic of a given style. Da F. Costa reasoned that musical style might be better pinpointed by focusing on the probability of pairs of notes of given durations occurring together. For example, one style of music might favour a quarter note being followed by another quarter note, while another genre would favour a quarter note being succeeded by an eighth note.

But there’s a problem with this taxonomy-by-analysis approach:

Barrington, however, believes that assigning genres to entire tracks suffers from what he calls the Bohemian Rhapsody problem, after the 1975 song by Queen which progresses from mellow piano introduction to blistering guitar solo to cod operetta. “For some songs it just doesn’t make sense to say ‘this is a rock song’ or ‘this is a pop song’,” he says.

(Now, doesn’t that remind you of the endless debates over whether a book is science fiction or not? A piece of music can partake of ‘rockness’ and ‘popness’ at the same time, and in varying degrees; I’ve long argued that ‘science fiction’ is an aesthetic which can partaken of by a book, rather than a condition that a book either has or doesn’t have, but it’s not an argument that has made a great deal of impact.)

This analyses of music are a fascinating intellectual exercise, certainly, but I’m not sure that these methods are ever going to be any more successful at taxonomy and recommendation than user-contributed rating and tagging systems… and they’ll certainly never be as efficient in terms of resources expended. And they’ll never be able to assess that most nebulous and subjective of properties, quality…

… or will they?

[ * Having just typed this rather flippantly, I am by no means certain that the future role of the critic/curator will be primarily one of recommendation. Will the open playing field offer more opportunity for in-depth criticism that people actually read and engage with for its own sake, or will it devolve into a Klausner-hive of “if you like (X), you’re gonna love (Y)”? ]