Tag Archives: emergent

Blog

Twitter’s mood predicts the stock market?

I can’t really reword this one to sound any less (or more) incredible, so I’m gonna go straight to quotes:

The emotional roller coaster captured on Twitter can predict the ups and downs of the stock market, a new study finds. Measuring how calm the Twitterverse is on a given day can foretell the direction of changes to the Dow Jones Industrial Average three days later with an accuracy of 86.7 percent.

“We were pretty astonished that this actually worked,” said computational social scientist Johan Bollen of Indiana University-Bloomington.

You and me both, Johan, you and me both… but then, it’s a weird old interconnected world we live in, isn’t it?

“We’re using Twitter like a psychiatric patient,” Bollen said. “This allows us to measure the mood of the public over these six different mood states.”

As a sanity check, the researchers looked at the public mood on some easily-predictable days, like Election Day 2008 and Thanksgiving. The results were as expected: Twitter was anxious the day before the election, and much calmer, happier and kinder on Election Day itself, though all returned to normal by Nov. 5. On Thanksgiving, Twitter’s “Happy” score spiked.

Then, just to see what would happen, Mao compared the national mood to the Dow Jones Industrial Average. She found that one emotion, calmness, lined up surprisingly well with the rises and falls of the stock market — but three or four days in advance.

As daft as it sounds on the surface, this is probably pointing at some sort of core truth; it’s pretty much established that markets are emergent systems born of human interaction, so why shouldn’t you be able to get an idea of where things are going by finding a way to sample the mood of the planet?

That said, I’d very much like to know how wide-ranging the Twitter sampling was: did they use multiple languages, for instance, or just English? I suspect that Twitter’s demographic in geographical terms is still very white, Western, male and middle-class, too; would these results be strengthened by using more data from wider sources, or has a sort of accidental cherry-pick taken place? (White Western middle-class males are more likely to be stock owners or investors of one stripe or another, I’m guessing, so there’s probably some sort of inherent bias in using Twitter as a sample source.)

Even so, I’m fascinated by research that treats human civilisation as a system-of-systems with observable properties, and the rise of social networking is probably the catalyst for this growing field. Whether knowing how the system reacts and correlates will allow us to control it more effectively is another question entirely, of course… feedback is a powerful thing, but as any guitarist will tell you, it comes with risks. 😉

Blog

The greying of Wikipedia

3 Comments

citation needed!Despite continued growth as one of the most-visited sites on the web, Wikipedia has a problem – it’s losing editors faster than it’s gaining new ones. Cue lots of veiled “told you so” from the Wall Street Journal [via /message]:

… as it matures, Wikipedia, one of the world’s largest crowdsourcing initiatives, is becoming less freewheeling and more like the organizations it set out to replace. Today, its rules are spelled out across hundreds of Web pages. Increasingly, newcomers who try to edit are informed that they have unwittingly broken a rule — and find their edits deleted, according to a study by researchers at Xerox Corp.

“People generally have this idea that the wisdom of crowds is a pixie dust that you sprinkle on a system and magical things happen,” says Aniket Kittur, an assistant professor of human-computer interaction at Carnegie Mellon University who has studied Wikipedia and other large online community projects. “Yet the more people you throw at a problem, the more difficulty you are going to have with coordinating those people. It’s too many cooks in the kitchen.”

What isn’t clear, at least from this article, is which editors are leaving. A few years ago, all you could find were articles complaining that Wikipedia had too many unskilled and uninformed editors, and that it was hence a valueless project; now that people are being deterred from fiddling because the cost of entry is too high for casual contributions, that’s the problem. C’mon, people; you can’t have it both ways.

Rather than unseating my faith in crowdsourcing, these developments at Wikipedia are pretty much in line with what I had expected to happen. The initial landslide of popularity was like a new frontier, and it inevitably attracted a lot of chancers and grifters – not least, I suspect, because the SEO Google-juice from outbound Wikipedia links is powerful stuff indeed. I’m inclined to see Wikipedia (and a lot of other web-based projects) as an emergent system, and this shedding of casual contributors makes perfect sense; not everyone cares enough to do it properly, and the system self-adjusts to exclude those low-value contributions. [image by mmetchley]

That said, Wikipedia isn’t completely emergent and spontaneous; the Wikimedia Foundation steers and directs it as it sees fit. But even so, it’s still surprisingly reliable by comparison to classically-produced encyclopedias… and those who accuse it of inherent bias have obviously never seen Conservapedia (which I’m not going to do the favour of linking to – just Google it if you fancy horrifying yourself with some ultra-conservative historical revisionism). Sure, it’s not perfect… but what is? I’d be interested to see a catalogue of the errors that a paper like the Wall Street Journal makes in the course of a year for comparison…

That said, there’s one statistic about Wikipedia that is fairly disappointing (though far from surprising):

A survey the foundation conducted last year determined that the average age of an editor is 26.8 years, and that 87% of them are men.

Um. Not so much a greying, after all.

Blog

Domed cities and super-squellettes

2 Comments

Here’s another classic science fiction trope being upgraded to serious proposition: the domed city. The Discovery Channel has apparently been doing a program about mega-engineering, and one of the subjects was a proposal to hide Houston beneath a dome to protect it from the effects of an increasingly erratic climate.

Houston dome concept

Sadly there’s not much detail about the hows and whys (they want you to watch the program, natch), and the sheer overload of Flash content on the DC site keeps crashing my browser. But the dome sure looks pretty – from the outside, at least. [via Technovelgy]

Meanwhile, if you want a more gritty and realistic look at the city landscapes of the near-future you should be tagging along with Bruce Sterling, who’s currently obsessed with emergent, repurposed and interstitial urban spaces and is producing a quality stream of links as a result. One of the latest nuggets is about the favelas of Caracas, Venezuela – built in and around a failed Modernist tower-block project and almost entirely maintained by its residents without government support or funding.

Blog

Keep watching the skies – tag clouds as predictors of emergent fads

1 Comment

relational tag cloudEvery day, I spend a couple of hours digging through my RSS subscriptions for interesting stories, some of which I use here at Futurismic and most of which I store away at del.icio.us as research material (you know, for those fiction pieces that I keep meaning to find time to write… ahem). [image by ottonassar]

I’m a big fan of tagging my links because it enables me to trawl through the stored pieces (mine, and other people’s as well) by context and related topics, but it turns out there’s a greater benefit – user folksonomies on social bookmarking sites can be used to track and predict emerging trends and fads using mass data analysis:

The researchers tracked different users and noted the submissions they made, as well as the tags used on those posts. Taking this data, they could see what tags were frequently used in correlation with one another. This created a “coocurrence network,” which assigns weight to tags based on how often the tag was used and how many different users applied it.

With this information, it was possible to conduct a random walk (stepping randomly from one tag to another) and note how tags that occur together can form an otherwise undetectable semantic chain. These tags, based on their association with one another, allowed the researchers to follow along as one popular trend gradually replaced its predecessor.

When comparing individual random walks with one another, researchers noted that tags that appear close together in a non-obvious semantic network were likely to be visited by the same user, and tags that were far apart were visited together less often. Although no individual user might be aware of following these obscure connections, they became obvious when the data was examined in bulk.

[…]

The applicability of Heaps’ law to Internet tags was noted in particular. Heaps’ law states that the number of distinct words used in a body of text grows sublinearly relative to the size of the text—the bigger texts have more diverse vocabulary, but there are diminishing returns as things scale up. Likewise, the number of unique tags on del.ici.ous and BibsSonomy grow nearly linearly relative to the total number of tags—that is to say, our interests and the vocabulary used to describe them grow directly along with the Internet. It isn’t all just lolcats and musical parodies, even though it might seem so sometimes.

This fascinates me, because it confirms as a real phenomenon something that I always dismissed as a fallacy born of close involvement; scanning close to a thousand RSS feeds a day from a variety of sources and covering a variety of subjects gives me a sense of being able to observe trends bubbling up out the web’s chaotic maelstrom. I get a real kick out of watching a story or meme moving from low-level niche sites into the wider world of the web, and seeing new obsessions gather popularity.

And talk about hindsight – if I’d thought about it, I’d have seen the economic collapse coming about six months or more before it bit in and shifted all my investments somewhere safer. If I’d had any investments, that is…

Of course, this sort of trend analysis could probably be used for profit or surveillance purposes as well as the more abstract goals of research and cultural analysis, but if you haven’t realised that the internet is the ultimate double-edged sword by now… well, you’ve not been following along with my links, have you? 😉

Blog

Shanty towns as architectural inspirations

Rio de Janeiro shantytownGOOD Magazine has a piece on architect Teddy Cruz, who plans to use the ad-hoc shanty towns of Tijuana, Mexico as the inspiration behind some new urban developments. The thinking is that what emerges out of necessity may actually have lessons to teach us about the efficient use of space and resources:

Behind the precariousness of low-income communities, says Cruz, there is a sophisticated social collaboration: People share resources, make use of every last scrap, and look out for each other.

[…]

Cruz’s plan aims to vault the income gap with developments on several lots that are integrated into the city. The developments will include 60 housing units, playgrounds, a market, urban agriculture, and job-training facilities, all managed by a coalition of nonprofit groups.

It’s certainly a nice idea, and I’d be the first to applaud any attempt to learn from emergent phenomena where human endeavour is concerned. But I can’t help but feel this might not work out quite as planned… possibly because the UK is littered with housing estates which were designed as self-contained communities, but which aren’t exactly examples of efficiency and harmony any more.

While there are surely lessons to be learned from shantytowns and other interstitial poor communities, I suspect the best lesson we can learn at present is that emergent systems are too complex to be copied easily. Necessity is the mother of invention, after all. [story via BoingBoing; image by Crucsou Barus]