datacombs

I had an idea a few weeks ago about using an online directory such as Yahoo or the Open Directory Project to help classify and organize bookmarks. You could connect to a bookmark server, and it would arrange your bookmarks into categories or whatever. Fortunately, the ODP provides their data as RDF, so I was able to parse it and index by URI. The information is now available through a XML-RPC interface. You can find more info here.

Ok, after posting my last post, I did some googling and found some other LJ members also interested in social network analysis type stuff. One of them is phyxeld who posted a perl script to get users with common interests here. Careful, the code is scary (perl makes my eyes bleed.) Anyways, I wrote some CGI scripts to wrap around it and put it up on the web. Check it out here. WARNING: I’ve modified the script to sleep for 3 seconds after every interest query, so if you’ve got 100 interests, it’s going to take AT LEAST 300 seconds to run. So open it up in another tab/window, and let it go. If there are any problems with running the scripts, let me know. It’s kinda fun to mess around with, but I really wish LJ offered a web services interface or something to get interests and other info.

There’s a field of study called social network analysis that I’m kinda obsessed with.

I don’t know if I’ve linked to any projects that dabble in the field, but I’m going to now. Social network analysis is analyzing the patterns that appear in social networks, usually applying graph theoretic techniques. I joined friendster recently hoping they’d offer up your network in a machine readable form. Unfortunately, they don’t. But there are some people who screen-scrape the friendster page, and do shit with the results. Here’s another project mapping his friends to a globe. The first guy has a link to some other ideas. Graphing networks of friends is cool and all, but I think we could take it further. Both LiveJournal and friendster allow you to specify interests or hobbies. I know a lot of people nowadays like to go on about how you’re more than the sum of your interests, your possesions don’t define you, etc. Baloney. If you find somebody who shares a large subset of your interests, or even a good number, there’s a good chance they’ll have other interests that you would be interested in.

So, how to apply this? Well, the easiest thing to do is a simple intersection of the interests on friendster or LiveJournal, and do something like assign a rating depending on how many interests are matched. LJ could have a different new “possible friend” everytime you login who shares many of the same interests you do. Or you could use it to introduce you to new hobbies/interests. Kinda like Amazon’s techniques. If you wanted to get more complicated, start using bayesian networks or some other pattern recognition technique to find matches.