March 2009 Archives

Fast and the Furious Twitter Hilarity

 
Last night I sent this tweet: "Finally watching The Fast and the Furious: Tokyo Drift."  Literally within a minute or two TokyoDriftMovie starts following me on twitter.  His bio says:

I'm looking for other people who enjoy The Fast and The Furious: Tokyo Drift Movie as much as I do, maybe that's asking too much. You at least have to enjoy it.

So does he (Rob Jones) have a bot set up that follows anyone who mentions the movie?  His profile links to an amazon store page where you can buy the movie.  Is he doing this for many movies?  If so, he takes really great pride in the details of his work--just check out his twitter stream, e.g. "I stopped keeping track after seeing Tokyo Drift 100 times..."

Duck Duck Go Architecture

 
I often get asked what Duck Duck Go "runs on."  This post basically answers that question by outlining the major moving parts that serve queries, i.e. its architecture.  I'll detail in another post what, in particular, makes it fast, i.e. tunables and other specifics.

Caveat: this architecture was designed for maximum query speed for our initial soft launch.  While also somewhat designed for eventual scalability, we don't have that much traffic yet (though we are growing at a nice clip).  So don't take this as advice like you might get at High Scalability.  It's really just for your amusement.  However, my last startup did have some scale (relatively speaking of course) so I know a bit about what I'm doing...

  1. DNS served by DNS Made Easy.  I used to serve it myself via djbdns, but DNS Made Easy is faster, makes it easier for me to deal with fail-over, and cheap.

  2. All requests come into nginx. I used to use two instances of Apache, one for dynamic requests and one for static files.  But nginx is faster, uses less memory, and is more stable.

  3. If a static file, nginx serves it directly, e.g. the home page.  It's really good at that.

  4. Otherwise, nginx checks my memcached store.  I hadn't used memcached before this, and find it a big win.

  5. If not in memcached store, nginx proxies to FastCGI processes that are running in the background.  I hadn't used FastCGI before this, as I always had used mod_perl with Apache. 

  6. The FastCGI processes are managed by daemontools (as is memcached).  At first I was worried about stability in these processes, but it hasn't proved to be an issue yet.

  7. Internally, the FastCGI scripts are written in Perl and run by the FCGI::Engine Perl module.

  8. The Perl scripts access a PostgreSQL database (when needed) to retrieve our zero-click information, among other things.

  9. The whole thing runs on FreeBSD.

  10. For fail-over and scalability purposes, I have EC2 images that replicate the above except that they run on Ubuntu (since, at the time, FreeBSD wasn't available).

  11. All of our site icons and zero-click info images are hosted on S3.

  12. We also reference some external YUI JS files.

Any questions?  

Also, I'd love any feedback on this architecture.  I'm always looking for ways to speed it up!

Update: additional comments can be found here.

Twitter: Please Charge me for Biz Tweets instead of Suspending my Account!

 

Short version: I violated twitter TOS, so my account was suspended (as it should have been).  However, I think what I was doing was worthwhile.  So what I suggest is a new, paid tweets section (akin to direct messages in functionality, but you pay to send them).  So if the receivers never wants to see them, they don't have to.  But if they do, the paid tweets are one-click away...  

***

Last night the Duck Duck Go twitter account was suspended. I was sleep deprived from the birth of my son, but yet still awake being a human baby monitor. So I quietly used the computer to pass the time (like I'm doing now!).

Lacking SSH access (hospital wifi blocks port 22 for some reason) I was essentially blocked from doing any real work. So I decided to mess around with twitter's search feature, which everyone seems to be talking about lately.

It really is cool. There were a ton of people talking about very specific things I'm really interested in, new search engines being one of them. So of course, I wanted to communicate with them.

So what did I do? Naturally, I found the relevant tweets, and replied to them. I'm still not sure if you can actually reply to a particular tweet, but I did click the reply link from each tweet's individual page. They were all tweets about checking out new search engines.

My reply said something like "@username here's another new search engine, which is live now: http://duckduckgo.com. What do you think? Yes, it's ours :)". Some people replied very quickly with feedback, so I know my tweets at least were somewhat relevant. I'm guessing others would have replied too, but I can't tell for sure because my account was suspended shortly afterwards.

So I naturally went to the twitter support page on account suspensions. It does look like I violated this term: "updates consist of duplicate or repeating links and/or text." That's exactly what I did. OK, fine. I sent an apology to them so hopefully my account will be reinstated soon.

To the bigger question though: what should I have done? Not sent the tweets of course...but is there some way to have the same effect without getting my account suspended? For example, should I have just changed up the message each time? Probably wouldn't have mattered. Sent to less people? But all those people seemed to express equal interest.

Thinking about it some more, I don't really see any way in the current twitter scheme to send tweets like this without risking suspension. And yet, there is value here (on both sides). I clearly want the right people to check out the search engine. And people who post tweets about checking out new search engines, presumably, on average, want to check out a new one!

So here's what I suggest, which I'm sure has been suggested before but I'm too lazy to find out where. (And one more nudge won't hurt anyone...):

Twitter should charge me to send tweets like these, i.e. unsolicited business related tweets. The charging part both would cut down on spam and make them money. But wait, isn't it still some form of spam? Agreed, and, as such, it both messes up the twitter user experience for those receiving them (their reply screen) and those people following our twitter stream, which is annoying.

To fix this issue, twitter should also make a new type of message (like DMs) for this stuff. Instead of 'd ' make it 'p '. Each time I make a paid tweet, charge me $0.05. And on the receiver end, put them in their own window so they don't clutter up anything else. That way only the receiver sees them, and only when they want to.

Update: additional comments can be found here.

About

   

I'm a solo founder of a new search engine and an angel investor. There is more about me on my home page.
I'm also doing a book on getting traction. Get notified when it's ready:

Online Karma

-
From a new search engine

Online Profiles

-
From a new search engine