Using external APIs to improve search

One of my original premises for starting DuckDuckGo was that for every query there is usually a vertical search engine somewhere that answers that query better than a general search engine. I still believe this premise to be true, and I'd like to do whatever I can to get you direct information from that optimal vertical search engine whenever possible.

Our bang syntax gets you to those other search engines directly, e.g. !cpan Net::Server. But in that case you have to know what vertical engine you want beforehand. You often don't, if for no other reason that you don't know hundreds of vertical search engines even exist. 

As such, I think it is more powerful to put a snippet of zero-click info right on the results page. Our stack overflow integration is a good example of this effort, which uses an index on our side derived from their creative commons dump

However, often times a dump (and resulting server-side index) is not possible. Reasons vary from fast-changing info to licensing to deep processing that needs to happen on the fly. So when this happens, an external call needs to be made. 

I've taken this external concept the furthest with our Wolfram Alpha integration, which will generate a lot of instant answers for you on DuckDuckGo. I think it has worked well, and so lately I've been working on integrating a lot more external APIs from other vertical search engines. Here are the latest integrations (still works in progress of course). 

Qwerly (example search: yegg). When twitter results appear in the results, we use qwerly's API to find other profiles for that particular person. You can click on the icons to go directly to those profiles. In the future, I'd like to expand this to other domains, as they add those into their API.

Numote (example search: glee). When you search for a TV show, we grab air time and episode information from the Numote API. You can click on the episodes to be taken to episode summaries. In the future, I'd like to add a link to watch the show online (if available).


Fanvibe (example search: flyers)When you search for a sports team, we get schedule and score info (including in progress games) from the Fanvibe API. You can click on a game to be taken to engaging chatter about that game. In the future, I'd like to expand this to college teams and other leagues.

SeatGeek (example search: chemical romance). When you search for a band, we get upcoming show info (including secondary ticket price info) from the SeatGeek API. You can click on a show to be taken to a seating chart and other ticket and venue info about that show. DuckDuckGo will get a commission if you subsequently purchase. In the future, I'd like to expand these listings to be location aware and also to show for sporting events.

Amazon (example search: modern perl book). When you search something shopping related, e.g. a book, we get product information from the Amazon Product Advertising API. You can click to Amazon for more info, and we've deep linked to other useful places on Amazon and to a WorldCat library lookup (for books). Like with SeatGeek, DuckDuckGo will get a commission if you subsequently purchase from Amazon. In the future, I'd like to expand the links for other products to link to manuals and other useful stuff.

There are several other integrations in various stages of development. I'd really appreciate suggestions for improvement as well as other information/services to integrate.

You'll notice that four out of five of these integrations are with startups. I really like working with startups not only because I'm one, but because they're often doing innovative things with data and because they're flexible such that we can produce the best possible search integration.

Using external APIs is not without its problems, however. Because these are supposed to display on top of the results, there are timing issues, and if they come in after the fact, things can jump. I've been working on ways to mitigate this problem by leaving appropriate space and setting various timeouts.

That said, I think it is definitely worth it. I think these types of integrations really improve the search experience, and it is where search is headed: more zero-click info, more of the time. The key is though it always has to be highly relevant, i.e. false positives and so-so info must be kept to a negligible amount.


If you have comments, hit me up on Twitter.
I'm the Founder & CEO of DuckDuckGo, the search engine that doesn't track you. I'm also the co-author of Traction, the book that helps you get customer growth. More about me.