Wednesday, July 30, 2008

The Search Engine That Could

The cool thing about the technology behind search engines like Google is that it is the source to find ANYTHING. Whether you want recipes, telephone numbers, latest news, or pictures of your favorite rock star, you can type it into a search engine and find the stuff you want. The technology is pretty complex, because you're not just looking for a phrase in a large text, sometimes you need to understand the semantics of the language to pick up on the key phrases or determine the relevancy of the sites you index. It's very hard to get it right. But when you do, you got a money maker on your hands.

Even though we use search engines for pretty much anything online, I think that search engines are not doing anywhere near enough what they could be. Here's what I mean. Search engines are natural and essential when you have a database, and you can market a search algorithm for almost anything.

If you have a database of people's information, you need a search engine to find people. If you have a database of clothes, you need a search engine to find the clothes you want. Amazon, iTunes, Netflix are all high profile web companies that would dissolve into chaos if their search algorithm went bad. And I would suggest that all such algorithms are connected in some way.

Suppose someone wrote on their blog analyzing the moral complexities in the relatively unknown comic Dodobirdman and it gets a lot of page links in the next few hours, getting on the front page of Digg the next day, naturally, when you are browsing for books and you type in birdman, it'll suggest Dodobirdman and automatically spit out the newest volume of the comic, as well as Dodobirdman backpacks, and of course the popular Dodoman key chains. While when the reverse happens, and a book gets rave reviews and becomes a bestseller, webpages that refer to the things related to the book online should get more attention.

So the first idea is that a search engine company should be a search engine company. Sure, website search engines are very different than Amazon's recommendation system or the Digg algorithm, but they are all related. Why should a company spend money to go make the hardware with enough computing power to analyze all of its data, including the staff required to maintain it, or write a svm multiple tree and constantly update it with the latest in machine language research, when they have the option of outsourcing it to a reliable company?

There are many ways of providing such a search engine service and it might be pretty tricky to find the right way. The content and all the "signals" (as google puts it) may be confidential information from the client, and the search algorithm would be the business secret of the service, but the algorithm needs the data to work and when you're optimizing something like that, you're bound to access a little bit of both. One way is to make servers with a collection of different search engine tools built-in, that you can put your data on, as well as getting it processed the way you want to. Even if the hardware isn't top-notch, if your software is unique, companies would buy them like hot cakes.

I can probably write more about the subject, but unless you're microsoft, yahoo, or google, you probably won't be interested, so let's move on to IDEA 2!!!

One of the problems with search engines is the trade-off between search engine effectiveness and privacy. You can find out all of the interests of a particular user, and recommend them everything they are interested in with high accuracy, but you would need training data that people don't really want big-name companies to be keeping in their database. If search engines now can already provide relevant results, imagine what they could do with your private data.

So idea 2 would be to create a client-side search engine the keeps track of things you buy, the searches you do, the kinds of website that you like that does not in any way reveal your information to anyone else. It will be your personalized search engine. When you encounter any kind of large collection, say search result from google or recommendations from buy.com, the local search engine would take those results and reorder the priorities depending on past behavior of the user. Of course, to protect the privacy of its user, it will have to non-discriminately download the information about the things it wants to look for, increasing the bandwidth in a world where bandwidth matters less and less to companies.

If people find what they want, they'll buy more things. So the development of this software (like a browser plug-in, or browser feature) can be a collaborative effort from major web companies that will give the sponsors an advantage of understanding the inner workings of the local search engine for optimization, while still giving the means for startups to take advantage of the local search with little investment.

If this program REALLY behaves itself, maybe it will also be able to expand to giving adword-like little popups when you aren't online at all. It could respond to the text that it sees you typing into your computer, or programs that you have running. If this idea evolves to this stage though, there are many other things you can do with the personal e-butler. It might become so good that you won't ever miss out on any of the Dodobirdman memorabilia, ever again! WOOT!

Monday, July 7, 2008

side view mirrors

Here's a quickee that's kinda related to the previous post.

In a car, you have a back-view mirror, and a rear-view mirror, and even with the 3 mirrors and windows on everyside, cars still have blindspots. Blindspots are dangerous.

If you have driven on mountain roads before, you should have seen some safety mirrors that show you what is around the corner. They are there so you can notice if there are any cars coming, but not necessarily to identify which kind of car it is. Those mirrors are not flat. They are convex so that you can see the widest range on the other side of the bend. So why are side-view mirrors flat?

When you look in your sideview, you may want to tell what is passing by behind you, so it should be flat for most of its surface area. But if the edges of these mirrors are rounded, even though you can't really see too clearly what is there, you can notice if there is something in your blind spot. That can make a difference between being 10 seconds late to your destination and flattening a scooter.... that has a whole family on it.

Tuesday, July 1, 2008

Bird's eye view glasses

Skiing goggles are pretty cool. Since they have to block out the UV rays, the goggles have a lot of reflexive paint, so they come in all different kinds of colors. Yet even if the goggles were yellow, your eyes can adjust to the color, and after a few minutes, everything will look normal again. When you take them off, since your eyes had to compensate for the color, everything looks blue.

Despite how cool it is, people wearing glasses for the first time definitely should not go skiing, in fact, they are a bit dangerous even when they walk. That is because glasses not only refocus what they see, the skew their vision. If you're wearing glasses right now, just take a look at the rim of your glasses, and you will see that the refocused vision doesn't match up with your original vision. Yet your eyes can still adjust to them.

Back in the days when quake 2 was popular, some people would change their field of view to a much higher angle than normal. Even though targets become smaller, you get to see more of what is going on. With training, this becomes an advantage when you can frag opponents before they see you, and not vice-versa.

So, chances are likely that our visual system will be able to adjust if we make a helmet that feeds a 360 vision into our eyes. It can be done with cameras and monitors on your helmet. Birds typically have 300~340 field of vision, but they have little overlap in their vision, which reduces clarity and sense of 3D. So, ideally, there would still be a high overlapping region, but the other regions would be at a reduced scale, to increase peripheral vision. It might look weird if a guy walks around with a ball on his head. But for vehicles, such a helmet would increase reaction time.

Anyone could be trained to use the helmet, probably within a week. You might have to spend some time on hands-eye coordination, like playing catch a few other people, adjusting to the change in vision.

Then you won't have an excuse when you crash into a tree when you're skiiing.