Posts Tagged ‘iPhone’

The killer app for iPhone 3GS

by Hang

You heard it here first folks, the killer app for the iPhone 3GS will be augmented reality. Two features in the new release makes the iPhone 3GS the perfect augmented reality platform. First, the inclusion of an onboard compass and second, the opening up of the API to full camera controls.

Almost everyone except developers were unaware of this but the iPhone 3G SDK painfully crippled the camera by forcing you to use the provided Apple API to take pictures (this meant no real time computer vision could run on it except when Apple conveniently didn’t notice you bypassing their allowed APIs). Now that the APIs are open, expect to see a bunch of innovative computer vision apps (face detection, object detection, tracking).

But the second breakthrough is that the compass finally provides a braindead 6 Degree of Freedom (DOF) estimation. Any rigid object in space can be defined by 6 parameters. 3 spatial ones and 3 rotational ones. The iPhone 3G could estimate 5 of those 6 parameters acceptable well with GPS providing rough spatial data, the acceleromter providing fine spatial data as well as the direction of gravity. The compass provides the final, missing degree of freedom that allows for complete pose estimation.

Augmented Reality needs to know two things: Where is the sensor and where is everything else. Once you know that, you can do all sorts of really cool shit. I have no idea what will eventually come out, that’s one of the exciting things about bringing this technology to a mainstream audience, but I can point to some interesting research directions that seem plausible:

  • Instant 3D modelling of everything: Wave your iPhone around an object and it’ll figure out how to create a crude 3D model of it in memory.
  • Interactive furniture arranging: Go to the store, scan a bunch of furniture you want to buy, go home, drag and drop virtual pieces of furniture in your living room to figure out which piece should go where.
  • Interactive tour guide: You’re walking through New York, you see a cool building, take a photo of it and all of a sudden you know it’s the flatiron building and you’re reading a wikipedia article about it.
  • Photosynth the entire damn world: nuff said.
  • AR Quake: nuff said.
  • History view: Point to a space and if there’s a security camera pointed at it, be able to review what happened at that spot at any point in time.
  • Invisible Ink: Leave messages on walls which only your friends can see. Send them on an easter egg hunt.
  • Virtual Ping Pong on the phone: Use the phone screen to see a virtual ping pong table and then swing the phone to make a hit
  • Physical, virtual avatar conferencing: Replicate the real world cocktail party acoustics in a virtual physical space. Cocktail parties are great because they allow the spontaneous formation of ad-hoc small conversation clusters within a larger conversation. Online tools do a poor job of replicating that dynamic but if we could bring physicality back into it, we might be able to bring some of this dynamism back into online conversation.
  • A million goddamn screens: This was a project I personally worked on that unfortunately, never got to far but has a dear place in my heart. Conventional computing is predicated on screens being expensive but if you stick a tracking marker onto a piece of paper, you can turn it into a screen. What would computing be like if you could produce screens out of some cardboard and a laser printer that could be any form factor and would cost 10 cents a pop? How would this help with information overload? Imagine you have a screen in the corner that represents the pile of unread emails. If you want to read an email, drag it onto a new screen. Important emails that you need to reply to each get their own seperate screen which you keep neat and tidy by arranging them in a pile. If you want to send a file to your coworker sitting next to you, you can drag it onto a screen and then physically hand him the screen. You can have a screen for each individual participant in an IM/voice/video conferencing and pulling the screen closer means you want to be alerted but pushing it away means you want to ignore them. What could you do if you had access to a million goddamn screens over the course of your lifetime?

Are some of these examples wildly unrealistic and totally unable to work in real life? Of course, they come from research inspirations. But they demonstrate the enormous power of augmented reality which is about to be unlocked within these next few years as developers grapple with just what’s possible with Augemented Reality.

February 23 2009

no provisioned iPhone device could be found

by Hang

I occasionalyl get this error in XCode. A quick reboot of the iPhone seems to fix the problem.

January 13 2009

iPhone apps that should have GPS integration

by Hang
  • Clock – Why can’t the time zone for my clock be set automatically to which time zone I’m in.
  • Weather – I want the default display to be the weather of my current location.
January 12 2009

Running videos in the background on an iPhone

by Hang

I’m too ADD to listen to audio lectures on my desktop but I’ve found it works when I have it running in the background on my iPhone while I’m playing games. One major annoyance was that I wasn’t able to listen to video lectures (such as TED talks) and do anything else at the same time on the iPhone.

It turns out there is a way to do it although it’s pretty obscure. First upload a 10 second silent audio clip onto your iPhone. Then, go into playlists and create an on-the-go playlist. Add the 10 second audio clip first and then any video clips you would like to watch. Start the playlist and then exit the iPod application before the audio ends. When the video starts up, it’ll be running in the background.

December 23 2008

Some tips for jailbreakers

by Hang

I’ve been developing a few ideas for iPhone applications and the app verification process is taking so long that it was just easier to jailbreak my phone than wait for Apple to finish “verifying” me. The applications for jailbreaking the iPhone all seem very well designed and easy to use but the documentation surrounding them is lacking. Here’s some tips that could save you a great deal of stress in case you decide to jailbreak your iPhone:

  1. If you care at all about how your apps are arranged, write down the order your apps appear on a piece of paper or something. The reason for this will be apparent later on.
  2. There’s no way of irrevocably damaging your iPhone via this procedure. No matter what happens to it, there’s always a way to get back to a virgin state so take a deep breath and chill if something unexpected happens.
  3. Backup your iPhone via iTunes immediately before jailbreaking it. To do this, right click on the phone and select backup
  4. Run through either Pwnage Tool or QuickPWN and follow all the steps. Hopefully, you should have a fully jailbroken iPhone at this step.
  5. If your phone comes out of the jailbreak at the apple boot logo screen and continually restarts, see this post for a fix
  6. If anything goes wrong with the jailbreak, you can always restore your phone to it’s original settings but the only thing that mysteriously seems to not get remembered is the order your apps are in, hence the handy cheat sheet.

Oct 22nd (day 10): Insights from the iPhone

by Hang

iPhone applications provide an unwitting peek at what companies believe is the real purpose of their product. Because of the limited form factor of the iPhone, developers are forced to release cut down versions of apps and what features they choose to include tells a lot about their priorities.

For example, the youtube iPhone app does not even suggest that comments exist wheras it devotes significant space to “similar videos”. Clearly, youtube believes their business success lies in encouraging people to browse through more videos. This is reflected in their benign neglect of the web based commenting system which I think has the potential to be one of the more interesting arenas on the web if it were handled well.

The facebook iPhone app features prominently the ability to take pictures and upload them directly to your profile as well as a well integrated commenting feature. Chat as well is featured heavily in the facebook app. Curiously enough, events are completely missing from the app (although not the web interface) which I regard as a major limitation. Also missing is the ability to forward messages or do anything other than a straight reply. Facebook mail has the potential to become the de facto email replacement but it seems relatively clear that this is an area that Facebook is not super interested in as there have only been a trickle of new functionality related to it’s messaging service.

Google maps allows you to plot a route via car but it doesn’t include the walking or public transit modes that it has on it’s website. Even more annoyingly, it doesn’t have the ability to dynamically change your route to a location as you move around which makes it effectively useless as a GPS navigation system.

Some of these insights would have been obvious without looking at the iPhone app version of websites but I think it does place these distinctions in starker contrast. It doesn’t just have to be the iPhone either, looking at any reduced functionality version of a site can give you a sense of what their developers regarded as essential and optional.

August 1 2008

Not impressed with the iPhone user interaction

by Hang

So I finally broke down and got an iPhone today and I have to say, so far, I’m not enormously impressed with the fit and finish of the user interface. This being my very first Apple product, I really didn’t know what were reasonable expectations but Apple has built it’s brand on producing polished, solid interfaces.

Don’t get me wrong, the interface is gorgeous and has a lot of cool tricks but there’s certain areas where the underlying lack of care shone through. The problems started from the moment after unboxing when registering for the very first time. iTunes software is required for registration but nothing was provided in the box. Downloading it was a trivial task for me but it was still an annoyance to have to download it.

Right at the *very* start of the registration process, there’s a series of relatively technical and fairly unimportant questions which really, I didn’t know what the hell I was supposed to answer and making me do that up front is just plain bad UI design. On the grand scheme of things, this is a relatively minor thing but first impressions count for a lot and, IMHO, Apple dropped the ball on this one.

Copyright ©2009 BumblebeeLabs — Theme designed by Michael Amini