Tag Archives: predictive keyboard

Note-Taking on OSX iPhone

Attended Dan Dennett’s “From Animal to Person : How Culture Makes Up our Minds” talk, yesterday. An event hosted by UQAM’s Cognitive Science Institute. Should blog about this pretty soon. It was entertaining and some parts were fairly stimulating. But what surprised me the most had nothing to do with the talk: I was able to take notes efficiently using the onscreen keyboard on my iPod touch (my ‘touch).

As I blogged yesterday, in French, it took me a while to realize that switching keyboard language on the ‘touch also changed the dictionary used for text prediction. Very sensical but I hadn’t realized it. Writing in English with French dictionary predictions was rather painful. I basically had to click bypass the dictionary predictions on most words. Even “to” was transformed into “go” by the predictive keyboard, and I didn’t necessarily notice all the substitutions done. Really, it was a frustrating experience.

It may seem weird that it would take me a while to realize that I could get an English predictive dictionary in a French interface. One reason for the delay is that I expect some degree of awkwardness in some software features, even with some Apple products. Another reason is that I wasn’t using my ‘touch for much text entry, as I’m pretty much waiting for OSX iPhone 2.0 which should bring me alternative text entry methods such as Graffiti, MessagEase and, one can dream, Dasher. If these sound like excuses for my inattention and absent-mindedness, so be it. 😀

At any rate, I did eventually find out that I could switch back and forth between French and English dictionaries for predictive text entry on my ‘touch’s onscreen keyboard. And I’ve been entering a bit of text through this method, especially answers to a few emails.

But, last night, I thought I’d give my ‘touch a try as a note-taking device. I’ve been using PDAs for a number of years and note-taking has been a major component of my PDA usage pattern. In fact, my taking notes on a PDA has been so conspicuous that some people seem to associate me quite directly with this. It may even have helped garner a gadget-freak reputation, even though my attitude toward gadgets tends to be quite distinct from the gadget-freak pattern.

For perhaps obvious reasons, I’ve typically been able to train myself to efficiently use handheld text entry methods. On my NewtonOS MessagePad 130, I initially “got pretty good” at using the default handwriting recognition. This surprised a lot of people because human beings usually have a very hard deciphering my handwriting. Still on the Newton, switching to Graffiti, I became rather proficient at entering text using this shorthand method. On PalmOS devices (HandSpring Visor and a series of Sony Clié devices), I was usually doubling on Graffiti and MessagEase. In all of these cases, I was typically able to take rather extensive notes during different types of oral presentations or simply when I thought about something. Though I mostly used paper to take notes during classes I’ve attended during most of my academic coursework, PDA text entry was usually efficient enough that I could write down some key things in realtime. In fact, I’ve used PDAs rather extensively to take notes during ethnographic field research.

So, note taking was one of the intended uses for my iPod touch. But, again, I thought I would have to wait for text entry alternatives to the default keyboard before I could do it efficiently. So that’s why I was so surprised, yesterday, when I found out that I was able to efficiently take notes during Dennett’s talk using only the default OSX iPhone onscreen keyboard.

The key, here, is pretty much what someone at Apple was describing during some keynote session (might have been the “iPhone Roadmap” event): you need to trust the predictions. Yes, it sounds pretty “touchy-feely” (we’re talking about “touch devices,” after all 😉 ). But, well, it does work better than you would expect.

The difference is even more striking for me because I really was “fighting” the predictions. I couldn’t trust them because most of them were in the wrong language. But, last night, I noticed how surprisingly accurate the predictions could be, even with a large number of characters being mistyped. Part of it has to do with the proximity part of the algorithm. If I type “xartion,” the algorithm guesses that I’m trying to type “cartoon” because ‘x’ is close to ‘c’ and ‘i’ is close to ‘o’ (not an example from last night but one I just tried). The more confident you are that the onscreen keyboard will accurately predict what you’re trying to type, the more comfortably you can enter text.  The more comfortable you are at entering text, the more efficient you become at typing, which begins a feedback loop.

Because I didn’t care that specifically about the content of Dennett’s talk, it was an excellent occasion to practise entering text on my ‘touch. The stakes of “capturing” text were fairly low. It almost became a game. When you add characters to a string which is bringing up the appropriate suggestion and delete those extra characters, the suggestion is lost. In other words, using the example above, if I type “xartion,” I get “cartoon” as a suggestion and simply need to type a space or any non-alphabetic character to accept that suggestion. But if I go on typing “xartionu” and go back to delete the ‘u,’ the “cartoon” suggestion disappears. So I was playing a kind of game with the ‘touch as I was typing relatively long strings and trying to avoid extra characters. I lost a few accurate suggestions and had to retype these, but the more I trusted the predictive algorithm, the less frequently did I have to retype.

During a 90 minute talk, I entered about 500 words. While it may not sound like much, I would say that it captured the gist of what I was trying to write down. I don’t think I would have written down much more if I had been writing on paper. Some of these words were the same as the ones Dennett uttered but the bulk of those notes were my own thoughts on what Dennett was saying. So there were different cognitive processes going on at the same time, which greatly slows down each specific process. I would still say that I was able to follow the talk rather closely and that my notes are pretty much appropriate for the task.

Now, I still have some issues with entering text using the ‘touch’s onscreen keyboard.

  • While it makes sense to make it the default that all suggestions are accepted, there could be an easier way to refuse suggestions that tapping the box where that suggestion appears.
  • It might also be quite neat (though probably inefficient) if the original characters typed by the user were somehow kept in memory. That way, one could correct inaccurate predictions using the original string.
  • The keyboard is both very small for fingers and quite big for the screen.
  • Switching between alphabetic characters and numbers is somewhat inefficient.
  • While predictions have some of the same effect, the lack of a “spell as you type” feature decreases the assurance in avoiding typos.
  • Dictionary-based predictions are still inefficient in bilingual writing.
  • The lack of copy-paste changes a lot of things about text entry.
  • There’s basically no “command” or “macro” available during text entry.
  • As a fan of outliners, I’m missing the possibility to structure my notes directly as I enter them.
  • A voice recorder could do wonders in conjunction with text entry.
  • I really just wish Dasher were available on OSX iPhone.

All told, taking notes on the iPod touch is more efficient than I thought it’d be but less pleasant than I wish it can become.


iPhone Wishlist

Yeah, everybody’s been talking about the iPhone. It’s last week’s story but it can still generate a fair bit of coverage. People are already thinking about the next models.

Apple has most of the technology to build what would be my dream handheld device but the iPhone isn’t it. Yet.

My wishful thinking for what could in fact be the coolest handheld ever. Of course, the device should have the most often discussed features which the iPhone currently misses (Flash, MMS, chat…). But I’m going much further, here.

  • Good quality audio recording (as with the recording add-ons for the iPod 5G).
  • Disk space (say, 80GB).
  • VoIP support (Skype or other, but as compatible as possible).
  • Video camera which can face the user (for videoconference).
  • Full voice interface: speech recognition and text-to-speech for dialing, commands, and text.
  • Handwriting recognition.
  • Stylus support.
  • Data transfer over Bluetooth.
  • TextEdit.
  • Adaptive technology for word recognition.
  • Not tied to cellular provider contract.
  • UMA Cell-to-WiFi (unlicensed mobile access).
  • GPS.
  • iLife support.
  • Sync with Mac OS X and Windows.
  • Truly international cellular coverage.
  • Outliner.
  • iWork support.
  • Disk mode.
  • Multilingual support.
  • Use as home account on Mac OS X “host.”
  • FrontRow
  • USB and Bluetooth printing.
  • Battery packs with standard batteries.

The key point here isn’t that the iPhone should be a mix between an iPod and a MacBook. I’m mostly thinking about the fact that the “Personal” part of the “PC” and “PDA” concepts has not come to fruition yet. Sure, your PC account has your preferences and some personal data. Your PDA contains your contacts and to-do lists. But you still end up with personal data in different places. Hence the need for Web apps. As we all know, web apps are quite useful but there’s still room for standalone applications, especially on a handheld. It wouldn’t take much for the iPhone to be the ideal tool to serve as a “universal home” where a user can edit and output files. To a musician or podcaster, it could become the ideal portable studio.

But where the logical step needs to be taken is in “personalization.” Apparently, the iPhone’s predictive keyboard doesn’t even learn from the user’s input. Since the iPhone is meant to be used by a single individual, it seems quite strange that it does not, minimally, adapt to typed input. Yet with a device already containing a headset it seems to me that speech technologies could be ideal. Full-text continuous speech recognition already exists and what it requires is exactly what the iPhone could provide: adaptation to a user’s voice and speech patterns. Though it may be awkward for people to use a voice interface in public, cellphones have created a whole group of people who seem to be talking to themselves. 😉

Though very different from speech recognition, text-to-speech could integrate really well with a voice-driven device. Sharing the same “dictionaries” across all applications on the same device, the TTS and SR features could be trained very specifically to a given user. While screens have been important on computers for quite a while, voice-activated computers have been prominent in science-fiction for probably as long. The most common tasks done on computers (writing messages, making appointments, entering data, querying databases…) could all be done quite effectively through a voice interface. And the iPhone could easily serve as a voice interface for other computers.

Yes, I’m nightdreaming. It’s a good way to get some rest.