Wednesday, August 25, 2004
Tablets
Mac mentioned tablets in a comment to my last post, and I know there's also been some rumor about this from Apple recently. I think tablet computers would be a great achievement, but they're not there yet. I think there's big problem they need to solve thoroughly before they become useful.
There are some places a computer just can't help you. I can't take notes on a laptop very easily, unless I'm just going to be writing things down. Some classes or meetings probably could lend themselves to that. For example, since I type very fast, a laptop would probably be useful for me if I were to take a history class. In fact, I often wanted a laptop when I was in history, as I was madly scribbling down everything that was being said. That's kind of a special case; I could never take notes for just about any other class on a laptop.
The value of having information in a computer is in the ability to search and manipulate it easily. Microsoft's tablet PCs are ridiculous because they miss this point entirely, they just let you draw pictures. There is no understanding of what you wrote, just a picture of it. This is a kind of shocking failure to understand why computers are useful.
Obviously, converting handwriting to text would be amazing. But if I could have a tablet that would just let me search my notes, written in my own handwriting, even in a simplistic way, it would be extremely useful. I don't know much about handwriting recognition, but I know enough to know that it is an extremely difficult problem. Still, perhaps a tablet computer has some advantages that can be exploited. Instead of facing the problem of recognizing a given image as a character or word, a tablet computer potentially has information on the order of the strokes, the velocity and acceleration used in making the strokes, and adjacent markings. Perhaps if you kept whatever the user drew as a picture, but kept this additional information around, then a query could be performed that would use knowledge of English words to try to guess what you wrote (like cell phones do with the letters on the number pad) and match your search word. Who knows? I sure don't, but I have to think that this technology has a lot of potential for improvement.
In any case, I think this is the big question in any tablet product. Of course, I'm only saying an answer to this question is necessary for tablets to take off, it's not clearly sufficient.
Update: Hm, it appears that I had some misconceptions about how Tablet PCs work. They DO have handwriting recognition, apparently, and it seems pretty decent. You CAN search your handwritten notes, too. That's great. I probably won't consider one until I could take notes for an econometrics class and then search for a Greek letter. But still, good stuff. It's gonna get there. I'm particularly impressed with software like OneNote and Grafigo, which will help recognize the shapes you draw. This sort of thing is vital. As good as the text recognition of Microsoft Journal seems, it doesn't appear to understand numbered lists or indentation of paragraphs.
Also, apparently handwriting recognition is way better than what you can get in current products. Check out this online demo, I was very impressed with how well it recognized my mouse-renderings of my already strange cursive. Hopefully Apple has something really great here.
Another Update: The more I look into this, the less I like Tablet PC. It is actually a full-on Windows XP system, just with some tablet-related additions. This might work, but I think it's probably too cumbersome an interface for that sort of general purpose use. I wouldn't want to have to use a pen to navigate around on my PC, that doesn't strike me as a very comfortable interface. What I would hope for is something a little more special-purpose. Something thin and light, and with software more like a simple notebook. Aside from writing, drawing, and querying it, I'd probably prefer to connect to my main computer to do anything more advanced with it. Perhaps this explains why those Tablet PCs are so unreasonably priced.
Who wants to sit there browsing the web with a pen? Chatting on AIM by writing instead of typing? Navigating heirarchical menus? Doesn't strike me as a particularly great user experience.
There are some places a computer just can't help you. I can't take notes on a laptop very easily, unless I'm just going to be writing things down. Some classes or meetings probably could lend themselves to that. For example, since I type very fast, a laptop would probably be useful for me if I were to take a history class. In fact, I often wanted a laptop when I was in history, as I was madly scribbling down everything that was being said. That's kind of a special case; I could never take notes for just about any other class on a laptop.
The value of having information in a computer is in the ability to search and manipulate it easily. Microsoft's tablet PCs are ridiculous because they miss this point entirely, they just let you draw pictures. There is no understanding of what you wrote, just a picture of it. This is a kind of shocking failure to understand why computers are useful.
Obviously, converting handwriting to text would be amazing. But if I could have a tablet that would just let me search my notes, written in my own handwriting, even in a simplistic way, it would be extremely useful. I don't know much about handwriting recognition, but I know enough to know that it is an extremely difficult problem. Still, perhaps a tablet computer has some advantages that can be exploited. Instead of facing the problem of recognizing a given image as a character or word, a tablet computer potentially has information on the order of the strokes, the velocity and acceleration used in making the strokes, and adjacent markings. Perhaps if you kept whatever the user drew as a picture, but kept this additional information around, then a query could be performed that would use knowledge of English words to try to guess what you wrote (like cell phones do with the letters on the number pad) and match your search word. Who knows? I sure don't, but I have to think that this technology has a lot of potential for improvement.
In any case, I think this is the big question in any tablet product. Of course, I'm only saying an answer to this question is necessary for tablets to take off, it's not clearly sufficient.
Update: Hm, it appears that I had some misconceptions about how Tablet PCs work. They DO have handwriting recognition, apparently, and it seems pretty decent. You CAN search your handwritten notes, too. That's great. I probably won't consider one until I could take notes for an econometrics class and then search for a Greek letter. But still, good stuff. It's gonna get there. I'm particularly impressed with software like OneNote and Grafigo, which will help recognize the shapes you draw. This sort of thing is vital. As good as the text recognition of Microsoft Journal seems, it doesn't appear to understand numbered lists or indentation of paragraphs.
Also, apparently handwriting recognition is way better than what you can get in current products. Check out this online demo, I was very impressed with how well it recognized my mouse-renderings of my already strange cursive. Hopefully Apple has something really great here.
Another Update: The more I look into this, the less I like Tablet PC. It is actually a full-on Windows XP system, just with some tablet-related additions. This might work, but I think it's probably too cumbersome an interface for that sort of general purpose use. I wouldn't want to have to use a pen to navigate around on my PC, that doesn't strike me as a very comfortable interface. What I would hope for is something a little more special-purpose. Something thin and light, and with software more like a simple notebook. Aside from writing, drawing, and querying it, I'd probably prefer to connect to my main computer to do anything more advanced with it. Perhaps this explains why those Tablet PCs are so unreasonably priced.
Who wants to sit there browsing the web with a pen? Chatting on AIM by writing instead of typing? Navigating heirarchical menus? Doesn't strike me as a particularly great user experience.
Comments:
You've been hitting the nail on the head. Microsoft's "Ink" format includes stroke order and pressure. As tablets get faster, it will be easier to do the good recognition required.
Also discussed (at a recent Micosoft Research summit, to which someone in my group was reporting) was voice recognition and the computer dealing better with ambiguity. Instead of just assuming one interprettation, for example, it could leave it open and/or ask the user. (Also voice recognition could be a part of it!)
Some of the technologies needed that were mentioned was better output and input dpi, hovering, cheaper hardware, more sturdy hardware, better battery life, and, um, some other things that I can't remember. The technology for the screens, however, are going to be very cheap. Just as every LCD became color, eventually every laptop LCD will take pen input. You can expect to see more flip-top designs in the future.
I think ultimately the laptop morphing into a tablet will be the right form factor for getting it used. Similarly, with the PDAs, the right form factor (in my opinion) will be the cellphone: it's getting cameras, pocket OSes (even Linux cellphones), basically what you'd need.
There was a theory that *in the limit* people only carry items of a particular size around unless there is a 5 times size difference. So, that means that someone would carry a PDA, but not a camera at the same time. The hierarchy goes something like: watch, PDA, laptop, desktop, car. (You also don't see people carrying two laptops around either.)
(I only used my laptop in college for social science courses. Math was too hard to write, and I wouldn't want to screw myself by writing TeX (as so many dumbasses say I could do) because I'd have to compile it in class and verify I got the denominator in the right place or something. Similarly, ASCII drawings or the Word equation editor take too long too.)
The word on the street is that most of the code required in the MacOS to get pens to work is already written, in order to support some obscure sounding photoshop artists tablet.
Also discussed (at a recent Micosoft Research summit, to which someone in my group was reporting) was voice recognition and the computer dealing better with ambiguity. Instead of just assuming one interprettation, for example, it could leave it open and/or ask the user. (Also voice recognition could be a part of it!)
Some of the technologies needed that were mentioned was better output and input dpi, hovering, cheaper hardware, more sturdy hardware, better battery life, and, um, some other things that I can't remember. The technology for the screens, however, are going to be very cheap. Just as every LCD became color, eventually every laptop LCD will take pen input. You can expect to see more flip-top designs in the future.
I think ultimately the laptop morphing into a tablet will be the right form factor for getting it used. Similarly, with the PDAs, the right form factor (in my opinion) will be the cellphone: it's getting cameras, pocket OSes (even Linux cellphones), basically what you'd need.
There was a theory that *in the limit* people only carry items of a particular size around unless there is a 5 times size difference. So, that means that someone would carry a PDA, but not a camera at the same time. The hierarchy goes something like: watch, PDA, laptop, desktop, car. (You also don't see people carrying two laptops around either.)
(I only used my laptop in college for social science courses. Math was too hard to write, and I wouldn't want to screw myself by writing TeX (as so many dumbasses say I could do) because I'd have to compile it in class and verify I got the denominator in the right place or something. Similarly, ASCII drawings or the Word equation editor take too long too.)
The word on the street is that most of the code required in the MacOS to get pens to work is already written, in order to support some obscure sounding photoshop artists tablet.
ghetto as it may sound, you can always load up photoshop and just jpg your hand scribbled notes to review later on a tablet pc....
Post a Comment
