307. Scientist’s gadgets: Tablet PC and handwriting formulae recognition
Save This Post as PDFAs usual, on Sunday I’ll try to feed you with as less technicalities as possible but still make the post useful for you… Today the subject of our discussion will be related to Tablet PCs and functionality they offer to a scientist (in particular, to a theoretical physicist).
As far as I remember, Steven Jobs once said that Tablet PC (and especially Windows XP Tablet PC Edition) is a joke and nobody really wants their computers to support handwriting recognition (“Who wants the stylus? Ehrrh” Sorry, Steven, I do 
). At that time he was probably right, especially regarding Windows XP. But nowadays, I think, Tablet PCs matured quite a bit, handwriting recognition support is built directly into Windows Vista – and this support is quite good as I can assure you based on my personal experience. Actually, I find that Tablet PC did become an ultimate scientist’s gadget. But before explaining why do I think so, let me first give you my personal answer to the question
0. Which one?
Although I am very well aware how much you love Apple, guys , I am afraid currently no Apple product supports handwriting recognition and related functionality (although rumours are circulating that Apple might eventually come out with Tablet PC). So, yes, you’ll have to kiss Bill Gates’ glasses to enjoy handwriting recognition support.
I am writing this post on Acer TravelMate C210 bought for me almost two years ago by the the Helsinki Institute of Physics (hey, guys, that’s another good reason to apply here, especially to Kari Enqvist’s group, if you are currently looking for a job – HIP is really well supported).
What can I say? Acer is terrific and reliable, it worked without a hitch for 2 years and hardware of C210 supported everything I wanted. Formfactor of C210 is especially nice which you’ll feel immediately as soon as you take this fellow to a flight in economy class with you.
Nevertheless… Tablet PC of the first choice for me was Fujitsu Siemens P1510, simply because it was so much smaller than Acer and the battery life was much better (8 hours, as far as I remember). Unfortunately, a single copy of any FS tablet pc was sold out in Helsinki at the time we were ready to make a purchase (I am not kidding), and hard decision had to be made. If I would have an option to buy a new Tablet PC today, Fujitsu Siemens would probably again be my first choice – a newer model of course, say, P1620.
So, what are Tablet PCs good for and why do I think the one’s purchase can be worth considering for a theoretical physicist?
1. Making notes
Have you ever had that unpleasant feeling after relocating to a place of your next postdoc position that you forgot to pick up some really important notes in your old office? Or that you lost a piece of paper containing some truly nice calculation (and somehow it’s hard for you to reproduce it)? Well, with Tablet PC you don’t need to worry about that staff anymore since you go completely digital. All your notes are made on your Tablet and stored there forever – your handwriting can be recognized, indexed and searchable.
As for note-taking software, currently, there are essentially two options on the market: Agilix GoBinder and MS Office OneNote (I am talking about ver. 2007). Sorry to say, but from my point of view Bill Gates rules and the latter way outsmarts the former, simply because it supports handwriting recognition, indexes your notes and allows full text search later. It allows many other things such as recording voice notes with later speech-to-text transformation (sic! and it does work even with my thick Russian accent), collaborating with other people in real time (that is, doing whiteboard meetings using OneNote) etc. etc.
Actually, I think that OneNote is the second most useful program in the Office Suite after PowerPoint.
If you are lucky enough, your IT department already has an Enterprise license for MS Office, that is, they will install MS Office to your Tablet for free (as it was in my case), but if you are unlucky – OneNote is included into MS Office Student Edition which costs about 90$.
Currently, all my notes (and not only science!) are getting written and saved in OneNote format.
2. Annotating papers from Arxiv
This is very convenient. To make annotating of downloaded PDF papers possible, you need either Adobe Acrobat Standard (or Professional) or FoxIt Reader. The latter is free and allows highlighting text in the PDF file – the way you do it by a marker on the hard copy of the paper (with your Tablet pen this is not just easy-peasy, this is fun). You do want to go for the former though, since Adobe Acrobat supports PDFs natively and handles them so much better than FoxIt. Acrobat does cost quite a bit, so ask your IT department guy if they have an enterprise license for Acrobat and can install it to your Tablet for free.
3. Online whiteboard session
If you want to chat with a friend or colleague and exchange some formulae during the conversation, there are three options I know of.
a) Create a notebook in MS OneNote and share it with your friend. Use phone or Skype to chat with her/him in the mean time This option can be problematic, if your IT department firewalls ports that OneNote uses to share notebooks.
b) Use MS Messenger Yes, that awful green shiny-looking crappy analogue of Skype The point is that it does natively support handwriting recognition during conversations. Personally, the best option ever – you can do real time collaborating online with collaborators overseas.
c) Use Skype Whiteboard Meeting support. I am going to write about it in details in one of the next “Scientist’s gadgets” posts.
4. Finally, the most interesting part – formulae handwriting recognition
Well, recognition of handwritten text is fine and fun, but what we really and ultimately want is recognition of handwritten formulae, don’t we? Would formulae handwriting recognition be supported in our Tablet PC (say, you can export written formula to TeX), it would really make our life and writing papers a bit easier.
That’s where Microsoft falls short. The reason, I guess, that technically formulae handwriting recognition is so much more complicated (roughly, 2 dim problem) than the recognition of handwritten text (roughly, 1 dim problem). Anyway, as it turns out, there exists a Japanese group (the leader is Masakazu Suzuki (Kyushu University)) working on handwriting formulae recognition for decades as well as OCR of scanned images of scientific papers. The group was able to come out with a real product, which is even free.
There are three products actually in the Infty suite. One is called InftyEditor (the last version is 3.06 released on the end of Dec 2008). The program supports formulae handwriting with subsequent export of your writing to LaTeX.
The second product is InftyReader (the last version is 2.7.9, dated Dec 2008 as well). This one supports recognition of scanned scientific papers and export of images into LaTeX, which is also quite useful (maybe, not directly of us, but for, say, libraries).
Finally, the third product is ChattyInfty that has a text-to-speech support of LaTeX source files (i.e., this fellow will read your LaTeX source files for you in the way which is understandable).
Among those three, InftyEditor is certainly the most interesting and useful one for me. My experience with it wasn’t smooth at all – while writing a formula, one should be really as accurate and precise as possible, since the hit/miss rate of the recognition software is terrible otherwise. InftyEditor (especially, its latest versions) does work though – in comparison, say, to redundant handwriting recognition support built into the latest version of Maple. I really hope that the Suzuki group will continue developing this extremely nice piece of software…
If you liked the post, please kindly consider to leave a comment, subscribe to the RSS feed or get new posts sent directly to your Inbox. If you want to chat with me in real time, you can find me on Twitter. The posts below are probably related to the subject of this one:
Loading ...
It is really nice to see more people using Tablet PCs, and thanks very much for sharing your experience with the different applications. I myself have had a Lenovo X61T for a few months now, and I am really enjoying the portability and easy access of pretty much everything I need for my studies and work. I use PDF annotator to annotate my PDF files, and it works like a charm.
One thing that will, I’m sure, greatly please you is to learn that you can have all your favourite reference/text books right on your tablet, for free! I left some comments at Gordon Watt’s post, so you might go and have a read there
Have fun!
Dear TM,
I’ve checked PDF Annotator out, unfortunately, it isn’t free. So it seems that the only option for people who want Annotator functionality for free is FoxIt.
Thanks for the link to Gordon Watts’ post!
Dmitry.
Dear Dmitry,
I just checked the PDF annotator website, and I am very surprised the price has gone up by that much! I paid 30 euros for the same software in late Nov 08. For me, that’s been one very noteworthy investment, considering I have lots of ebooks in pdf (not to mention the large N preprints that populate my desktop). Anyway, a quick google search easily reveals a non-empty set of possibly useful free annotator applications.
Best, TM
Windows 7 includes the new Math Input Panel which is capable of recognizing handwritten math equations and translating them to MathML that can be pasted into applications. If you have access to the Windows 7 beta you might want to check it out. Gottabemobile has some screenshots here:
http://www.gottabemobile.com/2.....reenshots/
Thanks for the link to Infty; I’ll check it out. The math recognition in Windows 7 works well for me, but I’m interested to see about other formats that are supported besides MathML.
Dear Loren,
thanks for pointing that out, I did not try Windows 7 yet. I do have a strong suspicion that Math Input Panel does not quite yet provide the functionality we need (see Lubos’ comment below) when writing papers
Cheers,
Dmitry.
I’ve only played with Cumrun’s tablet PC a few years ago, when I was fixing its hard disk or something like that. It’s fun and surely useful to make notes for the owner himself. Moving, copying, zooming, etc. are good tools to improve this drawing and writing over the old-fashioned paper.
However, I don’t believe that recognizing complicated handwritten data – like math equations – is a good idea that will be used in a systematic way in the future. There are too many things that may be misunderstood by the computer, too many conventions that the computer might misunderstand.
It still seems to me as a good pressure that humans are forced to rewrite equations in a mechanical LaTeX/MathML way by the computers. For example, if you look at the
http://www.gottabemobile.com/2.....reenshots/
link above, there are four equations. They’ve been surely optimized to be showable on the blog. Still, are they correctly interpreted? Not at all. Note that the square root in the solution of the quadratic equation (last eqn) doesn’t include “c” at the end which is left outside the square root: a wrong formula follows immediately and almost no one except for me would even notice.
This particular thing wouldn’t happen with hard TeX/MathML writing because there would be no reason to divide “a”, “c” inside/outside the square root. And the syntactically OK formulae that could be obtained by a mistake would look very different, and could be seen as mistakes.
Otherwise, Windows 7 is nice – I’ve tried it for a few hours now – but it is not that different from Vista, after all.
Dear Lubos,
My experience shows that several (even few) years do matter – progress goes fast nowadays. For example, recall how much handwriting recognition sucked back in Win XP years (if you tried that with Vafa’s tablet). Vista was quite a difference indeed – for example, in Vista handwriting recognition engine is able to learn and recognize your writing better with time (WinXP Tablet PC edition did not support learning).
On the other hand, I follow Suzuki group for a couple of years already and I do see that the recognition ability of their software is getting better… So, I would say – in a couple of years we will have this technology at least at the same level text handwriting recognition is now (of course, if demand will be high enough).
Cheers,
Dmitry.
Hi Dmitry,
Its would be really useful gadgets if it did work. I tried to convert a page in Coleman’s QFT lecture note and it gave me real silly stuff. I am not discouraging the technology, but its really hard for me to see how this would evolve into a something which would make things less time consuming. Anyways, I hope I am wrong.
Best,
Savan
Hi Savan,
what did you try – InftyEditor or Infty Reader?
Cheers,
Dmitry.
I tried the Inftyreader. It did not recognize much of anything except for few integrals. Perhaps, it does a better job with uncluttered documents.
Best,
Savan
I did not actually try Inftyreader, Inftyeditor was relatively fine from my point of view (especially compared to support of handwriting recognition in Maple). Early version did grow my stress level, though
Cheers,
Dmitry.
For PDF annotation of papers online, you might like to try our service at A.nnotate.com – there’s a free version (just needs a web browser, no plugins reqd), or you can get your institution to install it for unlimited use. Several people can annotate the same copy of a paper (e.g. for journal club discussions), and highlighted text and notes get indexed for you -
Fred
For whiteboarding, you may find our product, Dabbleboard even more useful. It does things like shape recognition to make whiteboarding really easy and fast. Doesn’t do character recognition itself, but you could certainly use Vista’s built-in support for it.
Actually OSX has handwriting recognition (called Ink) built in to the operating system, which then lets you write on your graphics tablet etc directly into any text box.