 |
Dan Bricklin's Web Site: www.bricklin.com
|
|
About Tablet Computing Old and New
A discussion of
PC tablet hardware and software from the 1990's, and why Microsoft's
pushing of the new Tablet PCs will bring renewed innovation.
|
In my "Tablet PC: First Impressions"
essay I made some assertions about the amount of progress in tablet
computing represented by the new Tablet PCs. Some sample quotes:
Bob and I were both struck with how little advance there had been since the last try for pen computers in some respects.
The advance in features, though, seems more like a "next release" or two of things rather than 10 years passing.
Things were pretty good in the old days.
...the
way Microsoft is doing this will hopefully encourage tablet-centric
innovation to start again throughout the hardware and software
industry, so we'll continue these advances, and the rate of improvement
will return to what it was in the early 1990's.
...[the new machines are] still basically a version 1 or 2...
I
think Microsoft and the hardware manufacturers who were willing to take
a chance trying to advance the state of mainstream personal computing
are to be commended for what they've done.
It's hard for people
who worked very hard bringing these new systems to market to hear me
say it only looks like a "next release", and at the same time it's hard
for others to understand why I believe things will advance so much
further because of Microsoft and the manufacturers' recent actions. The
purpose of this essay is to provide some of the reason for those
statements.
The old hardware and OS software
To understand why it
doesn't seem like such an advance, you have to be familiar with the
hardware and software of the early 1990's.
The use of pens and
tablets, and "light-pens" that you could point at the screen, goes very
far back in the history of computers. For example, the SAGE
air defense system from the 1950's used a "light gun" to interact with
the screen. CAD/CAM systems of the 1960's (like the pioneering SketchPad) and 1970's used light pens or pens on opaque tablets to manipulate items on the screen.
A pen-based desktop system that was part of the personal computer world came from Wang in 1988. Called Wang Freestyle, it let you annotate screen captures, faxes, and scanned images, with "ink" from an electronic pen
using an opaque tablet connected to a PC running normal applications,
and manipulate thumbnails of those images by dragging them around using
the pen. It let you synchronize recorded sound (using an attached telephone) to a recording of the pen motions.
It let you then print, email, or fax the results. Freestyle was a big
sensation at Comdex when shown. Even today, looking at a video of it in
action demonstrated by the project lead Stephen Levine, it is
impressive.
The first in the line of the "modern" tablet computers was the GRiDPad in 1989. Developed under R&D head Jeff Hawkins (who later founded Palm and Handspring), it was
about 9"x12"x1.4" with a 10MHz 8086 running MSDOS. It had a pen that was
at the end of a wire, and worked by making contact with a coating on
the screen. It could recognize hand printed characters, and was used
for data collection, like filling in forms.
The next really influential tablet system was from GO Corporation.
The prototype "Lombard" was 80286 based, and ran a new, GUI operating
system called PenPoint. GO was started in 1987. After announcing their
product in January 1991, GO upgraded the base system to require an
80386 for the first real customer release (which was in April 1992).
Later, as the company named EO, the processor for PenPoint was changed
again, this time to the AT&T Hobbit chip. Each time, software
developers had to upgrade their software.
After GO started on PenPoint, Microsoft reacted with enhancements to Windows 3.1 to create Windows for Pen Computing, better known as PenWindows. (The head of that project was Jeff Raikes,
who now heads Microsoft's Productivity and Business Services Group
which includes the Tablet PC.) Some machines produced at the time (such
as the 3 lb. NCR 3125 pure tablet) could boot to either PenPoint or
PenWindows. A variety of manufacturers made machines for PenWindows,
including Samsung and later Compaq. The most interesting PenWindows
computer, for me, was the GRiD Convertible, released in mid-1992.
(I still have a working one which I used for years -- most of the other
pen-enabled computers in my collection are stowed away in a warehouse.)
The GRiD Convertible was a normal Windows laptop, but when you closed
the screen, it folded down in such a way that the screen faced out --
like a tablet. It was started under Jeff Hawkins before he left to
found Palm Computing. (Notice how these two Jeffs' names keep coming
up.) NEC also made a variation on it's laptop with a screen you could
turn around, like some of today's tablets. Many other manufacturers
tried their hands at tablet computers, including Wang and IBM. These
computers all used either a Wacom digitizer and battery-less but
electronically active pen (the same used in many of today's Tablet PCs,
and very popular as a desktop accessory for graphic artists) or a
battery powered (or tethered on a wire) active pen from some other
manufacturer. The reason for a special pen is to let the computer track
the pen's location when it is held near, but not touching, the screen,
much like a mouse is moved before clicking. Windows depends upon the
ability to show different cursors, have "hovering" effects, etc.
Unfortunately, some of the digitizing technology of the day did not
work well with color screens which were just coming into a reasonable
price range, so digitizers were left off of most later machines.
Another computer of the day was the Momenta, but it had pretty much its own variant of Windows, and a pen like the original GRiDPad -- no hover.
One of the last of that crop of pen-enabled computers was the Apple Newton,
first shipped in 1993. While Apple had experimented with other tablet
computers, this was the one released to the most fanfare. The Newton's
pen, as I recall, did not have hover -- it was more like the later Palm
computers which just sensed pressure on the screen from any object.
In all cases, the
use of a pen as an input device was integrated into the operating
system to varying degrees. The pen could be used for most mouse
actions, such as clicking or dragging. Within almost any application,
instead of typing on a keyboard, you could write on the screen or tap
on a virtual keyboard. There were various "gestures" (special pen
movements) that invoked certain functions, for example, Undo, or, like
today's Tablet PCs, bring up a writing pad or virtual keyboard. All
systems had handwriting recognition of some sort.
Looking at the
machines of those days, and given the advances in hardware since,
today's Tablet PCs are not very surprising. They are somewhat lighter
and with much faster microprocessors and greater memory, but the pen
additions and form-factors are similar. The important thing, as I point
out in my First Impressions essay, is that today's machines come
into an environment where you read more on a computer screen, and
wireless connectivity to all of computerdom is commonplace. Now these
machines have a much more important reason to exist.
The old applications
The first applications for the GRiDPad were very basic, in line with the simple forms capabilities of a basic browser. With the advent of PenPoint, though, developers started producing much more sophisticated products,
pouring millions and millions of dollars into development. PenPoint
itself had a very sophisticated, pen-centric UI. Coming before the
convertibles, and trying to completely eliminate the keyboard, there
were all sorts of user interface advances. Some of those ended up
influencing Windows 95. It had OLE-like embedding well before it was
viable on Windows, it required just a "tap" to launch apps which
avoided the need for double-clicking, and more.
In early 1990, I
co-founded a company called Slate Corporation (along with other PC
veterans like Vern Raburn, Dottie Hall, and Tom Byers). Our mission was
to create application software for the upcoming PenPoint and other
tablet/pen operating systems. There were other companies that were
creating application software specifically for these machines, but ours
was the best funded, produced the most products, and is the one I know
best, so I'll talk about it first.
We demonstrated the first of our software when GO announced PenPoint (286 version) in January 1991, and shipped our products in shrink-wrapped boxes in 1992 for both PenPoint and PenWindows. The products we developed were:
 PenApps, an application development system somewhat similar to Visual Basic (which was being developed around the same time). It had an object oriented programming language (PenBasic) with support for ink as a data type,
a drag and drop interface builder, and more. You could write on a form
created with it, and when filling out forms it was smart about targeting the ink you wrote to the correct field (so you didn't have to carefully "writing inside of the lines"). It had "deferred translation"
where the ink was kept around so you didn't have to wait for each field
to translate as you filled out a form, and you could check translated
data against the original ink at any time. It had a built-in database.
It was a major product.PenBook,
an electronic book creation system. This was similar to Adobe's PDF
system (being developed around the same time), but tuned for reading on
a tablet computer. It could convert Postscript files output from most
any program into its format, and then you could read the "books" with a
special reader. The reader supported pen gestures for turning pages, annotating and highlighting, bookmarks that looked like paperclips, and more. It had searching and stored the "book" in a compressed format.At-Hand spreadsheet
for PenPoint only. This was a full-fledged spreadsheet (mainly created
by Bill Lynch who went on to work with Microsoft's Excel group for
years) complete with a BASIC-like programming language with special
spreadsheet data-types and operators to react to tapped buttons and
other events (developed by Bob Frankston before VBA came out from
Microsoft in 1994) and a full graphing package (developed by Buzz
Kelley, now with me at Trellix). It could read and write Excel and
1-2-3 files (thanks to Peter Levin, now with me at Trellix). In
addition to all this, it was completely operable with a pen, with lots of innovative features.
You could write on the spreadsheet cell grid, and it would target your
writing to the appropriate cell. If what you wrote was text, you got a
label cell; if it was a number, you got a numeric value, appropriately
formatted. If it couldn't recognize what you wrote well, you got ink reduced to fit in the cell
(ready for correction or to be left alone). For entering formulas, there
was a special input dialog tuned to the pen. A couple taps of the pen
selected a range of cells, and writing a "+" put the "sum" function where you wrote it. There was a markup layer
to annotate things with ink. The graphing system handled most of the
popular graph types (including 3D and contour) yet scaled appropriately
to work well when embedded on the sheet or elsewhere. A year or so
later we created an Excel plug-in called PenPower that added many of those pen-centric capabilities to Excel running under PenWindows.Day-Timer Pen Scheduler
for PenPoint or PenWindows. This was an electronic ink-based version of
Day-Timer, Inc.'s organizer, with calendar-based day/week/month/year
views, notetaking pages, to-do lists, and a name/address book (which
used text and ink). With easy zooming, you could use "tiny text" to fit
lots of data in any space ("Your pages are uncluttered, yet full of
valuable information"). You could circle something of interest, and
then file that snippet away in an index by topic, linked back to the
original, all with a quick gesture.LooseLeaf Notetaker for PenWindows. This was an ink-based notetaking application for the GRiD Convertible with a variety of pens and markers.In addition to Slate's products, there were deep, innovative products from other companies. For example, Pensoft produced a personal information manager that used recognized text and a data base. A later company (founded in 1991) was Aha!, which created an ink-based notetaking product with extensive ink editing features. Among other things, it could "word wrap" text still in ink, and do background translation for later conversion or searching. Aha! was bought by Microsoft in 1996, and you can see how the Windows Journal program comes from it (without some of the cool word-wrapping features).
One of the issues we were working on at Slate in the mid-1990's was evangelizing the use of digital ink
created with a digitizing pen as a normal data type among applications.
We also had to deal with making the ink look true enough to your quick
scribbles, so that even when you used a 6"x8" screen to mark up an 8
1/2"x11" fax shrunk to fit, it would look "normal" printed out or
re-faxed at full size. We did lots of work with growing and shrinking
ink, and related issues. (When you shrink, you don't want the lines to
get below a certain thickness or else it sometimes looks weird.) We
also worked on some early pocket sized prototypes, as well as software
for Apple's Newton when it first shipped.
Learn from old patents
To learn more about
the level of thinking that went into these old products, you can read
some of the patents that came out of those efforts. Since patents are supposed to teach you what is novel and important, reading them should be like reading a techie-to-techie whitepaper about what's special and why.
(Unlike when looking for infringement, just read the main body of these
patents, not the claims. I list them here not to say whether or not
they apply today, but rather as a source of learning about what was
thought about in the past.)
5,613,019: System and methods for spacing, storing and recognizing electronic representations of handwriting, printing and drawings. [Based on filings to the Patent Office done in May 1993.] This has the text of Aha!'s description.
I found the middle section (it's a long patent) where it discusses how
to determine what's a "word" (getting the dots over the "i"s to be part
of the right word, even if written much later) interesting. The patent
mentions that the digitizers of those days sampled the pen's position
about 200 times a second -- faster than most Tablet PCs today.
5,455,901: Input device with deferred translation.
[Based on filings from November 1991.] This describes keeping the ink
around to translate later, as well as for verification or instead of
translation. It's the Slate PenApps patent. This and the other
Slate patents are now owned by Compaq/HP. (Compaq bought Slate
Corporation when we ran out of money when people refrained from buying
the computers that ran our software.)
5,717,939 and 5,848,187: Method and apparatus for entering and manipulating spreadsheet cell data. [These are based on filings from November 1991.] These are the Slate At-Hand spreadsheet
patents (the two have similar text, but different claims). They describe
targeting ink to cells, special spreadsheet gestures, improved
recognition for a spreadsheet, and more.
5,867,150: Graphic indexing system. [Based on filings from February 1992.] One of the Slate Pen Scheduler
patents. This relates to selecting something on the screen by circling
it and then quickly adding it to a graphical index or gallery.
Sometimes it's easier to just put the image of a piece of a page into
an index for quick scanning with your eyes than to type a description.
This patent relates to such a feature.
5,231,578: Apparatus for document annotation and manipulation using images from a window source, 5,625,833: Document annotation & manipulation in a data processing system. [Based on filings in 1988.] Some of the Wang Freestyle patents.
So as you can see, the thinking 10 years ago was quite deep, with applications on par with anything being shown today.
Why the machines are version 1 or 2
Looking at some of the machines, you can see that we still haven't learned all the tricks necessary to make a tablet without rough edges.
For example, on the Toshiba, which is supposed to have one of the
better pen holders, when you put the pen back in its holder, the pen
tip is close to the side of the screen and entices the mouse cursor to
move over to it, away from where you left it. (This might be when you
put the pen away in keyboard mode and use the touchpad, or in tablet
mode to use just the arrow buttons for reading.) Worse yet, putting the
pen in the holder often presses the tip, signaling a mouse click. If
there are buttons or icons on that side of the screen, they sometimes
get selected.
The screens vary in
their feel and the pens in their weight. The perfect paper-like feel of
drag for writing, without muddying up the image with ground glass,
hasn't been perfected.
Some of the machines
have built-in prop-up stands for reading on a desk in portrait mode,
and others don't...yet. (I find that important.) We don't know enough
about how many buttons are best, nor how to place them, though
manufacturers are experimenting. I'm sure there are other physical
attributes to be worked out.
As I pointed out in my First Impressions essay, the default values for things aren't always tuned to tablet use.
On top of all this,
the weight and battery life still isn't down far enough, though the
4.25 lbs. of the Toshiba Tablet PC is much better than the 6 lbs. of
the old GRiD Convertible. (Since both are normal convertibles with
similar batter life, they are a good comparison.)
Why we'll see renewed advancement
In the early 1990's,
innovation in tablet and pen computing moved at a rapid rate. Once the
hardware and operating systems companies stopped pushing it, though,
independent software developers stopped. Without constant trying of new things, and testing them in the marketplace, it is hard to have advancement.
The main "new" thing you hear from Microsoft has been about their book
reading software, developed for other purposes. While functionally
similar to Slate's old PenBook and other products, Microsoft persevered
in the image quality area to solve various problems and get a nice
improvement in the eyes of many people. This improvement shows what
happens when you keep trying.
The big thing from
my viewpoint, though, is Microsoft's trying again and going to the
trouble of integrating basic tablet and pen functionality into the
latest version of Windows, and simultaneously driving better hardware
with some minimum requirements. In addition, they are spending the time
and money to upgrade their Office products with tablet-specific
features and to provide a complete set of APIs for developers of other
products.
If developers learn
about what was done in the past, they can move ahead and produce better
solutions to the problems we were addressing, and discover new areas to
be covered. Software development is a continuous process of building on what came before, and then testing with real use.
By Microsoft starting with an advanced ink application of the last
generation, they've set the bar high enough to give people a boost. If
they really leave things open for outside development (from both a
technical and business viewpoint), and continue innovating themselves,
new ideas can be tested and evaluated by the market. The fact that we
now have good hardware with lots of marketing behind it means there
will be at least some market for new software.
Remember what
happened with the Internet as developers experimented with HTML after
the early browsers came out. Compare what web sites looked like in 1994
and 1996 to today. (For example, compare the early browser-based web
site authoring systems to later ones like Trellix's, and you'll see
huge improvement.) Now that we have a basis to build upon, that type of
advancement, like we saw in tablet and pen computing in the early
1990's, can resume where it left off.
- Dan Bricklin, 22 November 2002
|
|
|
© Copyright 1999-2002 by Daniel Bricklin
All Rights Reserved.
|