A New Script for Searching Texts Written by Hand
GOOGLE’ S experimental service Google Scholar (www.scholar.google.com ) scours the Web for academic papers. But while it may prove an asset for students and those who work in areas like science, neither it nor any other search engine is very useful for historians whose research involves plowing through documents from the time when handwriting ruled.
By IAN AUSTEN
December 30, 2004, New York Times, Page 86, Section Circuits, Vol CLIV, Number 53,079
GOOGLE’ S experimental service Google Scholar (www.scholar.google.com ) scours the Web for academic papers. But while it may prove an asset for students and those who work in areas like science, neither it nor any other search engine is very useful for historians whose research involves plowing through documents from the time when handwriting ruled.
Even after handwritten documents have been scanned and made available through the Web, it is not possible for Google or any other widely used searching technology to read them. The only method in use for searching such documents involves first having them typed into standard computer text, a costly and time-consuming process.
“There is an enormous amount of handwritten stuff locked away in many archives, libraries and museums,” said R. Manmatha, a research assistant professor with the Center for Intelligent Information Retrieval at the University of Massachusetts, Amherst. “Most of the time when people do research they just ignore this stuff because it’s not accessible.”
With handwritten documents, it is not just scholars who are frustrated by their inaccessibility. People tracing their family histories often find the process involves reading a lot of mostly irrelevant documents to find relatively small amounts of information.
Most of Dr. Manmatha’s work in computer science has involved using computers to search for photographs or bits of film in digital databases. When he turned to the problem of searching handwritten texts, he found his experience to be very complementary to the task.
“Handwritten manuscripts are just images,” he said.
Some sophisticated handwriting recognition systems are in use. The United States Postal Service and postal agencies around the world use them to read addresses at sorting stations. And forms filled out during the last United States Census were scanned and read by computers.
But Dr. Manmatha said the experience developed from those systems was not particularly useful when he and two graduate students, Toni Rath and Victor Lavrenko (who is now a postdoctoral research associate), began work on their manuscript searching project. The current systems have to cope with only a limited range of material - for example, names and addresses - written in a consistent format. On top of that, postal systems have large numbers of human readers as backups, something that wouldn’t be possible for a manuscript search engine.
To develop their system, Dr. Manmatha and his students obtained about 1,000 pages of George Washington’s correspondence that had been scanned from microfilm by the Library of Congress.
They began by working on a variation of an approach used to search digital photographs, trying to match specific typewritten letters with digital images of their handwritten counterparts. But Dr. Manmatha said the inherent variations in handwriting quickly made that approach too cumbersome.
The problem was also made more difficult by the fact that Washington dictated to several secretaries and wrote some of his letters personally. The result is that his papers contain at least five handwriting styles.
The breakthrough came from looking at research into how people read, Dr. Manmatha said. Rather than analyzing individual letters, he said, people look at words and even parts of sentences as whole units.
To develop software that would take a similar holistic approach, Dr. Manmatha turned to an idea developed to let search engine users enter queries in their own language to find Web pages written in another language. Rather than mapping words between the two languages one for one, Dr. Manmatha said, those systems rely on software that is trained to spot common ground. For example, he said, English-language search engines that can look through French Web pages use training software that compares pages and pages of debate transcripts from the Canadian House of Commons, which publishes all of its proceedings in both languages.
To train his manuscript software similarly, Dr. Manmatha had a portion of Washington’s papers converted into computer text. His software is given some help as it makes its learning comparisons. For example, the system is designed to eliminate the slanting in the handwriting and to rescale words to make them a consistent size.
Even after training, the resulting software lacks the accuracy of programs used to read and digitize printed books, Dr. Manmatha said. But that it is not a significant problem.
“That’s the difference between having to recognize a thing and having to search it,” he said. “You don’t have to get every word right.”
Right now, Dr. Manmatha believes that the system is about 65 percent accurate. Some of that has to do with the variations in handwriting between Washington and his secretaries. The inconsistent ink fading in letters written with quill pens is another problem. And the fact that the Library of Congress’s digital images were created from microfilm copies rather than original documents was also an issue.
Dr. Manmatha believes that refinements to the software will increase its accuracy, but that even the current level could be a useful tool for researchers. It could allow them to reduce their reading time by at least broadly indicating if a group of papers is potentially useful, he said.
Outside of accuracy, Dr. Manmatha said there were two things that needed improvement in the system. It can generally cope with the difference in the writing styles in Washington’s papers because they contain broad similarities. But somewhat like voice recognition software, the program has to be retrained before it can digest documents in a significantly different hand. Dr. Manmatha said that eliminating or minimizing that retraining step would be difficult, but that he believed it would be possible.
In addition, before the system can move from its current 1,000 pages to the full 140,000 pages of Washington manuscripts in the library’s digital collection, it will also need to operate more quickly. That may be as simple as adding more computing power, Dr. Manmatha said.
Though no library or archive has yet approached Dr. Manmatha about the system, he will brief Google about it early next year. With sufficient funds for software development and document scanning, Dr. Manmatha said, it may be possible within a decade for people to search historical manuscripts from home as easily as they now locate anything else on the Web.
“A tool like this will help people access such material and make possible new discoveries,” he said.
Tags: history, research, Technology