[CentOS] Document Scanning and Storage

Thu Sep 13 00:09:20 UTC 2007
gjgowey at tmo.blackberry.net <gjgowey at tmo.blackberry.net>

I guess it boils down to what someone needs out of such a system.  If you're looking to store text data from ocr conversion (for searching) then you could use a clob column as well as a blob for the image based version.  As for storing these in the db, php makes that easy since all file uploads are automatically stored in temporary files and it's just a matter of reading the contents and storing it in the db.  User rights can be regulated using a separate column for the entry.  Basically 6 columns would be all that's needed: record id (incrementing integer), a blob, a clob, group access field (I'd suggest an integer), date stored, and user who created the record. 

Of course a separate table to convert user logins to what groups they can access.

Now if you want to get more elaborate you could add a version column to the db so that "updates" to the record will create another row with the version number increased so the original isn't modified (ala subversion, cvs, etc).  Another column that stores a Boolean value could be used to store if the record is "deleted" (hidden in reality) or not.

Logins can be stored in the db and an apache mod can be used to use that information.

It's really not elaborate in creating the storage part.  It's always the ui that is the worst part of any project (both the admin and end user ui), but eclipse with struts and tiles or XML and xsl instead of tiles (personally, I prefer the xml/xsl approach, but to each their own) can make it less painful.  I've had rather good experiences with eclipse and the redhat plugin (formerly exadel) for working with struts and tiles.


Sent from my BlackBerry wireless handheld.

-----Original Message-----
From: Bill Campbell <centos at celestial.com>

Date: Wed, 12 Sep 2007 16:26:00 
To:centos at centos.org
Subject: Re: [CentOS] Document Scanning and Storage

On Wed, Sep 12, 2007, Dennis McLeod wrote:
>I'd like to start scanning our boxed up documents. I'd say about 30,000
>files total.
>Mostly to eliminate the boxes of paper we have.
>I'd like to scan them, store them, Have some sort of index, and be able to
>retrieve them on multiple machines. I think PDF would be the desired format.
>I'd like be able to set some permissions as well. (not a deal breaker...)
>I've searched Sourceforge, and have seen knowledgetree, myDMS, contineo,
>etc, but really would like to hear from someone that is using something

This is not a trivial operation.

I was a principal in a company that developed a Linux based system to do
this about 8 years ago, with a product good enough that it made national
news when Bill Gates' home town of Medina Washington bought a system from
us, not a Windows based system.

The scanning can be done pretty nicely using a scanner with and ADF
(Automatic Document Feeder), and xsane has the ability to number pages
skipping numbers so one can can both sides of two-sided documents in two
passes.  The biggest issue is probably doing the OCR conversion to get text
for indexing.  We used proprietary software from Vividata for this which
worked pretty well.  I haven't looked seriously at gocr or other open
source OCR software for Linux so don't know how well it would work.

I've been using the ReadIris OCR software on Macs recently, which has some
very nice features such as handling multi-page PDF files well.

If I were to tackle this today, I would probably do it using Plone since it
handles things like indexing and organization well.

INTERNET:   bill at celestial.com  Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/  PO Box 820; 6641 E. Mercer Way
FAX:            (206) 232-9186  Mercer Island, WA 98040-0820; (206) 236-1676

Breathe fire, slay dragons, and take chances. Failure is temporary, regret
is eternal.
CentOS mailing list
CentOS at centos.org