Print

Print


I got an email that this got rejected to the listserv yesterday when I sent it. Resending. Sorry if it comes in twice.

Stephanie,
Before you pull the plug or purchase an Aware license you have 2 options (at least for testing):
1. Change the settings file so that it uses TIFF images rather than jp2's. This works but is VERY slow.
2. I think the only thing you might need to do is convert the tiff images to another format (jpg for example) and then update the xml files to point to those files instead. For example, if you edit this file: http://chroniclingamerica.loc.gov/data/batches/batch_dlc_jamaica_ver01/data/sn83030214/00175039259/00175039259.xml and have it point to target1.jpg instead of target1.jp2 (and do that for subsequent xml files) I THINK it will work out of the box. You can probably pull off a massive find and replace using a program like notepad++ to test it out.

I did a lot of custom work on chronam for the Northern NY Library Network (http://nyshistoricnewspapers.org) for their digital newspaper collection and I know the Historic Oregon Newspapers has done a lot of work as well (http://oregonnews.uoregon.edu/) so between the LOC and others you might find this feasable. It's really great software!

Mike Beccaria
Systems Librarian
Head of Digital Initiative
Paul Smith's College
518.327.6376
[log in to unmask]<mailto:[log in to unmask]>

From: Data, API, website, and code of the Chronicling America website [mailto:[log in to unmask]]<mailto:[mailto:[log in to unmask]]> On Behalf Of Williams, Stephanie
Sent: Wednesday, July 17, 2013 8:53 AM
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: JPEG 2000 Library and Alternatives?

Thanks very much for all of the swift responses. This is really helpful.

It's encouraging to know that others have had success adapting the software to use other image types/libraries, and that's something we'd definitely be interested in down the line. The sticking point right now is that we were hoping to evaluate the software for some specific needs before we devoted a huge amount of development/administration time to it.  Changing the image type/library at least *sounds* like it would require a significant amount of development and troubleshooting; am I wrong about that?

I remember some discussions from awhile back about the possibility of an Amazon Machine Image becoming available again; does anyone have any news about that?  Otherwise, it sounds like our options for evaluating the software will require either 1) purchasing a license from Aware or 2) committing to some custom development work.

Thank you again!

Stephanie Williams
North Carolina Digital Heritage Center
http://www.digitalnc.org

P.S. - Has anyone had any success contacting Aware re: an SDK license? We've both called and emailed and haven't heard anything in about a week.


________________________________
From: Data, API, website, and code of the Chronicling America website [[log in to unmask]] on behalf of Summers, Ed [[log in to unmask]]
Sent: Thursday, July 11, 2013 8:50 AM
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: JPEG 2000 Library and Alternatives?
Hi Stephanie,

We do have a license to the Aware SDK, and it's what we use in production at chroniclingamerica.loc.gov. I personally think that releasing chronam as opensource with the proprietary dependency is somewhat problematic.

Mark Phillips led a session at the code4lib 2012 NDNP pre-conference, where we did some user testing to see whether there was any noticeable difference when generating thumbnails and image tiles from a JPEG derivative of the TIFF compared to the equivalent generated from the JP2. We couldn't notice any difference at all, even when zoomed in all the way on a newspaper page. The obvious advantage is that the JPG takes up significantly less space (both on disk and in memory) than the TIFF, and it is usable by opensource image libraries like PIL [1] which chronam is already using.

So, in theory, I think we could re-work the batch loader so that it converts the TIFF to a JPG. This would mean you could keep your batches on spinning disk using 6-7 times less storage. Since they are smaller the files also will read in faster than the TIFF, and decrease the memory footprint of the webapp.  Is this something that would be of value? I seem to remember Karen Estlund and Duncan Barth (University Oregon) and Luis Baquera (UC Riverside) were interested in this option at the pre-conference--but I don't know if they are still interested. I think it should be possible to keep the batch loader working the way it currently does (using the JP2 if present, or the TIFF) unless a --convert-to-jpeg option is given.

Does this sound like something worth pursuing?
//Ed

[1] http://pythonware.com/products/pil/

From: Data, API, website, and code of the Chronicling America website [mailto:[log in to unmask]]<mailto:[mailto:[log in to unmask]]> On Behalf Of Williams, Stephanie
Sent: Wednesday, July 10, 2013 5:24 PM
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: JPEG 2000 Library and Alternatives?

Hello!

We have just begun our attempts to install/use Chronam and have a question about the JPEG 2000 image handler. It looks like the software is dependent on Aware's JPEG 2000 library/SDK.

Does LC have a license to allow the use of the SDK?  Is everyone else going this route?  (We've requested information about using it, but if it's costly we're interested in trying to make this work in an open source way--even if that means serving a different type of image).

Thanks very much!

Stephanie Williams
North Carolina Digital Heritage Center
http://digitalnc.org