In addition to the good advice you got from David and Chris, you might want to try the Django Debug Toolbar:
Once enabled you get handy information about database queries, template rendering, etc as you browse through your web app. It is a generally useful tool for any Django project, if you happen to have others.
One thing I noticed is that images take a long time to render as well. This can prove to be a problem even when you put Varnish in front of Chronam, since researchers often interact with a long tail of content. Are you using the TIFF files? Also, how much memory does your server make available to Chronam?
From: Data, API, website, and code of the Chronicling America website [mailto:[log in to unmask]] On Behalf Of Michael Beccaria
Sent: Tuesday, July 02, 2013 6:50 PM
To: [log in to unmask]
Subject: Slow Page Load Times and Production Settings
I'm using the new code base and have a collection of newspapers that total over 200,000 pages. I know the LOC uses caching software (varnish) to speed up page loads and we currently don't have that running. On our site, the load time for the newspaper list page is probably close to 5 minutes (http://nyshistoricnewspapers.org/newspapers/) but the other pages load relatively quickly.
Is there something wrong in the code or is that expected given the queries that django is trying to execute and I need to install a cache to make it work faster?
This is part of a bigger question. What general recommendations do any of you have for putting this software into production with multi-million page collections? This could be server/network specific (Ram, virtualization, multi-server, etc.) or software specific (caching, settings, etc.). We want to release this site to the public relatively soon and want to gear it up to be ready to get hit by users.
Thanks so much for your suggestions and guidance.
Head of Digital Initiative
Paul Smith's College
[log in to unmask]
Become a friend of Paul Smith's Library on Facebook today!