LISTSERV mailing list manager LISTSERV 16.0

Help for CHRONAM-USERS Archives


CHRONAM-USERS Archives

CHRONAM-USERS Archives


CHRONAM-USERS@LISTSERV.LOC.GOV


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

CHRONAM-USERS Home

CHRONAM-USERS Home

CHRONAM-USERS  July 2013

CHRONAM-USERS July 2013

Subject:

Re: Slow Page Load Times and Production Settings

From:

"Brunton, David" <[log in to unmask]>

Reply-To:

Data, API, website, and code of the Chronicling America website" <[log in to unmask]>

Date:

Wed, 3 Jul 2013 08:25:12 -0400

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (35 lines)

Hi Michael,

The site looks really nice!

My guess about what is happening: your MySQL index size exceeds the amount memory you have available to MySQL.  This could be either because of a configuration in your my.cnf, or it could be a physical limitation of the machine (e.g. there is too little RAM physically on the machine to fit the index).

If you can tweak your MySQL configuration to add RAM, that should help, but if you can't, I think putting a reverse proxy / cache in front of it should alleviate most of the problem.  We have some similarly slow pages on one of our instances of the software that load in less than a second from the cache.

How many newspapers do you have listed on that page?  Other options would be to either remove some of the query-heavy fields from the table or to paginate the list, depending upon how much you want to dig into the templates.

Best,
David.

-----Original Message-----
From: Data, API, website, and code of the Chronicling America website [mailto:[log in to unmask]] On Behalf Of Michael Beccaria
Sent: Tuesday, July 02, 2013 6:50 PM
To: [log in to unmask]
Subject: Slow Page Load Times and Production Settings

I'm using the new code base and have a collection of newspapers that total over 200,000 pages. I know the LOC uses caching software (varnish) to speed up page loads and we currently don't have that running. On our site, the load time for the newspaper list page is probably close to 5 minutes (http://nyshistoricnewspapers.org/newspapers/) but the other pages load relatively quickly.

Is there something wrong in the code or is that expected given the queries that django is trying to execute and I need to install a cache to make it work faster?

This is part of a bigger question. What general recommendations do any of you have for putting this software into production with multi-million page collections? This could be server/network specific (Ram, virtualization, multi-server, etc.) or software specific (caching, settings, etc.). We want to release this site to the public relatively soon and want to gear it up to be ready to get hit by users.

Thanks so much for your suggestions and guidance.

Mike Beccaria
Systems Librarian
Head of Digital Initiative
Paul Smith's College
518.327.6376
[log in to unmask]
Become a friend of Paul Smith's Library on Facebook today!

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

June 2019
April 2019
March 2019
February 2019
October 2018
August 2018
July 2018
June 2018
May 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
May 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
March 2015
January 2015
November 2014
October 2014
September 2014
June 2014
May 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
February 2013

ATOM RSS1 RSS2



LISTSERV.LOC.GOV

CataList Email List Search Powered by the LISTSERV Email List Manager