Adjust queues on Ingestion 2 workers

Description

The resque workers are very large – starting life at 200+ MB and growing to multiple gigabytes. A decent-sized mapping can take up half of the server's real memory and cause paging.

Do some combination of the following:

  • Adjust the queues on the production workers so that only one or two mappers run on one box.

  • Add production worker instances.

  • Upsize the memory of worker-prod2.

Activity

Show:
Mark Breedlove
June 23, 2015, 5:34 PM

worker-prod2 has been upsized, and the worker counts have been adjusted, but I still plan to add another worker instance.

Mark Breedlove
June 23, 2015, 8:07 PM

A third worker instance has been added. Let's see how this goes.

Assignee

Mark Breedlove

Reporter

Mark Breedlove

Labels

None

Priority

High
Configure