Adjust queues on Ingestion 2 workers
Description
The resque workers are very large – starting life at 200+ MB and growing to multiple gigabytes. A decent-sized mapping can take up half of the server's real memory and cause paging.
Do some combination of the following:
Adjust the queues on the production workers so that only one or two mappers run on one box.
Add production worker instances.
Upsize the memory of worker-prod2.
Activity
Show:
Mark Breedlove
June 23, 2015, 5:34 PM
worker-prod2 has been upsized, and the worker counts have been adjusted, but I still plan to add another worker instance.
Mark Breedlove
June 23, 2015, 8:07 PM
A third worker instance has been added. Let's see how this goes.
Assignee
Mark Breedlove
Reporter
Mark Breedlove
Labels
None
Priority
Configure