Thread: Go Fetch Jobs not executing all schedules

Go Fetch Jobs not executing all schedules

Hi Bruno,
Me again!
I have been having an issue with all of the schedules running. Let me explain where I'm at.
- I have 35 schedules set-up, 35 templates, all different. So 1 schedule per template.
- It runs daily
- 120 seconds between schedules
- currently about 7 to 8,000 jobs loaded, all from this plugin.
- Daily, between the 35 schedules, I am loading about 150 to 250 new jobs.
- Running manually, each schedule takes between 40 seconds up to about 1 min, 45 secs

The main issue I have is that I get through a certain number of schedules and then it just stops, leaving several schedules un-executed. The schedules run from the top of the list to the bottom, so it's always the top bunch that run, while the bottom do not.
The issue is not related to any particular schedule or template. After running the debug, it is always a php timeout or memory limit error. Increasing the limits has the effect of having most (or all) of the schedules run.
BUT, over time, fewer and fewer schedules run. It would seem that the more data (jobs) on the site, the longer it takes to execute, and hence the errors.
What is weird is that the schedules stop running part way through, but when run manually, they work fine. Well, until today when even a manual run wouldn't work. I had to up the memory limit again!!

As of now, my php execution timeout is 240 seconds, and my php memory is 768M.

My question is this:
Is this normal or expected based on the amount of data we're talking about here?
OR,
do you think there is some other host issue or plugin issue causing the requirements for really high limits?

Thanks,
Nick

Urls for each schedule