BoldMinded

Publisher does not support the Fluid field type. Please do not contact asking when support will be available.

If you purchased an add-on from expressionengine.com, be sure to visit boldminded.com/claim to add the license to your account here on boldminded.com.

Ticket: DataGrab Cron stuck

Status Resolved
Add-on / Version DataGrab 4.0.4
Severity
EE Version 6.3.4

XL Chia

May 23, 2022

Hi, 

I’ve created a template to run several grabs from Eventbrite API, mainly attendee information, in a channel Orders. There are 10 ticket types, so I have set up a template to run these imports

{exp:datagrab:run_saved_import id="11"}
{exp:datagrab:run_saved_import id="12"}
{exp:datagrab:run_saved_import id="3"}
{exp:datagrab:run_saved_import id="4"}
{exp:datagrab:run_saved_import id="2"}
{exp:datagrab:run_saved_import id="5"}
{exp:datagrab:run_saved_import id="8"}
{exp:datagrab:run_saved_import id="9"}
{exp:datagrab:run_saved_import id="6"}
{exp:datagrab:run_saved_import id="7"}

The url for this template is https://afcc.com.sg/account/tickets-sync

The latest log file gives this

08:06:00 05/23/2022 Import #11 Started
08:06:01 05/23/2022 Import #11 Initialized
08:06:01 05/23/2022 Begin Importing [36355742395705369349001]
08:06:01 05/23/2022 [36355742395705369349001] already exists. Skipping entry per import setting.
08:06:01 05/23/2022 Import #11 End

The import doesn’t seem to run past the first line {exp:datagrab:run_saved_import id="11"}

Alternatively, I’ve also used the import urls separately in my cron tab. The cron fires but the import don’t finish. (screenshot attached with caption datagrab – cron not finishing)

My goal is to run these imports hourly, any help debugging will be great!

Thanks,
XL

 

#1

BoldMinded (Brian)

May 23, 2022

Thanks for reporting this, but I don’t have enough information to take any action on.

Can you share with me an example file you’re trying to import?

What happens if you execute just 1 import at a time manually, do they finish?

#2

BoldMinded (Brian)

May 23, 2022

Most importantly, lets forget about the crontab for a bit and focus on my last question and just manually executing the imports one at a time.

#3

XL Chia

May 23, 2022

Hi Brian,

Thanks for the quick reply.

All of them are variations of grabbing the JSON feed from Eventbrite, something like this https://www.eventbriteapi.com/v3/events/322534087137/attendees/?status=attending&changed_since=2022-05-21T03%3A14%3A58Z&token=XXX

Manually running the import within the addon, and running the import urls work, and they will complete. But they don’t complete on Cron, as shown in Screenshot_2022-05-23_at_10-00-32_DataGrab_Configuration_ExpressionEngine.png

#4

BoldMinded (Brian)

May 24, 2022

I think I know what the issue is. You need to change the limit value. The cron is only executing 1 at a time then stopping. Or you need to change your cron so it actually loads a web browser in the background so it can make separate http requests for each item to import if you want to keep the limit to 1.

#5

XL Chia

May 25, 2022

Hi Brian, you are right. We can close this as it’s not a Datagrab, but a cron request issue.

Thanks for the insight!

Login to reply