Publisher does not support the Fluid field type. Please do not contact asking when support will be available.
If you purchased an add-on from expressionengine.com, be sure to visit boldminded.com/claim to add the license to your account here on boldminded.com.
Ticket: Batching and CRON for import
| Status | Resolved |
| Add-on / Version | DataGrab Data Grab 4.0.2 |
| Severity | |
| EE Version | EE 6.3.2 |
Neil Evans
May 30, 2022Hey Support,
General Overview of what we are doing..
Running a fairly large import, that needs to be updated periodically over the day. So the batch amount was set to 10, and set to run every 15 minutes through a Cron…
I am using a simple Curl in the CRON, based on your docs:
curl—silent—compressed “http://example.com/index.php?ACT=20&id=id&limit=10&passkey=12345x”
The cron runs and the output emailed to myself shows that only the first batch of 10 is run. This makes sense as the batches are processed by a Meta Refresh which work within a browser… But do not work within a Curl request!
I saw in a previous response that you suggest launching a browser to process the batch which will then follow the meta refresh… But this is unlikely to exist on a production web server, at least it does not on mine!
Therefore, I am just asking if this is normal practise to process batches? Or am I just processing them?
Anyway, it explains why i was having a success importing from the browser, yet the client complains about the cron based ones not working!
BoldMinded (Brian)
May 31, 2022
If you need to import via Curl request then you’ll have to set the limit to 0, or a really high number so it tries to import them all at once.