All add-ons currently require PHP 7.4 or greater.
On July 4th 2024 PHP 8.2 will be the new minimum requirement for all add-ons. Expect any add-on released after that date to require 8.2 or greater. Some releases may not immediately take advantage of 8.x specific features in PHP, which means you might, be able to continue using new releases in PHP 7.4, however, if you experience an error the first thing you should do is update to PHP 8.2 then create a support ticket if the error persists.
Please read about the changes to BoldMinded add-on licensing.
Ticket: Issues with import
Status | Resolved |
Add-on / Version | DataGrab 4.0.5 |
Severity | |
EE Version | 6.3.4 |
Blue Fish
Jul 28, 2022I am having 2 issues.
1. A simple cron job has been set up to trigger the import. When using the Import Url provided by DataGrab or the method via template, the import isn’t triggered within the Control Panel. The cron does point to a shell script.
The cron command is below:
*/30 * * * * /bin/sh /path/to/import/script/jobs.sh >> /path/for/logging/jobs.log 2>&1
Within the shell script I have the following:
/usr/bin/curl "https:/domain.tld/?ACT=28&id=35&limit=0"
Again I have used both the Import Url and template method within the shell script.
The output of the cron is simply Import has finished. I receive the same output when running the import via ssh. However, the Control Panel and channel that entries are to be imported to isn’t updated.
2. When running the import from within the Control Panel, visiting the Import Url or url to import template, the import runs. The issue is it isn’t importing everything into EE. It is recognizing duplicate entries but isn’t using the field designated within the import configuration to check for duplicates but rather it is looking at potentially the title as the default.
Logs for DataGrab are being stored within EE’s Developer Log
This is an EE2 to EE6 upgrade, fyi. The site was previously using AJW Datagrab
BoldMinded (Brian)
Did you see this ticket? https://boldminded.com/support/ticket/2421
Blue Fish
I have and I have tried a combo of limit counts, ranging from 0 to 500 to no limit being declared at all. No dice. Issue 2 persist with or without the cron working.
Blue Fish
Comment has been marked private.
Blue Fish
After some clean up, uninstalling and re-installing the add-on, resetting up the import with same configuration settings and running the import we have found what is possibly part of issue 2. It seems a key feature from EE2, auto-incrementing of url_titles, has been removed so when the import runs, it finds entries that are different based on the field designated to check for duplicates but is unable to create the new entry due to the title of the entry matching a pre-existing entry. Because of no auto-incrementing the entry cannot be created.
So consider this partially resolved. Issue 1, DataGrab not triggering the import via cron still persists.
BoldMinded (Brian)
Thanks for the update. I’m on vacation for the next few days, so it might be a week or so until I can really take a look at this.
BoldMinded (Brian)
FYI the CP login you provided isn’t working.
Blue Fish
Comment has been marked private.
Blue Fish
Comment has been marked private.
BoldMinded (Brian)
Comment has been marked private.
Blue Fish
Afetr some continued digging, I was able to get the cron job to work. The only way it will work is when line 268 of models/datagrab_model.php is commented out. This is
When the import runs via cron new entries are not able to be imported and I receive the following message:
DataGrab: Import error with entry {ENTRY TITLE} with the status field: Invalid Selection
The channel is set to a default status of open. I have changed the config setting within DataGrab to both Channel Default and open, getting same result both times.
With this install, a brand new clean install not upgraded, it seems I am unable to import within the Control Panel as well. The only way I was able to get the entries to import before the cron was hitting the import url within my browser.
I’ve updated the cron to run every 30 mins. Logging is in the Developer Logs in the EE Control Panel
Blue Fish
Comment has been marked private.
BoldMinded (Brian)
Try the build in the next comment, and make sure the import is configured to have a Default Author. Based on the last error message I’m guessing you didn’t have a member with the ID of 1 in your database, otherwise it shouldn’t have thrown that error.
BoldMinded (Brian)
Comment has been marked private.
Blue Fish
the import had a default author and originally there was a member with the ID of 1 and still the issue persisted. I will try the build and let you know of any updates, etc.
Thanks!
Blue Fish
Update!
The build provided previously resolved the issue. I added it to a clean install with a member with the an ID of 1. Also tested on the upgraded install (EE2 -> EE6) and it worked there as well. The cron runs as intended.
Please note, on the upgraded install, there isn’t a member with an ID of 1 and DataGrab still works
Appreciate the help with this.
Blue Fish
Comment has been marked private.
BoldMinded (Brian)
I have this on line 142 of the model file, is that what you’re seeing as well?
BoldMinded (Brian)
If that is what you’re seeing then I think the actual fix is this, which is to add the conditional at the beginning of the function on line 147