Sorry to come back with another problem. Same site - this time trying to run the big import on the staging server. The import on old DG had to be split into two parts - first part is 3000 entries, the second is the remainder, about 1200 or so entries. On the upgraded DG these are still split in this way.
Previously with DG4.0.5, it ran the 3000 entry update without overt issues that weren’t my own fault (the Assets images thing). And once I got the Assets thing squared away, re-running it was successful.
This time, on DG4.2.0, it’s getting stuck at about 2866* entries and not completing. The logs don’t report any errors, but if I dig into the HTML of the CP, it does end up with one.. it’s exhausting the memory.
Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 622592 bytes) in
on line 112
*This is my best guess with rough math—the logs end at 8602 lines, there are roughly 3 lines per-entry, the first two lines are the ‘start’ phrases, and the last two lines are the last entry it was trying before it gave up. So 8598/3 is 2866.
The second attempt with the same file got slightly further - at 2946 entries, but did not complete entry #2946 and halted.
When I ran the import of 3000 entries on 4.0.5, the logs ended at 9166 lines - 47 entries added, 2944 updated & 9 that were flagged as the duplicate url-titles, totaling 3000.
Additionally, in the cell of the table in the DG settings under ‘Status’, it is showing a ‘NOTICE’ warning, instead of the ‘completed’ badge.
Undefined variable: timestamp
user/addons/datagrab/models/datagrab_model.php, line 800
To be frank, this error seems to appear whether the import runs successfully or not (like on a two entry test), so I’m not entirely sure if it’s even helpful, or if it’s contributing to why the import isn’t finishing.
I did try to revert the install back to 4.0.5 and re-run the import with the same CSV file that was getting stuck on 4.2.0, and while the Notice errors do not appear, the import is still getting stuck with the memory exhaustion. We did include a .user.ini file to set the memory limit to -1 and time limit to 100 minutes, but it does not seem to make any difference - the exhaustion errors were the same both before and after adding the ini file.
Additional note—that same CSV file seemed to run fine on the production site which is still on EE5.3.0 & DG3.x.
Alternately, Caroline is trying to run the CLI importer but is running into other issues, I don’t know if it’s all related or what, but that may need to be a separate ticket if it isn’t.
Let me know if you want access to the staging server in this case since it is publicly accessible.