Publisher does not support the Fluid field type. Please do not contact asking when support will be available.

If you purchased an add-on from expressionengine.com, be sure to visit boldminded.com/claim to add the license to your account here on boldminded.com.

Ticket: CLI: Import works in CP, but CLI has endless loop of 0% reported to log

Status Resolved
Add-on / Version DataGrab 4.2.1
Severity
EE Version 6.4.3

Paul Larson

Jan 09, 2023

php /var/www/vhosts/mysite.com/httpdocs/system/ee/eecli.php import:run—import_id=53 && cat ../system/user/cache/DataGrab-import.log

Screenshot:
https://www.dropbox.com/s/7t5vabep2zu2mjp/2023-01-09_11-38-37.png?dl=0

Data File:

https://www.dropbox.com/s/apdbcspd2aw0v82/is_LGTNG_parent.tsv?dl=0

Settings:

https://www.dropbox.com/s/8lidac7rmpxjdu1/2023-01-09_11-42-25.png?dl=0

When run from CP, log is:

17:43:38 01/09/2023 Import #53 Started
17:43:38 01/09/2023 Import #53 Initialized
17:43:38 01/09/2023 Begin Importing [25PR54]
17:43:38 01/09/2023 [25PR54] already exists. Updating.
17:43:40 01/09/2023 Calling after_channel_entry_save() hook.
17:43:40 01/09/2023 Updated 1 entries

#1

Paul Larson

Note: Conveniently, the import via CLI works. (updates field data)

It’s just the endless loop that occurs in logging/execution.

#2

BoldMinded (Brian)

That is interesting b/c using the 3 entry .tsv file you gave me the log stops and both the CLI and CP imports stop accordingly, no loops.

#3

Paul Larson

Correct! The 3-liner works.

#4

BoldMinded (Brian)

The .tsv file you linked in the ticket is empty…. all it has is the heading row.

#5

Paul Larson

Argh. Fixed!

#6

BoldMinded (Brian)

I mentioned this in Slack but posting here for posterity. The issue seems to only occur with CSV files with 1 or 0 rows to import. A CSV with 1 row to import actually has 2 rows, one of them being the header, which is ignored. This scenario is causing the looping logic to not be able to escape the loop. I’m looking into a solution without having to heavily refactor things and potentially cause regressions with other imports.

#7

BoldMinded (Brian)

I’m not going to spend time finding a hack fix for this. Instead I’m going to spend time into improving DataGrab’s entire import system to use pub/sub. https://eecms.slack.com/archives/C04CUNNR9/p1674828964829489

Closing this out.

Login to reply