Publisher does not support the Fluid field type. Please do not contact asking when support will be available.

If you purchased an add-on from expressionengine.com, be sure to visit boldminded.com/claim to add the license to your account here on boldminded.com.

Ticket: Import fails

Status	Resolved
Add-on / Version	DataGrab 4.2.1
Severity
EE Version	6.4.5

Zack Morgan

Feb 21, 2023

Hi,

Just purchased and installed DataGrab, set up a test import, and it fails, with an error in the logs:

17:20:03 02/21/2023 Import #1 Started
17:20:03 02/21/2023 Import #1 Initialized
17:20:03 02/21/2023 Begin Importing [Housing Association]
17:20:03 02/21/2023 Call to undefined function BoldMinded\DataGrab\Service\directory_map()
17:20:03 02/21/2023 #0 /home/accountnameredacted/public_html/system/user/addons/datagrab/models/datagrab_model.php(1095): BoldMinded\DataGrab\Service\DataGrabLoader->fetchModuleHandlers()
#1 /home/accountnameredacted/public_html/system/user/addons/datagrab/models/datagrab_model.php(901): Datagrab_model->handleThirdPartyAddons(Array, Array, Array, ‘create’)
#2 /home/accountnameredacted/public_html/system/user/addons/datagrab/models/datagrab_model.php(292): Datagrab_model->importItem(Array, Array)
#3 /home/accountnameredacted/public_html/system/user/addons/datagrab/mod.datagrab.php(149): Datagrab_model->do_import(Object(Datagrab_csv), Array)
#4 /home/accountnameredacted/public_html/system/ee/legacy/libraries/Actions.php(175): Datagrab->run_action()
#5 /home/accountnameredacted/public_html/system/ee/legacy/libraries/Core.php(657): EE_Actions->__construct(true, Object(Closure))
#6 /home/accountnameredacted/public_html/system/ee/legacy/controllers/ee.php(51): EE_Core->generate_action(true)
#7 [internal function]: EE->index()
#8 /home/accountnameredacted/public_html/system/ee/ExpressionEngine/Core/Core.php(266): call_user_func_array(Array, Array)
#9 /home/accountnameredacted/public_html/system/ee/ExpressionEngine/Core/Core.php(122): ExpressionEngine\Core\Core->runController(Array)
#10 /home/accountnameredacted/public_html/system/ee/ExpressionEngine/Boot/boot.php(161): ExpressionEngine\Core\Core->run(Object(ExpressionEngine\Core\Request))
#11 /home/accountnameredacted/public_html/index.php(164): require_once(’/home/positivea…’)
#12 {main}

The site is on the latest EE6, PHP Version 7.4.22, so not sure what the issue might be?

BoldMinded (Brian)

Feb 21, 2023

This must be an EE 6 issue b/c I don’t see it in 7. In there Service/DataGrabLoader.php file line 146 add this to the beginning of the function:

public function fetchModuleHandlers(): array
    {
        // Add this conditional:
        if (!function_exists('directory_map')) {
            require_once APPPATH . 'helpers/directory_helper.php';
        }

See if that fixes it.

Zack Morgan

Feb 22, 2023

Thanks, that has allowed the import to run. However, it has raised another interesting issue. I am importing a test csv (one header row and one data row) and on running the import, I get the response ‘1 entries added’, and the logs show this:

11:41:13 02/22/2023 Import #1 Started
11:41:13 02/22/2023 Import #1 Initialized
11:41:13 02/22/2023 Begin Importing [Housing Association]
11:41:13 02/22/2023 [Housing Association] is a new entry
11:41:13 02/22/2023 Added 1 entries
11:41:13 02/22/2023 Clearing all cache

However…

Multiple identical entries are created. On first run I had 10, on the second I had 24, and on the third attempt I had 6.

PS Another small issue - whenever I update the import settings, the description field is overwritten with a single character: ‘7’. I don’t care about this too much but I guess it’s worth letting you know.

BoldMinded (Brian)

Feb 22, 2023

CSV’s with a single entry (e.g. 2 row CSV files where the first row is the header) or CSV files with no entries (e.g. just the header) have been an issue lately, and I think an issue for the entirety of DataGrab’s existence as far as I can tell from the code. It doesn’t handle it correctly. Try adding a second dummy row of data to the CSV and see if that corrects the issue. It’s an issue with how DG constantly reads from the import file and tries to skip rows and it’s not correctly skipping the heading row.

To be entirely honest I’m not putting more effort into correcting this in DataGrab 4.x b/c the effort is not worth it. I’m instead focusing effort into DG 5 that is based on Laravel’s Queue service, so issues like this shouldn’t be a problem anymore. I’m hoping to have a beta version of v5 available in the next couple of weeks. If you’d like a copy of that let me know, but I won’t be spending any additional time trying to fix this issue in v4. Sorry.

Zack Morgan

Feb 22, 2023

OK, importing more than one row at a time is normal, so I can live with that. Thanks

BoldMinded (Brian)

Feb 22, 2023

Sounds good. Let me know if anything else comes up.

BoldMinded (Brian)

Feb 27, 2023

Zack, if you want you can try v5. I’ll post the links in the next comment then close this ticket. If you encounter any issues just open a new ticket.

BoldMinded (Brian)

Feb 27, 2023

Comment has been marked private.