Publisher does not support the Fluid field type. Please do not contact asking when support will be available.

If you purchased an add-on from expressionengine.com, be sure to visit boldminded.com/claim to add the license to your account here on boldminded.com.

Ticket: DataGrab doesn’t import Grid data

Status	Resolved
Add-on / Version	DataGrab 4.2.2
Severity
EE Version	7.2.10

Daniel Szczukiewicz

Mar 03, 2023

Hi Brian,

I have a strange issue (probably PHP configuration). I try to grab the data into Grid field and it works well but on localhost only. Unfortunately it doesn’t work on live server. After configuration the Grid fields in DataGrab are always empty. I removed the Grid field, created a new one etc… Everything is the same as on localhost anyway.

What it can be?

PHP 8.0.28 (same issue at 7.4 & 8.1.9)

Thanks.

BoldMinded (Brian)

Hi, Daniel. I’m going to be honest, I’m going to need a lot more information. Screenshots of the configuration, small sample of the import file, more info about the Grid field itself, the DataGrab-import.log file. Right now I don’t have any information I can use to help with. I would say if it works on one server but not another, then it likely will be a configuration issue. Are there any PHP errors in the import log? Can the live server actually read the import file correctly?

Daniel Szczukiewicz

Sure, I’ll try to provide everything you need.

Yes, DataGrabb can read json file as I can see data on the first configuration screen (screen-1). Then I can choose the data for grid field (screen-2) and save it. But when I back to settings the grid field is empty (screen-3).

I was trying to change grid field from string to number - no impact. I deleted the grid and created a new one - no impact. Increased memory_limit and post_max_size - no impact.

BoldMinded (Brian)

What about there output from the DataGrab-import.log file?

Daniel Szczukiewicz

Well, I switched on my computer today and tried once again and.. it’s working! I do not know why. So configuration passed but this time import doesn’t work. All fields in grid have been set up as TEXT type with ALL allowed content. Logs return this one:

10:18:14 03/04/2023 Import #4 Started 10:18:14 03/04/2023 Import #4 Initialized 10:18:14 03/04/2023 Begin Importing [Ekstraklasa] 10:18:14 03/04/2023 Unsupported operand types: int + string …

I played with different types/allowed content but no results.

Example of JSON:

{
"rank": 15,
"team": {
"id": 349,
"name": "Piast Gliwice",
"logo": "https://media.api-sports.io/football/teams/349.png"
},
"points": 23,
"goalsDiff": -3,
"group": "Ekstraklasa",
"form": "WLWLD",
"status": "up",
"description": null,
"all": {
"played": 22,
"win": 6,
"draw": 5,
"lose": 11,
"goals": {
"for": 23,
"against": 26
}
},
"home": {
"played": 11,
"win": 3,
"draw": 2,
"lose": 6,
"goals": {
"for": 12,
"against": 13
}
},
"away": {
"played": 11,
"win": 3,
"draw": 3,
"lose": 5,
"goals": {
"for": 11,
"against": 13
}
},
"update": "2023-03-03T00:00:00+00:00"
},

BoldMinded (Brian)

Unsupported operand type was in the log file?

What version of php are you using?

BoldMinded (Brian)

Never mind you already said v8

BoldMinded (Brian)

Is there anything after the unsupported operand type error? Line numbers of files or a call stack?

Daniel Szczukiewicz

FUll content of LOG file:

08:51:30 03/06/2023 Import #4 Started 08:51:31 03/06/2023 Import #4 Initialized 08:51:31 03/06/2023 Begin Importing [Ekstraklasa] 08:51:31 03/06/2023 Unsupported operand types: int + string 08:51:31 03/06/2023 #0 /var/www/vhosts/sleepy-yonath.77-68-101-165.plesk.page/httpdocs/production/app/addons/datagrab/models/datagrab_model.php(295): Datagrab_model->importItem()

1 /var/www/vhosts/sleepy-yonath.77-68-101-165.plesk.page/httpdocs/production/app/addons/datagrab/mod.datagrab.php(149): Datagrab_model->do_import()

2 /var/www/vhosts/sleepy-yonath.77-68-101-165.plesk.page/httpdocs/production/system/ee/legacy/libraries/Actions.php(181): Datagrab->run_action()

3 /var/www/vhosts/sleepy-yonath.77-68-101-165.plesk.page/httpdocs/production/system/ee/legacy/libraries/Core.php(678): EE_Actions->__construct()

4 /var/www/vhosts/sleepy-yonath.77-68-101-165.plesk.page/httpdocs/production/system/ee/legacy/controllers/ee.php(51): EE_Core->generate_action()

5 [internal function]: EE->index()

6 /var/www/vhosts/sleepy-yonath.77-68-101-165.plesk.page/httpdocs/production/system/ee/ExpressionEngine/Core/Core.php(268): call_user_func_array()

7 /var/www/vhosts/sleepy-yonath.77-68-101-165.plesk.page/httpdocs/productionsystem/ee/ExpressionEngine/Core/Core.php(124): ExpressionEngine\Core\Core->runController()

8 /var/www/vhosts/sleepy-yonath.77-68-101-165.plesk.page/httpdocs/production/system/ee/ExpressionEngine/Boot/boot.php(184): ExpressionEngine\Core\Core->run()

9 /var/www/vhosts/sleepy-yonath.77-68-101-165.plesk.page/httpdocs/production/public/index.php(164): require_once(‘…’)

10 {main}

BoldMinded (Brian)

Daniel I have never seen this error occur with DataGrab, and what’s more confusing is that around line 295 there is no mathematical operations or concatenations of variables of any kind, so I’m not even sure why it’s throwing an error on line 295.

Would you be willing to test version 5 of DataGrab? I’m just wondering if time is better spent on the new version vs tracking down this issue in an old version. I already have a couple of people testing it, you would be the 3rd.

#10

Daniel Szczukiewicz

Brian, sure I can test ver 5.0 of DataGrab!

#11

BoldMinded (Brian)

Are you in the EE Slack?

#12

Daniel Szczukiewicz

Yup, sent you a message already.

#13

Daniel Szczukiewicz

Brian, I think I found out an issue…

This error occurs if “Publish date offset (in seconds)” is empty. When I put “0” there then import finished without issues.

#14

Daniel Szczukiewicz

However I found out a different issue…

While importing to grid any data is “null”, like example below,…

{
"id": 384472,
"name": "P. Chadala",
"age": 19,
"number": null,
"position": "Goalkeeper",
"photo": "https://media.api-sports.io/football/players/384472.png"
},

… then rest of rows have empty “number” column.

#15

BoldMinded (Brian)

I think for the null values, especially if it’s defined as a number column, your import file is going to need to contain a 0 instead of null.

#16

Daniel Szczukiewicz

Yup, I had to modify endpoint and to replace “null” by other value. Now working well.

I just suggest to set up “Publish date offset (in seconds)” field as required 😉

Thanks for your help.

#17

Daniel Szczukiewicz

I added a new grid field and… the same issue like in #2. No data save in grid field at all :(

#18

BoldMinded (Brian)

Is this using DG v4 or v5?

#19

Daniel Szczukiewicz

v5 at the moment. But it didn’t work on v4 too.

#20

BoldMinded (Brian)

And there isn’t any info in the DataGrab-import.log file? I have some test imports with Grid fields and they’re working fine, so I’m not sure where the disconnect is.

#21

Daniel Szczukiewicz

The thing is that I do not run imports so LOG file is empty.

I can’t save data chosen in grid file and that’s the issue. When I chose the data, save configuration and back there I can see empty grid fields.

#22

BoldMinded (Brian)

Can you please share the full json file you’re trying to import, along with screenshots of your Grid field settings so I can try to replicate this locally.

#23

Daniel Szczukiewicz

Brian, please check new import that I added already (Top Scorers). Try to set up everything and back to settings again - there is no data in grid.

#24

BoldMinded (Brian)

Daniel, can you share with me a copy of the full import file? The login info to your environment is no longer in the ticket, so I can’t get to it.

#25

Daniel Szczukiewicz

Brian,

I added new data to ticket. Please, check now.

Thanks.

#26

BoldMinded (Brian)

Last update on Slack is “In my local env I imported 300 entries and it took maybe 30s.”

I’ll keep this ticket around for a few days just in-case, but I think it’s technically resolved.

#27

Daniel Szczukiewicz

Brian,

I updated access to my server so please check the last two imports (Top Scorers & Transfers). DataGrab doesn’t keep data in Grid field there. I was playing with field types in grid but it doesn’t help at all.

Thanks.

#28

BoldMinded (Brian)

Comment has been marked private.

#29

Daniel Szczukiewicz

Comment has been marked private.

#30

BoldMinded (Brian)

I noticed that one of the imports, not the Transfers one, the other one, didn’t have any Grid fields chosen in the settings. So I edited the DG settings and ran an import and it seemed to import fine.

#31

Daniel Szczukiewicz

Comment has been marked private.

#32

BoldMinded (Brian)

I can’t login at the moment b/c you’re in there but I just chose which values from the import file to assign to the Grid field.

#33

Daniel Szczukiewicz

I logged out already… Did grid work on my server env? I was trying to do it second ago and still nothing.

#34

BoldMinded (Brian)

The Top Scorers import created an entry with Grid rows /cp/index.php?/cp/publish/edit/entry/422 - This is the one that I changed the config on, rand the import, and then the Grid rows appeared.

#35

Daniel Szczukiewicz

The entry in the channel always will be created but with empty grid data. The data you saw there I added manually.

Just go into Import setting of Top Scorers, set up data into grid and SAVE. Then back into the settings and you will se that all settings you have done is gone - grid is empty again.

#36

BoldMinded (Brian)

I can’t login b/c you’re logged in.

#37

Daniel Szczukiewicz

Ugghh.. Sorry. Now.

#38

BoldMinded (Brian)

Ok, this appears to be an issue with the json file format, not the import or configuration. You’ll need to reformat your JSON file b/c you’re importing multiple rows into a single Grid field in a single Entry. The file as you had it was re-updating the same entry 18 or more times with just 1 row in the Grid.

https://www.dropbox.com/s/t1sa9908ejgi1k1/ticket-2621-a.png?dl=0 https://www.dropbox.com/s/h7fz7urbmm5cc82/ticket-2621-b.png?dl=0

#39

Daniel Szczukiewicz

Ufff… :( I created JSON as you requested and… still nothing. Please check this endpoint: https://sleepy-yonath.77-68-101-165.plesk.page/api/top-scorers/ekstraklasa.json

The same is in CMS already so you can enter and try to grab the data (grid is still empty). No idea what’s wrong this time… :(

#40

BoldMinded (Brian)

Ok, so this is weird. It didn’t want to save because the Grid field was the only one added to that channel. I added another random custom field to the Top Scorers channel, saved the DG import settings, and it kept the column assignments, then removed the custom field from the Top Scorers channel and it still saves fine. I can see why it’s doing this in the code so I’ll try to fix it. But you should be good to go now.

#41

Daniel Szczukiewicz

Wow, I didn’t expect that at all 😊 I did the same for Transfers and it’s working.

Good you know what’s cause the issue.