Publisher does not support the Fluid field type. Please do not contact asking when support will be available.
If you purchased an add-on from expressionengine.com, be sure to visit boldminded.com/claim to add the license to your account here on boldminded.com.
Ticket: Grid Field import appending instead of updating
| Status | Resolved |
| Add-on / Version | DataGrab 5.1.0 |
| Severity | |
| EE Version | 7.3.14 |
hopstudios
Dec 18, 2023The import file contains rows of data for a price table, with each row in the source corresponding to rows in a grid field. The first column has a date that corresponds to the entry’s title. Import settings are set on the grid field to append new rows. So with a 59-line import source, we get 58 grid rows in the entry (header row is ignored).
In the “Determine what happens if the import is run again,” we have the title field set as the field to check for duplicates and have checked “Update existing entries.”
In the older version of DataGrab, running the import again resulted in the old data being replaced with the source of the import. In this newer version, the data is appended to the existing data. So instead of a 58 row grid entry, we get a 116 row grid entry.
hopstudios
Comment has been marked private.
BoldMinded (Brian)
Did you try the “Update row if this column matches:” option?
hopstudios
Hello,
We made the change you suggested, but are still having issues with imports if they’re triggered remotely. The import seems to only evaluate part of the source file before quitting.
Source File: https://drive.google.com/file/d/1puIVdRzMosgsLUr2wHVjVC9TIvZwkAXg/view?usp=sharing
Import log (1/18/23, import triggered remotely): https://drive.google.com/file/d/1LhFbNJgR1Zxe6AE9Wc53OxmB_Fr-0890/view?usp=sharing
Import log (1/19/23, import triggered manually in control panel): https://drive.google.com/file/d/1kLo-cPbOVeEyvmsXVCPdU7KXiy7rnrZy/view?usp=sharing
BoldMinded (Brian)
What do you mean by triggered remotely? Via the Action URL or CLI command?
hopstudios
Via the Action URL.
BoldMinded (Brian)
What happens if you run the import via the CLI command?
hopstudios
The client uses the Action URL as part of an automated workflow they’ve got in place.
Will running the CLI command help with debugging the possible Action URL problem we’re experiencing?
BoldMinded (Brian)
I don’t know if it’ll help with anything, but I’m curious if it yields a different result.
BoldMinded (Brian)
The “Determine what happens if the import is run again” setting is for the main entry itself, not the Grid rows. So if you’re using the date as the unique identifier in the Title, then it will update that entry based on the date value. If you want to update existing rows in a Grid field, then you’ll need some unique identifier in each grid row that it can use to determine if the row should be updated or if the row in the import file is new and should be appended. You mentioned this was different in the old DataGrab, but I’m pretty certain I have not changed any of this behavior. FWIW I don’t think there is a bug here, I think you need a unique ID on each row of your import file, and you’ll need to change the Grid field import settings to use it to determine if it should append or update. When looking at your import file I don’t see anything that sticks out as a unique ID. Something as simple as a timestamp column would work though.
hopstudios
Hello,
There is a unique identifier on each row of imported content, the fund code, and the issue we’re having is that the import seems to stall and not traverse the entire import file when run remotely via the Action URL.
If I log in and manually click the start import button, the import will run as expected and populate the entry as desired. It’s only when run via the Action URL that we encounter these partial imports.
BoldMinded (Brian)
According to the screenshot you shared though, you aren’t using the “Update the row if this column matches” option, you have it set to “Keep existing rows and append new”.
BoldMinded (Brian)
What do you see when the import runs? Can you share a video?
BoldMinded (Brian)
What other add-ons do you have installed?
BoldMinded (Brian)
Comment has been marked private.
BoldMinded (Brian)
Might be a related ticket https://boldminded.com/support/ticket/2837
hopstudios
Hello,
This issue was resolved by making the adjustment to the batches setting as suggested in the linked ticket above. Once I set it to a number higher than the number of records, our issue disappeared and we’ve been successfully updating and importing entries for a couple of weeks now.
Thanks for sticking with me and helping resolve this issue!