Publisher does not support the Fluid field type. Please do not contact asking when support will be available.

If you purchased an add-on from expressionengine.com, be sure to visit boldminded.com/claim to add the license to your account here on boldminded.com.

Ticket: Duplicated rows (relationship field) from EE2 to EE3

Status Resolved
Add-on / Version Publisher 2.8.3
Severity
EE Version 3.5.16

Hop Studios

Aug 13, 2018

Description:
We’re in the process of updating the site from EE2 to EE3. We’ve updated the publisher to 1.7.5 and used the playa matrix importer to convert the fields before running the EE upgrade.

After the upgrade, we upgraded publisher to 2.8.3 and we found that the relationship fields all have extra rows with no related entries. Can it has something to do with the languages?

Detailed steps to reproduce the issue:
1. Have a few languages (in our case 8) and a playa field relating an entry to another
2. Update publisher to 1.7.5 and use the matrix importer
3. Run EE upgrade from 2 to 3

If this ended up needing you to login to our staging server, I’ll provide you the logins.
Thanks!

#1

BoldMinded (Brian)

Can you provide more info… did you delete some languages in the EE2 install before upgrading to EE3? If there are extra rows in the Rel field with no related entries, what exactly is the Rel field showing? It needs a title of the related entry, but if there are no related entries, then there won’t be a title (screenshot)?

#2

BoldMinded (Brian)

Backup your database and try these queries. You might have some orphaned data.

DELETE FROM exp_publisher_relationships WHERE field_id NOT IN(SELECT field_id from exp_channel_fields);
DELETE FROM exp_publisher_relationships WHERE parent_id NOT IN(SELECT entry_id from exp_channel_titles);
DELETE FROM exp_publisher_relationships WHERE child_id NOT IN(SELECT entry_id from exp_channel_titles);

DELETE FROM exp_relationships WHERE field_id NOT IN(SELECT field_id from exp_channel_fields);
DELETE FROM exp_relationships WHERE parent_id NOT IN(SELECT entry_id from exp_channel_titles);
DELETE FROM exp_relationships WHERE child_id NOT IN(SELECT entry_id from exp_channel_titles);
#3

Hop Studios

Hello there,

Thanks for the quick response. Also I think I’ve made a mistake, it is the matrix field that has duplicated rows (it contains a relationship field)

I did not delete any language before the upgrade. I didn’t do anything other than upgrade the publisher to the latest version for ee2 and running then importer.

Here are the before and after screenshot (sorry but this is a matrix field containing a playa field i’m showing but after the importer it remains only 1 row:

https://lh3.googleusercontent.com/Gc78n-NRkHbNyHge_HFN0jvRBvLEC0PgNxFcHARsGXGJODLBcCB0fL0sGqvKI1hwAUu2ld2XVZLoEFOHG2zvuAVHUznc5X7BoE5F-JzsCKAfEh7QdgPcPSbE89Y83d4t8BL1Y7TB

https://lh4.googleusercontent.com/YCOCSjo_9DYVaoIQfjEmw0JfgDlSowRqp_S3TfsQIiMhjf338ik9WBnCkdZCkdo5Kv1S9syQFWRaRdL9SqQPXewbfHSnznJ40NoHEL5l4W2YMYgOt0xgei7nGK_FTKCEXYxH4lqC

The second row continues and made 15 duplicated rows. It’s not the same amount of duplicated rows generated for other entries.

#4

BoldMinded (Brian)

Check your exp_matrix_data table for that entry to see if there are actual duplicate rows, or if its showing the same row repeatedly. Then check your Publisher settings to see what the Persistent Grid setting is set to (change it to the opposite value and see if that changes anything).

#5

BoldMinded (Brian)

Also your first screenshot shows Matrix in EE2. What does it look like after you converted it to a Grid field in EE2? It would help to know if the Grid field in EE2 has the correct number of rows or if the issue first starts happening in EE3.

#6

Hop Studios

I looked in the database and confirm that there are duplicated rows. The query I ran is:

SELECT * FROM exp_matrix_data WHERE entry_id=24536 ORDER BY row_id;

The result set has 16 rows. 8 ‘open’ status and 8 ‘draft’ status. I have 8 languages.

I did do a successful conversion of matrix to grid before running the ee2 to 3 upgrade and the rows count was correct.

I also did try the setting the persistent setting on and off and both returns the same thing (I’ve actually been going at it for quite a few days doing re-upgrade after re-upgrade with different process/variants).

#7

BoldMinded (Brian)

Can you provide FTP and CP access to this environment? If so please check that both logins work and that I can edit files in the publisher folder.

TBH this sounds like a complicated problem, I don’t know what the issue is or how long it will take to resolve. It seems like the field isn’t getting the language parameter in its query, which doesn’t make any sense b/c it wouldn’t be working for anyone else and I’d have multiple similar bug reports right now, but I don’t.

#8

Hop Studios

That’s what I thought. Before I provided you with the logins, do you want me to revert the staging site back to right before I ran the ee2 to 3 upgrade?

#9

BoldMinded (Brian)

The odd thing about this is Publisher doesn’t modify the Matrix tables at all during the upgrade. So if in EE2 entry 24536 has just 2 rows, one for open and one for draft, but after the upgrade it has 16 rows, something else is doing that. Are you positive that entry wasn’t saved in the CP after upgrading to EE3? Can you provide the full SQL result from that query with the columns in EE2 before the upgrade, and then in EE3 after the upgrade?

#10

Hop Studios

Thanks for pointing me in the right direction.

Here’re my steps and what I found: 1. I rebuild the EE2 environment and ran the Playa Matrix Importer 2. While there’s only 1 row in the matrix field and 1 row in the grid field, I saw 16 rows in the database when I

SELECT * FROM  `exp_channel_grid_field_255` WHERE entry_id =24536

half draft and half open status 3. I then ran

SELECT entry_id, field_id_160 FROM  `exp_channel_data` WHERE entry_id =24536

and see that the matrix field contains:

A NEW PAGE|[24534] [home_page_video] Home Page Video - Plan of Love|
UNE NOUVELLE PAGE|[24534] [home_page_video] Home Page Video - Plan of Love|
A NEW PAGE|[24534] [home_page_video] Home Page Video - Plan of Love|
A NEW PAGE|[24534] [home_page_video] Home Page Video - Plan of Love|
A NEW PAGE|[24534] [home_page_video] Home Page Video - Plan of Love|
A NEW PAGE|[24534] [home_page_video] Home Page Video - Plan of Love|
A NEW PAGE|[24534] [home_page_video] Home Page Video - Plan of Love|
A NEW PAGE|[24534] [home_page_video] Home Page Video - Plan of Love

Is there any sort of clean up that I can do to remove the duplicates? as I’m not entirely sure how Publisher (for ee2) and Matrix interact with each other.

Really appreciate your help.

#11

BoldMinded (Brian)

Its hard to say how to cleanup the data. Can you run those queries again but as an export so it includes the column names and values of each row?

mysql -e "SELECT * FROM  `exp_channel_grid_field_255` WHERE entry_id =24536" -u myuser -pxxxxxxxxx mydatabase
#12

Hop Studios

Here is the result set:

+--------+----------+-----------+----------+----------+----------------------------+----------+----------+-----------+-------------------+------------------+
| row_id | entry_id | row_order | col_id_5 | col_id_6 | col_id_7                   | col_id_8 | col_id_9 | col_id_10 | publisher_lang_id | publisher_status |
+--------+----------+-----------+----------+----------+----------------------------+----------+----------+-----------+-------------------+------------------+
|   4215 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     |           |                 1 | open             |
|   7055 |    24536 |         1 |          |          | <h2>UNE NOUVELLE PAGE</h2> |          | NULL     |           |                 3 | open             |
|   6901 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     |           |                 2 | open             |
|   7077 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     |           |                 4 | open             |
|   7091 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     | NULL      |                 5 | open             |
|  13610 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     | NULL      |                 6 | draft            |
|   9016 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     | NULL      |                 6 | open             |
|  34923 |    24536 |         1 |          |          | <h2>UNE NOUVELLE PAGE</h2> |          | NULL     |           |                 3 | draft            |
|  17004 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     | NULL      |                 7 | open             |
|  31656 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     | NULL      |                 7 | draft            |
|  35350 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     |           |                 8 | draft            |
|  35079 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     |           |                 8 | open             |
|  35349 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     |           |                 2 | draft            |
|  34924 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     |           |                 4 | draft            |
|  31657 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     | NULL      |                 5 | draft            |
|  35439 |    24536 |         1 |          |          | <h2>A NEW PAGE</h2>        |          | NULL     |           |                 1 | draft            |
+--------+----------+-----------+----------+----------+----------------------------+----------+----------+-----------+-------------------+------------------+

I think the problem is more on the Matrix side? What I don’t understand is why there is only one row showing when there are multiple rows stored in the matrix field. The only thing I think has something to do with is the set of languages.

#13

Hop Studios

I ran

SELECT * FROM  `exp_matrix_data` WHERE entry_id =24536

I got 16 rows like the above result set but each with different publisher_lang_id (also half are draft and half open).

#14

BoldMinded (Brian)

Is this happening for just this one field/entry or is it happening with others? E.g how wide spread is this? If its just 1 entry, I’d just delete those extra rows manually.

#15

Hop Studios

This is happening to all the matrix fields that I checked and all the related entries.

From what I’m seeing now, each language (publisher_lang_id) has a row in exp_matrix_data for 1 channel entry. When I select different language, the different matrix row get’s used to displayed.

When I ran the importer, it basically took all the matrix rows and converted them to grid rows. However, in EE3 the publisher 2 doesn’t collapse the multiple grid rows (using the status and language to determine which one is displayed) like EE2 Publisher 1.7.5 does but rather shows all the rows. How does Publisher 2 handle multiple languages?

#16

Hop Studios

Hello Brian,

I think I might have found the problem. When I view the entry in ee3, I don’t see the language selection: https://lh6.googleusercontent.com/WLCtNha8Orng5V6G22zzV0npQmAURehXXjX5TFdtnWgkeItNWrF69ceNtgebhCn5DaSO1zgfXqrwQjuHP4wQb2plTPmurWoAKh8NeIjzjYeAgult72I3yefZXqPWh4WuWlvN2L4j

But I’m not too sure why. Can you help?

#17

BoldMinded (Brian)

Comment has been marked private.

#18

Hop Studios

The language selection is showing up now but all rows are still showing up and all languages have a check mark: https://lh6.googleusercontent.com/fm8hX2zP8nZMDf7Yyz0l9i0Zbp8-zacgemCmk0u9uG00vMLzoI6X8yQ3pTG5rVl5WgdA4dHoDKzZLMyHrNvQqUOp-MVoWMRFmJfAc0qOPdmbeYpWhweSWH8CWBMKam4TQsn4JyPF

#19

Hop Studios

I tried to create a new entry and at first all languages don’t have the check mark. I saved the record and English has the check mark indicating that it’s got record. I then switched to French and made some changes to the grid field and save. In the back end the rows count didn’t change but the field is updated with what I typed/saved in French (publisher_lang_id remains 1 which is English).

#20

BoldMinded (Brian)

I’m not sure I’m following your last comment. Can you share a video of what you’re seeing?

In general, If there are duplicate rows after running the migration in ee2 then that isn’t really a Publisher problem to solve. That’s a migration problem. Kevin made it support Publisher days, but it’s possible that it doesn’t properly support Relationships inside of Matrix/Grid.

#21

Hop Studios

I’ll try to capture a screenshare video later when I’m in a quieter environment.

Please take a look at this list of information and confirm if this is not a Publisher’s issue (and maybe suggest where I should be looking): – in EE2 – 1. In the db, there are 16 rows (with different lang_id and status) in a matrix field attached with an entry 2. When editing the entry, only the current row (with the selected language and status) will be shown 3. After the importer, in the db the grid field also has 16 rows exactly matching the matrix field 4. When editing the entry, only the current row (with the selected language and status) will be shown – after EE2 to EE3 upgrade and Publisher upgrade– 5. In the db, the grid field still has 16 rows for that entry with the exact same language and status data 6. When editing the entry, all grid rows show and selecting any language/status won’t hide the other rows like it did in EE2

Thank you!

#22

BoldMinded (Brian)

Publisher 2 handles the Grid data the same as it did in EE2, but clearly something is out of whack.

Yeah I’m going to need FTP and CP access for this. Check the FTP info before sending and make sure the publisher files are editable by the user. It’ll probably be a few days until I can sort this out with everything I have going on right now.

#23

BoldMinded (Brian)

How many rows do you see in your exp_extensions table for Publisher_ext? Should be 56. See if there is anything thing in this list that you don’t have.

sessions_start
core_boot
custom_field_modify_data
channel_form_entry_form_tagdata_start
channel_form_entry_form_tagdata_end
publish_form_entry_data
channel_entries_query_result
channel_search_modify_search_query
channel_search_modify_result_query
channel_entries_tagdata_end
channel_module_categories_start
channel_module_category_heading_start
core_template_route
cp_custom_menu
cp_js_end
cp_member_login
channel_entries_tagdata
template_post_parse
email_send
pagination_create
relationships_query
relationships_display_field
relationships_post_save
relationships_modify_rows
grid_query
grid_save
assets_save_row
assets_field_selections_query
assets_data_query
structure_get_data_end
structure_get_selective_data_results
structure_reorder_end
structure_create_custom_titles
structure_modify_publish_tab_settings
structure_get_custom_variables
blocks_get_blocks
blocks_disassociate
blocks_create_block
blocks_delete_block
blocks_set_block_order
blocks_set_atom_data
blocks_pre_save_blocks
blocks_post_save_block
blocks_post_save
before_channel_entry_save
after_channel_entry_delete
after_channel_entry_save
after_channel_field_save
after_channel_field_delete
cp_css_end
blocks_delete_blocks_by_entry
low_events_archive_events
low_events_calendar_events
blocks_update_field_data
publish_live_preview_route
before_channel_field_save
#24

BoldMinded (Brian)

Make sure all those have “y” in the enabled column too.

#25

Hop Studios

There were some extensions not enabled! Setting them to enabled fixed the issues. Thank you so much for your help!

#26

BoldMinded (Brian)

Which ones were set to not enabled? More importantly how did they get that way? :D

#27

Hop Studios

I think what happened was: when the upgrade to ee3 first finished, I saw a blank page rather than the cp. I then turned on the debug and saw some extensions errors. So I started setting some of them to ‘n’. Sorry but I don’t really remember which ones that was set to ‘n’. But the lesson that I learned is that after the upgrade process I need to make sure the list of extensions are all enabled. Sorry for the trouble, Brian.

#28

BoldMinded (Brian)

No worries, a few back and forth comments is better than a real bug 😊

Login to reply