EE4/5 compatibility updates:

  • Publisher is EE4 and EE5 compatible, but it does not currently support the Fluid field (it's in the works).

Ticket: Publisher Search

Status Client Wait
Add-on / Version Publisher 2.10.8
Severity
EE Version 5.1.3

Hop Studios

Mar 06, 2019

Hello Brian,

In earlier version of Publisher (1.7.5), when we do a channel entries loop, this happened:
1. we get all items that fit the search parameters
2. the items will display in the current language (say French) and for those who don’t have French translation they will display in the default language (English)

Since we upgraded to EE 5 and Publisher 2, we only get the items that have the translation in the current language.
Setting the Persistent Entries to yes/no doesn’t change the result.

Is there any way we can get the same behaviour as before?
1. The channel entries loop fetch all entries as is (no matter what language you are in)
2. When rendering, display the translated version if applicable or display the default language

Thanks!

#1

BoldMinded (Brian)

Mar 07, 2019

I’m not sure what would cause you to be getting different results now, b/c the latest version of Publisher Low Search didn’t change any behavior, it was just an upgrade to change how it got the language and status variables from Publisher.

#2

Hop Studios

Mar 07, 2019

Sorry Brian, I marked the wrong add-on. This is for Publisher 2.

#3

BoldMinded (Brian)

Mar 07, 2019

Can you please provide more details? Perhaps the template tag you’re using (without any extra html markup), the results you’re currently getting, and the results you expect to get?

#4

Hop Studios

Mar 07, 2019

Comment has been marked private.

#5

BoldMinded (Brian)

Mar 08, 2019

It’s been awhile since I’ve done anything with the search: parameter. This may be one where I’ll have to replicate the issue locally so I can debug it properly. I’ll try to get back to you early next week, or sooner, if I can find time this weekend to take a look at it.

#6

Hop Studios

Mar 08, 2019

Hello Brian,

I did more tests. So in our case, the search:field_name doesn’t work if the field is ignored in the Ignored fields setting. The search parameter worked when I tried searching an not ignored field. Ideally the channel entries loop should search in both EE native channel data table and Publisher’s table in my opinion.

#7

BoldMinded (Brian)

Mar 09, 2019

If it is working the way you just described, then it sounds like it is working correctly to me. An ignored field means it always uses the default language value, and in this case it is searching the default exp_channel_data or exp_channel_data_field_x field, not Publisher’s tables.

#8

Hop Studios

Mar 11, 2019

So the situation is:
1. A product channel has a field product_language that is an ignored field
2. Product entry 28801 has product_language set to “fr”
3. When the language is set to /fr in the url, entry 28801 doesn’t show up in {exp:channel:entries search:product_language=”=fr”}
4. Entry 28801 doesn’t have a french translation
5. When I check in the DB SELECT * FROM exp_channel_data WHERE entry_id=28801; I found that the field column for product_language and it has the value “fr”

I suspect the ignored fields are not getting searched or the entry is not displaying because it doesn’t have the translation for the viewing language (Show content fallback (FE) is on).

Expected outcomes (the behaviours from previous version of Publisher):
1. Ignored field or not, the search:field should pull the entries that have the value specified (if in the CP the field is set to “fr” it should be grabbed when search:product_language=”=fr”)
2. The channel:entries loop shouldn’t care about the viewing language and grab the entries by the parameters specified.
3. The default language will be rendered if a translation for the viewing language is not found.

#9

BoldMinded (Brian)

Mar 11, 2019

If it was working that way before hand then it was probably a bug b/c the current functionality is correct. An ignored field is meant to be ignored in every aspect.

Have you tried the search tag? http://docs.boldminded.com/publisher/template-tags

It can do similar searches and will return a list of entry_ids that you can then pass to an entries tag.

{exp:publisher:search field_name="keyword|another keyword"}
    {exp
:channel:entries entry_id="{entries}"}
        {title}
    {
/exp:channel:entries}
{
/exp:publisher:search} 
#10

Hop Studios

Mar 11, 2019

Are you saying that if a field is ignored it cannot be searched in {exp:channel:entries} loop?

I don’t think that’s correct. Shouldn’t the ignore field settings just make it so that the field value will be consistent across all translations?
I think when someone writes search:field, the search will look for a match in channel_data table if the field is ignored and publisher_data if the field is not ignored.

We set that field to be ignored so that we can search without the restraints of the viewing language. Reason is: we have multiple products with different translations but we want to be able to compile a list of items that have one particular field set to a certain value and display it either with the translated version or the default translation.

{exp:publisher:search product_language="fr"}...{/exp:publisher:search} 

I tried the search tag you suggested, the expected entries are still not displaying. They all have product_language set to fr but they don’t have a French translation; and I’m viewing in /fr of the site. I expected the default English translation of the entry to be displayed.

#11

BoldMinded (Brian)

Mar 11, 2019

Are you saying that if a field is ignored it cannot be searched in {exp:channel:entries} loop?

It can be searched on, it’ll just always search the default language field value.

Shouldn’t the ignore field settings just make it so that the field value will be consistent across all translations? I think when someone writes search:field, the search will look for a match in channel_data table if the field is ignored and publisher_data if the field is not ignored.

That is exactly what it is doing. It’s consistently expected to be the default language value across all languages.

I don’t know how it was working the way it was in EE 2, but I don’t see how I can refactor this to work the way you’re wanting it to without breaking the concept of an ignored field (or break someone else’s site who is using search:field parameters).

Go to line 880 of mod.publisher.php, and change the line ->where(‘lang_id’, $this->request->getCurrentLanguage()->getId()) to ->where(‘lang_id’, YOUR_ID), and use the exp:publisher:search tag to see if you get the expected results. You may need to also use the publisher_lang_id parameter in the entries tag.

{exp:publisher:search field_name="keyword|another keyword"}
    {exp
:channel:entries entry_id="{entries}" publisher_lang_id="ID"}
        {title}
    {
/exp:channel:entries}
{
/exp:publisher:search} 

If adding the language ID in the query of the exp:publisher:search tag works, then I’ll add a parameter so it can be used as

{exp:publisher:search field_name="keyword|another keyword" publisher_lang_id="ID"}
    {exp
:channel:entries entry_id="{entries}" publisher_lang_id="ID"}
        {title}
    {
/exp:channel:entries}
{
/exp:publisher:search} 
#12

Hop Studios

Mar 11, 2019

Hello Brian,

I dug deeper and I found that the results are searched but cut from the returned results set from line 403

if (
        ((
$statusFilter || $langFilter || $categoryFilter) && !isset($entries[$entry['entry_id']])) ||
        (!isset(
$entries[$entry['entry_id']]) && !$isPersistenceEnabled)
    )
{
        
unset($results[$k]);
    
else {... 

In this case, there’s a categoryFilter. Why does Publisher unset the result when there’s a categoryFilter and the entry is not from the translated results?

#13

BoldMinded (Brian)

Mar 11, 2019

I’ll try to take a look at that soon. I still think the behavior I described sounds logically correct knowing how the ignored fields are supposed to work, but if restoring it to the behavior you’re described is fairly trivial I will try to do that.

#14

Hop Studios

Mar 11, 2019

Thanks, Brian. So for us the category is set in the url https://staging.lucistrust.org/fr/store/category/alice_bailey_books_p

#15

Hop Studios

Mar 11, 2019

Sorry I forgot to say the code I pasted is from Service/Entry/EntryResult.php line 403

if (
        ((
$statusFilter || $langFilter || $categoryFilter) && !isset($entries[$entry['entry_id']])) ||
        (!isset(
$entries[$entry['entry_id']]) && !$isPersistenceEnabled)
    )
{
        
unset($results[$k]);
    
else {... 

The results are searched correctly so I confirmed that the search:field is working as it should (the ignored field is searched using the default value).
I dumped the $results before and after ext.publisher.php line 798

$results $this->entryResult->getAll($entryIds$results); 

and the first dump has 24 (which matches the current site in EE2 and Publisher 1.7.5) and the second dump only has 3 (the 3 entries from the publisher entries results).

I think 24 native channel entries should display and the 3 particular entries that have the french translations should replace the respective native channel entries results and display.

#16

BoldMinded (Brian)

Mar 11, 2019

Comment has been marked private.

#17

BoldMinded (Brian)

Mar 11, 2019

In my test I have 3 entries, all assigned to the “black” category listed at the url “/de/karten/category-de/schwarz” which is /de/cards/category/black in English. But only 2 of the entries have a field with the value of “fr” in it, so this query returns two results, as expected, b/c it’s matching the category AND the custom field value of “fr”. This is the new query generated from the build in the previous comment. If you’re expecting a different result then I’ll need an example of a working query that EE core generates returning the expected rows (e.g. reproduce the results without Publisher installed), b/c I’m pretty sure this is the same result that EE would produce given the search parameter and category. I’m not sure how this would work any other way to be honest, other than changing that last AND to an OR, which fundamentally changes the search query and I think would differ from what EE would generate itself.

SELECT `ct`.*, `t`.*, `d`.*, `channel_title`, `c`.`channel_name`, `c`.`channel_url`, `c`.`comment_url`, `c`.`comment_moderate`, `c`.`channel_html_formatting`, `c`.`channel_allow_img_urls`, `c`.`channel_auto_link_urls`, `c`.`comment_system_enabled`, `username`, `m`.`email`, `m`.`screen_name`, `m`.`signature`, `m`.`sig_img_filename`, `m`.`sig_img_width`, `m`.`sig_img_height`, `m`.`avatar_filename`, `m`.`avatar_width`, `m`.`avatar_height`, `m`.`photo_filename`, `m`.`photo_width`, `m`.`photo_height`, `m`.`group_id`, `m`.`member_id`, `ct`.`url_title` AS default_url_title, `ct`.`status` AS status, `t`.`title` AS title, `t`.`site_id` AS entry_site_id
FROM 
(`exp_publisher_titles` AS t)
JOIN `exp_publisher_data` AS d ON `t`.`entry_id` = `d`.`entry_id` AND t.lang_id d.lang_id AND t.status d.status
JOIN 
`exp_channel_titles` AS ct ON `ct`.`entry_id` = `t`.`entry_id`
JOIN `exp_channels` AS c ON `c`.`channel_id` = `t`.`channel_id`
JOIN `exp_publisher_category_posts` AS cp ON `cp`.`entry_id` = `t`.`entry_id` AND cp.lang_id t.lang_id AND cp.status t.status AND cp.cat_id IN (4)
JOIN `exp_members` AS m ON `m`.`member_id` = `t`.`author_id`
WHERE `t`.`status` =  'open'
AND `t`.`lang_id` =  2
AND `t`.`entry_idIN (901312
AND `
t`.`channel_idIN ('3') AND ( d.field_id_21 'fr' ); 
#18

BoldMinded (Brian)

Mar 11, 2019

For comparison sake this is the query that EE generates to find the 3 entries, however, it isn’t taking into consideration that only 2 queries are of the requested language.

DISTINCT t.entry_id exp_channels.channel_id t.entry_datet.sticky FROM exp_channel_titles AS t
                LEFT JOIN exp_channels ON t
.channel_id exp_channels.channel_id LEFT JOIN exp_channel_data AS wd ON wd.entry_id t.entry_id INNER JOIN exp_category_posts ON t.entry_id exp_category_posts.entry_id
                         INNER JOIN exp_categories ON exp_category_posts
.cat_id exp_categories.cat_id LEFT JOIN exp_channel_data_field_21 ON exp_channel_data_field_21.entry_id t.entry_id WHERE t.entry_id != '' AND t.site_id IN ('1')  AND t.entry_date 1552359397  AND (t.expiration_date OR t.expiration_date 1552359397) AND t.channel_id IN (3,5)  AND exp_categories.cat_id '4' AND t.status 'open' AND ( exp_channel_data_field_21.field_id_21 'fr' )ORDER BY t.sticky desct.entry_date desct.entry_id desc LIMIT 0100 
#19

BoldMinded (Brian)

Mar 12, 2019

While those 2 queries look different they are effectively doing the same thing. The second one (from EE’s mod.channel.php file) is fetching just they entry_ids, then it later does another query that very closely resembles the first one (from Publisher) Publisher just isn’t doing the extra query b/c it doesn’t need to.

#20

Hop Studios

Mar 12, 2019

Comment has been marked private.

#21

BoldMinded (Brian)

Mar 12, 2019

Comment has been marked private.

#22

Hop Studios

Mar 12, 2019

Hello Brian,

This time I got this error

Fatal errorUncaught ErrorCall to undefined method Channel_model::f() in /chroot/home/lucistru/staging.lucistrust.org/html/contentzsystem/user/addons/publisher/Service/Entry/EntryResult.php:850 Stack trace#0 /chroot/home/lucistru/staging.lucistrust.org/html/contentzsystem/user/addons/publisher/Service/Entry/EntryResult.php(764): BoldMinded\Publisher\Service\Entry\EntryResult->generateSearchFieldSql(Array) #1 /chroot/home/lucistru/staging.lucistrust.org/html/contentzsystem/user/addons/publisher/Service/Entry/EntryResult.php(640): BoldMinded\Publisher\Service\Entry\EntryResult->getQueryBuilderResult(Object(EllisLab\ExpressionEngine\Service\Database\Query)) #2 /chroot/home/lucistru/staging.lucistrust.org/html/contentzsystem/user/addons/publisher/Service/Entry/EntryResult.php(354): BoldMinded\Publisher\Service\Entry\EntryResult->getAllQuery(Array, Array) #3 /chroot/home/lucistru/staging.lucistrust.org/html/contentzsystem/user/addons/publisher/ext.publisher.php(789): BoldMinded\Publisher\Service\Entry\EntryResult->getAll( in /chroot/home/lucistru/staging.lucistrust.org/html/contentzsystem/user/addons/publisher/Service/Entry/EntryResult.php on line 850 

I think your original search was working correctly (build e9838c43). It’s only when Publisher is trying to match and replace with the translated version the default language entries are stripped out.

In Service/Entry/EntryResult.php line 403
$results (default native results) has 24
$entries (publisher results) has 3 (the 3 that has French translation from the 24 entries)

if (
    ((
$statusFilter || $langFilter || $categoryFilter) && !isset($entries[$entry['entry_id']])) ||
    (!isset(
$entries[$entry['entry_id']]) && !$isPersistenceEnabled)
)

$statusFilter is ‘’
$langFilter is ‘’
$categoryFilter is an array
and $entries[$entry[‘entry_id’]] is empty for the 21 results that don’t have the French translation
so the conditions are met and the following code is to unset that entry from the $results list. Why?

- Gilbert

#23

BoldMinded (Brian)

Mar 12, 2019

They’re being removed b/c it doesn’t exist in the $entries array, which is populated by the query that the last update is supposed to fix.

Set line 850 to this:

$fieldsSql .= ee()->channel_model->field_search_sql($terms$searchColumnName$siteId); 

Looks like I botched the code and somehow removed “ield_search_sql”

#24

Hop Studios

Mar 12, 2019

The page loads but the results are still missing.

// If any filters were used and we don't have an Publisher entry, cut it.
                // If its not in the translated query result and persistence is off then
                // we need to remove the entry from the result set. 

Reading this, I still think the problem is with the filters (category).

The second condition

!isset($entries[$entry['entry_id']]) && !$isPersistenceEnabled 

will not be met because $isPersistenceEnabled is true. In this case, even the entry (from $results) we want is not in $entries (publisher translated results), the row will not be unset($results[$k]);

I tried removing the first condition

if (
                    
//(($statusFilter || $langFilter || $categoryFilter) && !isset($entries[$entry['entry_id']])) ||
                    
(!isset($entries[$entry['entry_id']]) && !$isPersistenceEnabled)
                )

and I got the result set that I wanted: 24 results and the 3 that does have translation are displayed in french and the 21 that doesn’t are displayed in english.

#25

BoldMinded (Brian)

Mar 12, 2019

Can you create a new EE install with Publisher and the necessary entries and settings to re-create the issue, then I can install it locally and debug it. I can’t debug something this complex from your current site b/c I need to be able to step through the code execution to see what is happening and when. I can’t just remove a large conditional like that b/c it is going to have adverse side affects possibly to your site and other customer sites.

I don’t know why this was “working” for you in EE2, but based on what I’ve uncovered it feels like it is working correctly in it’s current state. Yes it is removing the item from the result b/c it is doing a category filter “AND cp.cat_id IN (4)” and it is also searching a custom field “AND ( d.field_id_21 = ‘fr’ )”, so both of those can’t be true. If it finds the right entries by category, 1 or more of them may be immediately removed by the query b/c of the custom field lookup. Seeing this in a clean EE environment they way you’re seeing it would help me dig into this further. Just please clearly indicate in the template you use what the expected results should be (e.g. exactly which entry_ids should or should not be returned in the result)

#26

Hop Studios

Mar 12, 2019

Hello Brian,

I tried a fresh install and I noticed something that’s super odd… I created 2 ignored fields
1. Product Language
2. Product Language in field group
They behave the same way.

http://publisher.lucistrust.org/fr/site/category/alice-bailey-books
Is kind of correct
- Product 1 and 4 have no French translation but their language field is set to fr so they display in default language English. CORRECT
- Product 3 and 5 have French translation and their language field is set to fr but don’t have English translation so they don’t display. CORRECT (it’s trying to search for the field value from the default language which doesn’t exist)
Here’s where it gets weird:
Go to Product 1 edit page to add the French translation http://publisher.lucistrust.org/admin.php?/cp/publish/edit/entry/1&lang_id=1
Go to http://publisher.lucistrust.org/fr/site/category/alice-bailey-books and now the page only loads Product 1 and Product 4 is gone
You can then delete the Product 1 French translation and see that Product 4 shows up again.

* I did notice that the site I was reporting issues with has the legacy channel_data table structure and the new one has a table for each field.

Login information included.

- Gilbert

#27

BoldMinded (Brian)

Mar 12, 2019

Can you package that site up with an SQL dump so I can debug it locally?

#28

Hop Studios

Mar 12, 2019

Comment has been marked private.

#29

BoldMinded (Brian)

Mar 13, 2019

Gilbert, thanks for sending the files. This helps a lot. I have it running locally and will let you know what I figure out in the next day or two.

#30

BoldMinded (Brian)

Mar 14, 2019

Comment has been marked private.

#31

Hop Studios

7 days ago

Comment has been marked private.

#32

BoldMinded (Brian)

7 days ago

The main issue was that the result was getting removed from the array as you pointed out when the categoryFilter was present. I added an additional check to see if there are search fields, and if there are, to skip the unset() line that you mentioned previously. The search fields resulted in query with more results than the native EE query, thus the need to call the unset().

It will be committed to the main branch, but I don’t feel 100% confident about it. It solves your problem, but I’m trying to think of scenarios where it might have adverse side affects for someone else. With that said, I’m going to take another look at the code changes and see if this is indeed the best solution. If I end up changing anything, I’ll try to send you a test build so you can test in your environment too.

Login to reply

Contact

For add-on support, please use the Support section. General inquries and pre-sale questions can be sent to support@boldminded.com.