Publisher does not support the Fluid field type. Please do not contact asking when support will be available.

If you purchased an add-on from expressionengine.com, be sure to visit boldminded.com/claim to add the license to your account here on boldminded.com.

Ticket: 25 to 60 second entry save time

Status Resolved
Add-on / Version Publisher Lite 2.7.3
Severity
EE Version 3.5.15

Fabcom

Mar 07, 2018

Migrating from EEv2.x to EEv3.5.15 and all seems to work but it takes anywhere from 25 seconds to 60+ seconds to save an entry after editing. EllisLab support looked at it and they say there are hundreds of SQL queries generated by Publisher and suggested we contact you.

Just to compare apples to apples, here’s the URL that Ellis referenced in their response:
http://atsu.newatsudev.com/admin.php?/cp/publish/edit/entry/240&S=e322a5e99b0405b8caf44ffc1bc7a52b

Can you please take a look and advise. We have a fresh back-up of the site/dB so no worries about breaking anything.

#1

BoldMinded (Brian)

I can take a look when I can, but adding urgent to the ticket won’t expedite anything.

You should update to 2.7.6 before I take a look at this and make sure you have Publisher’s cache enabled in its settings page. If you’re still seeing a lot of queries please provide the full list of queries (link to a txt file on dropbox or something).

#2

BoldMinded (Brian)

The urls you provided are throwing a 403 forbidden error.

#3

Fabcom

ok we’re updated to 2.7.6 and your IP has been whitelisted for access.

#4

BoldMinded (Brian)

Comment has been marked private.

#5

Fabcom

IP added.

If you log in and hit the URL: http://atsu.newatsudev.com/admin.php?/cp/publish/edit/entry/240&S=e322a5e99b0405b8caf44ffc1bc7a52b

EE’s debug is turned on and you will list all the queries listed (1105). If you’d like I can probably cut/paste them from there to a file, but may be easier for you to just see them in place. The account provided is superadmin access.

#6

BoldMinded (Brian)

Comment has been marked private.

#7

BoldMinded (Brian)

If you disable Diffs it’ll greatly cut down on the number of queries until I get a fix.

#8

BoldMinded (Brian)

Comment has been marked private.

#9

Fabcom

Comment has been marked private.

#10

BoldMinded (Brian)

Have you tried disabling Diffs?

#11

Fabcom

Comment has been marked private.

#12

BoldMinded (Brian)

I don’t think the truncation error is b/c of Publisher. That appears to be more of an MySQL error, e.g. the column in the database has a max length on it. If field_id_356 is set to “text” then it shouldn’t have any problem accepting that amount of content.

#13

BoldMinded (Brian)

The error is for field field_id_749, which looks to be trying to save a blank string, so that doesn’t make sense.

#14

Fabcom

Comment has been marked private.

#15

Fabcom

Comment has been marked private.

#16

BoldMinded (Brian)

I know what was changed in 2.7.6 that may cause the error, but the error shouldn’t happen if field_id_749 is set to the text type. Text db columns can hold a ton of data.

In the Service/Entry/Entry.php file line 489, add this writeToFile line

https://www.dropbox.com/s/8xj0cxdsssgd3vh/Screenshot 2018-03-08 13.24.08.png?dl=0

And in addon.setup.php at line 315, uncomment the writeToFile function. Save an entry, and see what is saved to the update.txt file in your cache folder.

#17

BoldMinded (Brian)

Lets focus on the sql error first, then look at the slowness after the sql thing is fixed.

#18

Fabcom

Comment has been marked private.

#19

BoldMinded (Brian)

field_id_749 doesn’t have a value, so there is no reason it would throw a truncation error on that, and field_id_328 should easily fit in a text column. Did you look at the database structure to see what type of column field_id_328 and field_id_749 are set to?

#20

Fabcom

Comment has been marked private.

#21

BoldMinded (Brian)

The CP login info you provided doesn’t work anymore.

Which one of those is the Grid field? 749 or 328?

#22

Fabcom

Comment has been marked private.

#23

BoldMinded (Brian)

One line 466 of that Entry.php file mentioned above, add float to the array

$intTypes = ['int', 'tinyint', 'smallint', 'mediumint', 'bigint', 'float'];
#24

Fabcom

Comment has been marked private.

#25

BoldMinded (Brian)

Awesome. Can’t believe I didn’t include float originally.

#26

Fabcom

Should I comment out the file writer?

#27

BoldMinded (Brian)

Yes. Now for the slow save times. Backup your DB and un-install the Pages module. I’m curious if the slowness is coming from how Publisher saves page data or somewhere else.

#28

Fabcom

Comment has been marked private.

#29

BoldMinded (Brian)

Ok, so lets see what queries are running when saving. Deep dive into EE…

system/ee/legacy/database/drivers/mysqli/mysqli_driver.php line 138, add the writeToFile line…

function _execute($sql)
 {
  $sql = $this->_prep_query($sql);
  $query = $this->connection->query($sql);
writeToFile('queries', $sql);
  $this->last_query = $query;
  return $query;
 }

Make sure the writeToFile function is still uncommented in addon.setup.php. Then save an entry and see whats in that queries log file and share it with me. Don’t add the writeToFile line when loading the entry. Open it in EE, then add the line, and save it. I want to see only the queries performed when its saving an entry.

#30

Fabcom

Comment has been marked private.

#31

BoldMinded (Brian)

Comment has been marked private.

#32

BoldMinded (Brian)

Comment has been marked private.

#33

Fabcom

Comment has been marked private.

#34

BoldMinded (Brian)

Excellent! I still see a lot of these queries, so I’ll look into it more.

SELECT *
FROM (`exp_publisher_diff_settings`)
#35

Fabcom

Comment has been marked private.

#36

Fabcom

Comment has been marked private.

#37

Fabcom

Comment has been marked private.

#38

BoldMinded (Brian)

Well obviously. Publisher has to do a lot of work to make translations viable. What I’m concerned about now is those publisher_diff_settings queries. There is no way that many should still be executed. In your Service/Diff.php file, do you see the getSettings function and does it look exactly like this?

public function getSettings($fieldId)
    {
        $fieldId = preg_replace('/field_id_(\d+)/', '$1', $fieldId);

        $db = ee('db');
        $settings = [];

        if (!$this->requestCache->get('diffSettings')) {
            /** @var CI_DB_result $qry */
            $qry = $db->get('publisher_diff_settings');

            foreach ($qry->result() as $row) {
                $snippetContents = false;

                if ($row->snippet_id) {
                    $snippets = ee()->publisher_template->get_snippets(false);

                    if (array_key_exists($row->snippet_id, $snippets)) {
                        $snippetContents = ee()->publisher_template->get_snippet_contents($row->snippet_id);
                    }
                }

                $settings[$row->field_id] = [
                    'snippet'  => $row->snippet_id,
                    'template' => $row->template_custom,
                    'use_template' => ($snippetContents ? $snippetContents : $row->template_custom),
                    'style'    => $row->style
                ];
            }

            $this->requestCache->set('diffSettings', $settings);
        }

        $cacheResults = $this->requestCache->get('diffSettings');

        if ($cacheResults && isset($cacheResults[$fieldId])) {
            return $cacheResults[$fieldId];
        }

        return [
            'snippet' => '',
            'template' => '',
            'use_template' => '',
            'style' => ''
        ];
    }

Can your disable Diffs and then do the query logging when saving an entry?

#39

Fabcom

Comment has been marked private.

#40

BoldMinded (Brian)

In that same mysqli_driver.php file, line 277, add this writeToFile, then save one of those same entries you’ve been testing with.

function _list_columns($table = '')
 {
     writeToFile('show-columns', $table);
  return "SHOW COLUMNS FROM ".$this->_protect_identifiers($table, TRUE, NULL, FALSE);
 }
#41

Fabcom

Comment has been marked private.

#42

BoldMinded (Brian)

Ok, so the excessive queries to SHOW COLUMNS FROM exp_channel_grid_field_* don’t appear to be coming from Publisher, but from EE itself. At this point I don’t know what else I can do to make it faster.

#43

Fabcom

Comment has been marked private.

#44

BoldMinded (Brian)

There is no guarantee those queries are causing the issue. You’d have to stop those queries to see if it is them, but if you disable Publisher and it saves in 3 seconds then obviously there is something slow in Publisher but I’m not sure what it is, or when I’d be able to identify it, let alone fix it. Regardless having Publisher installed is going to make things a bit slower out of the fact of what it’s trying to do. I don’t even have an EE 2 site running anymore so I can’t compare.

#45

BoldMinded (Brian)

One thing you could do is setup a New Relic account, which could help identify slow performance areas. If you install it on your server and configure your site, and can provide me a login to New Relic I could take a look and maybe see what the issue is and if I can fix it.

https://docs.expressionengine.com/latest/monitoring/new-relic.html https://newrelic.com/

#46

Fabcom

Comment has been marked private.

#47

BoldMinded (Brian)

Try the build in the next comment. I found another batch of unnecessary queries this morning. In my install it reduced the amount of queries by about 200. I don’t know if it’ll have drastic effects on your times, but its worth a shot.

#48

BoldMinded (Brian)

Comment has been marked private.

#49

Fabcom

Thanks Brian, we will give it a try!

#50

Fabcom

Comment has been marked private.

Login to reply