Jump to content
mr-fan

beginnerquestion on performance with a textformatter

Recommended Posts

Hi guys,

i've just a simple question of a little textformatter that i use for replacing different textvalues.

I'm not that experienced PHP professional so i asking before i get in possible troubles...

Setup is a "Glossar" like page holder with glossar_entries of various types like (abbr, internal link, external link)

In my textfields i use pipes to set a term of that glossary like ||PW|| is great! i can preset ||Internal|| and ||External|| links and reuse them in every textblock...if i change the glossar entry on every page i use this it will automagic change...

so far so simple (i've tested autolink and bought Profields...but i wanna give the user the power to edit this entries and have more control where these kind of autolinks work and could used)

Here is my first more advanced textformatter on this and my simple question is - will this produce any overload or trouble if i get around about 0-10 terms on a content page?

Second Question is on caching with textformatter replacements? How this is handled?

<?php
/**
 * ProcessWire TextformatterGlossary
 *
 * module made by mr-fan.
 * 15.09.15 basic class and wrapper configuration added
 *
 */
class TextformatterGlossary extends Textformatter
{
    /**
     * getModuleInfo is a module required by all modules to tell ProcessWire about them
     *
     * @return array
     *
     */
    public static function getModuleInfo()
    {
        return array(
            'title' => 'Autolink from Glossar',
            'version' => 101,
            'author' => 'mr-fan',
            'summary' => "Allows to use tags in textareas to autolink to specific glossary links."
            //'href' => 'http://processwire.com/talk/topic/1182-module-image-tags/?p=57160',
        );
    }
    /**
     * Format the given text string.
     *
     * @param Page $page
     * @param Field $field
     * @param string $value
     */
    public function formatValue(Page $page, Field $field, &$value){

        // use fast strpos check to make sure that $value contains pipes ||
        if (stripos($value, '||') === false) return;

        //get all terms in ||pipes|| in an array
        $matches = array();
        preg_match_all('/\Q||\E[^|]+\Q||\E/', $value, $matches);

        //the multidimensional array holds the single strings in the second array['0']
        foreach ($matches['0'] as $key => $match) {

            //get all glossary pages in a pagearray
            $entry = wire('pages')->find("template=glossar_item,title=$match")->first();

            if ($entry) { //entry is found in our glossar pages

                //rip the pipes
                $term = str_replace('|', '', $match);

                //set the replacement depending from the item type
                switch ($entry->glossar_type) {
                    case '1': //abbr
                        $replacement = '<abbr title="' . $entry->headline . '">' . $term . '</abbr>';
                        break;
                    case '2': //external link
                        $replacement = '<a rel="help" target="blank" href="' . $entry->extern_link->url . '" data-original-title="' . $entry->headline . '"><span class="fa-globe" aria-hidden="true"></span> ' . $term . '</a>';
                        break;
                    case '3': //internal link
                        //internal link need to get the url
                        $internLink  = wire('pages')->get("$entry->page_link");
                        $replacement = '<a rel="help" href="' . $internLink->url . '" data-original-title="' . $entry->headline . '">' . $term . '</a>';
                        break;
                    default:
                        $replacement = $term;
                }
                //works the part inside the tags are changed ||test|| on every match
                $value = str_replace($match, $replacement, $value);

            } else { //the entry for  ||term|| is not found and get renderd without pipes just as normal text
                
                //rip the pipes
                $term  = str_replace('|', '', $match);
                //replace the matches of ||term|| with the cleaned value
                $value = str_replace($match, $term, $value);
            }
        }
    }
}

Best regards mr-fan

Share this post


Link to post
Share on other sites

thanks, nice idea and good example.

sometimes i have a replacements table or textarea right in the editor; then i use {{tokens}} in the body to grab those values;

(template does the runtime replacement).

Sometimes you might have some boilerplate content that is reused in various places but needs to have certain terms replaced contextually to the page..

+---------+----------+
| search1 | replace1 |
+---------+----------+
| search2 | replace2 |
+---------+----------+

Share this post


Link to post
Share on other sites

to be honest i'm on a tight dealine for a project i'll use this textformatter first and like i wrote i'm not the great PHP dev.
... :wacko:

so may someone could write if the code seems to be ok or will this give me a mess or performance problem in the long run and important for me while this is nowhere in the doc's are textformatter's output/replacements cached?

Share this post


Link to post
Share on other sites

The whole system itself works great i've in the pagetree

-glossar (rootpage with BatchChildEditor in Lister Edit mode)

--item 1

--item 2

item could be actual an <abbr> or an internal link (with a page field with choosen select) or a external link...but could be everything i will setup on this replacements

itemtype is simple choosen by a optionsfield

only condition is the title of the glossaritem have to match the term.

this could be much more develeopped to get real footnotes or additonal contentypes to replace....

Share this post


Link to post
Share on other sites

hi mr-fan,

what kind of caching do you use? if you are using template cache then performance should not be a problem as the textformatter will only be executed once (when the cache is created). in that case you would have to flush your cache every time your glossar is changed...

i think you could also use markupcache for this special field: https://processwire.com/talk/topic/7-new-markupcache-module/

Share this post


Link to post
Share on other sites

Thanks for reading...

Your thoughts are exactly what i need..since I bet that I forgot some details like to setup flushing if glossary entries change.

And I will take a look into markupcache. Until now I only worked with template cache.

Best regards mr-fan

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By MoritzLost
      This is a textformatter module that will automatically replace titles of other pages on your site with links to those pages. For example, if you have a template glossary-term, and mention the exact title of one page using that template in a textfield, the title will be automatically linked to that page (if the textfield uses that formatter). This is good for SEO, and saves you some manual labour. You can configure which templates should get automatically linked, and of course the formatter is only active for the fields you add this formatter to. Note that if you need more manual control over when and where titles are automatically linked, you're probably better of using Autolink from a Glossary by @mr-fan.
      Features
      Allows you to limit the automatic links by template. Only includes published & visible pages by default, with an option to include hidden pages. Automatically excludes the current page, with an option to change that behaviour. Allows you to configure the minimum title length for linked pages. Doesn't overwrite existing links, and detects most edge cases (titles inside other tag's attributes, titles inside existing links et c.). Supports multi-language sites. Titles will only be linked if a title in the current language is set. Can add configurable attributes to all automatically created links. Includes the ability to use page fields as replacement patterns for attributes. For example, you can create CSS classes that include the name of the template of the linked page. Prefer oldest or newest page in the case of duplicate titles. Queries the database directly for improved performance. Has options to switch between case sensitive and case insensitive modes, and force case sensitive behaviour even for case insensitive database collations. Allows you to overwrite the module configuration via the API to call the module with different settings for different requirements on the same page. Download & Documentation
      The module is now available in the modules directory: https://modules.processwire.com/modules/textformatter-page-title-links/
      You can download & install it through the backend using the classname TextformatterPageTitleLinks.
      To install it manually, download or clone the module from the Github repository into your site/modules folder and install through the backend.
      The complete documentation can be found in the README in the repository. Make sure to check out the module configuration page after installing the module.
      Requirements
      PHP 7.1 or higher ProcessWire 3+ (it will probably work in older versions, I haven't tested those though). This is my first module, I hope it may become useful to some of you 🙂
      If you find any errors or have some other suggestions or feedback, let me know!
    • By Robin S
      File Info
      A textformatter module for ProcessWire. The module can add information to local Pagefile links in two ways:
      As extra markup before, within or after the link As data attributes on the link (handy if you want to use a Javascript tooltip library, for instance) Screenshots
      Module config

      Example of output

      Installation
      Install the File Info module.
      Add the textformatter to one or more CKEditor fields.
      Configuration
      Add markup action (and general)
      Select "Add markup to links" Select the Pagefile attributes that will be retrieved. The attribute "filesizeStrCustom" is similar to the core "filesizeStr" attribute but allows for setting a custom number of decimal places. If you select the "modified" or "created" attributes then you can define a date format for the value. Enter a class string to add to the links if needed. Define the markup that will be added to the links. Surround Pagefile attribute names in {brackets}. Attributes must be selected in the "Pagefile attributes" section in order to be available in the added markup. If you want include a space character at the start or end of the markup then you'll need >= PW 3.0.128. Select where the markup should be added: prepended or appended within the link, before the link, or after the link. Add data attributes action
      Select "Add data attributes to links" Select the Pagefile attributes that will be retrieved. These attributes will be added to the file links as data attributes. Attributes with camelcase names will be converted to data attribute names that are all lowercase, i.e. filesizeStrCustom becomes data-filesizestrcustom. Hook
      If you want to customise or add to the attributes that are retrieved from the Pagefile you can hook TextformatterFileInfo::getFileAttributes(). For example:
      $wire->addHookAfter('TextformatterFileInfo::getFileAttributes', function(HookEvent $event) { $pagefile = $event->arguments(0); $page = $event->arguments(1); $field = $event->arguments(2); $attributes = $event->return; // Add a new attribute $attributes['sizeNote'] = $pagefile->filesize > 10000000 ? 'This file is pretty big' : 'This file is not so big'; $event->return = $attributes; });  
      https://github.com/Toutouwai/TextformatterFileInfo
      https://modules.processwire.com/modules/textformatter-file-info/
    • By MoritzLost
      UPDATE: I have published a stable version of this module!
      Discussion thread:
      Github: https://github.com/MoritzLost/TextformatterPageTitleLinks
      ---
      Hello there,
      I'm working on a tiny textformatter module that searches the text for titles of other pages on your site and creates hyperlinks to them. I'm not sure if something like this exists already, but I haven't found anything in the module directory, so I wrote my own solution 🙂
      It's not properly tested yet and is still missing some functionality I would like to implement, so at the moment it should be considered in BETA. Features include limiting the pages that will get searched by template, and adding a custom CSS class to the generated hyperlinks. As I'm writing this I noticed that it will probably include unpublished and hidden pages at the moment, so yeah ... it's still in development alright 😅
      You can download the module from Github:
      https://github.com/MoritzLost/TextformatterPageTitleLinks
      There's some more information in the readme as well.
      Anyway, let me know what you think! I'm happy about any feedback, possible improvements or ideas on how to improve the module. Cheers.
    • By Mobiletrooper
      Hey Ryan, hey friends,
      we, Mobile Trooper a digital agency based in Germany, use ProcessWire for an Enterprise-grade Intranet publishing portal which is under heavy development for over 3 years now. Over the years not only the user base grew but also the platform in general. We introduced lots and lots of features thanks to ProcessWire's absurd flexibility. We came along many CMS (or CMFs for that matter) that don't even come close to ProcessWire. Closest we came across was Locomotive (Rails-based) and Pimcore (PHP based).
      So this is not your typical ProcessWire installation in terms of size.
      Currently we count:
      140 Templates (Some have 1 page, some have >6000 pages)
      313 Fields
      ~ 15k Users (For an intranet portal? That's heavy.)
      ~ 195 431 Pages (At least that's the current AUTOINCREMENT)
       
      I think we came to a point where ProcessWire isn't as scalable anymore as it used to be. Our latest research measured over 20 seconds of load time (the time PHP spent scambling the HTML together). That's unacceptable unfortunately. We've implemented common performance strategies like:
      We're running on fat machines (DB server has 32 gigs RAM, Prod Web server has 32gigs as well. Both are running on quadcores (xeons) hosted by Azure.
      We have load balancing in place, but still, a single server needs up to 20 sec to respond to a single request averaging at around about 12 sec.
      In our research we came across pages that sent over 1000 SQL queries with lots of JOINs. This is obviously needed because of PWs architecture (a field a table) but does this slow mySQL down much? For the start page we need to get somewhere around 60-80 pages, each page needs to be queried for ~12 fields to be displayed correctly, is this too much? There are many different fields involved like multiple Page-fields which hold tags, categories etc.
      We installed Profiler Pro but it does not seem to show us the real bottleneck, it just says that everything is kinda slow and sums up to the grand total we mentioned above.
      ProCache does not help us because every user is seeing something different, so we can cache some fragments but they usually measure at around 10ms. We can't spend time optimising if we can't expect an affordable benefit. Therefore we opted against ProCache and used our own module which generates these cache fragments lazily. 
      That speeds up the whole page rendering to ~7 sec, this is acceptable compared to 20sec but still ridiculously long.
      Our page consists of mainly dynamic parts changing every 2-5 minutes. It's different across multiple users based on their location, language and other preferences.
      We also have about 120 people working on the processwire backend the whole day concurrently.
       
      What do you guys think?
      Here are my questions, hopefully we can collect these in a wiki or something because I'm sure more and more people will hit that break sooner than they hoped they would:
       
      - Should we opt for optimising the database? Since >2k per request is a lot even for a mysql server, webserver cpu is basically idling at that time.
      - Do you think at this point it makes sense to use ProcessWire as a simple REST API?
      - In your experience, what fieldtypes are expensive? Page? RepeaterMatrix?
      - Ryan, what do you consider as the primary bottleneck of processwire?
      - Is the amount of fields too much? Would it be better if we would try to reuse fields as much as possible?
      - Is there an option to hook onto ProcessWires SQL builder? So we can write custom SQL for some selectors?
       
      Thanks and lots of wishes,
      Pascal from Mobile Trooper
       
       
    • By Mike Rockett
      As I mentioned in this issue, I've create a new textformatter for ParsedownExtraPlugin, which adds some oomph to your markdown.
      Repo: Parsedown Extra Plugin
      Unlike the built-in textformatter for Parsedown and Parsedown Extra, this should be used when you want to use Extra with additional configuration/customisation.
      Some examples:
      ### Test {.heading} - A [external link](https://google.com/){.google} with `google` as a class that opens in a new tab if the config property is set. - [Another link](/page){target=_blank} that opens in a new tab even though it isn't external. ```html .html <p>Test</p> ``` There's some config options available to you, such as setting attributes on all/external images and links, setting table and table-cell alignment classes, adjusting footnote classes and IDs, adding <code> attributes to their parent <pre> elements, and changing the <code> class if your syntax highlighter does not use language-*.
      I was thinking about adding the ability to make links open in a new tab by appending a plus to the link syntax, but only external links should be opening in a new tab anyway. Further, this would add extra, unnecessary processing time.
      Please let me know if you bump into any problems. ☺️
×
×
  • Create New...