Jump to content

Using SmartyPants and HTML Entity Encoder Textformatters together


Robin S
 Share

Recommended Posts

Are the SmartyPants and HTML Entity Encoder textformatters compatible with each other?

If I apply Entity Encoder first and SmartyPants second then the quotes stay "dumb".

If I apply SmartyPants first and Entity Encoder second then I get code like

’

in my field output.

Link to comment
Share on other sites

I believe it depends on the particular textformatters that you are using. Some will work together and some won't.  ProcessWire's Wiki briefly talks about textformatters and their use.

Additional Link that explains Textformatters:

http://www.flamingruby.com/blog/taking-apart-a-simple-textformatter-module/

Link to comment
Share on other sites

Some will work together and some won't.

I can understand that, but in the case of SmartyPants and HTML Entity Encoder these are both essential formatters in my book. SmartyPants for proper quote mark typography and HTML Entity Encoder for valid HTML from plain text fields. I'm surprised they don't work together, which makes me think I must be missing something.

Link to comment
Share on other sites

From the first link I gave you:

You may also apply multiple text formatters at once. Simply select more than one, and drag them to the order you want them to process the text. This doesn't necessarily mean that you should add more than one though. For instance, if you added both "entity encoder" and "Markdown" you'd end up with double encoding, since Markdown already does it's own entity encoding (as do most lightweight markup languages).

An example of where you might want to apply two text formatters would be to combine Markdown with Smartypants. These two are designed to run together. According to the author, Smartypants is meant to run after Markdown, so we'd drag Smartypants to be below Markdown.

  • Like 2
Link to comment
Share on other sites

Thanks, but I'm not clear on what you're getting at. My first post explains that using the textformatters in either order gives unsatisfactory results.

I believe the problem is that SmartyPants encodes the quote mark, dash and ellipsis punctuation as HTML entities (but no other characters, so it can't be used as a replacement for HTML Entity Encoder unfortunately), and then HTML Entity Encoder double-encodes the ampersand character in those entities.

Two possible solutions I can see:

  1. Modify HTML Entity Encoder to allow the "double_encode" parameter to be used in htmlspecialchars, which avoids the double-encoding of existing HTML entities. The downside is that it wouldn't be possible to have literal text like "&" appear in the output. Funnily enough, this problem manifests in the PW wiki article you linked to, with phrases like

    Note how the ampersand is now entity encoded, being "&" rather than "&".

  2. Modify SmartyPants so that it inserts UTF-8 characters rather than HTML entities. I tried this and it seems to be a good solution. If there are no major downsides to this maybe Ryan will consider adapting the core TextformatterSmartyPants module to allow this as an option.
Link to comment
Share on other sites

  1. Markdown and Smartypants are meant to work together.  They are designed to work together.
  2. Markdown does it own "entity encoding" so there is no need to use HTML Entity Encoder with Smartypants.
  3. Use Markdown first then have Smartypants apply.

That's what I got from the information that I linked to.

  • Like 1
Link to comment
Share on other sites

Ah, I get you now, and it works. :)

Though it does seem like using the Markdown parser on text that does not intentionally contain Markdown is risky - could create as many problems as it solves. I can imagine situations where an editor enters a plus, minus or asterisk at the start of a line and a list item inadvertently results.

I guess I could always add a strip tags textformatter at the end to remove any HTML tags added by the markdown textformatter, but that is getting kinda crazy.

Link to comment
Share on other sites

Why not create a copy and remove the encoding functionality from smartypants? As long as you make sure everything is encoded afterwards there shouldn't be any concerns with that.

Thanks, yes, those were my thoughts also a couple of posts up. Seems to work well with UTF-8 characters instead of HTML entities. There's no need to entity encode the characters inserted by SmartyPants for modern websites using charset UTF-8.

I think this should be the default for TextformatterSmartypants, or at least an option, if this makes the module compatible with TextformatterEntities - using them together is pretty important for anyone who wants both valid HTML and correct punctuation from plain text fields. I'll try my hand at modifying TextformatterSmartypants to give the option of UTF-8 or HTML entities and submit a pull request.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...