Dizkus

Edit: Full article here

Up until now, PostNuke URLs have consisted of an index file in the site's root with a long convoluted query string that is both user- and search-engine unfriendly, meaning the search engines choke on them and thus doesn't index PostNuke sites well, while making the URLs hard to post to people in email or in forums. For instance, a news link looks like this:

For some time now, PostNuke users have cried out for better Search-Engine Friendly URLs, and for the past few years, the only thing available has been a theme hack first detailed by Karateka (possibly E. Soysal before that, the links in the article are dead) way back in 2002, since worked on by ColdRolledSteel (Craig Saunders), and consequently me.
The advent of the ShortURL hack has seen sites hosted on Apache servers with the URL Rewriting module (mod_rewrite) enabled get URLs like

for the above link, where certain assumptions have been made about the default settings for mode, thread and threshhold. A big improvement, but not very descriptive, and it comes at the cost of heavy post-processing of the site's content for links. Also, Search Engines use link keyword relevance in their rankings, and Article123 doesn't say much about the link, except that it's an article with the id 123.
As Karateka pointed out at the time in his article, a problem in implementing friendlier URLs with virtual directories is that all paths in PostNuke are relative, ie relative to the site root folder where index.php is located, and fixing it then would have required extensive changes in the core. That is, a URL like /Example/view.html would result in the browser looking for all links relative to its present location, ie in the nonexistant subfolder called Example, and subsequently it would fail to find the linked stylesheets, images etc, and all links from the page would similarly fail.
Unfortunately this situation has not changed in the intervening years, but as PostNuke modules are becoming API-compliant, they reference the same system function to build their URLs, so fixing this function and other associated functions to use root-relative links will fix all compliant module URLs. But that leaves all other links, like images, Javascript, and stylesheets. The move to templating with Xanthia (for themes) and pnRender (for modules) is also making it easier, since Xanthia templates use a Xanthia variable to reference the theme's image directory path. So fixing Xanthia and pnRender will fix most paths in Xanthia themes. The exception are stylesheet and Javascript link paths and any links in the theme header, for which new path variables need to be introduced, so some updating of Xanthia themes is required. This makes the transition period to PN 0.8 an ideal time to introduced these changes, since few Xanthia themes have been released so far, and core modules are only just being converted to pnRender.

My proposed implementation still retain the Xanthia filter for backwards compatibility with older themes, modules and blocks, but has been wholly rewritten and pared down to 24 rules, including a rule to fix all links to be root-relative. As PostNuke is in transition to be fully pnAPI-compliant by PostNuke 0.8, the remaining ones can gradually be removed altogether as themes, modules and blocks are updated. It seeks to reduce the reliance on the Regular Expression post-processing for links and introduce more user-friendly URLs that have more relevance for people and search engines alike by using virtual directories to visually distinguish sections of the site by module and function, such as

and for the News articles introduce Category, Topic, and Title information in the link:

For instance for a news story in the category Computers and the topic PostNuke called "PostNuke Shorturls", you'd have the URL

This is a clear, concise and informative link that tells the user and search engine alike something about the link before going there, while retaining backwards compatibility with links of the old ShortURL scheme. It more closely emulates the way we think and organise information, using the folder analogy where we have a clearly-labelled Computer category folder, under which we have the various sub-categories – Topics – with various articles. In this case, we're using a virtual file anchored by the word “Article”, clearly identifying it as such, followed by the article number and title.
In this instance I've excluded the News keyword altogether for brevity in favour of the Category and Topic keywords which insinuate News anyway, though there is nothing against being consistent with all the other ShortURLs and having the Module appear first, as in

This is for the special case of the core News module though, a more generic method is needed overall for URLs with various unknown parameters passed in the query string. This implementation uses the scheme:

where the Query string parameters are tagged onto the virtual filename grouped by colons and separated by hyphens, the idea being to use commonly-used characters we might normally use in a list to make it look as natural and readable as possible. It may be a less-commonly used character than the hyphen is needed, like the tilde ~ character. The extension is not necessary, but used for convenience. The 3 types used are either one of HTML, htm, or phtml, the latter useful to distinguish when you want to link to real HTML files on the site.
There are any number of possible ShortURL systems, the simplest being to simply chop the URL into virtual directories, like /News/123/ from the above example as some do. Xaraya uses a variant of this for news, though it doesn't use mod_rewrite, so appears like

Again, this is concise, but contains few meaningful keywords other than the module name News. You can combine the two methods for News and have

which works very well, but loses some of the elegance of the above philosophy, since the latter part breaks up the virtual file into 3 with no anchor words, which is not how we organise information.
For generic URLs, there are a number of methods; for instance Mambo, another CMS, use generic ShortURLs like

for a News URL like

where the querystring values are grouped by commas and separated by forward slashes (virtual directories). It is a ShortURL, though in this case not shorter, and doesn't have any useful keywords, other than “newsfeed”, and is not very human-readable. For a generic URL, this is somewhat unavoidable, but can be better than that.
This implementation also contain a way to customise ShortURLs on a per-module basis through a file called shorturls.php placed in the module folder (see the Example module), such as the News URLs, or 3rd party modules like PostCalendar, which instead of the full URL like

with the above generic ShortURLs would be rendered as

but with customised URLs become

The beauty is, though, once we've created the groundwork in the core of PostNuke, any implementation will be fairly easy.



-Martin

If you want to try it out on a test site (what works for me doesn't necessarily work for you), there are download links below.

Note: I use Apache2, and whereas I've tried to take Apache1.3's older Perl interpreter into account, the htaccess file isn't tested on it. Possible issues on Apache 1.3: In the 3rd Party Rules section, the 4th PostCalendar rule uses non-capturing parentheses (?:...), and more generally, at the top of the htaccess file, I set a PHP directive: php_flag session.use_trans_sid off. I don't know if that could cause any trouble, but that's something that can be commented out (add # in front of the line).

Please backup your site before installing these patches, since a lot of system files and templates are replaced. The PostNuke 0.76rc4 ShortURL package is rather large, consisting of some 400 files in a 1Mb zip file. The PN0.75 package has some 170 files and is around 800kb. Most of the changes are drop-in changes that doesn't necessitate updating of modules, but there are some exceptions in the PN0.76 package, in particular the Settings and Polls modules, where you need to first go to the Module list, regenerate, and update. Specific patches for popular 3rd party templated modules like AutoTheme and PNphpBB2 are included, but only a limited number of 3rd party modules have been tested with this package. No changes are made to the database, but it is still a good idea to back that up as well. You have been warned.

PostNuke 0.75 ShortURL package (833kb)
Mega PostNuke 0.76rc4 ShortURL package (1Mb) (contains various module and template updates as well)
Short-ShortURL package for pn0.76rc4 (596kb)

Updated core themes:
PostNukeBlue (249kb)
Screenshot: The main adminpanel in PostNukeBlue with the Aqua-themed tabs
SeaBreeze (120kb)
Screenshot: Settings panel in SeaBreeze

A PDF of the ReadMe included in the package, but with additional screenshots, is found here (570kb).

I've also written a more technical ReadMe on installing ShortURLs, included in the package under the docs folder, and also found here.

here's a test of the tab system using the Aqua theme. It also comes with an XP-styled theme and the default-CSS-based one.

Martin,

Firstly good work (as usual!). A couple of comments

1) Would it be possible to get a package containing just the short urls stuff. The current downloads contain a lot of different changes not all short URL related.
2) The theme modifications to fix the paths for javascript and stylesheets etc. can all be fixed in one go by setting a base URL. e.g. .

-Mark

1. Yes, though you would need the modified (and templated) Settings module, since it's enabled there, it is after all a system feature in this implementation independent of templating. And, there are a lot of plugins where I've added pnGetBaseURI() before paths, and the pager and pagerabc plugins will now use ModURL when ShortURLs are enabled. The fixed Xanthia filter will correct paths to be root-relative, but the whole system must move away from only using relative links if it wants more advanced ShortURLs. But I'll have to try using the BASE tag, see how that goes.
And yes, I didn't quite know when to stop, I'm a bit of a perfectionist. I did pn0.75 first before I downloaded pn0.76rc, so I'd already made a tabs-enabled Settings module (man, does it scream out for it...), and so it seemed natural that the pn0.76 Admin area needed the same. And the Tabs theme is set from the Adminpanel config. It's a lot better. But having done that, all the admin templates needed updating, since they used the adminmenu with the minitabs, which cluttered up their admin panels. There's also some 3rd party stuff in there, like AutoTheme and PNphpBB2. So, yes, it's pretty big, at 400 files.
However I'll make a package with the modded core files and plugins and the Settings module. I'll throw the Admin module in for good measure, since I'd otherwise have to change the Admin config back.
Oh, and check out the new Polls template. I know you might make an Advanced one, but also make a better-looking one. The background scale-image can change with the chosen scale for the polls.
I though of making a separate package with the templates, but since they rely on the modified pnRender, I just put 'em all together.

2. You may be right, using a BASE tag could solve a few problems. There is no Xanthia pngetbaseurl filter, there's one for pnRender though. I suppose themes can use pnRender plugins? The Javascript and Stylesheet links in the themes are set by Xanthia, right?
On the whole, I've used baseuri, since it's neater and less sourcecode; the modded pnModUrl checks if ShortURLs are on, if it is, it adds the base URI, unless SSL is enabled, in which case it provides the full BaseURL. Changes to pnBaseUri is critical for getting the right path, the old one gets the wrong one, ie in the ShortURL /Example/view.html it provides /Example as the base URI
I fixed Xanthia to add the variables $baseurl and $nukepath, didn't think too hard about a name for the latter frankly, I thought of $baseuri, but wanted something less technical, not too close to or easily confused with the former, and more userfriendly. Having a Xanthia variable ready is frankly better than having to use a Xanthia plugin, which entails Smarty verifying there is such a plugin, then call its function, which then calls a pnAPI function, which (if it's like the pnRender plugin) checks the URL for email addresses to encode via pnVarPrepForDisplay (rather than just using htmlspecialchars). It's the long way of going about it, especially when you consider that all the other paths in the template use a Xanthia variable as a prefix, $imagepath. Only makes sense to do the same for the baseurl or baseuri, whatever the variable is called.
Same goes for pnRender, it is technically possible to get the theme colours, but via an awkward plugin call. Much better to use the same variable names that Xanthia uses, $bgcolor, $color1, etc. Right now in pn0.76rc4, the templated modules look, well, unprofessional. I downloaded Mambo, and they have done some very nice work on their Admin area.

If you downloaded the package, then you have the ShortURL ReadMe that details the above post and reasoning.

Also, I've amended the parent post with a link to the News article on this, I've asked ppl for comment. It's more detailed than this post.

So, I urge you (the dev team) to consider an overhaul of the templated modules along the lines of the ones in the package. That is, use theme colours and the themetable functions (I had to fix the pnRender plugins).

Oh, small niggle. Templates aren't indented like PHP code, since this is the "presentation layer", all the extra spacing appears in the sourcecode. HTML has its own indentation convention.

EDIT: Just tried

in the theme templates and header_footer module (using the Xanthia variable I added), works just great with my modded files and themes as it should, even after disabling some of the Xanthia filter rules. I'll leave the path filter rule in for now (it's the last one), but it appears a BASE tag can make it redundant (IF people use updated Xanthia themes with the Base tag; older themes would use the Header_Footer module).

Question:

We already use a short URL setup (from dev-PostNuke) and get "displayarticle674.html" for example. We have also edited some of the core files so that the story page displays the titlebar as - Article Title - Topic - Sitename.

The question would be this - if we were to implement this alternate way of achieving short urls would we be penalized in any way from the search engines for changing our urls? We rank very high on google, msn and yahoo for our chosen keywords and we would hate to ruin our placing.

One last thing, the short URL system we already use via .htaccess rewrite does a very good job at rewriting pretty much every type of module output - both API compliant and not. There was no need to change any module files to get our short urls, though this new method does spell out in the URL better just what it is the URL is pointing at.

Would it also be a leap of faith to say that PostNuke as it evolves is heading this new way anyways?

--
My PN's:

Health News

Canadian Domain Names

Free Press Releases

Politics Journal

This implementation, experimental as it is, is backwards compatible with the current "Articlexxx.html" ShortURLs. To get "displayarticle674.html" to work the same requires a small adjustment in the htaccess file.
So, where it reads

you'd change it to
I've added the optional (display)? part and corrected the backreference numbers in the rewritten URL to compensate. The rule is case-insensitive, so the case of Article doesn't matter.
The rewrite rules are all on one line, the forum wraps them.

As for other customised URLs, you would probably require similar tweaking. The extra info like Title and Topic are for humanand search-engine eyes only, not used in the full URL, so they just need to be accounted for. The titlebar is another issue relating to a modded header.php file, the pn0.75 package does include hacks like the Titlehack to provide similar info for modules, and PN already has story title in the News titlebar, but not Topic. It currently does Title - Sitename - Slogan. The tweak should be straightforward though (if you know PHP) if you compare the two header.php files. PN 0.76 is different, it uses a templated header/footer output. It sets the title in the Xanthia plugin modules/Xanthia/plugins/function.title.php. The change is the same, though, just different place.

PN hasn't seen much movement on this arena so far, the Xanthia filter in PN0.75 is basically just a bloated version of the original hack posted by Karateka in 2002. Which prompted me to have a go, but if Mark wants to take it on, then yes it may well go that way now.

The main difference in my proposal is that I'm moving the creation of ShortURLs to the core where they belong, which will only work for API-compliant modules, the rest is taken care of by the Xanthia filter with a reduced ruleset. As I mention in the article, the idea is to slowly move away from the filter altogether, as it's not an efficient way of doing it. The core modules are all being made API-compliant, and 3rd party modules and blocks are following suit. That part is about long-term planning, a transitional phase is needed, as is the backwards compatibility to current ShortURLs.

I will spell out again this is an experimental package, you should only try it out on a test site with the same modules etc as your site, and there are always the possibility of different server setups puttting a spanner in the works as well. If you have modded core modules, that will be affected too. Even so, back up. Always a good idea. I ought to do it myself sometime :)

To get the titlbar to read NewsTitle : Topic : Sitename, for Xanthia themes you'd change /modules/Xanthia/plugins/function.title.php to read

That also adds the Module name to other URLs, that can help with search engines too. Not to mention the back-button history!

I updated the packages with the Base tag added to the Header_Footer module, and the same for the Seabreeze and PostNukeBlue themes. And the PNphpBB2 header template.
And I added the Title plugin modification above. Just because it's a good idea.

I'll see about a ShortURL-only package maybe tomorrow. With the Base tag, I'd have to worry less about paths in module plugins. I'd keep the pnRender and Xanthia plugins in though. Doesn't hurt to have the base URI added to their URLs.

I must say great job, it looks incredible. I would just like to say I would be very interested in the Short URL only package, as I don't understand/know what the rest is. I'm still learning this stuff for the most part. My understanding is that if correctly installed this short URL package would make the articles have URLs that included the name of the article and such?
Thanks!
john

That's right, the News module in PN 0.76rc4 (and Pn0.75) is still an old-style module that needs to be manually fixed for this functionality (it's in the package), but I'm sure the templated version is in CVS somewhere, probably for Pn0.8.
PostNuke is in a transition stage, and the core modules, and third-party modules as well, are gradually being converted. I believe the core-module transition is scheduled to be complete by Pn0.8, hence this is a good time to suggest some changes to the ShortURLs, moving them into the core.

I'll get to the ShortURL-only packages shortly. It will still have some changes in template output, since the ShortURL settings have been moved to the main Settings module, where they belong. I would rather leave too much in than neglect something that might be needed, like a language define, plugin, or something. That's not going to hurt. But since the main changes involve replacing core files, and based on a Release Candidate at that, it's not recommended for a production site.

All the extra stuff is an overhaul of the template output of mainly the Admin section, various bug fixes I came across, and some code cleanup 'coz I wanted to. It's intended for those with a pn0.76rc test install that don't mind experimenting, and as a showcase for the developers. It's largely template changes for beauty's sake.


I had some time on a Mandrake Linux 9.1 server today to test from scratch, and it threw up some minor bugs, one with the pn0.75 package (the Settings module), which prevented it ffrom working. The other two was with PostNukeBlue, to do with the case of some Xanthia tags, which doesn't affect Windows, since it is a case-insensitive system. So I've re-uploaded those.
However it had Apache2 like my Windows server, so I still haven't tested the htaccess on Apache 1.3.

Excellent work! I was hoping for a package like this for a long time. I'll try it on a test site as soon as there is the "ShortURL-only package".

Thank you very much, msandersen! :)

The PN0.75 package is essentially a "ShortURLs-Only" package, with the addition of AutoTheme and PNphpBB2 patches and example plugins for PostCalendar and pnEncyclopedia (they are not necessary) in case people wonder. I added a Contents index with a brief description of the change to the included RedaMe for people who wanna know.
Here's the one for pn0.75:


The full pn0.76RC4 package (a test release, after all), has the following:


The "ShortURLs-Only" package for pn0.76rc4 I've just uploaded HERE has the following:


Note the themes folder has a changed theme.php file as used in Xanthia themes, it goes in the actual theme folder, however the updated path in question doesn't actually seem to be used anymore, as far as I know, so you probably don't need to pay any attention to it.
Same with the language folder, there is one addition which is mainly used in the templates in the larger package, but I've kept it in just in case.
Some of the older core modules, like News, Topics, and Sections needed to be fixed manually to support shortURLs.

Again, I've only tested it on Apache2, I have a sneaking suspicion one or two of the PostCalendar htaccess rules might be invalid for Apache 1.3's older RegEx interpreter, sadly out of step with PHP's. You'll know if you get a Server Error.

Short-shortURL package for pn0.76rc4

Martin,

Thanks for that package - that makes it a bit easier to see what's happening. I take it the admin tabs aren't strictly necessary either? Another question related to the use of the title in the URL. How well has this been tested? Are there likely to be any problems relating to the use of non standard character sets (e.g. cyrillic, chinese etc.) or, possibly, even accented characters?

-Mark