This is a log of new features of note in DWiki.
New: DWiki can generate Google Sitemaps
I added the ability for DWiki to generate Google Sitemap format XML files, using the view 'sitemap'. The information included is very basic currently: just the URL for each file page, all of them set to priority 0.8 (in the hopes that Google will decide that all of the directories are priority 0.5 and prefer returning file page results).
Google does not say what Content-Type you should return sitemaps in, so I have opted for 'application/xml'.
In the future, something as elaborate as Atom rendering may be done. For now, everything is hardcoded in the
sitemap::minurlsetrenderer.Updated: now directories are shown too, at priority 0.6. This feature is clearly going to be in flux for a while.
New: https:// now supported in plain text
As part of fixing Atom feeds to not break embedded https:// urls, I decided that we should support plaintext https:// URLs, like say https://bugzilla.redhat.com/.
DWiki should now support non-HTTP URLs much better in general (before, there were a number of problems and issued). You can even include mailto: links if you really want to.
New: Atom feed autodiscovery
There's a standard for autodiscovery of Atom feeds, involving
<link rel="alternate" type="application/atom+xml" href="...">element in your<head>. Now DWiki has aatom::autodiscrenderer to create them.The current code only generates 'recently changed pages' Atom feed links, and so disappears entirely when there isn't one. In theory one can have multiple autodiscoverable feeds (the first is the default, and they get
title="..."elements), but I don't quite feel like being that daring just yet.(I am also not confidant that clients have the UI issues involved sorted out. I'm not sure I have the issues sorted out; for example, should file pages have only the comment feed in the autodiscovery, or should they also have the recent changes for their directory feed in? Which better matches practical user expectations? Can I expect users to be aware of the difference between directory pages and file pages?)
New: linktocomments renderer
This is a little new renderer that creates a link to a page in the view necessary to show comments. In turn, this has caused the 'your comment has been posted' template page to be tarted up so that it uses it, thereby letting people who have posted comments see them in the page they go to.
(I have decided not to have it link to the comment section of the page, just on general principle. I may change my mind about this.)
New:
feed-max-sizeandfeed-max-size-ipsThis is all because LiveJournal has undocumented size limits on incoming syndication feeds, limits that DWiki can easily blow past. Since I actually wanted LiveJournal to be able to get syndication feeds from me, DWiki has grown two new configuration settings.
feed-max-sizeis an integer kilobytes. It is a rough limit on how large any feed can be; once DWiki generates a feed that is this many kilobytes or larger it stopps adding more entries, regardless of the setting foratomfeed-display-howmany. If unset, there is no size limit.
feed-max-size-ipsrestrictsfeed-max-sizeto the whitespace separated list of IP addresses or tcpwrappers style IP address prefixes (eg '66.150.15.' to get all of66.150.15.*). Syndication fetches from other addresses will behave as if there was nofeed-max-size.Strictly speaking,
feed-max-sizelimits only the size of theatom::pagesoratom::commentsoutput to that size. Whatever else is tacked on to make up a feed (hopefully not very big) will add some extra size.Moral: undersize
feed-max-sizea bit. For LiveJournal, the limit is apparently 150 kilobytes (currently), so setting it to '120' or so should provide a comfortable safety margin.Although I'm not entirely fond of this (to put it one way), the documentation has been updated appropriately, making this feature more or less official.
New:
/oldest/virtual directory restrictionDWiki has long been able to give people the latest N things in a virtual directory context (as
latest/<N>). Now it can give them the oldest N things, using the obvious syntax:oldest/<howmany>.Just to show off, ranges properly convert themselves into 'oldest/<N>' at the end of their run, just as they convert themselves into 'latest/<N>' at the start.
Documentation has been updated appropriately.
New: Better Last-Modified handling
Over the past while it has become increasingly obvious that it's useful for as many responses as possible to carry a
Last-Modified:header. (The last straw was wanting Google's index to show modification dates for DWiki pages.)My reason for killing
Last-Modified:was so that things like logging in and logging out, which can't be reflected in the timestamp, would still have conditionalGETs be served new pages. But since the conditionalGETlogic is in DWiki itself, I can have DWiki be smarter about it.DWiki now separates the page timestamp from the idea of whether the page timestamp is reliable or simply vaguely useful information. The page timestamp will always be served if it exists at all, but conditional
GETs only look at the page timestamp if it's reliable (which means that if authentication is on, the answer is generally 'not').This should work much better.
New: Page Titles
Pages now have accessible 'titles', sort of. A page's title is taken to be the value of the header that starts the page, if said header is on the very first line. (So this page's nominal title is 'New: Page Titles'.) The header level doesn't matter; a
<h6>is as good as a<h1>, so long as it's the first line on the page.This info is available only after the page has been rendered, in the new global context variable
:wikitext:title. Fortunately for us, Atom feed entries can have their fields in any order, so we are free to generate<title>after<content>.Why did I do this? First, it's suitably low rent, and second I decided I wanted some vague way to generate semi-real page titles in Atom feeds instead of the current full path to the page (ever so helpful and informative as it is).
The only tricky bit was making sure that only the appropriate magic wikitext renderers set the page title, and not all the times that we spin through wikitext looking for, eg, permissions. (Especially important in Atom feeds, as Atom feeds look at everyone's permissions before they do the real rendering.)
A DWiki page (technically, any wikitext, so comments too) can now start with the line '
#pragma pre' to declare that the entire rest of the page is simply preformatted text and should be barfed out as such (minus the #pragma line, which is swallowed). '#pragma plaintext' is accepted too.This is a much more convenient and maintainable way to stick plaintext files (such as program source or something) into a DWiki than indenting the entirity of their text one space.
Note that this does not make the page come out as text/plain. The page is still text/html and fully templated, it's just that the wikitext is one big <pre> lump, instead of more sophisticated formatting.
It's unlikely that DWiki will acquire any other sorts of pragmas (eg to say 'format this as nicely HTML-ized Python code'), partly because ChrisSiebenmann is dubious about the 'nicely HTML-ized' bit of any formatters since they invariably involve aesthetic decisions that people (eg, him) can and do object to. Having an easy way of including plaintext is the 80%-90% solution, and that is the DWiki way.
DWiki has a new template handling scheme: the core idea is that we now have a way of a) picking the first existing template from a list of them and b) generating candidate templates by variable substitution and 'all parent directories' expansion. This gives DWiki a simple and general framework for doing things like 'template injection', which lets us skin an entire directory hierarchy (but not the entire wiki) with things like blog sidebars.
This also gives us a single top-level template that generates all normal HTML-based pages, thereby giving us a single place to skin the entire site. The per-view templates in
views/*(now only a convention) now just generate view-specific information, leaving all of the rest up to the top-level template.The clarity and lack of stupid template piece duplication of the result is a clear indication of how it is a better scheme. (And no more silly things like splitting a <div> start and end into different files and hoping they get included in the right spots.)
TemplateSyntax and TemplatesUsed have been revised appropriately.
One obvious way of handling blogs with categories is to create appropriate directory hierarchies for each category, then hardlink a page's file into all of the appropriate 'category' directories. However, this raises a problem: DWiki's idea of a page's identity is its path.
Directories can now have Readme files, called
__readme. Readme files are injected into pages via the new rendererinject::readme(probably the first of several injectors).The current templates don't inject
__readmein normal directory views, but do inject them for blog and blogdir views (as you may see from this directory). Blog and blogdir views now drop all files starting with__, taking out__readmeand__accessand any future special magic files.
DWiki can now generate Atom feeds for recently changed pages and recently made comments, either for the entire DWiki or for some subtree of it. For comments, this can be down to an individual article.
At the moment, pages in the Atom feed are rendered without macros except for
CutShort, for efficiency reasons. All of the links are turned into absolute links (with http:// et al), since this is basically required. Nulled-out macros produce a small message to that effect in the generated content, so that people reading the Atom feed can tell that something is going on.
The primary way of getting nested lists is now to indent the nested list entries relative to the parent list (entry). This looks visually better in plain ASCII for cases when there is a decent amount of text.
Although ChrisSiebenmann thought he wasn't going to, the old style of nesting lists (multiple list start characters, eg
***) still works. It turns out the GNU Emacs will properly autoindent for these lists but not for real indented lists, plus sometimes they actually look visually better.The amount of old-style nesting is ignored in an indented context; it's treated as just a new level.
You can now use LinkAbbrevs by name (not by URL) without a
|; ie, instead of writing[[<text>|]], you can just write[[<text>]]. This only happens if<text>wouldn't result in a link to a real page or an external URL if there was no abbreviation.Thus, one can write
[[Google http://www.google.com/]]in the page once, and later write[[Google]], and have it work out.
DWiki now lets you use spaces to separate things in
[[....]]links instead of|. If you do this, the last word is taken as the link URL or page, and the rest are the link name. (|has priority over this; DWiki tries space-separation only if there is no|.)Thus
[[Google Rules The Web http://www.google.com/]]turns into Google Rules The Web.You can use either side as an abbeviation later, for example: Google Rules The Web, Google Rules The Web. (See View Source.)
LinkAbbrevs done this way don't have to use
|, as long as there is a space in the value:[[Google Rules The Web]]still turns into Google Rules The Web.This allows somewhat more aesthetic long link name things.
Note that the opening
[[and the closing]]have to be on the same line in the wikitext.
A DWiki RedirectFile can now point to absolute URLs as well as local DWiki pages. Absolute URLs are written like they would be in
[[...]]; either http:// or<...>.
DWiki now supports generating links to URLs on the wiki's web server that are outside the DWiki itself. These are written as links in the format
<...>and have to happen inside[[...]]. For example:See [[the root of this web server|</>]].generates a link to the DWiki's web server's root, which is all but certain to be outside the DWiki space if you're running DWiki as a CGI-BIN.
DWiki now remembers the name and URL for links that you write with
[[...|...]]and thereafter allows you to omit one side of the|, at which point it will fill in the remembered values. This means that if you want to link to Google a lot in your document, you only have to write (eg) the URL once, and can thereafter refer to Google much more compactly.This does collide a little bit with using
[[...|]]to write totally unstyled text: it only comes out unstyled if it's not already been used as the name of a link.For now I will live with that.
The
RecentChangesDWikiText macro now accepts additional arguments that are the list of directories to restrict the listing to, or with a dash at the front, directories to exclude from the listing. If you combine both, both criteria must match: in an included directory and not excluded.This lets a RecentChanges listing exclude areas that churn too much or are otherwise less interesting to list. (Perhaps, for example, a blog sub-hierarchy in a DWiki.)
If there is a common directory prefix that scanning is limited to, the scan is efficient: only that directory and lower is looked at at all.
Template expansion via
#{...}now removes a final newline if such a final newline is present. (It doesn't remove more than one newline.)The final newline is really an implementation artifact of files; it's there because lines end with newlines, not because people consider it to be part of the file's real content. Deleting it thus brings template expansion closer to inserting people's idea of the file's contents into place.
It also means that we avoid having templates introduce whitespace into undesired places. For example:
[There's more starting at %{blog::seemore}#{blog/rangemore}]and
blog/rangemoreof:or %{range::moreclip}doesn't introduce a gap between the end of %{range::moreclip}'s output and the ']' in what the browser displays. (See how we didn't think of
blog/rangemoreas actually having a newline at the end as part of it?)
If {{RecentChanges}} is invoked in a {{Striped:...}} context, it now names the links to the pages by the name of the page instead of the page's full path. This turns out to be much less annoying if many (or even some) of the changes are deep in a hierarchy.
Rational: you asked for a compact display so you're going to get it.
All wikitext macros and template renderer functions now have Python docstrings. This is because there is a new macro, {{DocAll}}, that spits out a list of macros or renderers and their docstrings, thereby creating a one-stop shop for a list of them all and documentation for them.
Formatting and TemplateSyntax have been changed to use this, thereby killing several birds and a fix-me with one somewhat extended stone.
In the process of writing docstrings, I fixed several irritating limitations and renamed a few renderers.
ChrisSiebenmann likes this approach best because it keeps the documentation closely attached to the function, thereby serving as a clear visual reminder that a) change the function, update the documentation that's right there in front of him and b) write a new function, write a docstring to go with it.
Directories can now be displayed as a 'blog', which treats all file descendants of the directory as if they were blog entries and attempts to be somewhat intelligent about how much to show and how to navigate things, supporting navigation by page date or Nth to Mth most recent pages.
ChrisSiebenmann doesn't believe this makes BlogDir obsolete. BlogDir is an excellent view for ChangeLog style situations, which is exactly what NewFeatures uses it for; however, it makes a bad way of displaying a true blog-style environment because of eventual information overload. BlogView is designed to deal with that by trimming what gets displayed and providing time-based and range-based navigation.
Default directory views, mentioned before in DefaultDirViews, are now inherited from parent directories if the child specifies no particular desire about the whole thing.
This is convenient for setting an entire hierarchy to a BlogView (qv).
DWiki has a new formatting feature. If you write just '* * *' on a line by themselves, they center and become a separator. Like, well, this:
* * *
You can't indent them, because I decided that would run too much risk of confusing things with pre-formatted text blocks.
Why? I just decided I liked that style of chunk separator. (Maybe if I could get a <hr> variant that didn't stretch the entire width...)
Note: feature subject to me changing my mind.
Symbolic links inside the DWiki page area now cause redirections if their value would be a valid redirection in a REDIRECT line.
For example: FrobTig, which is a symlink in /Aliases with the value '
../People'.We could have tried using os.path.realpath() on the symlink and taking the result relative to the store root, but I think that that has more subtle explosive breakages.
Symbolic links that don't resolve to a real DWiki page this way are interpreted normally, so you can still have symlinks that point to files outside the DWiki pagestore root.
DWiki now supports 'virtual directories': directories that don't really exist but instead serve to limit what's shown for a real directory. For example, you can limit what's shown for a real directory to only the most recent 5 things, or to only things written on 2005/05/29.
DWiki's tables can now have rows that span multiple lines, using indentation to continue them on subsequent lines just like with lists. The appearance is straightforward; just write:
| start your table | another cell that is continued on another line | the end cell | | cell one | cell two | cell three |A row is closed by having a '
|' at the end of a line, or just by starting a new row.
DWiki now supports definition lists (<dl>, <dt>, <dd> in HTML). I started with what I think was Wikipedia's syntax but decided it was ugly in plain text so came up with my own that I like better.
DWiki now has a simple hierarchical way of handling access to pages for various things (both access and commentability), so you can give (or take away) permissions for things to entire directory trees at a shot. We use a simple implementation where directories can have a magic file called
__access, which creates default permissions for everything under them.
DWiki now has an extremely low-rent generic from-the-web search functionality. It's so low-rent I'm not sure I'm going to keep it, but we'll have to see.
As part of the fallout of other work, I fixed the nit where 'View Source' would show as a page tool on pages where viewing the source would get you a permission or unavailability error.
With blog-style things now decently supported, there is the problem that some pages want to be longer than is comfortable for display in a blog setting. Now wikitext can signal that only the front should be shown in some contexts, like so:
Directories can now have default view types associated with them, by the simple yet sleazy method of touching a file in them called
.flag.prefview:<viewname>.This lets us auto-set a directory to default to a blogdir view.
The page tools now autogenerate a list of 'See As <whatever>' links for alternate directory formats.
This required some innards magic to distinguish a normal view being explicitly specified from no view being specified and us just defaulting to a normal view.
Directories can now be displayed as a 'blogdir', which treats the files in the directory as if they were blog entries: they are sorted by modification time, most recent first, and then they are all displayed inline with a title.
A chunk of this behavior is controlled by templates and new renderers. 'blogdir' is a new view, only valid for directories, which uses
blogdir.tmpl. Theblog::blogdirrenderer does the direct expansion, running each file through the templateblog/blogentry.tmpl.There are also new renderers for the new blog-like time format and for showing the owner of a file. (Currently the Unix and/or RCS owner.)
This is ... how shall we say it ... not hugely scalable over the long run in terms of time structure and number of entries.