Daisy Wiki Implementation

The Daisy Wiki is build on top of the Apache Cocoon framework. Cocoon is historically strong in XML-based web publishing, which is very useful when implementing a CMS frontend. Next to that, Cocoon also gained powerful webapp development support, in the form of innovative flow-control solutions and a powerful and easy-to-use form handling framework.

Since Cocoon is one of the lesser-known frameworks, an introduction is provided below that should give a head-start into the wonderful world of Cocoon.

So what is Cocoon?

Sitemaps and pipelines

Two important concepts of Cocoon are the sitemap and pipelines. The sitemap is defined in an XML file (called sitemap.xmap) and it is based on the sitemap that Cocoon decides how to handle an incoming request (a browser HTTP request or a programmatic internal request). We'll come back to that in a moment.

A pipeline is an XML-processing pipeline, and consists of three types of components: a generator, followed by one or more transformers and ended by a serializer. These three are connected by SAX events. SAX events represent things you find in an XML document: for example a start tag with attributes, character data, and end tags. A generator produces SAX events, a transformer changes SAX events (one example is an XSLT-transformer), and a serializer serializes the SAX events to a byte stream (an XML document, a HTML document, or possibly an image file if the events represent an SVG picture, or a PDF is the events describe a XSL-FO document).

The sitemap contains matchers that match on request properties, most often this is the URL path. By walking through the sitemap, a pipeline is constructed. As a simple (but very common) example, an extract from a sitemap can look like:

<map:match pattern="/pages/mypage">
  <map:generate src="somefile.xml"/>
  <map:transform src="some_stylesheet.xsl"/>
  <map:serialize type="html"/>
</map:match>

Suppose you enter in the location bar of your browser a request for " /pages/mypage". When this request is handled by Cocoon, Cocoon will run over the sitemap till it encounters a matcher that matches the path "/pages/mypage". Inside the matcher, it will find the definition of a pipeline to be executed. When it encounters the "map:serialize", it knows all it needs to know, it stops the evaluation of the sitemap and will start executing the pipeline. To compare with Java, you could imagine the sitemap to be one big method where the matchers are if's, and where map:generate, map:transform and map:serialize instructions set/add properties to a pipeline object. After the map:serialize, it is as if there were a return instruction that stops the execution of the method.

The generator used in the example above is the default one (because, unlike the serializer, it is missing a type attribute), and the default one is the file generator. This generator generates SAX events by parsing an XML file. The transformer is again the default one, which is the XSLT transformer. The serializer then is the html serializer (which is also the default one, but I've just added the type attribute for illustrative purposes).

Something about the pattern attribute of the map:match: there can be different types of matchers, the default one is the wildcard matcher. This means that in the pattern, you can use wildcards, and then later on refer to the values of those wildcards. The following example illustrates this:

<map:match pattern="/pages/*/*/**">
  <map:generate src="/myrepository/{1}/documents/{2}-{3}"/>
  ...
</map:match>

One star in the pattern matches any character except the forward slash. Two stars match any character, including the forward slash. You can then later on refer to the matched text using the {star-position} syntax, as shown in the src attribute of the map:generate. Next to the default wildcard matcher, Daisy also uses the regexp matcher, which is basically the same but uses regular expressions to perform the matching.

Apples

Next to executing XML-transforming pipelines, Cocoon can also call controller logic. There are several implementations of the flow-engine: a javascript based one, one called "javaflow", and one called "apples". The first two feature a special thing called continuations, but since we don't use that, just ignore them. Apples sure is the least-used one (it might well be that we're the only ones using it), but its simplicity and stability make it a good choice.

So, how do these apples work?

An apple controller is a Java class that implements the interface AppleController:

public interface AppleController {
    void process(AppleRequest req, AppleResponse res) throws Exception;
}

This should speek pretty much for itself, it is a servlet-like interface. We'll go into the details further on.

In the sitemap, an apple controller can be called as follows:

<map:match pattern="/mypage">
  <map:call function="<name-of-the-apple-class>"/>
</map:match>

So when you enter the path /mypage in the browser, Cocoon will run over the sitemap and encounter the matcher matching /mypage. When it then sees the map:call, it will instantiate the class mentioned in the function attribute (the name function for this attribute is a bit weird, it is from the time the javascript flow solution was the only one). Then it will call the process(req, res) method of the apple.

Now let's look a bit more at the AppleRequest and AppleResponse objects. The AppleRequest object gives access to the Cocoon Request object, which is about the same as the Servlet Request object (Cocoon defined its own interface to be independent from servlet containers, you can also run Cocoon on the command line for example). Next to that, it offers access to sitemap parameters, which are extra parameters which can be defined using <map:parameter name="..." value= "..."/> tags inside the map:call element.

The AppleResponse object tells Cocoon what it should do after the process() method returns. There are two possibilities:

• either a Cocoon pipeline is called to render a page (in which case an object, typically a Map, can be passed to that pipeline)
• or a HTTP redirect is performed to bring the user/browser to another page

Knowing all this, you can see how this gives a web-MVC-style of working: the sitemap is used to determine a controller to call, the controller then does it works and finally lets Cocoon render a page (based on a pipeline described in the sitemap), or redirect to another URL.

There's still one important thing that we didn't mention: Apples are stateful objects. After Cocoon instantiates the Apple, it will generate a random ID and store the apple, whereby the random ID can later be used to retrieve the apple instance. This random ID is called the continuation id. How is this useful? Well, let's first look at how an existing apple instance can be called from the sitemap:

<map:match pattern="/mypage/*">
  <map:call continuation="{1}"/>
</map:match>

When you do a request for e.g. /mypage/123, the above matcher will match it, and the value "123" is passed as continuation ID to the map:call statement. This will cause Cocoon to look up the apple instance stored under the given ID, and call the process() method of the Apple. So you could say that the flow continues in the same apple instance.

The advantage of this Apple-approach is that the instance variables of the apple can store objects related to a certain flow instance. In contrast to plain servlets, you don't need to store them in the session and afterwards retrieve them from there. Also, in contrast with plain session-based solutions, you've got separate flow data for each instance of the Apple, and thus the same flow can be run multiple times in parallel even if there's only one session.

If you don't need the flow logic to be stateful, let the Apple implement the interface StatelessAppleController in addition to the AppleController interface (StatelessAppleController is only a marker interface).

First look at the Daisy sitemap

The logical structure of the Daisy sitemap (and of a sitemap in general) is as follows:

<map:sitemap>
  <map:components>
    ...
  </map:components>

  <map:resources>
    ...
  </map:resources>

  <map:pipelines>
    <map:pipeline internal-only="true">
      ...
    </map:pipeline>

    <map:pipeline>
      <map:parameter name="expires" value="access plus 5 hours"/>
      ...
    </map:pipeline>

    <map:pipeline>
      ...
    </map:pipeline>

  </map:pipelines>
</map:sitemap>

Now some explanations about all this:

Cocoon allows to have multiple sitemaps and to mount one sitemap into another, this allows to split off the handling of a part of a site to a separate sitemap. There is one root sitemap, this is the one where it all starts and this is the one located in the root of the webapp directory. Daisy ships with the default Cocoon root sitemap, which contains a lot of component definitions and automatic mounting of sitemaps located in subdirectories of the webapp directory. The Daisy sitemap itself is meant to be used as a subsitemap of this root sitemap (you could use it directly as the root sitemap if you copied over all required component definitions).

In the above given fragment, the first element inside the map:sitemap element is the map:component element. This element contains the declaration of various types of sitemap components: generators, transformers, serializers, and a couple of others.

The map:resource element contains resources definitions, these are reusable pieces of pipeline defintions.

Then the map:pipelines element contains all the statements that Cocoon will run over when processing an incoming request, i.e. the matchers, generators and so on. It is however first divided using map:pipeline (without s at the end) elements. The map:pipeline elements allow to set some options, some of which are illustrated in the above example: the first map:pipeline has an internal="true" attribute. This means that this part will only be considered for internal requests: thus not requests coming from a browser, but requests coming from for example an Apple.

The second map:pipeline element is the one which contains all the matchers for static resources, therefore we specify a special parameter there to tell Cocoon that it should tell the browser that these files won't change for the next five hours. This heavily reduces requests on the webserver, since things like images, css and js files only need to be retrieved once.

The last map:pipeline is the one containing the matcher for all public non-static requests.

So if you want to figure out how a certain URL that you see in your browser is handled by Cocoon, you open Daisy's sitemap.xmap file, and look top-to-bottom at the matchers inside the last map:pipeline element. Often, these matchers will contain a map:call statement that calls an Apple. You can then go look at the source code of that Apple to see how it handles the request, and what method it calls on the AppleResponse object. Often, this will be the sendPage(String path, Object viewData) method, which will cause Cocoon to start evaluating the sitemap to match the given path (but now it starts in the first map:pipeline element because it is an internal request).

Daisy repository client integration

If you read our introduction to Merlin, you'll remember about the Avalon Framework interfaces like Configurable, Initializable and Serviceable. Well, Apples (and all Cocoon components in general) can also implement these interfaces and Cocoon will honour them. This is because Cocoon is also Avalon-based, though it doesn't use Merlin as runtime container.

The Daisy repository client (the remote implementation of the RepositoryManager interface) is available inside Cocoon's component container, and you can get a handle at it in an Apple by implementing the Serviceable interface so that you get access to a serviceManager from which you can retrieve it. (Note that in contrast to Merlin, where each component needs to define its dependencies, the component container in Cocoon is less strict and allows you to retrieve any component).

If you look at the implementation of a Daisy Apple, you'll see that the work of getting a repository instance is done by the method LoginHelper.getRepository(request, serviceManager). This method will first check if there is a Repository object stored in the session (which is stored there when logging in) and if so return that, otherwise it will create a default guest-user Repository object.

Actions

In the Daisy sitemap, you'll also see a couple of map:act statements. These call a so-called Cocoon action, which is basically a Java class that can do whatever it wants. In contrast with an Apple called by a map:call statement, the evaluation of the sitemap continues after encountering a map:act statement.

JX Templates

In the Daisy sitemap many pipelines start with a map:generate with type="jx" . This is the JXTemplate generator, which executes a template. This template can access the data that you passed as the second parameter in the AppleResponse.sendPage(path,viewData) method. The syntax for JXTemplates is described here.

Cocoon Forms (CForms)

For handling of web forms, Cocoon makes use of the CForms framework, which is documented over here.

CForms is a widget-oriented high-level forms framework. It is based on the concept of having a form definition (which defines the widgets that are part of a form) and a corresponding form template (to define how the widgets are placed in a HTML page). The form definition and form template are simply XML files. One handy side-effect is that for Daisy's document editor, for which the structure depends on the document type, we can simply generate the form definition and form template by performing an XSLT transform on the XML representation of a Daisy document type (see doctype_to_form.xsl and doctype_to_formtemplate.xsl)

This last sentence isn't entirely correct anymore, since the implementation of the editor has changed a bit. It is now only the fields form which is dynamically generated, using doctype_to_fieldedit_form.xsl.

Extending the Daisy Wiki application

The default Wiki application offers an extension mechanism and sample, showing you how to include outside functionality inside the Wiki application (and default look and feel), this is discussed elsewhere. Also, here is a very simple example showing you how to integrate external RSS feeds into a Daisy webpage.