LynxWikiDevelopment

Enter a topic name to show or a new topic name to create; then press Enter
Summary
describes how LynxWiki has been and continues to be developed
LynxWiki Development
Developer John Walker Davidson
Location South Mountain, Ontario Canada
Development Started 2013-02-25
Current Version (at time of writting) v 1.1.7.1
HomePage Source Code Repository

LynxWiki Background

Before LynxWiki, there was FlexWikiFlexWiki was an early open source project sponsored by Microsoft and included the unique feature of a user accessible embedded scripting language to generate dynamic content in the wiki. This scripting language was known internally as WikiTalk and had similarity to SmallTalk in that all objects were immutable after initial creation. However as an early .Net project started in 2003 it lacked architectural maturity, that would make it easy to maintain and extend. The engine was heavily tied to the ASP.Net event processing structures.

I wanted LynxWiki to have the same dynamic content generation capabilities, but it had to run in MVC rather than the ASP.Net environment. The capability target for LynxWiki is to have the same basic capabilities as are available in MediaWiki, but those capabilities to created independently and not directly interoperable with MediaWiki data.

In deciding to have LynxWiki as an Open Source project it was necessary to review a number of potential open source solutions for external requirements that could be used in providing functionality to LynxWiki. It was determined that an Apache Version 2.0 license would be appropriate for development of LynxWiki.

LynxWiki Internal Resources

The main internal resource is a parser that takes raw input from the user and eventually outputs that as HTML5 text for a browser. There are 3 elements to this parser: 200 plus regular expressions, an engine to apply these regular expressions and output an intermediate XML file and finally an XSLT file to convert the intermediate data to actual HTML5 output. This was created as an experiment within FlexWiki and did not meet the threshold for deployment due to performance issues. It turns out a number of years later that the performance problem is only an issue in 64-bit regular expression processing and does not exist in 32-bit processing.

Eric S Raymond wrote in The Cathederal and the Bazaar that "Smart data structures and dumb code works a lot better than the other way around."[ 1 ] This concept is key to the capabilities in LynxWiki in that smart data enhances the speed of the application and the ability to provide features within the application with minimal side-effects. The data for a version is always stored in one file and is duplicated in the current version file. This way the current version is always known and the new version is always written as a discrete file. A minor update needs to update both of these files. The data file stores the intermediate output as processed by the parser, so that a request to display a topic only needs to process the XSLT step rather than parse the full topic data. The use of XSLT processing for final output virtually guarantees valid HTML5 output. Use of a hierarchy of regular expressions in the parser enables rapid development of new output features.

LynxWiki External Resources

The selection of the Apache version 2.0 license was important for the successful development of LynxWiki. There were two possible choices for a license: the GPL version 3.0 or Apache Version 2.0.

Source Repository

The choice of license would determine where the repository was housed, with the choices there being CodePlex or GitHub. If GPL was the license then the repository would be GitHub as CodePlex did not allow GPL v3 licenses, otherwise it would be CodePlex. Either would integrate with Visual Studio 2012, but I was more familiar with the TFS support provided by CodePlex[ 2 ]. I had no real experience with Git as a repository.

Scripting Engine Provider

There are any number of wiki applications available[ 3 ][ 4 ]. I needed to be able to create something new and unique in this field. LynxWiki needed to have the ability for the user to create scripts that would run as a topic is loaded and modify the contents of the topic dynamically. Of course this scripting language would have to be securable so that the ability to create malicious topics would be severely reduced, if not eliminated. The other requirement beyond security was the ability to expose .Net objects to the script language easily, so that the user could manipulate topic content.

The Microsoft sponsored Dynamic Language Runtime (DLR) seemed to offer the most capability to provide the glue to interface the wiki application to the scripting capability. The DLR is licensed with the Apache v2.0 license. As the Apache license is not compatible with the GPL license[ 5 ] all the software for LynxWiki would have to be non-GPL. The final piece of the scripting puzzle was to select the actual scripting implementation and for this I chose IronPython It took about 2 days of effort to integrate IronPython into LynxWiki and get the first scripts working. Iron Python in Action[ 6 ] by Micheal J. Foord and Christian J Muirhead was instrumental in getting this all working so quickly.

Full Text Search Provider

After the Apache v2.0 license was chosen it became relatively easy to start filling in functionality with other Open Source components. One of the more important elements of a wiki is a fast search capability. Lucene.Net was an obvious first choice. It took about 6 hours to get a test of the search functionality going and that proved that Lucene.Net would be more than adequate. There are administrative functions available through the wiki interface for managing search indexs. the search functionality has also been exposed to the script language so that dynamic queries can be executed by users to create topic content.

Other Functionality Providers

The capability to perform a side-by-side compare of versions of a topic is provided by " DiffPlex":http:diffplex.codeplex.com. The ability to export an archive of the wiki topics for offline use is facilitated by DotNetZip Display cues for menu items are provided by an icon set, Silk Icons by Mark James.

Javascript Providers

Javascript is important in implementing a responsive web application and reduces the load on the web server itself. Microsoft began distributing the Open Source jQuery as part of their MVC product. Similarly, the product Modernizr is also distributed by Microsoft. This means those 2 products are already integrated into ay MVC-based application. Code samples are easily included in the wiki and the enhanced display is a result of SyntaxHighlighter by Alex Gorbatchev. The ability to include properly formatted mathematical formula is a contribution of jsMath from Union University.

On-going Development

Functionality still to be provided are lists of incoming and outgoing links on a topic, a table of contents. a script-based picture gallery and completion of the script-based survey creation and result display system.

Conclusion

Many of the external components carried Apache v2.0 licenses, and those that do not are compatible. It is amazing that such disparate functionalities can easily be combined to produce an application much greater than the individual sums. Previously such functionality took multiple person-years to complete (see history of FlexWiki while I, as a single developer, have been able to accomplish all of this in less than 2 months. This would not have been possible without the enormous contribution of all the other Open Source projects that have provided critical functionality to this application.


  1. Smart data - http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s06.html
  2. Source Repository - http://lynxwiki.codeplex.com
  3. Wikipedia Comparison - http://en.wikipedia.org/wiki/Comparison_of_wiki_software
  4. Wiki Matrix - http://www.wikimatrix.org
  5. Apache / GPL Compatibility - http://www.apache.org/licenses/GPL-compatibility.html
  6. Iron Python in Action - http://www.ironpythoninaction.com/

Version: 30   Revised: 2013-04-19 14:13:19 Last Updated by: 208.114.148.38 Show Links to Topic