dmccallum | 03/05/08

Placing new tooling on existing worksites en masse is a classic Sakai administration problem. Creating new worksites as copies of other sites is easy enough, but subsequent changes to the original site are not reflected in the copy. So if an institution decides, for example to push Melete to all course worksites midway through a semester, its options are somewhat limited since neither Sakai's APIs or its UI expose first-class bulk tool placement features. Thus institutions generally fall back to database or webservice scripting to cascade new tool placements to subsets of existing sites. As Ian Boston pointed out in his reply to the thread linked above, SASH, i.e. the SAkai SHell, is one such ad hoc scripting option.

SASH is a Sakai tool implemented in the RSF stack which presents a terminal-like UI:

Deployment against a 2.5+ Sakai instance is quite straightforward. Checkout the source from the Contrib repo and kick off a Maven build:

  $> cd sakai-trunk-wc
  $> svn co -q https://source.sakaiproject.org/contrib/sakaiscript/sash/trunk/ sash
  $> cd sash
  $> mvn clean install sakai:deploy

Following deployment and a Sakai restart, place the "HTML Terminal" tool on the super-user's worksite. You'll find that you'll need to be a super user to interact with SASH and that you must not be using the default password.

SASH ships with several bash-like builtins at varying levels of completeness. For example, here we see a simple execution of "echo":

More interestingly, though, SASH also ships with a collection of "add-ons" which include a set of administrative utility commands and script interpreter launchers. For example, here we see an administrator using the "site" command to create a new worksite and place a new page and tool on it:

1. Browse command usage

2. Create a new site

3. Create a new page and place the schedule tool on it

4. Navigate to the site and confirm the presence of the schedule tool.

One option, then, for scripting tool placements across multiple sites would be to find the Java class implementing the "site" command (SiteUtility) and extend it to support additional arguments. Or, we could implement our own command altogether, possibly leveraging the SiteUtility API where appropriate. It turns out that registering new commands is quite straight forward.

At initialization time, a SpringBeanCommandFinder collaborator injected into the SashInterpreter seeks out beans registered with its ApplicationContext which implement the BaseBuiltin or BaseCommand types.

So, to enable a custom SASH command, we can code an extension of BaseCommand and bundle it into a Sakai component. For example, a Hello World command complicated for demonstration purposes might be implemented as follows:

      
package org.sakaiproject.sash.extension.demo;

import org.sakaiproject.sash.api.BaseCommand;

public class HelloWorldCommand extends BaseCommand {

	public String getUsage() {
		return getName() + 
			" - Prints an optionally modified Hello World message \n" + 
			"Usage: \n" +
			"\t " + getName() + " MODIFIER \n\n";
	}
	
	public String getName() {
		return "hw";
	}
	
	public void execute(String[] argv) {
		String modifier = argv.length >= 1 ? argv[0] : null;
		modifier = modifier == null ? " " : " " + modifier + " ";
		if ( " ? ".equals(modifier) ) {
			println(getUsage());
		} else {
			println("Hello" + modifier + "World");
		}
	}

}

The component definition consists of a single bean:

      
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
<beans>

  <bean id="org.sakaiproject.sash.extension.demo.HelloWorldCommand"
  	class="org.sakaiproject.sash.extension.demo.HelloWorldCommand" />

</beans>

Following Maven-executed build and deployment of this command and a Sakai restart, we can invoke it from the SASH command line using the name defined by HelloWorldCommand.getName():

While the command coding and registration process is relatively painless, it's not an appealing workflow from a system administrator's point of view. Setting up the source code project is not exactly brainless and deploying new components to a production Sakai system is potentially a non-starter since SASH will not detect new or modified command implementations without a restart. More convenient would be the ability to actually script SASH to perfrom administrative tasks without involving the standard compile -> deploy -> restart process. Enter SASH's "scripting support" add-ons. These commands launch dynamic language interpreters, passing script streams read from Sakai's content hosting service. Out-of-the-box support exists for bootstrapping BeanShell, Jython and Groovy scripts.

To demonstrate a SASH-launched Groovy script we'll skip yet another pass at "Hello World" and look at solving our original problem of pushing new tool placements to existing worksites.

Here's an attempt at solving the problem in Groovy by retrieving the appropriate components from the ComponentManager cover and exercising their APIs. Note that Groovy has no trouble interacting directly with Sakai's Java interfaces. No special bridging configuration is necessary. The syntax should be relatively accessible to Java developers:

      
// addtool.groovy

// Import directly-referenced interfaces 
import org.sakaiproject.component.cover.ComponentManager
import org.sakaiproject.site.api.SiteService

// SASH will send the script name as the first arg, hence the arg
// list size must be one greater than the required arg count
assert args.size() >= 3 , "Must specify a tool ID and a page name"

// Retrieve references to Sakai components
siteService = ComponentManager.get("org.sakaiproject.site.api.SiteService")
toolManager = ComponentManager.get("org.sakaiproject.tool.api.ToolManager")

// A block that tests for the given tool's presence on the given site
def toolAlreadyPlaced = { site, toolId ->
  site.getToolForCommonId(toolId) != null
}

// A block that creates a new page on the given site and places
// the referenced tool on that page. The page and tool will have
// the same name
def placeTool = { site, toolId, pageName ->
  siteEdit = siteService.getSite(site.id)
  sitePageEdit = siteEdit.addPage()
  sitePageEdit.setTitle(args[2])
  sitePageEdit.setLayout(0)
  toolConfig = sitePageEdit.addTool()
  toolConfig.setTool(toolId, toolManager.getTool(toolId))
  toolConfig.setTitle(pageName)
  siteService.save(siteEdit)
}

// Now the script's "main" section. Caches CLI args and passes
// a block to SiteService.getSites() for site-by-site processing
def toolId = args[1]
def pageName = args[2]
println "Adding tool to sites [tool ID: ${toolId}] [page/tool title: ${pageName}]"
siteService.getSites(SiteService.SelectionType.NON_USER, ["course","project"], null, 
 null, null, null).each { site ->
  println "Processing site [title: ${site.title}] [id: ${site.id}]"
  if ( site.id == "citationsAdmin" ) {
    println "Skipping Citations Admin site..."
  } else if ( toolAlreadyPlaced(site, toolId) ) {
    println "Skipping site: tool is already placed on this site..."
  } else {
    println "Adding tool to site..."
    placeTool(site, toolId, pageName)
  }
}

Following upload via the Resources tool, SASH can execute the script by passing its path to the "groovy" command:

1. Upload the Groovy source to the /user/admin directory

2. Execute the script from SASH

3. Navigate to an affected site and verify new tool placement

Suppose, though, we were concerned that site lookups and saves don't occur with a single database transaction and in fact we'd like the entire script to execute in a single transaction. We can take advantage of the fact that the default SiteService implementation delegates to SqlService, modify our Groovy script, re-upload it, and re-execute it from SASH without bouncing Sakai. For reference, here's what such a modified script might look like:

      
// addtool.groovy (transactional)

// Import directly-referenced interfaces 
import org.sakaiproject.component.cover.ComponentManager
import org.sakaiproject.site.api.SiteService

// SASH will send the script name as the first arg, hence the arg
// list size must be one greater than the required arg count
assert args.size() >= 3 , "Must specify a tool ID and a page name"

// Retrieve references to Sakai components
siteService = ComponentManager.get("org.sakaiproject.site.api.SiteService")
toolManager = ComponentManager.get("org.sakaiproject.tool.api.ToolManager")
sqlService = ComponentManager.get("org.sakaiproject.db.api.SqlService")

// A block that tests for the given tool's presence on the given site
def toolAlreadyPlaced = { site, toolId ->
  site.getToolForCommonId(toolId) != null
}

// A block that creates a new page on the given site and places
// the referenced tool on that page. The page and tool will have
// the same name
def placeTool = { site, toolId, pageName ->
  siteEdit = siteService.getSite(site.id)
  sitePageEdit = siteEdit.addPage()
  sitePageEdit.setTitle(args[2])
  sitePageEdit.setLayout(0)
  toolConfig = sitePageEdit.addTool()
  toolConfig.setTool(toolId, toolManager.getTool(toolId))
  toolConfig.setTitle(pageName)
  siteService.save(siteEdit)
}

// A block that implements the site-by-site processing loop
def placeToolOnSites = { toolId, pageName ->
  println "Adding tool to sites [tool ID: ${toolId}] [page/tool title: ${pageName}]"
  siteService.getSites(SiteService.SelectionType.NON_USER, ["course","project"], null, 
   null, null, null).each { site ->
    println "Processing site [title: ${site.title}] [id: ${site.id}]"
    if ( site.id == "citationsAdmin" ) {
      println "Skipping Citations Admin site..."
    } else if ( toolAlreadyPlaced(site, toolId) ) {
      println "Skipping site: tool is already placed on this site..."
    } else {
      println "Adding tool to site..."
      placeTool(site, toolId, pageName)
    }
  }
}

// A block that wraps placeToolOnSites in a Runnable implementation
// such that it can be passed to SqlService.transact() and execute
// as a single db transaction.
def txn = {
  def toolId = args[1]
  def pageName = args[2]
  placeToolOnSites(toolId, pageName)
} as Runnable

// Now the script's "main" section. 
sqlService.transact(txn, "Adding Tools")

Having the ability to script Sakai with a JVM-hosted dynamic language is, I must admit, just downright cool. And to the extent that being able to upload and execute arbitrary scripts in a Sakai environment empowers system administrators and prototypers to quickly get work done, SASH is certainly a Good Thing.

Of course, SASH scripting it isn't entirely without risk. The SASH engine requires that it run as a Sakai super-user, giving its commands and scripts plenary access to any Sakai API. Additionally, since SASH doesn't run in a specialized ClassLoader or otherwise protected space, it can effectively "do" anything your JVM can do, including access the server side file system. Script development in general also tends to be somewhat naive, especially in terms of error handling and recovery (just look at my Groovy code!), and is rarely subjected to the same kind of automated testing one expects from "mainline" development. It's easy to forget that scripts which are so easy to put together are themselves pieces of "production code" and as such may require somewhat more planning than simply "hack, upload and run." For example, in our tool placement script, we cannot predict the size of the transaction resulting from any given execution. What impact would an exceptionally large tool placement transaction have on a production-deployed Sakai instance? Are we drastically increasing the likelihood of live- or deadlocking on database resources? Perhaps the script should throttle transactions into predictable "batches" to better amortize the cost of the larger process.

As they say... with the power comes the responsibility...