It took me about 50 pages to get into the Play Framework Cookbook. The stated goal of the book is to pick up where the beginner’s documentation leaves off. Having already written a couple applications using Play, I was hoping the book would jump right into some nice advanced topics that I wasn’t familiar with. Instead, the first chapter (Basics of the Play Framework) dealt with exactly the beginner issues that are better handled with a Getting Started guide. If you’ve spent much time with the Play framework, you’ll probably be best served by jumping ahead to Chapter 2 (Using Controllers). Here, the author makes an emphasis on keeping your controllers lean by generous use of helper classes and methods. The renderPDF and renderRSS methods presented are good examples of this.

The similar concept that your application should remain lean and functionality should be extracted into modules is introduced in Chapter 3 (Leveraging Modules). The author shows a nice selection of built-in and downloadable modules that can be used to clean up your code (dependency injection), lock down your application (security), and interact with other databases (MongoDB). This would have been a nice place to jump right into writing your own modules, but first there’s the small interlude that is Chapter 4 (Creating and Using APIs). This chapter included a nice example for creating a custom tag to display charts from the Google Chart API, but it still seemed out of place.

Chapters 5 (Introduction to Writing Modules) and 6 (Practical Module Examples) provide a solid overview of the module system and how it can be used. Even more important, it discusses the limitations of Play modules and how they can be avoided. The author also gets his hands dirty with topic of bytecode enhancement, a technique that Play uses to provide the optimum developer experience. Writing the code to provide that experience (non-type-checked code in quotes, eek!) is something I’d like to avoid, but at least I know now how Play does it. Other examples include relevant topics such as indexing with Solr and interacting with ActiveMQ.

The final chapter (Running in Production) addresses some issues that pop up when moving from developer machine to the data center. A number of recipes demonstrate deployment with popular web servers (Apache, Nginx, Lighttpd) if you don’t want to use the built-in Play server in production. Integration with the Jenkens and Calimoucho continuous integration systems is also discussed in this chapter, although I think these topics would have been better suited to a separate chapter focusing on testing. Rounding out the production chapter are a number of topics related to monitoring and logging.

Overall, I think the Play Framework Cookbook would be a reasonable purchase for someone who’s written a couple smaller applications using Play and is looking towards something larger and more complex. The suggestions for keeping your classes and application lean will surely be useful. The second chapter is available as a sample from the publisher’s website.

Disclosure: I was provided with an eBook copy of the Play Framework Cookbook by Packt Publishing for the purposes of this review.

The JSV below calculates the time at which the incoming job will reach the hard runtime limit and if it is after the configured maintenance window begin, modifies the hard runtime limit and notifies the user. This ensures that all jobs running on the cluster will end before the maintenance window begins and that the users know that.

Without further ado, here’s the code. It looks for a file named maintenance in the $SGE_ROOT/$SGE_CELL/common directory that contains the unix timestamp that the maintenance window begins. It’s executed client-side from the cluster-wide sge_request file in the same directory. The script requires the Date and Time libraries for some date math that it does. I’m not a perl programmer, so bear with me.

#!/usr/bin/perl

use strict;
use warnings;
no warnings qw/uninitialized/;

use Env qw(SGE_ROOT SGE_CELL);
use lib "$SGE_ROOT/$SGE_CELL/common/perl/lib/perl5/site_perl/5.8.8/";
use Date::Format;
use Time::Duration;
use lib "$SGE_ROOT/util/resources/jsv";
use JSV qw( :DEFAULT jsv_sub_is_param jsv_sub_add_param jsv_sub_get_param jsv_send_env jsv_log_info jsv_is_param jsv_get_param );

sub hms2s {
	my $input = shift;
	if( $input =~ m/(\d*):(\d*):(\d*)/ ) {
		my $h = $1 || 0;
		my $m = $2 || 0;
		my $s = $3 || 0;
		return $h*3600+$m*60+$s
	} elsif( $input =~ m/(\d+)/) {
		return $1;
	} else {
		return 0;
	}
}

jsv_on_start(sub {
	jsv_send_env();
});

jsv_on_verify(sub {

	my $data_file="$SGE_ROOT/$SGE_CELL/common/maintenance";
	my $success = open(DAT, $data_file); 

	if( !$success ) {
		jsv_accept("No maintenance window scheduled.");
		return;
	}

	my $maintenance_begin = <DAT>;
	close(DAT);
	if( !$maintenance_begin ) {
		jsv_accept("No maintenance window scheduled.");
		return;
	}

	# Allow a 5 minute window for jobs to die before the maintenance officially starts.
	my $delta = 300;
	my $now = time();

	if( $maintenance_begin-$delta < $now ) {
		jsv_log_info('*'x81);
		jsv_log_info('* Maintenance is currently in progress');
		jsv_log_info('* For more information, see http://blahblah');
		jsv_log_info('*'x81);
		jsv_reject();
		return;
	}

	my $requested_rt = hms2s(jsv_sub_get_param('l_hard', 'h_rt'));
	if( $now + $requested_rt > $maintenance_begin - $delta ) {
		my $time_to_run = $maintenance_begin - $delta - $now;
		jsv_sub_add_param('l_hard','h_rt',$time_to_run);
		jsv_log_info('*'x81);
		jsv_log_info('* A maintenance window is scheduled for '.time2str('%m/%d/%Y %H:%M:%S', $maintenance_begin, 'EST'));
		jsv_log_info('* Your job will be allowed to run for '.duration($time_to_run));
		jsv_log_info('* For more information, see http://blahblah');
		jsv_log_info('*'x81);
	}
	jsv_correct('Job is accepted');
	return;
}); 

jsv_main();

The Script

#!/bin/bash

# Configuration values
SGE_ROOT=/usr/share/gridengine
SGE_CELL=default
SVN_URL=file:///home/user/subversion
SVN_IMPORT_DIR=gridconfig
SVN_LOAD=/usr/share/doc/subversion-1.4.2/svn_load_dirs.pl

# The real work
TMPD=`mktemp -d`
source $SGE_ROOT/$SGE_CELL/common/settings.sh
$SGE_ROOT/util/upgrade_modules/save_sge_config.sh $TMPD
find $TMPD -name accounting -exec rm -v {} \;
sed -i "/^load_values/d" $TMPD/execution/*

$SVN_LOAD -no_user_input $SVN_URL $SVN_IMPORT_DIR $TMPD

rm -fr $TMPD

Run this regularly from cron to enable the real power that I’ll discuss below.

Explanation

Lines 4-8 define configuration variables used later on. The SGE_* variables might already be defined for your shell. If not, you know what they should be. SVN_URL is the URL of the Subversion repository you’d like to store the configuration in. It should have already been created using svnadmin create <dir>. SVN_IMPORT_DIR is the name of the directory in your repository that will store the configuration information. The real magic of this script comes from svn_load_dirs.pl, which comes with Subversion. Provide the path to it in the SVN_LOAD variable.

Lines 11-13 setup the SGE environment and dump the current configuration to a temporary directory. The save_sge_config.sh script comes with Grid Engine and is a gem. It writes the grid configuration to a directory structure. It’s worthwhile to run it on your own and browse the output.

Lines 14 and 15 do some post-processing on the dumped configuration. In particular, the accounting file is deleted because in our case, it’s huge and dynamic, so I don’t want to store multiple copies of it in the repository. Another piece of dynamic information is the load_values for each execution host. These will change each time you run the script and don’t provide any useful historical information that you can’t get in better form from Ganglia, so I get rid of them. One last piece of dynamic information is the backup_date file which contains the date and time that save_sge_config.sh. I like to keep this around because it provides some context, but you could also delete that file here.

With the configuration gathered, line 17 simply calls svn_load_dirs.pl to load it into Subversion. A commit message in the form of ‘Load /tmp/sakfjaskfj into gridconfig.’ is attached to the import.

Cool stuff

Now that all the configuration information is stored in Subversion, the world is our oyster. We can use any of the normal Subversion tools to learn more about the configuration.

Finding out what’s changed

By using svn diff, you can find out what’s changed between different dates. For example, here are the changes that I made between October 2nd and 3rd:

user@host ~$ svn diff file:///home/user/subversion/gridconfig -r {2009-10-02}:{2009-10-03}
Index: backup_date
============================================
--- backup_date (revision 177)
+++ backup_date (revision 181)
@@ -1 +1 @@
-2009-10-01_00:10:01
+2009-10-02_00:10:01
Index: usersets/superusers
============================================
--- usersets/superusers  (revision 177)
+++ usersets/superusers  (revision 181)
@@ -2,4 +2,4 @@
 type    ACL
 fshare  0
 oticket 0
-entries johnny
+entries johnny,billy

You’ll notice the backup_date is there, as I mentioned previously. Additionally, you’ll notice I added billy to the superusers userset. Good to know.

WebSVN

WebSVN is an online Subversion repository browser. Set it up correctly to point at your new gridconfig repository and you can get the same diff information as before through your browser.

Rollback your grid configuration

Did something go horribly wrong with your configuration? Rollback to a previous version using the load_sge_config.sh that also comes with Grid Engine and is located in $SGE_ROOT/util/upgrade_modules. Simply checkout the version of the repository you want to load and pass it to load_sge_config.sh. If you do this successfully, let me know. So far we haven’t had any catestrophic configuration changes.

Something else?

If you manage to do something else interesting with your versioned configuration information, leave a comment below or send me a message.

Update: Fixed a typo in the code with the $TMPD variable.

Introduction

Some of these ideas are taken from a page I started on the WordPress Codex to mull over workflow ideas.

WordPress is built to cater to the single user running a personal blog up to the global company running an entire website. In the big cases such as the latter, some controls and processes need to be in place to restrict what users can do and to assure that certain events take place in the correct order. Collectively, I’ll call these controls and processes a workflow and they’re the heart of the project I’d like to do for Google Summer of Code.

Wikipedia defines a workflow as:

A workflow is a reliably repeatable pattern of activity enabled by a systematic organization of resources, defined roles and mass, energy and information flows, into a work process that can be documented and learned.

This definition can be trimmed and refined a bit to better suit its application to WordPress, yielding:

A workflow is a reliably repeatable pattern of activity enabled by a systematic organization of resources, defined roles, and information flows, into a work process that can be documented and learned.

Use Cases

• category editors – to be published in a category, a post/page must be approved by an editor for that category
• writer->editor->designer – the writer writes text, which is edited by the editor, and then made pretty by the designer
• the big boss – everything that gets published must get the ok of the boss

System Design

I envision the workflow system being implemented as a plugin. I don’t think the number of people needing workflow controls justify putting it in the core. I do, however, think there is a critical mass of sites that would benefit from the plugin.

My initial thinking about the plugin leads to three main components: design, notification, and moderation.

Design

Under ‘design’, I categorize the definition of the workflow. This would be a ‘Settings’ page and would utilize some kind of GUI to create a funnel-like representation of the workflow. Posts/pages enter the system at the wide end of the funnel, go through a number of workflow stages, and emerge published at the other end. Designing the workflow would consist of creating vertices and edges in this tree. Vertices are moderation checkpoints that must be passed in order to move towards the root (publication). Edges are the path that documents travel along.

Another facet of the workflow design is assigning users to the vertices. A user at a vertex is the gatekeeper (moderator) of that stage, allowing documents to pass unscathed, modifying them, and/or sending them back to the previous stage. Here, I picture a drag/drop interface to assign users to vertices. Multiple users can occupy a vertex, as can one user occupy multiple vertices.

Notification

Notification is simply the idea that a user be notified when a document reaches his/her stage. This would naturally occur when a document has passed from the previous stage, but also when a document has been turned back from the next stage. Some options would be useful here to prevent a user from getting swamped with notification emails. For example, only notify a user once between logins or at most once per day.

Moderation

This is really the heart of the plugin. In order to pass through a stage, a moderator for that stage has to approve a document. The moderator can optionally modify the document before approving it. Once approved, a document either moves to the next stage or is published. A rejected document is sent back to the previous stage. Comments can be attached to the document to provide a justification for the decision.

Implementation

The design component will be implemented as a settings page that allows the user to graphically design the workflow. When saved, the workflow will be captured as a tree structure and written to the options table.

Each stage of the workflow will have a shortname (think post slug). This short name will be used to call an action when a post {enters, exits, …} that stage. For example, the action ‘workflow_editor_entered’ would be called with the post/page id when an author submits a document to the editor stage. Default action handlers would be included that would email the users in that stage. This is the notification component. These default action handlers could be disabled if a user wanted to just notice the waiting posts, or a different action handler that twittered or SMSed a user.

Forcing posts to go through the workflow would be largely accomplished with the current capabilities system. Many of the capabilities are meta-capabilities and are also sent through a user_has_cap filter before being processed. The plugin would add a handler to this filter that looked up in the workflow to see if that action was allowed. For example, in the case of an author X working on post 100 in the editing stage, the filter would receive the values (X, 100, editing). The plugin would determine if the user is able to pass posts through that stage.

There are a couple open questions about the implementation. If a published document is edited, where does it enter the workflow pipeline? Is the published version un-published? How does the revision control introduced in WP 2.6 play factor in?

Existing Functionality

WordPress introduced the concept of a ‘pending review’ state in version 2.3. This was a big step towards providing a workflow system, but it doesn’t go far enough. Indeed, it only covers the last use case above. In that case, the ‘big boss’ would need to be the only user with permission to publish. All other users would be able to create posts/pages and submit them for review. This only provides the moderation component of the design. There’s no built-in mechanism to notify the boss that a post is waiting for review. The only way to design the workflow is to designate the boss as having the publishing capability. There’s no further way to design such a trivial workflow.

The pending state doesn’t provide a workflow, but it makes it much easier to implement a full workflow system. Basically, the entire workflow exists inside of the pending state. When the original author thinks that a document is ready to be published, it is submitted for review, at which point it enters the workflow system and is given the pending state. The editors at that first level of the workflow are notified that a document is waiting for them. An editor makes any necessary changes and passes the document through to the next stage in the workflow. It is still in the pending state at this point. Until the document reaches the root of the workflow and is green-lighted to be published, it’s in the pending state.

When Mark Jaquith posted about the new pending state in June 2006, he wrote that “There are obviously a lot more ‘editorial process’ types of features that we could implement”. In addition, the comments on that post requested many of the features that this GSoC project would provide.

Previous Work

I’ve previously implemented a much simpler 1-stage workflow in a plugin called Zensor. It hasn’t been updated in quite a while due to external time constraints. This work was done for a high school in Germany where all changes to the website needed to be approved by the headmaster.

Public Comment

I solicited comments for this idea on the wp-hackers mailing list, but didn’t receive too much feedback. Maybe more comments will appear below.

Updates

4/10/2009: I added the Existing Functionality section and expanded on the Implementation details in response to comments from Jane Wells on the GSoC site.

I don’t think this has enough information. There is a review process built into WP core already… what actual functions are you proposing to built, how will they differ from current functions, and how will you approach it technically? Those are the questions that the application needs to answer.

Top 20 WordPress actions

As is clear from the graph, the most popular WordPress action is admin_menu. This makes sense, as one of the first things a plugin developer might want to do is have their plugin show up in the WordPress backend. Not surprisingly, many of the other top 20 actions are used just to get the plugin up and running: init, plugins_loaded, activate_, deactivate_, etc. The other well-represented group of actions are those dealing with the post life-cycle: save_post,publish_post, edit_post, delete_post.

In total, there were 659 unique actions used 12373 times. The graph above represents about 72% of the actions.

Top 20 WordPress filters

The shape of the top 20 filters graph is a bit different. the_content is the clear favorite of plugin developers, along with a couple other content filters: the_excerpt, the_content_rss, the_excerpt_rss. The query filters are also well-represented: posts_where, query_vars, wp, posts_join. The filter graph drops off pretty quickly so that most of the filters are used a maximum of 20 times.

In total, there were 512 unique filters used 5071 times. The above graph represents about 55% of the filters.

Method

I’ll post more about my method in the future. The numbers and graphs should be current as of SVN revision #85627 on 1/10/2009. A number of errors were encountered along the way and some corners were cut, so the numbers aren’t perfect.

Pine bark

I drove along I-85 to get to the parking area, which had only a couple cars in it at 11am on a Saturday. Indeed, I only saw a few other people on the 2.2 mile trail. The first half a mile or so, there’s still a direct line-of-sound to the highway, but after curving around the back of the mountain, it dropped away and I was left with just the quiet of the woods. There’s much more pine around here than in Roanoke, so I snapped a couple pictures of the interesting bark texture. At this point, it was overcast, but bright. As I progressed on the path, it got darker and more overcast.

Interesting fungus

The thing that most caught my eye on the first half of the trail was this bit of fungus growing on a stump. There weren’t too many mushrooms around. It’s too cold for them at this point, I suppose. While walking around a different park in the area a couple months ago, there were lots of types of mushrooms popping up all over the place. In general, there wasn’t too much green around. On the back side of the mountain, there was a certain leafy tree/bush that was doing well for itself, but that was it. This is visible in the Google Map.

Busted fridge

Also around back, the park butts up against a powerline right-of-way. I walked under and around the structures. A long time ago (months, years, decades?) someone decided to use one of the towers as his personal dumb. On that day, he had two tires, a refrigerator, and random metal bits with him. It’s all slowly becoming part of the landscape, but I think the fridge will be around for a while. From the right-of-way, there was a view of a quarry a mile away and a smaller road. It was quieter, but they weren’t the most peaceful or natural surroundings. Luckily, the trail quickly led back into the woods.

Warning sign

The next adventure on the path was this warning sign which, while intimidating, didn’t keep me from walking the forbidden 20 feet to take a peek from the overlook. It was indeed a steep drop and it wasn’t until I went around to the bottom that I saw how undercut the overlook itself was. It does look like it’ll disappear to the bottom of Panther’s Den some day. However, today wasn’t that day and I live to disobey warning signs yet another day. Further along the bottom of the cliff was a fire circle and a pile of tent stakes giving the appearance that someone had braved the risk of an overlook falling on them and spent the night there.

Bumpy tree

At this point, the trail goes along the Eno River for a short stretch. At its widest, the Eno was about 20 feet across and if I squinted, I could see it flowing. A women walked by me impatiently calling her dog, who had managed to get on the other bank. Further upstream and downstream, the park ends and private land begins, giving one the sense that they really had to shoehorn the park into this space. Reading the history of the (somewhat new) park seems to confirm this. I don’t know whether they’ve added the additional 250 acres yet, but it doesn’t seem like there’s too much room to grow from it’s current size.

Ever-present interstate

From the Eno River, I came back around to the front of the mountain and was confronted with the sound of I-85 again. Perhaps in the Summer when all of the trees have grown in, the aural assault isn’t as brutal. On this stretch, I found a couple interesting trees and started passing more people coming from the other direction. A couple steps later, I was back at the parking area and on my way home.

In general, Occoneechee Mountain was a nice, quick getaway. It’s closeness almost makes up for the intrusiveness of the interstate while at the park. Maybe I’ll try back in the Spring when there’s a bit more foliage on the trees. As for Winter, I think I’ve seen all the park has to offer.

My first thought went to Kurt Vonnegut. Not only was his writing amazing and funny and poignant, but having died recently, the media was full of articles singing his praises, which I thought I could use to pique my students’ interest. Unfortunately, in the end, I wasn’t able to find a piece of suitable length with a suitable vocabulary.

After further consideration, my mind went to David Foster Wallace. Of course, his >1000 page tome wouldn’t do, but his much shorter essay on the ethics of eating/cooking/celebrating lobster fit the bill for length and humor well.

We started reading it and soon enough we were immersed in a world of lobster fests, melted butter, … and footnotes. DFW is famous for his footnotes and although I tried to stress the importance of the footnotes and the suggestion that they should be read with the text, I never quite convinced the students to give them the respect they deserved.

On my last day of class at Siebold Gymnasium, we only had another page or two to read. It was definitely a length that we could have tackled in the hour that we had. However, the students had other plans and we ended up celebrating my departure with two cakes, one in the shape of… a lobster. Little did I know, but in the weeks that we had read the essay, the students had started to wonder if I had some kind of lobster fetish.

This is why it’s especially sad to read today that DFW hanged himself last night.

As discussed previously in this series, you’ll need to have WP-Crontrol installed and go to the Add PHP Cron Entry screen under Manage->Crontrol. For Hook Code, enter the following snippet:

    $email  = 'There are currently ';
    $email .= akismet_spam_count();
    $email .= ' spams identified by Akismet, of which ';
    $email .= akismet_spam_count('comment');
    $email .= ' are comment spams and ';
    $email .= akismet_spam_count('trackback');
    $email .= ' are trackback spams.  Akismet has caught ';
    $email .= number_format_i18n(get_option('akismet_spam_count'));
    $email .= ' spams since you first installed it.';
    wp_mail('scompt@scompt.com', 'Spam statistics', $email);

Notice that all we’re doing is tossing a string together and sending it off to the wp_mail function to be emailed. You’ll of course want to change the email address. I personally don’t need to know how much spam you’re getting… I’ve got problems of my own.

• A Shortcode for use in posts and pages
• A template function for use in templates
• A dashboard widget

An example download graph looks like this:

Download Plugin-Stats Version 1.1

PHP Cron Entry Screen

The first requirement is to install WP-Crontrol. This will add the Manage->Crontrol admin screen. Head there and you’ll get a quick insight into what’s going on behind the scenes in the WP-Cron system. We want to add a new PHP cron entry, so click on the small link next to ‘Add Cron Entry’. Now you should have something like the screen to the right.

The code that we’re going to use for the hook is below. I won’t go deep into it. Suffice to say, it looks at the date of the most recent post. If it’s more than 2 days old, it sends out an email reminder.

    global $post;
    $recent = new WP_Query("showposts=1");
    if( $recent->have_posts() ) {
        $recent->the_post();
        if(time()-strtotime($post->post_date)>60*60*24*2)
            mail('scompt@scompt.com', 'Post reminder',
                 "Hey!  You haven't posted in a while.");
    }

The only other options to set are when to run the hook and how often. After saving the hook, it should run immediately. Depending on how good you’ve been, you’ll get an email.