Click on our Sponsors to help Support SunWorld
Regular Expressions by Cameron Laird

New choices for scripting

Language designers offer novel possibilities for scripting developers

February  1999
[Next story]
[Table of Contents]
Subscribe to SunWorld, it's free!

February 1, 1999: New choices for scripting
Interested in exploring new frontiers in scripting? Get a quick overview of Ficl, FIJI, REBOL, Ruby, and WebL -- the latest languages that have the scripting world buzzing. (1,700 words)
February 15, 1999: Scripting's challenges
What are scripting's trouble spots? Cameron and Kathryn examine three biggies. (1,300 words)

Mail this
article to
a friend
It's a good time to be a scripter.

The first half-year of Regular Expressions has documented the robust solutions savvy developers create with high-level "gluing" languages that support rapid development. During the last decade or more, leading languages have yielded efficient implementations and idioms for their use.

Other riches are available, though, to those ready to explore the technological "bleeding edge." In the next few months, we'll give particular attention to new programming possibilities that are just now opening up.

New scripting languages
New scripting languages are the subject of this installment. Even though each of the senior scripting languages -- Perl, Rexx, and so on -- has plenty of life left in it, several engineers have chosen to make a fresh start. Let's survey a few of their creations: Ficl, FIJI, REBOL, Ruby and WebL. In future installments, we'll report on new developments in several older languages.

Ficl embeds easily
Ficl is the Forth Inspired Command Language written in 1997 by John Sadler, a Hewlett-Packard bioscience products manager. It's now at version 2.02.

Ficl is like Tcl in that it's written in C and easily embeddable. That is, while Forth is inherently relatively easy to embed, Ficl's implementation makes it automatic wherever ANSI C is available. Its syntax is ANS Forth, so there's a large community of developers who already know how to use it. Unlike most Forth systems, though, Ficl supplies an object model, which makes it even easier to glue in pieces coded in assembly-language or C++.

Other pertinent Ficl features are its thread-safety and reentrancy, ROMability, and extensibility.

Ficl's most prominent applications are in the FreeBSD boot loader and a PalmOS debugger.

FIJI juggles Java objects
Ficl is real Forth in syntax and much of its semantics. Jack J. Woehr's ForthIsh Java Interpreter (FIJI) is a more distant relative: It's a Forth-like language interpreter/compiler whose entities are actually Java objects, so Java is fully accessible from within FIJI.

Woehr, an independent consultant and entrepreneur with enough standing in Forth to be on the Standards Committee, wanted a (scripting) shell for Java work he's planning. In November, he decided to create his own. It's now ready for release, as version 1.0 appeared on the last day of January. He expects to use it to accelerate Java development on any of several special-purpose devices such as the JavaPC, and plans to re-implement it in the native object system of at least one palmtop.

As Woehr explains, "FIJI is useful as a command shell, as a Java scripting language, as a tool for interactively exploring details of the Java environment, and, as it comes with full source, a tutorial example of a fairly complex Java program."

The 0.9 distribution includes a few example usages, including a "Hello, world" variant we trimmed slightly:

value MyFrame
value MyTextArea

: newFrame  \ -- \
   java.awt.Frame class new ( ) to MyFrame

: newTextArea \ -- \
   java.awt.TextArea class new ( ) to MyTextArea

: openWindow \ -- \
   MyFrame add ( MyTextArea java.awt.Component () , )          
   MyFrame setSize ( 480 Long>intparam , 240 Long>intparam , ) 
   MyFrame setTitle ( " Hello from FIJI!" , )                  
   MyTextArea append ( " Hello from FIJI! " , )                
   MyFrame setVisible ( true (primitive) , )                   

: closeWindow \ -- \
   MyFrame setVisible ( false (primitive) , )
   MyFrame dispose ( )

: doIt \ -- \
    newFrame newTextArea openWindow
    " To close the window, type" . cr
    "     closeWindow   "        . cr

" Type doIt to open the Hello window." . cr

Two months is a short gestation period for a full-blown language. This testifies to Woehr's experience and skill, though, more than the modesty of FIJI's scope. Early users have voiced few complaints about its functionality. The most frequent plea from the correspondence Woehr has received has been for "theory papers" that explain how he's implemented FIJI's surprisingly complete object reflection system so compactly and robustly.

REBOL with a cause
REBOL is a sophisticated newcomer. It has its own domain name (, Web site, company (REBOL Technologies), self-conscious founding evangelist (Carl Sassenrath), publicity (several magazine articles have already appeared), novel self-image ("network messaging language") and plenty of technical strengths. Sassenrath is well known for the wizardry he built into Amiga computers' multi-tasking kernel, and now he has his own company devoted to making "simple things simple to do" while offering "a smooth learning curve over a wide range of uses, from simple to sophisticated."

REBOL is reminiscent of Scheme or another LISP derivative in several ways; among them is that REBOL programs can be written to look like almost anything else. The "REBOL in ten easy steps" tutorial on the Web site closes with these examples of the potential of REBOL's dialect construct:

Here are a few examples of other dialects that might be written to solve particular problems. The first is a video studio control script:

         [queue tape to 0:36:35]
         [roll tape]
         [wait until 0:37:07]
         [wipe tape to image with effect 3]
         [key title-text]

A graphical user interface:

         button [200 100] style-4 [play cruse-narration]
         button [*   120] style-4 [start scene 27]
         window [260 100] text-80 [select window area]

A soccer scheduler:

         [team-12 plays team-32 on saturdays at 10:00]
         [team-12 must drive]
         [team-32 can play any day]
         [team-17 does not play on mondays]
These examples also hint at the uniformity of REBOL's syntax for managing both code and data. Other REBOL primitives such as objects and messages are remarkably powerful enough to code interesting processors concisely. Here's the skeleton for a filter that automates REBOL-to-REBOL communications by way of e-mail:
foreach mail-msg email-port [
    message: import email-template mail-msg
    if message/x-rebol [
	remove mail-msg
	do message/content

Ruby aims to surpass Perl
Ruby's inventor, Internet engineer Yukihiro Matsumoto, tells us that "I really love the philosophy presented by Larry Wall ...[that] `there's more than one way to do it.'" He took this so much to heart that he came up with another language to do many of the chores Perl usually shoulders.

Ruby aims to better Perl, Python, Tcl and Java. At least those are the comparisons made on its Web site. Ruby is most like a streamlined Perl -- not "too cryptic and weird," in Matsumoto's words -- with emphasis on correct object orientation. The examples available through the Ruby home page illustrate Ruby's ties to Perl. Here, for instance, is an implementation of the finger command:

#! /usr/local/bin/ruby
require "socket"
$*.push "" if $*.size == 0
  name, dom = adr.split("@")
  if dom                 
    remote =, 'finger')
    print "[#{dom}]\n"                                                  
      remote << if name then "#{name}\n" else "\n" end    
      while remote.gets                
        # you should use read instead of gets
        # to defend the buffer from overflow.
    print `finger #{name}` # I always choose easier way!

Long-time language theory fan Tim Peters detects hints of Eiffel, CLU, Modula and Tcl in this. For all its similarities to Perl, Ruby seems to attract more attention from those now using Python than those coming directly from Perl. In Japan, Ruby has more programmers than Python. Among the products written with Ruby are SGmail (GUIfied IMAP4 e-mail reader) and Meeting 2000 (Web-based scheduler).

WebL scripts Web applications
WebL is the outlier in this group. We include it as a testimony to the importance of Web applications and the market awareness of Compaq Computer Corporation, which sponsors it.

WebL automates Web operations. It's not appropriate for general-purpose computing, as are the other languages we've mentioned here. WebL exposes several interesting concepts for manipulating distributed information, including a markup algebra aware of both HTML and XML. WebL addresses many of the dynamic challenges of Web programming with its use of Luca Cardelli's service combinators.

Cardelli is the well known Digital Equipment Corporation theoretician of mobile computation, language types, and distributed processing. Service combinators naturally express "human reflexes" including retries, abandonment of stalled links, and parallelized requests.

WebL is implemented in Java, and is correspondingly portable. While its home page sometimes goes months without updates, it provides a readable introduction to the language.

New-crop choices
The scripting languages described here are evolutionary, not revolutionary. Whether that's good or bad depends on your own situation. Among these, REBOL is the one most commercially encumbered, and still in the most technical flux; it looks as though Sassenrath plans to fit at least a couple more of his innovative good ideas into REBOL before damping its rate of change. The other languages are easier to evaluate -- you'll be able to identify quickly whether they're likely to match your interests and needs. While none of these languages suggest the imaginative potency that went into the early florescence of, say, COBOL, FORTRAN, LISP, Algol, Simula, and APL, each has the potential to change the way you think about problems, and also to save you many hours on your next project.

Special thanks to Alex Ferrieux, Andreas Kupries, Steven Majewski, Tim Peters, Mark Roseman, Peter Salus, Larry Virden, and Jean-Claude Wippler for their helpful discussions on language theory.

Real-world Forth
In the first installment of each month, we spotlight a use of scripting in "the real world." The easy explanation is that it's "evidence" to shake up those who persist in believing that expressive, interpreted languages are only toys. The deeper truth is that we find it educational to see the range of problems scripting addresses.

In any case, an anonymous reader ridiculed December's entry on Python. He didn't detail his dismissal of scientific study of shock-induced plasticity and crack blunting, which is certainly part of our real world. Perhaps, though, he'll find this month's offering more suitable.

In honor of the two Forth-derived languages sketched in this installment, let's focus on "some corporate applications stories." That's the unassuming title of a page maintained by FORTH Inc. It mentions a few of the clients this computing pioneer has served during the last quarter century, and briefly describes the Forth-based applications on which their businesses rely, including: the FedEx SuperTracker, EuroPay's smart-card point-of-sale terminals, and a couple of Space Shuttle subsystems.

Robert T. Caffrey of NASA was particularly detailed in his descriptions of how FORTH Inc.'s products "dramatically help in the quick generation of error-free software." Check out the link to Forth's application stories below.



February 15: Scripting's challenges
We've figured out what's wrong with scripting.

Even after years of confronting it, the popular belief that scripting is "unsafe" or "only for toy problems" continues to surprise us. Part of the difficulty is that many of the specific details for which critics fault such languages are in fact strengths or, at worst, non-issues. (Example: Yes, there are far fewer software engineering utilities sold for Perl than for C. Any dispassionate observer, though, would agree it's a good thing that Perl semantics don't permit memory violations, and thus don't deserve all the checking that is a practical necessity when coding in C.)

However, in reviewing several collections of scripted code recently, a few consistent sore spots emerged:

In this installment, we take a look at these pitfalls.

Where the magic bullets are needed
First, let's be utterly clear: The problem is not that scripted languages can't express complex algorithms, or that scripting style seems like algorithmic sloppiness. Our impression is that coders in C are far more likely to muddle their computational constructs. In fact, many of the people thinking most seriously about algorithms prototype in languages such as Python.

Beside algorithmic expressiveness, one of the best things about scripting is that it encourages practitioners to exploit code-data dualities. This is crucial in such examples as Tk's remarkable success as a cross-platform, cross-language graphical toolkit. The same principle of communication through exchange of simple, human-readable encodings, has powered scripting's dominance in network domains like CGI and exploitation of new protocols.

Problems start to appear, though, when managing such data. Let's consider a specific example. Suppose someone comes by your desk and asks, "How many spammers have we rejected today?" With any luck, you'll know which log file records these events, and then it'll take only a couple of lines of your favorite scripting language to select the entries of interest and count them. Scripters are notorious for doing this sort of work in less than 10 minutes -- and sometimes less than 10 seconds. Remember, too, that you can be this efficient under any of the major operating systems; most scripting languages work equally well with all of them.

The temptations of fragile architecture
The follow-up is predictable: "How long would it take to create an alarm so we know in real-time when a spammer has targeted us?" The realistic answer seems to be in the hours, if not days. Here's why: It's apparently an overwhelming compulsion for scripters in this situation to proliferate a couple of log files (one for filtered events, one for filtered events that have already been reported), at least one daemon or system service, a scraper or two that periodically put themselves asleep, a watcher to ensure that the log files don't fill up a file system, heartbeats to watch the other processes, etc., etc. Something's out of balance.

What's remarkable is that these Rube Goldberg-esque contraptions actually work. They do; we've seen them behind the computing scenes of many of the biggest and best known corporations and government agencies. Scripting languages have so much going for them that they can keep even grossly imbalanced architectures like the one described above standing.

The problem seems to be this: High-level coding is so easy and even fun that programmers quickly become expert at it. They become expert at clever and supple construction of subtle algorithms. Designing robust architectures, though, is very different, and many of the same programmers never learn the necessary skills. Their instinct is to just keep programming until a problem gives up from exhaustion.

Thinking about architecture: Two books and a toolkit
Douglas Bennett's Designing Hard Software provides a thorough, theoretical framework for these architectural concerns. Even simpler steps would help many scripting programmers, though. Sriram Srinivasan uses grown-up language for the topics that appear in Advanced Perl Programming, and persistence is one of those topics he covers so well that his chapters 10 and 11 are worthwhile reading for scripters of any language.

Neither of these, however, expresses the excitement of good scripting. Persistence still feels more like an add-on than a solution. A long-term solution might be brewing, though, in the work of researchers such as Jean-Claude Wippler. His Equi4 Software offers a family of persistence products called MetaKit working across languages and platforms. He knows first-hand how application development usually progress:

"One problem is: data on disk is rigid. So when you extend/alter an application, you have to deal with getting that last data-structure/-format, converting it, and saving it again. This is tricky, and often hinders evolution. New insights on how to better perform a task get shelved because the process of change is so difficult, error-prone, risky, or just tedious. I think that's a huge problem. It is obvious to me that most designs are the result of learning what the core problems and issues are. Development is a learning process (and scripting makes it a pleasant and effective one). We need solutions which support/enhance learning, instead of today's decide/sigh/fix cycles.

Another problem, is the total mismatch between data when stored and data when in use. What is this concept of "saving" and "restoring"? Why do we not simply use data more or less? I think one reason this has not been properly solved, is that we need to do this to be able to evolve our software. Without an "on" and an "off" phase, we have no way to experiment with new approaches and alter fundamental structures. SmallTalk is an example of a purely persistent environment, yet it has to artificially incorporate a save/restore mechanism to allow changes.

Scripting, especially string-oriented languages such as Tcl, have the potential to merge stored and loaded data "formats", and get rid of this whole mess. The missing link is the ability to reorganize live data. ...

MetaKit can reorganize live data and is an attempt to provide this missing link."

You don't have to wait for a whole new language with smart persistence. You don't even have to use any of the formal persistence mechanisms, including DBI adaptors, ODBC, and so on, that are available. It'll help your designs, though, if you start thinking that you want data-on-disk and data-in-process to work better together, and give them a chance.

For the problems of process management, we can offer even fewer consolations. There's really no satisfying conceptual treatment of how to think about crons, watchdogs, heartbeats, system services, and so many other special process constructs. We'll look for chances in future columns to explain several of these ideas.

Scripting in public
Last month we noted the rising number of scripting stories in print these days. Barry Warsaw of the Corporation for National Research Initiatives insightfully observed the complementary importance of tutorials held during professional meetings, and launched a Web page, "Conferences hosting Python Tutorials", to encourage ones targeted at that particular language. Barry's right; conferences are good times to learn new languages, learn more about languages you already know, and/or teach others. Take advantage of these opportunities.

In particular, we recommend Toronto in the springtime for your next working vacation because one of us (Laird) is chairing the "Web Scripting Language Forum" at the Eighth International World Wide Web Conference. See the Resources, below, for details.

Coming up
In March, we'll look at other little-known languages that might help with your programming challenges.

Click on our Sponsors to help Support SunWorld


About the author
Cameron Laird and Kathryn Soraiz manage their own software consultancy, Network Engineered Solutions, from just outside Houston, TX. Reach Cameron at Reach Kathryn at

What did you think of this article?
-Very worth reading
-Worth reading
-Not worth reading
-Too long
-Just right
-Too short
-Too technical
-Just right
-Not technical enough

[Table of Contents]
Subscribe to SunWorld, it's free!
[Next story]
Sun's Site

[(c) Copyright  Web Publishing Inc., and IDG Communication company]

If you have technical problems with this magazine, contact

Last modified: