New choices for scripting Language designers offer novel possibilities for scripting developers

February  1999

February 1, 1999: New choices for scripting Interested in exploring new frontiers in scripting? Get a quick overview of Ficl, FIJI, REBOL, Ruby, and WebL -- the latest languages that have the scripting world buzzing. (1,700 words)

February 15, 1999: Scripting's challenges What are scripting's trouble spots? Cameron and Kathryn examine three biggies. (1,300 words)

Mail this article to a friend

t's a good time to be a scripter.

The first half-year of Regular Expressions has documented the robust solutions savvy developers create with high-level "gluing" languages that support rapid development. During the last decade or more, leading languages have yielded efficient implementations and idioms for their use.

Other riches are available, though, to those ready to explore the technological "bleeding edge." In the next few months, we'll give particular attention to new programming possibilities that are just now opening up.

New scripting languages
New scripting languages are the subject of this installment. Even though each of the senior scripting languages -- Perl, Rexx, and so on -- has plenty of life left in it, several engineers have chosen to make a fresh start. Let's survey a few of their creations: Ficl, FIJI, REBOL, Ruby and WebL. In future installments, we'll report on new developments in several older languages.

Ficl embeds easily
Ficl is the Forth Inspired Command Language written in 1997 by John Sadler, a Hewlett-Packard bioscience products manager. It's now at version 2.02.

Ficl is like Tcl in that it's written in C and easily embeddable. That is, while Forth is inherently relatively easy to embed, Ficl's implementation makes it automatic wherever ANSI C is available. Its syntax is ANS Forth, so there's a large community of developers who already know how to use it. Unlike most Forth systems, though, Ficl supplies an object model, which makes it even easier to glue in pieces coded in assembly-language or C++.

Other pertinent Ficl features are its thread-safety and reentrancy, ROMability, and extensibility.

Ficl's most prominent applications are in the FreeBSD boot loader and a PalmOS debugger.

FIJI juggles Java objects
Ficl is real Forth in syntax and much of its semantics. Jack J. Woehr's ForthIsh Java Interpreter (FIJI) is a more distant relative: It's a Forth-like language interpreter/compiler whose entities are actually Java objects, so Java is fully accessible from within FIJI.

Woehr, an independent consultant and entrepreneur with enough standing in Forth to be on the Standards Committee, wanted a (scripting) shell for Java work he's planning. In November, he decided to create his own. It's now ready for release, as version 1.0 appeared on the last day of January. He expects to use it to accelerate Java development on any of several special-purpose devices such as the JavaPC, and plans to re-implement it in the native object system of at least one palmtop.

As Woehr explains, "FIJI is useful as a command shell, as a Java scripting language, as a tool for interactively exploring details of the Java environment, and, as it comes with full source, a tutorial example of a fairly complex Java program."

The 0.9 distribution includes a few example usages, including a "Hello, world" variant we trimmed slightly:

value MyFrame
value MyTextArea

: newFrame  \ -- \
   java.awt.Frame class new ( ) to MyFrame
   ;

: newTextArea \ -- \
   java.awt.TextArea class new ( ) to MyTextArea
   ;

: openWindow \ -- \
   MyFrame add ( MyTextArea java.awt.Component () , )          
   drop                                                        
   MyFrame setSize ( 480 Long>intparam , 240 Long>intparam , ) 
   drop                                                        
   MyFrame setTitle ( " Hello from FIJI!" , )                  
   drop                                                        
   MyTextArea append ( " Hello from FIJI! " , )                
   drop                                                        
   MyFrame setVisible ( true (primitive) , )                   
   drop                                                        
   ;

: closeWindow \ -- \
   MyFrame setVisible ( false (primitive) , )
   MyFrame dispose ( )
   ;

: doIt \ -- \
    newFrame newTextArea openWindow
    cr
    " To close the window, type" . cr
    "     closeWindow   "        . cr
    ;

" Type doIt to open the Hello window." . cr

foreach mail-msg email-port [
    message: import email-template mail-msg
    if message/x-rebol [
	remove mail-msg
	do message/content
    ]
]

#! /usr/local/bin/ruby
require "socket"
                                                        
$*.push "" if $*.size == 0
 
$*.each{|adr|
  name, dom = adr.split("@")
 
  if dom                 
    remote = TCPsocket.open(dom, 'finger')
    print "[#{dom}]\n"                                                  
    begin                                                             
      remote << if name then "#{name}\n" else "\n" end    
      while remote.gets                
        # you should use read instead of gets
        # to defend the buffer from overflow.
        print                                 
      end
    ensure
      remote.close
    end               
  else          
    print `finger #{name}` # I always choose easier way!
  end                     
}

• Ficl 2.0 release notes http://www.taygeta.com/ficl.html
• Palm Debuffer http://www.pagesz.net/~sessoms/debuffer/
• FIJI ForthIsh Java Interpreter http://www.well.com/user/jax/SoftWoehr
• JavaPC Overview http://www.sun.com/software/javapc
• REBOL http://www.rebol.com/
• Ruby home page http://www.netlab.co.jp/ruby/
• SGmail home page http://www.sgmail.org/
• WebL home page http://www.research.digital.com/SRC/WebL/
• Forth Inc.'s "some corporate applications stories" http://www.forth.com/Content/Stories/Stories.htm
• Full listing of past Regular Expressions columns http://www.sunworld.com/sunworldonline/common/swol-backissues-columns.html#regex
• The SunWorld Topical Index -- a comprehensive listing of all SunWorld articles by subject http://www.sunworld.com/common/swol-siteindex.html
• sunWHERE -- launchpad to hundreds of online resources for Sun users http://www.sunworld.com/sunwhere.html
• Back issues of SunWorld http://www.sunworld.com/common/swol-backissues.html

February 15: Scripting's challenges
We've figured out what's wrong with scripting.

Even after years of confronting it, the popular belief that scripting is "unsafe" or "only for toy problems" continues to surprise us. Part of the difficulty is that many of the specific details for which critics fault such languages are in fact strengths or, at worst, non-issues. (Example: Yes, there are far fewer software engineering utilities sold for Perl than for C. Any dispassionate observer, though, would agree it's a good thing that Perl semantics don't permit memory violations, and thus don't deserve all the checking that is a practical necessity when coding in C.)

However, in reviewing several collections of scripted code recently, a few consistent sore spots emerged:

• Scripters have the same problems as everyone else in the industry when reading specifications and deciding how to handle error conditions
• Data persistence becomes fragile in scripted applications with frightening speed
• Process management is often a mess

In this installment, we take a look at these pitfalls.

Where the magic bullets are needed
First, let's be utterly clear: The problem is not that scripted languages can't express complex algorithms, or that scripting style seems like algorithmic sloppiness. Our impression is that coders in C are far more likely to muddle their computational constructs. In fact, many of the people thinking most seriously about algorithms prototype in languages such as Python.

Beside algorithmic expressiveness, one of the best things about scripting is that it encourages practitioners to exploit code-data dualities. This is crucial in such examples as Tk's remarkable success as a cross-platform, cross-language graphical toolkit. The same principle of communication through exchange of simple, human-readable encodings, has powered scripting's dominance in network domains like CGI and exploitation of new protocols.

Problems start to appear, though, when managing such data. Let's consider a specific example. Suppose someone comes by your desk and asks, "How many spammers have we rejected today?" With any luck, you'll know which log file records these events, and then it'll take only a couple of lines of your favorite scripting language to select the entries of interest and count them. Scripters are notorious for doing this sort of work in less than 10 minutes -- and sometimes less than 10 seconds. Remember, too, that you can be this efficient under any of the major operating systems; most scripting languages work equally well with all of them.

The temptations of fragile architecture
The follow-up is predictable: "How long would it take to create an alarm so we know in real-time when a spammer has targeted us?" The realistic answer seems to be in the hours, if not days. Here's why: It's apparently an overwhelming compulsion for scripters in this situation to proliferate a couple of log files (one for filtered events, one for filtered events that have already been reported), at least one daemon or system service, a scraper or two that periodically put themselves asleep, a watcher to ensure that the log files don't fill up a file system, heartbeats to watch the other processes, etc., etc. Something's out of balance.

What's remarkable is that these Rube Goldberg-esque contraptions actually work. They do; we've seen them behind the computing scenes of many of the biggest and best known corporations and government agencies. Scripting languages have so much going for them that they can keep even grossly imbalanced architectures like the one described above standing.

The problem seems to be this: High-level coding is so easy and even fun that programmers quickly become expert at it. They become expert at clever and supple construction of subtle algorithms. Designing robust architectures, though, is very different, and many of the same programmers never learn the necessary skills. Their instinct is to just keep programming until a problem gives up from exhaustion.

Thinking about architecture: Two books and a toolkit
Douglas Bennett's Designing Hard Software provides a thorough, theoretical framework for these architectural concerns. Even simpler steps would help many scripting programmers, though. Sriram Srinivasan uses grown-up language for the topics that appear in Advanced Perl Programming, and persistence is one of those topics he covers so well that his chapters 10 and 11 are worthwhile reading for scripters of any language.

Neither of these, however, expresses the excitement of good scripting. Persistence still feels more like an add-on than a solution. A long-term solution might be brewing, though, in the work of researchers such as Jean-Claude Wippler. His Equi4 Software offers a family of persistence products called MetaKit working across languages and platforms. He knows first-hand how application development usually progress:

"One problem is: data on disk is rigid. So when you extend/alter an application, you have to deal with getting that last data-structure/-format, converting it, and saving it again. This is tricky, and often hinders evolution. New insights on how to better perform a task get shelved because the process of change is so difficult, error-prone, risky, or just tedious. I think that's a huge problem. It is obvious to me that most designs are the result of learning what the core problems and issues are. Development is a learning process (and scripting makes it a pleasant and effective one). We need solutions which support/enhance learning, instead of today's decide/sigh/fix cycles.

Another problem, is the total mismatch between data when stored and data when in use. What is this concept of "saving" and "restoring"? Why do we not simply use data more or less? I think one reason this has not been properly solved, is that we need to do this to be able to evolve our software. Without an "on" and an "off" phase, we have no way to experiment with new approaches and alter fundamental structures. SmallTalk is an example of a purely persistent environment, yet it has to artificially incorporate a save/restore mechanism to allow changes.

Scripting, especially string-oriented languages such as Tcl, have the potential to merge stored and loaded data "formats", and get rid of this whole mess. The missing link is the ability to reorganize live data. ...

MetaKit can reorganize live data and is an attempt to provide this missing link."

You don't have to wait for a whole new language with smart persistence. You don't even have to use any of the formal persistence mechanisms, including DBI adaptors, ODBC, and so on, that are available. It'll help your designs, though, if you start thinking that you want data-on-disk and data-in-process to work better together, and give them a chance.

For the problems of process management, we can offer even fewer consolations. There's really no satisfying conceptual treatment of how to think about crons, watchdogs, heartbeats, system services, and so many other special process constructs. We'll look for chances in future columns to explain several of these ideas.

Scripting in public
Last month we noted the rising number of scripting stories in print these days. Barry Warsaw of the Corporation for National Research Initiatives insightfully observed the complementary importance of tutorials held during professional meetings, and launched a Web page, "Conferences hosting Python Tutorials", to encourage ones targeted at that particular language. Barry's right; conferences are good times to learn new languages, learn more about languages you already know, and/or teach others. Take advantage of these opportunities.

In particular, we recommend Toronto in the springtime for your next working vacation because one of us (Laird) is chairing the "Web Scripting Language Forum" at the Eighth International World Wide Web Conference. See the Resources, below, for details.

Coming up
In March, we'll look at other little-known languages that might help with your programming challenges.

Resources

• Douglas Bennett's Designing Hard Software http://www.manning.com/Bennett/index.html
• Sriram Srinivasan's Advanced Perl Programming http://www.amazon.com/exec/obidos/ISBN=1565922204/sunworldonlineA/
• MetaKit for Python (despite the title, this is really a philosophical analysis of data design issues) http://www.equi4.com/jcw/mkpython.html
• Conferences hosting Python Tutorials http://www.python.org/psa/teachers.html
• Eighth International World Wide Web Conference http://www8.org/
• Developers' Day at Eighth International World Wide Web Conference http://www8.org/developers.html
• Full listing of past Regular Expressions columns http://www.sunworld.com/sunworldonline/common/swol-backissues-columns.html#regex
• The SunWorld Topical Index -- a comprehensive listing of all SunWorld articles by subject http://www.sunworld.com/common/swol-siteindex.html
• sunWHERE -- launchpad to hundreds of online resources for Sun users http://www.sunworld.com/sunwhere.html
• Back issues of SunWorld http://www.sunworld.com/common/swol-backissues.html

If you have technical problems with this magazine, contact webmaster@sunworld.com

URL: http://www.sunworld.com/swol-02-1999/swol-02-regex.html
Last modified: