Cinderella languages
Tcl and Perl: Uninvited belles of the enterprise ball
August 1999
![[Next story]](/sunworldonline/icons/tb-next.gif){kind=icon}
![[Table of Contents]](/sunworldonline/icons/tb-toc.gif){kind=icon}
![[Search]](/sunworldonline/icons/tb-search.gif){kind=icon}
|
|
Cinderella languagesTcl and Perl: Uninvited belles of the enterprise ball |
August 1999
|
August 1, 1999: Cinderella languages
More stories of scripting's successful integration. (1,500 words)August 16, 1999: IBM takes lead in database competition
Cameron and Kathryn discuss the language aspects of DB2 and also highlight some interesting conferences for scripters. (1,200)
 Mail this article to a friend |
any of the most important, most mission-critical applications in
today's computing world are integrated with the help of scripting
languages. To discover this, though, you often have to go to the
working developers, not their managers or vendors.
We continue our look at scripting the enterprise this month with two tales of datastores in the news, and how scripting helps realize their potential.
In last month's column, Tom Poindexter, president of Talus Technologies, explained that database vendors have traditionally taught that C-coded embedded Structured Query Language (eSQL) is the proper and natural approach to developing database applications. As we see in each installment of Regular Expressions, though, many jobs are best suited to more expressive languages. Here's how Poindexter recalls his own experience:
Long before Tcl, Perl, Python, et al., became popular for programming and accessing databases, scripting was a longtime staple of database programmers and administrators. Scripting had not been mentioned or promoted as an "approved" access method by database vendors, but folks who had to solve problems in short order quickly found that scripting was indeed the ticket.Traditional tools have been Unix shell, plus the database vendor's SQL language processor ...
Unfortunately, database vendors never really promoted scripting as a method to quickly get results from a database ... Anything done with shell was considered slow, ugly, and a hack.
Most database vendors offer some sort of third- or fourth-generation language for database applications in addition to C interfaces. The trouble with these 4GL systems [include the fact that] proprietary languages [are] single focused and inflexible ... 4GLs are worthless for any other type of database scripting, like database admin chores.
In textbooks, the right way to develop database applications involves strict separation of the roles of designer, administrator, and coder. Specific kinds of performance are emphasized. From that perspective, attention to scripting, and especially administrative programmability, are nearly inexplicable. The realities of commercial programming are quite different, though. Poindexter recalls how, for example, he hacked together bits of Unix shell, Postscript templates, and reverse-engineered SQL to finish in a week what was scheduled for an "army" of fourth-generation language (4GL) developers and data-entry temporary workers.
Modern scripting: Perl and Tcl
This sort of success made Poindexter ripe at the beginning of this
decade for the Usenet announcements of Perl and Tcl releases. After
several early adventures, he eventually started in 1992 a long-term
project that he continues to maintain: Sybtcl. The architecture is
developed through a typical sequence: first, batched invocation of
SQL commands by an external process; then a slightly more dynamic
conversation between Tcl and Sybase's ISQL (Interactive SQL); and
finally, direct connection to Sybase's native C library.
Coincidentally, Michael Peppler, working independently, released
Sybperl at the same time.
Confirmation of the value of Sybtcl came when, according to Poindexter,
another programmer -- neither a Unix/C programmer, or X11 programmer, but with a background instead in Tandem systems with Cobol -- was hired by my client. The fun part about Tcl and scripting was seeing this programmer begin to produce workable Tcl/Tk/Sybtcl code in less than two weeks. He certainly wasn't going to produce applications with C, DB-Lib, and Motif in two weeks like he was with Tcl.
Poindexter's career continued along these lines. Scripting's ease of use and expressivity rescued project after project, as he constructed Tcl bindings to Oracle, network management software, GUI builders, and other large, obstreperous libraries. He has built or salvaged billing systems, reporting software, and much else for several very large and well-known corporations.
Tcl at Oracle
As Poindexter developed these extensions, he generally released each
one to the world at large under a "BSD-style license -- you are free
to do just about anything with the software, on the condition that
my copyright and license notice remain intact." This made for
interesting surprises about who was using the software he wrote.
One day, for example, he received a patch from a developer at Oracle. As it turned out, Oracle was developing its Oracle Enterprise Manager (OEM) -- a new GUI panel for Oracle servers -- with Tcl/Tk, Oratcl, and perhaps other Tcl extensions. Despite the evident benefit Oracle derives from his work, though, Poindexter characterizes his relationship with vendors as "nonexistent." Apart from one attempt by Oracle lawyers to have him sign a release for liability for Oratcl, the vendors largely ignore him:
In fact, I believe that database vendors "still don't get it" when it comes to supporting scripting. Unfortunately for end users and developers, database vendors still develop and promote development and administrations tools that either help to lock a user more into the vendor's own product line, or jump on the latest craze in software ...Oracle doesn't really advertise the fact that they use Tcl, and appear to only use it for what they consider a niche segment (automating administration with OEM). Sybase uses Tcl heavily for a testing framework internally (the QUASAR project which does automated regression testing on Sybase data server is over 1 million lines of Tcl), but doesn't make public mention of Tcl. Sybase did ship SybPerl briefly for Web development.
I've also had the chance to work with Oracle consultants at various clients. They were also ignorant of Tcl being used in their own products.
Despite vendors' rather puzzling attitude, Poindexter is confident of his approach:
Tcl makes me a more productive programmer. It's just the right mix between programming a custom, low-level solution versus a tool that is too high level to be much use outside a specific problem domain ... John Ousterhout [Tcl's inventor] proposed the "two-language" programming model, using a high-level language for control (Tcl), and a low-level language for specific system functions (C or Java). After using Tcl for seven-plus years, I can heartily agree with the benefits."
Poindexter also lauds the Tcl community as a strength of the language. Along with his own contributions of free software, he serves that community in such roles as conference cochair for Tcl/2k: The Seventh Usenix Tcl/Tk Conference (see Resources).
DB2 gains ground
Not all vendors appear so indifferent to productivity of front-line
programmers. In a few weeks, we'll look at IBM's DB2 relational
database management system (RDBMS). By the time you read this
column, DB2 will be available in its 6.1 release for all platforms.
This will be an ideal opportunity to review DB2's technical
features, including its connections to scripting.
Scripting in the real world: Whole-enterprise intelligence
ERP. XML. RDBMS. LDAP. Legacy integration. Magazines and
seminars agree these are among the hottest topics in contemporary
software development. If you've worked with them yourself, you've
surely experienced the large undertakings involved. Enterprise
resource planning (ERP) projects, for example, notoriously drag on
through months and years of expensive, esoteric consultation and
programming.
This isn't the case for Computerized Processes Unlimited Inc. (CPU), though. This month's example of scripting in the real world is CPU's IntelliPlant. IntelliPlant unifies dataflow throughout an entire industrial organization, so that ERP systems, plant operations, and all other information-technology functions share strategic data in realtime. It is "high tech" for many organizations to bring their process control or supervisory control and data acquisition (SCADA) variables to a state of automation where report data can be exchanged with decision support systems (DSSs) every month or week. Projects that achieve this level of integration frequently last for many months.
The man doing much of the work on IntelliPlant is CPU Senior Consultant Gerald Lester. Like Poindexter, Lester is well known in the Tcl programming community. Since the time of the very first Tcl/Tk Conference, he's contributed code and ideas. He's enthusiastic about his work with CPU and about Tcl's role in that work.
Scripting is a major contributor to IntelliPlant's capabilities. IntelliPlant's customization interface is Tcl. Extensibility with Tcl pervades IntelliPlant's capabilities, from its firewall-respectful networking and over-the-wire XML data format (programmable with Tcl), to its datastore connections (by way of RDBMS and LDAP extensions to Tcl), and its highly productive graphical configuration client (written in Tcl/Tk) that establishes translation tables between different third-party products. Specialized data transforms can be written in Tcl, C, or Java because, of course, good scripting glue plays well with other technologies.
CPU Business Unit Manager Charlie Graffeo summarizes the business significance of these technical achievements: "IntelliPlant's nonintrusive, publish-subscribe messaging middleware foundation combined with IntelliPlant's tools for rapid plant application integration significantly reduce the duration of plant-centric integration projects." From an executive perspective, much of the value of IntelliPlant lies in its ability to unify operations so that all the separate processes of a single plant can appear as one datastream to an ERP monitor. That facility of integration is exactly the openness Ousterhout preaches, and which has served Poindexter so well.
While CPU's client and partnership lists remain largely confidential, Graffeo reports that three Monsanto factories have installed IntelliPlant and that CPU's 20-person plant-integration group is growing rapidly.
Summary
Even the most factual descriptions occasionally sound like
cheerleading. Scripting is particularly prone to this because
there's such a cognitive chasm between reality and the way it's
regarded. Many conscientious management information systems
professionals sincerely believe that scripting is limited to "toy"
problems. Despite this, the most successful practitioners of
enterprise programming consistently harness the strengths of
scripting in their work -- and you can, too.
Over the next few months, we'll report several more stories from designers and coders of mission-critical applications of the largest scale. Read for yourself the lessons they've learned about scripting languages.
Resources
|
|
|
|
|
August 16: IBM takes lead in database competition
Earlier this month, we saw that Oracle and other database management
system (DBMS) vendors have tried repeatedly to differentiate
themselves with proprietary development aids. Similar strategies
are taken by Unix vendors and Microsoft; the result is that, despite
demonstrable successes with open technologies, programmers usually
end up locked into to a technically inferior approach.
One leading DBMS vendor has distinguished itself, though, in its recent exploration of reliance on standards. Intriguingly, this vendor -- IBM -- just ascended to first place among all DBMSs, according to a study issued by Dataquest earlier this year. Almost a third of all license revenues in 1998 for DBMS products running on mainframes, Unix, or Windows NT were for DB2, IBM's leading database product. Those revenues just beat out Oracle's portion of the market (IBM's 32.3 percent to Oracle's 29.3 percent).
Language dimensions of DB wars
July's 6.1 release of DB2 does plenty to make programmers' lives
easier; we particularly like its availability for every significant
flavor of Unix, including Solaris and Linux, as well as AS/400,
S/390, and NT. Complementing this attraction is IBM's approach to
development tools. Oracle, for example, generally promotes its
PL/SQL procedural language and other proprietary add-ons as
productivity aids. In contrast, IBM has thrown its considerable
weight behind Java as a safe and productive language.
DB2 articulates business and technical exposure of Java in intriguing ways. First, IBM promotes Java for scripting database operations. While Regular Expressions generally favors lighter languages for scripting, Java can also fill that role. We'll devote a future column to the implications of using a systems language in a scripting style. However, the advantages in integrated productivity over C or a proprietary fourth-generation language are evident.
Equally interesting is DB2's support for stored procedures, with Java as the language for such procedures in DB2. This brings all the advantages that an open technology should have: a well-populated developer pool, an abundance of freely available working code, and plenty of documentation and training choices. At the same time, IBM is able to contribute its own value -- a stored procedure builder tool in the DB2 product line translates between SQL queries and corresponding Java-coded stored procedures. Programmers can use the same language -- Java -- in all three tiers of the standard business architecture, with a choice of tools in each. Moreover, DB2 now supports the ISO SQL3-standard SQL/PSM (persistent stored modules) scripting language for stored procedures.
Our conclusion is that IBM is managing DB2 as a model for standards-based technology. IBM is the leader in turning the RDBMS business into a commodity market, one that it can legitimately dominate through DB2's portability and adherence to standards. With its move toward Linux, IBM freely gives away developer kits, and charges only for deployment. Herschel Harris, IBM's director of database technology, proudly reports that a third of the 50,000 downloads recorded so far are to .edu sites. It's easy to guess that the next generation of RDBMS programmers might regard DB2 as the default choice for serious work.
DB2 certainly hasn't reached perfection yet. However, programmers who want to concentrate productively on their RDBMS work without petty proprietary restrictions will certainly want to include DB2 on their short list.
Summertime fun
Bruce Eckel, coach for the Python Project Workshop |
Late summer is a traditional season for the intellectual hijinks of industry conferences. This month's LinuxWorld Expo included sessions on scripting Perl and Python, and TOOLS USA '99 (TOOLS stands for technology of object-oriented languages and systems) spotlighted several of the best thinkers on language theory. This installment of Regular Expressions appears just before O'Reilly's Open Source Software Convention, with its tracks for Perl, Python, and Tcl. There'll also be plenty of scripting conversation at the Linuxbierwanderung, Microprocessor Forum, Linux Showcase, Australian Unix Users Group, and Information Hiding gatherings.
Two events deserve special mention, though. The Python Project Workshop in early October promises to be extraordinary. Well-known author and speaker Bruce Eckel opens the doors of his own laboratory in Colorado. The occasion is a five-day, hands-on intensive program on using Python.
Why is Eckel, famous for his expertise with C++ and enthusiasm for Java, teaching Python? He doesn't need an excuse to lead hikes through the Rockies, though his advance publicity lists that as one of the collateral recreations. His actual motivation strikes some of the same chords as those pushing DB2 forward. Eckel told us, "I only care about productivity ... I've lately become interested in raw productivity, completely ignoring any issues other than 'can we get it working at all?' When you look at the miserable failure rates of programming projects, this is a really important question ... Python takes the philosophy of Java to the next step: everything is about aiding the programmer in Python."
Logo on t-shirt for 7th Annual International Python Conference, November |
On his Web site, he clearly expresses the laudably experimental plan for the week: "I think Python has the potential of being an exceptionally productive language, so much so that it would be possible to get a group of people together and create something valuable in five days. I think it would be terrific fun to try to build an open source project during a five-day workshop, so I want to try it."
Our best wishes to Eckel. Regular Expressions's aim is the same: to help programmers write applications that meet human needs. We expect the Python Project Workshop will mark a historic milestone. Its emphasis on power, simplicity, rapid development, and the Python and UML technologies is very timely.
Finally, Usenix's Second Conference on Domain-Specific Languages (DSLs) in October brings together a high-powered group comfortable with operating at an interesting level of abstraction. Here's an example: judging by headlines in the mainstream press, XML (Extended Markup Language) seems to have become a sort of financial instrument, related most closely to such other acronyms as IPO and DJIA. Invited DSL speaker Philip Wadler, though, knows far better: "XML is little more than a notation for trees and for tree grammars, a verbose variant of Lisp S-expressions coupled with a poor man's BNF." The conference program goes on to make it clear that his deep experience with language theory yields meaningful and even practical insights into XML's use and prospects.
This kind of incisive thinking is characteristic of DSL conferences. We also have a particular soft spot for Peter Lee's treatment of the topic, "Language technology for performance and security; or, making life better, not just easier." His emphasis on design principles that promote program correctness neatly complements Eckel's focus on "raw productivity."
More generally, the pervasive ideology among DSL researchers is one that's taught us valuable lessons: consider computer-human interactions as instances of linguistic encounters. Express requirements in terms of special-purpose languages that capture a domain's character. Exploit available technologies to build and manipulate these special-purpose languages.
There's still a lot to learn about how best to get applications to
do what we expect of them. Programmers live in exciting times.
Conferences such as these are invaluable for diffusing ideas and
personal contacts that are deep enough to make a difference.
|
|
Resources
|
|
|
|
![[(c) Copyright Web Publishing Inc., and IDG Communication company]](/sunworldonline/icons/b-copyright99.gif){kind=icon}
If you have technical problems with this magazine, contact webmaster@sunworld.com
URL: http://www.sunworld.com/swol-08-1999/swol-08-regex.html
Last modified: