Planet Stratego

Eelco Visser: Talk about the Researchr Digital Library

Mon, 15 Mar 2010 11:04:07 +0000

At the recent IFIP WG 2.11 meeting in St Andrews, I gave a talk about researchr, the digital library application I have been working on in the past year. Here is the recording of that talk, which gives an introduction to the features of the tool.

Download: 768x576

Eelco Visser: Spoofax: The Language Workbench

Sun, 14 Mar 2010 22:20:38 +0000

At the recent IFIP WG 2.11 meeting in St Andrews, I gave a demonstration talk about the Spoofax language workbench. I made a recording. The talk is, eh, a little, eh, rough, but should be useful to give an impression of the basic features of Spoofax.

Download: 512x384, 768x576, 1024x768

Eelco Visser: The Unbearable Fragility of Software Configuration

Sat, 13 Mar 2010 17:44:13 +0000

Issue 52 of the Spoofax project in YellowGrass is a perfect illustration of the fragility of software configuration; the presence or absence of a line break makes the difference between a working and a failing system. What is worse is that the error is very hard to detect and requires a hunch from an expert. If the eclipse.ini file would have a proper syntax and static semantic constraints, the error could be caught right on load time, or preferably when editing the file in the first place. Configuration files are models in a domain-specific language that should be treated just like the classes of a Java program.

Eelco Visser: Publication Clouds

Sat, 27 Feb 2010 21:01:00 +0000

Previously researchr supported 'tag cloud' views for bibliographies providing a quick overview of the important and lesser important topics in a set of publications collected in a bibliography. Similarly, an 'author cloud' gives an overview of the authors of a bibliograpy, emphasizing authors with many papers.

Now clouds are not just provided for tags and authors, but also for year, publication type, and publication venue. Furthermore, cloud views are provided for all 'publication lists' in researchr: author profile, alias page (publications authored by author with same name), bibliography, tag, and also for the publications of the editions of a conference. The latter provides a nice overview of the community and topics of a conference series.

Eelco Visser: Coloring Language Aspects

Sat, 20 Feb 2010 20:17:04 +0000

I was looking for a way to illustrate the integration of different language aspects in WebDSL. After tedious experimentation with graphical editors to highlight the parts of a program, it dawned on me that the syntax coloring configuration of the Spoofax editor for WebDSL is the perfect tool for the job. Spoofax provides DSL for syntactic editor services, such as syntax coloring. The Spoofax generator, generates default configurations for the various services. For instance, for syntax coloring the following settings (left) are generated. To illustrate the role of data models, user interface templates, and expressions/actions I added the following customization of the default (right):

The result can be seen in the following fragments consisting of an entity declaration (data model) and a page definition with form and action:

I find it hard to choose good colors (e.g. yellow provides bad contrast on white), but this example nicely show how Spoofax allows assigning colors to syntactic categories, not just tokens.

Eelco Visser: WebDSL Applications

Fri, 19 Feb 2010 09:48:39 +0000

Eelco Visser: Building IDEs for Domain-Specific Languages with Spoofax/IMP

Fri, 12 Feb 2010 15:00:48 +0000

At the IFIP WG 2.11 meeting in March I'll give a talk on Spoofax/IMP; I'll try to record the talk and post it here.

Domain-specific languages are a key component of program generation. While we have ample experience building code generators and compilers, modern software developers expect integrated development environments such as Eclipse and Visual Studio to boost their productivity. To achieve the productivity gain promised by domain-specific languages, it is required that they come with strong IDE support. Since the production of DSLs cannot afford the effort that is put into IDEs for general-purpose languages, better tools are needed.

In this talk I present Spoofax/IMP, an Eclipse plugin for creating Eclipse plugins for custom (domain-specific) languages. Spoofax integrates several domain-specific languages for language definition. SDF supports modular, declarative definition of syntax with arbitrary context-free grammars integrating lexical syntax. A new implementation of the SGLR parser for SDF supports sophisticated error recovery. Stratego supports model transformation, code generation, static analysis, and refactoring with rewrite rules and programmable strategies. Editor service DSLs support declaration and customization of syntactic editor services such as syntax highlighting, folding, outline views, and semantic editor services such as error checking, cross references, content completion, and refactorings.

I'll introduce Spoofax by discussing a definition for a subset of the WebDSL web programming language. The talk will be partly slides and partly demonstration.

Eelco Visser: Composing Domain-Specific Languages

Fri, 13 Nov 2009 00:06:00 +0000

Last week I was in Bergen (Norway) for the PhD thesis defense of Anya Bagge and the opening of the Bergen Language Design Laboratory. For the occasion of the opening a few people where asked to talk about the history and future of programming language design. Don Sannella talked about the failure of the Extended ML formal methods project. Bjarne Stroustrup talked about social issues of language design such as the (enormous) impact of C++ and the hassles of language standardization (and took my picture during the BLDL dinner). Paul Klint gave a flavour of his new Rascal meta-programming language. Horacio Bouzas and Carl Seger talked about the role of languages in oil exploration and chip design.

As a long time collaborator of the Bergen group, I also was asked to shine my light on language design. Instead of giving a straight up talk about Stratego or WebDSL, I decided to try to give a somewhat more general slant on the currently dominant theme of our research at TU Delft: composition of domain-specific languages. Today I extended the talk for delivery at the TU Eindhoven CS colloquium. I like the style of the talk and it gave rise to a good discussion about the trade-offs between internal and external DSLs afterwards. This is also the start of the development of a lectures series for my MDSD course in the Spring semester. Since there was an expression of interest, I've uploaded my slides. They're presentation-zenish, so there's not a whole lot of explanatory text and may not be useful without me talking about them. Feedback is welcome.

Composing Domain-Specific Languages

View more documents from eelcovisser.

Abstract The history of programming languages shows a progressive development from low-level programming languages close to the machine, to high-level languages close to the problems being solved with software. Domain-specific languages take this a step further than general purpose programming languages by making assumptions about the class of applications for which the language is intended. Complete applications typically require programs in multiple (technical) domains, which can be catered for by separate domain-specific languages. While such separation of concerns is beneficial for domain expressivity, it often leads to loose coupling and lack of static verification. Hence, the design of individual DSLs needs to be complemented with their linguistic integration.

In this talk, I illustrate these ideas with the design of WebDSL, a domain-specific language for data centric web applications. WebDSL linguistically integrates the definition of data models, user interfaces, actions, access control rules, data validation rules, styling rules, and workflow definitions. While maintaining separation between these concerns through specialized sub-languages, linguistic integration ensures static consistency checking and correct code generation. The language allows developers to concentrate on the essential design of web applications, abstracting from accidental complexity, such as the details of data persistence. The combination of high-level and low-level constructs ensures high expressivity, while supporting customization to application requirements.

Eelco Visser: An Eclipse Editor for WebDSL

Fri, 23 Oct 2009 23:16:00 +0000

This afternoon I created an Eclipse editor for WebDSL from just its syntax definition using Spoofax/IMP in a matter of minutes. The editor provides syntactic editor services such as syntax highlighting, folding, outlining, error recovery; all out of the box. Here's a screenshot with an impression.

Note how my decision to include 'sections' in WebDSL turns out to play very nice with code folding. Spoofax/IMP also supports semantic error services such as displaying type check errors and cross-references, but that requires the WebDSL compiler to be refactored from a batch compiler to an incremental compiler.

Eelco Visser: Take the red PIL (or choose any other color you like)

Sun, 18 Oct 2009 20:35:00 +0000

The Platform Independent Language PIL now has its own website with a manual for the language and downloads of the compiler. You can use PIL as target for your domain-specific language and have it work on all PIL supported platforms. Currently only Java and Python are supported, but creation of PIL back-ends is cheap, probably much cheaper than making dedicated back-ends for your own DSL. You can get involved and contribute a new back-end for a new platform. To start we would like to have back-ends for PHP, C#, C, and Objective C, but any others are welcome too.

Eelco Visser: The Big Scheme of Things: Episode 2 -- Templates

Sat, 10 Oct 2009 13:53:23 +0000

While attending GPCE, SLE, and MoDELS in Denver, I recorded episode 2 of The Big Scheme of Things screencast series. Using a simple to-do application as example application, I illustrate various aspects of the use of WebDSL, a domain-specific language for web applications.

In this episode, I show the use of templates to divide a page definition into smaller fragments, the definition of action in templates, and the use of icons as actions.

Download: 512x384, 768x576, 1024x768

[This is as much an experiment in screencasting, as it is a WebDSL tutorial. Feedback is welcome. Thanks for the feedback on the previous episode. For this recording I have used a proper microphone (Rode Podcaster) instead of the built-in microphone of my MacBook; sound quality should be better. I still have to look into other formats than Quicktime; suggestions for useful formats are welcome. An iTunes feed is in the planning as well. There was a request for a start and end tune; apart from the requirement that such a tune should be copyright free, music is not my strong point; suggestions are welcome.]
Continue reading "The Big Scheme of Things: Episode 2 -- Templates"

Eelco Visser: Integration of Data Validation and User Interface Concerns in a DSL for Web Applications

Mon, 05 Oct 2009 09:34:00 +0000

One of the tricky aspects of getting a web application right, is data validation. That is, checking that inputs to forms is correct in some sense that is relevant for the application. For example, the input for a year field should be an integer that represents a year that sensible in the context of the application. Data validation actually goes beyond checking validity of single input fields in a form and includes invariants over the data model that should be maintained, and assertions that should hold at some point in processing input data.

The problem of data validation is two-fold. First, we'd like a declarative way to specify validation rules. Second, the checking of these rules should be integrated in the rest of the application. In particular, injecting error messages in the UI. In a paper that Danny Groenewegen will present this afternoon at the Software Language Engineering (SLE 2009) we describe how we have extended WebDSL with declarative data validation rules that are checked automatically and for which error messages are injected in the UI.

Danny M. Groenewegen, Eelco Visser. Integration of Data Validation and User Interface Concerns in a DSL for Web Applications. In Mark G. J. van den Brand, Jeff Gray, editors, Software Language Engineering, Second International Conference, SLE 2009, Denver, USA, October, 2009. Revised Selected Papers. Lecture Notes in Computer Science, Springer, 2009. [researchr]

Abstract: Data validation rules constitute the constraints that data input and processing must adhere to in addition to the structural constraints imposed by a data model. Web modeling tools do not address data validation concerns explicitly, hampering full code generation and model expressivity. Web application frameworks do not offer a consistent interface for data validation. In this paper, we present a solution for the integration of declarative data validation rules with user interface models in the domain of web applications, unifying syntax, mechanisms for error handling, and semantics of validation checks, and covering value well-formedness, data invariants, input assertions, and action assertions. We have implemented the approach in WebDSL, a domain-specific language for the defi- nition of web applications.

Eelco Visser: Quine In Stratego

Sun, 04 Oct 2009 21:18:00 +0000

Tonight I was reminiscing with Tijs van der Storm about conferences of the past, in particular the day that I wrote a Quine in Stratego at OOPSLA 2004. I reported about that in my pre-blog and at the Stratego/XT site, but not in this blog. Since it was a fun example, I thought it deserved a place on my official blog. Here goes:

After GPCE/OOPSLA in Vancouver Tijs van der Storm challenged me to write a Stratego program that prints its own source. So I set to work, with the following result. I'll make it educational and reconstruct the design process.

The basic idea

The core of a self printing program is that it should contain its own source and duplicate that. The following program implements this idea, but with 'prefix' and 'suffix' instead of the

----------------------------------
module quine 
imports lib
strategies
  main =
    !["prefix", "suffix"]
    ; \ [x, y] -> [x, x, y, y] \ 
    ; concat-strings
    ; (stdout, [])
    ;  0
-----------------------------------

Compiling and running this program produces:

prefixprefixsuffixsuffix

Quoting the source

Next we replace 'prefix' and 'suffix' by the prefix and suffix of the program with respect to the list. Note that we don't bother with the layout of the quoted fragment. Note that the backslashes of the anonymous rewrite rule need to be escaped in the string.

------------------------------------------------------------------------------------------
module quine 
imports lib
strategies
  main =
    !["module quine imports lib strategies main = ![", 
      "]; \\ [x, y] -> [x, x, y, y] \\; concat-strings; (stdout, []);  0"]
    ; \ [x, y] -> [x, x, y, y] \ 
    ; concat-strings
    ; (stdout, [])
    ;  0
------------------------------------------------------------------------------------------

The output of this program is

------------------------------------------------------------------
module quine imports lib strategies main = ![module quine 
imports lib strategies main = ![]; \ [x, y] -> [x, x, y, y] \; 
concat-strings; (stdout, []);  0]; \ [x, y] -> 
[x, x, y, y] \; concat-strings; (stdout, []);  0
------------------------------------------------------------------

which is starting to look good, but not quite there, since it doesn't compile.

Getting the quotes right

What we have to do now is introduce quotes in the printed program, such that the pieces of code in the list are actually parsed as strings.

------------------------------------------------------------------------------------------
module quine 
imports lib
strategies
  main =
    !["module quine imports lib strategies main = ![\"", 
      "\"]; \\ [x, y] -> [x, x, y, y] \\; concat-strings; (stdout, []);  0"]
    ; \ [x, y] -> [x, x, "\",\"", y, y] \ 
    ; concat-strings
    ; (stdout, [])
    ;  0
------------------------------------------------------------------------------------------

The output of this program is

------------------------------------------------------------------
module quine imports lib strategies main = !["module quine 
imports lib strategies main = !["",""]; \ [x, y] -> [x, x, y, y] \; 
concat-strings; (stdout, []);  0"]; \ [x, y] -> 
[x, x, y, y] \; concat-strings; (stdout, []);  0
------------------------------------------------------------------

which still does not compile because of the doublequotes embedded in the string.

Getting the quotes more right

We need to escape the embedded strings. This can be achieved easily by using the escape strategy from the library, which escapes doublequotes and backslashes.

------------------------------------------------------------------------------------------
module quine 
imports lib
strategies
  main =
    !["module quine imports lib strategies main = ![\"", 
      "\"]; \\ [x, y] -> [x, x, \"\\\",\\\"\", y, y] \\; concat-strings; (stdout, []);  0"]
    ; \ [x, y] -> [x, x, "\",\"", y, y] \ 
    ; concat-strings
    ; (stdout, [])
    ;  0
------------------------------------------------------------------------------------------

This produces (without the newlines)

-----------------------------------------------------------------------------
module quine imports lib strategies main = !["module quine imports 
lib strategies main = ![\"","\"]; \\ [x, y] -> [x, x,
 \"\\\",\\\"\", y, y] \\; concat-strings; (stdout
, []);  0"]; \ [x, y] -> [x, x, "\",\"", y, y] \; 
concat-strings; (stdout, []);  0
-----------------------------------------------------------------------------

Bootstrapping

Now the last program is not exactly the same as its predecessor, but if we compile and run it, it produces its own source literally.

Eelco Visser: Abstractions for Mobile Applications (Postdoc Position)

Sat, 03 Oct 2009 01:06:00 +0000

(Update: the position has been filled)

Yesterday, I had an interesting phone discussion with researchers from T-Mobile, which strengthened me in my plan to make the next MoDSE case study about abstractions for mobile applications, and to make that the focus for the open postdoc position in the project.

The Software Engineering Research Group of Delft University of Technology has a vacancy for a postdoc position in the “Delft University of Technology has a vacancy for a postdoc position in the “Delft University of Technology has a vacancy for a postdoc position in the “Delft University of Technology has a vacancy for a postdoc position in the “Model-Driven Software Evolution (MoDSE)” project funded by the Jacquard Software Engineering Research program of NWO.

Job description: The candidate will lead the next MoDSE case study in domain-specific language engineering into the domain of mobile applications. The case study will consist of finding high-level, platform independent abstractions for mobile applications and creating a collection of domain-specific languages covering the domain, building on the experience and tools from the WebDSL case study.

Requirements: The candidate should have a PhD degree in Computer Science with a strong background in software engineering, specifically software language engineering, model-driven engineering, and/or programming languages. Ideally the candidate has experience with language design and implementation.

Contract: 2 years, 8 months Salary: Maximum of €3755 per month gross depending on past experience. TU Delft offers an attractive benefits package, including a flexible work week, free high-speed Internet access from home, and the option of assembling a customized compensation and benefits package (the 'IKA'). Salary and benefits are in accordance with the Collective Labour Agreement for Dutch Universities.

How to apply: Send applications consisting of an application letter and a detailed CV in PDF by email to Eelco Visser

Eelco Visser: The Big Scheme of Things: Episode 1

Mon, 21 Sep 2009 08:14:00 +0000

WebDSL is a domain-specific language for building web applications. WebDSL avoids the boilerplate that is common in other web technologies by providing tuned domain-specific notations for the concerns of web development. Furthermore, the consistency of these concerns is verified at compile-time. WebDSL is available from webdsl.org. There you can also find instructions for installing the compiler and the other components that you need to run WebDSL applications. In this series of screencasts I will illustrate the use of WebDSL, by building a web application from scratch. The application we are going to build is called 'The Big Scheme of Things', a to-do list application. In this first episode I keep it really simple and create a single page app that allows us to add and remove tasks. First I show how to get an even simpler "Hello web!" app running.

Download: 512x384, 768x576, 1024x768

(This is as much an experiment in screencasting, as it is a WebDSL tutorial. Feedback is welcome.)
Continue reading "The Big Scheme of Things: Episode 1 "

Eelco Visser: PIL: A Platform Independent Language for Retargetable DSLs

Wed, 09 Sep 2009 18:25:06 +0000

WebDSL is not the first domain-specific language I have designed and implemented. With each language the question is which platform to develop the language for. That is, which language should the compiler generate or should the interpreter be written in and which libraries to use with that language as run-time system. After the first generation of the ASF+SDF MetaEnvironment that was written in LeLisp, we decided in the 1990's that the second generation should target C as a fairly portable and ubiquitous 'assembly language'. Thus, the prototype implementation of the SGLR parser for SDF was written in C. For my next project, the transformation language Stratego, I also chose C as platform with the ATerm library as run-time system providing term representation and garbage collection. When I started work on WebDSL in 2007, Java and the JVM provided the best platform for web application development. Thus, Java is the target language generated from WebDSL applications. (The set frameworks used to create the run-time system is subject to change.)

SDF, Stratego, and WebDSL transcend the platforms they are implemented on. Each of the languages provides abstractions that are independent of the implementation platform and could easily be used on other platforms as well. However, creating back-ends for all sorts of platforms is a tedious and laborious undertaking, and then maintaining the consistency of these back-ends is worse. This problem is not unique to languages that I come up with, but is rather a universal problem for domain-specific languages.

To address this problem as part of our work on a general approach to the development and evolution of domain-specific languages, we are developing PIL, a platform independent language that abstracts from the differences between object-oriented languages. Back-ends that translate PIL to languages such as Java and Python are much smaller than the back-end of the WebDSL compiler, which encapsulates a lot of domain-specific implementation knowledge about the domain of web applications. With PIL we hope that we can not only develop domain-specific platform-independent abstractions, but also provide portable implementations for a variety of platforms. A paper about PIL has been accepted for the second conference on Software Language Engineering:

Zef Hemel, Eelco Visser. PIL: A Platform Independent Language for Retargetable DSLs. In Mark G. J. van den Brand, Jeff Gray, editors, Software Language Engineering, Second International Conference, SLE 2009, Denver, USA, October, 2009. Lecture Notes in Computer Science, Springer, 2009. (to appear) [researchr]

Abstract: Intermediate languages are used in compiler construction to simplify retargeting compilers to multiple machine architectures. In the implementation of domain-specific languages (DSLs), compilers typically generate high-level source code, rather than low-level machine instructions. DSL compilers target a software platform, i.e. a programming language with a set of libraries, deployable on one or more operating systems. DSLs enable targeting multiple software platforms if its abstractions are platform independent. While transformations from DSL to each targeted platform are often conceptually very similar, there is little reuse between transformations due to syntactic and API differences of the target platforms, making supporting multiple platforms expensive. In this paper, we discuss the design and implementation of PIL, a Platform Independent Language, an intermediate language providing a layer of abstraction between DSL and target platform code, abstracting from syntactic and API differences between platforms, thereby removing the need for platform-specific transformations. We discuss the use of PIL in an implemementation of WebDSL, a DSL for building web applications.

Eelco Visser: Weaving Web Applications with WebDSL

Thu, 27 Aug 2009 07:59:08 +0000

At OOPSLA this year, Danny Groenewegen and I will be giving demonstrations of WebDSL. There will probably be two slots scheduled in the program for the demonstration, and we'll be happy to give unscheduled demonstrations throughout the conference. We wrote a little paper for the proceedings that gives an impression of what we will be showing.

Abstract: WebDSL is a domain-specific language for the development of web applications that integrates data-models, userinterface models, actions, validation, access control, and workflow. The compiler verifies the consistency of applications and generates complete implementations in Java or Python. We illustrate the key concepts of the language with a small web application.

Eelco Visser: Stratego/XT 0.17 Released

Wed, 08 Jul 2009 09:35:20 +0000

Stratego/XT 0.17 is now available from

http://www.stratego-language.org/Stratego/StrategoRelease017

Stratego/XT 0.17 introduces major improvements across the board, including language additions, a new compiler library, numerous improvements to the compiler, significant changes to the library handling, new libraries for parsing, pretty printing and term validation, 64-bit support, stack traces and more.

For this release, over 200 outstanding issues have been addressed, much thanks to the efforts of external bug reporters and contributors.

The manual has been updated to reflect the changes made to the language and libraries, and the new libraries come with up-to-date source code documentation.

This release of Stratego/XT has taken a while to be promoted to a major release. After the release of 0.16 we had decided that 'major releases' were an archaic notion, since most users subscribe to the continuous integration builds produced by the buildfarm. However, major releases do provide stable baselines and a better indication of progress of the project. To improve clarity and stability we are changing the release policy. Our goal is to release a new major version of Stratego/XT every few months.

Eric Bouwers: Migration to the nine-headed monster

Tue, 07 Jul 2009 22:19:47 +0000

After some recent activity on the psat-dev mailinglist I became aware of the (lack of) available builds for php-sat. Even though the development speed is not what I would want it to be (so much fun things to do, so little hours in a day!) I still believe it is important to release early and often.

Fortunately, Eelco Dolstra had some time to migrate php-front, php-sat and php-tools to Hydra, the new Nix-based continuous build system. After some tweaking we now again have access to unstable build for all PSAT-projects. Go Hydra!

Eelco Visser: Preventing injection attacks with syntax embeddings

Wed, 10 Jun 2009 08:38:03 +0000

Our paper on StringBorg is being published by Science of Computer programming:

M. Bravenboer, E. Dolstra, and E. Visser. Preventing Injection Attacks with Syntax Embeddings. A Host and Guest Language Independent Approach. Science of Computer Programming, 2009.

StringBorg is a technique for embedding 'string' languages in general purpose languages in a safe way, to avoid injection attacks. The paradigmatic example is the embedding of SQL queries, which typically is done using string literals as in the following example:

  String userName = getParam("userName");
  String password = getParam("password");
  String query = "SELECT id FROM users "
                      + "WHERE name = ’" + userName + "’ "
                      + "AND password = ’" + password + "’";
   if (executeQuery(query).size() == 0)
      throw new Exception("bad user/password");

In these approaches it is very easy to forget to escape SQL meta characters in the values obtained from the client. This opens the door to an attack through a query that escapes from the programmed query. StringBorg prevents such attacks by syntactically embedding the query language in the host language. For example, the query above can then be written as follows:

  SQL q = | SELECT id FROM users
                    WHERE name = ${userName} AND password = ${password} |>;
  if (executeQuery(q.toString()).size() == 0) ...

Now, the syntax of the query is checked statically. But more importantly, at run-time the query is constructed by a query API that ensures that the query constructed has the same syntactic structure as the one defined by the programmer. Furthermore, it enforces escaping meta-characters in values spliced into the query, thus guaranteeing that no injection attacks can occur.

The paper does not just provide a solution for embedding SQL in Java, but offers a generic approach for embedding any guest language in any host language with little more effort than providing syntax definitions for host and guest language.

Abstract: Software written in one language often needs to construct sentences in another language, such as SQL queries, XML output, or shell command invocations. This is almost always done using unhygienic string manipulation, the concatenation of constants and client-supplied strings. A client can then supply specially crafted input that causes the constructed sentence to be interpreted in an unintended way, leading to an injection attack. We describe a more natural style of programming that yields code that is impervious to injections by construction. Our approach embeds the grammars of the guest languages (e.g. SQL) into that of the host language (e.g. Java) and automatically generates code that maps the embedded language to constructs in the host language that reconstruct the embedded sentences, adding escaping functions where appropriate. This approach is generic, meaning that it can be applied with relative ease to any combination of context-free host and guest languages.

Eelco Visser: Stratego/XT Machine for Code Generation 2009

Sat, 30 May 2009 17:41:44 +0000

For the participants of our hands-on tutorial on Creating DSLs with Stratego/XT at the Code Generation 2009 conference, we (i.e. Rob Vermaas) created a VirtualBox image with all the software needed during the tutorial. In particular, it contains a full installation of Stratego/XT with compiler, libraries, and auxiliary packages such as java-front. In addition, the image contains a built-from-source installation of the WebDSL language for building web applications, and an installation of tomcat for deploying created web apps.

The image and instructions for its installation are available from http://strategoxt.org/Stratego/CodeGeneration2009Tutorial

During the tutorial additional material will be handed out with concrete exercises.

The virtual machine is also useful for exploring Stratego/XT and WebDSL outside the context of this particular tutorial.

Eelco Visser: Domain-Specific Languages for Composable Editor Plugins

Sat, 23 May 2009 11:29:42 +0000

Spoofax/IMP is a toolset for the creation of interactive development environments for custom languages based on domain-specific languages for editor services. The toolset is especially aimed at the developers of domain-specific languages, allowing them to provide IDE support for their specialist language under development. An important feature of Spoofax/IMP is the support for language composition, i.e. for languages consisting of multiple, syntactically different, sub-languages. Furthermore, the toolset allows the customization of heuristically generated editor services without loosing the ability to regenerate these services when a language evolves.

At LDTA 2009 we presented a paper about Spoofax/IMP. The final version of that paper is now finished, and a pre-print is available.

L. C. L. Kats, K. T. Kalleberg, and E. Visser. Domain-Specific Languages for Composable Editor Plugins. In T. Ekman and J. Vinju, editors, Proceedings of the Ninth Workshop on Language Descriptions, Tools, and Applications (LDTA 2009), Electronic Notes in Theoretical Computer Science. Elsevier Science Publishers, April 2009. [pdf]

Abstract: Modern IDEs increase developer productivity by incorporating many different kinds of editor services. These can be purely syntactic, such as syntax highlighting, code folding, and an outline for navigation; or they can be based on the language semantics, such as in-line type error reporting and resolving identifier declarations. Building all these services from scratch requires both the extensive knowledge of the sometimes complicated and highly interdependent APIs and extension mechanisms of an IDE framework, and an in-depth understanding of the structure and semantics of the targeted language. This paper describes Spoofax/IMP, a meta-tooling suite that provides high-level domain-specific languages for describing editor services, relieving editor developers from much of the framework-specific programming. Editor services are defined as composable modules of rules coupled to a modular SDF grammar. The composability provided by the SGLR parser and the declaratively defined services allows embedded languages and language extensions to be easily formulated as additional rules extending an existing language definition. The service definitions are used to generate Eclipse editor plugins. We discuss two examples: an editor plugin for WebDSL, a domain-specific language for web applications, and the embedding of WebDSL in Stratego, used for expressing the (static) semantic rules of WebDSL.

Eelco Visser: Adding Error Recovery to Scannerless Generalized-LR Parsing

Sun, 10 May 2009 22:19:59 +0000

We just got the notification that our submission to OOPSLA 2009 has been accepted. The paper presents a solution to error recovery for the SGLR parsing algorithm. Here's the full citation and abstract (pre-print will follow later):

Lennart C. L. Kats, Maartje de Jonge, Emma Nilsson-Nyman, and Eelco Visser. "Providing Rapid Feedback in Generated Modular Language Environments. Adding Error Recovery to Scannerless Generalized-LR Parsing" In Gary T. Leavens, editor, Proceedings of the 24th ACM SIGPLAN Conference on Object-Oriented Programing, Systems, Languages, and Applications (OOPSLA 2009), New York, NY, USA, October 2009. ACM. (to appear).

Abstract: Integrated Development Environments (IDEs) increase programmer productivity, providing rapid, interactive feedback based on the syntax and semantics of a language. A heavy burden lies on developers of new languages to provide adequate IDE support. Code generation techniques provide a viable, efficient approach to semi-automatically produce IDE plugins. Key components for the realization of plugins are the language's grammar and parser. For embedded languages and language extensions, constituent IDE plugin modules and their grammars can be combined. Unlike conventional parsing algorithms, scannerless generalized-LR parsing supports the full set of context-free grammars, which is closed under composition, and hence can parse language embeddings and extensions composed from separate grammar modules. To apply this algorithm in an interactive environment, this paper introduces a novel error recovery mechanism, which allows it to be used with files with syntax errors -- common in interactive editing. Error recovery is vital for providing rapid feedback in case of syntax errors, as most IDE services depend on the parser -- from syntax highlighting to semantic analysis and cross-referencing. We base our approach on the principles of island grammars, and automatically generate new productions for existing grammars, making them more permissive of their inputs. To cope with the added complexity of these grammars, we adapt the parser to support backtracking. We evaluate the recovery quality and performance of our approach using a set of composed languages, based on Java and Stratego.

Eelco Visser: A Textual DSL for creating Visual Diagrams

Sat, 02 May 2009 09:28:33 +0000

I have been playing around for a couple of minutes with yUML, an online service by Tobin Harris for creating UML diagrams using a textual input language. The diagram below is generated while you load this page. The input needed to generate the diagram is the following list of relations:

  [Publication]++->*[Author], 
  [AbstractAuthor]^[Author], 
  [AbstractAuthor]*->1[Person], 
  [AbstractAuthor]*->1[Affiliation], 
  [Person]*->*[Publication], 
  [Publication]^[PrintPublication], 
  [PrintPublication]^[Article], 
  [PrintPublication]^[InProceedings], 
  [Publication]^[PublishedVolume], 
  [PublishedVolume]^[Proceedings], 
  [InProceedings]->[Proceedings], 
  [AbstractAuthor]^[Editor], 
  [PublishedVolume]++->*[Editor], 
  [Person]*->*[PublishedVolume], 
  [PublishedVolume]^[Book], 
  [PrintPublication]^[InCollection], 
  [Book]->*[InCollection] .

This diagram documents a (small) subset of the data model underlying the researchr.org application for bibliography sharing and reviewing. *[Author], [AbstractAuthor]^[Author], [AbstractAuthor]*<->1[Person], [AbstractAuthor]*<->1[Affiliation], [Person]*->*[Publication], [Publication]^[PrintPublication], [PrintPublication]^[Article], [PrintPublication]^[InProceedings], [Publication]^[PublishedVolume], [PublishedVolume]^[Proceedings], [InProceedings]<->[Proceedings], [AbstractAuthor]^[Editor], [PublishedVolume]++->*[Editor], [Person]*->*[PublishedVolume], [PublishedVolume]^[Book], [PrintPublication]^[InCollection], [Book]<->*[InCollection] ." />*[Author], [AbstractAuthor]^[Author], [AbstractAuthor]*->1[Person], [AbstractAuthor]*->1[Affiliation], [Person]*->*[Publication], [Publication]^[PrintPublication], [PrintPublication]^[Article], [PrintPublication]^[InProceedings], [Publication]^[PublishedVolume], [PublishedVolume]^[Proceedings], [InProceedings]->[Proceedings], [AbstractAuthor]^[Editor], [PublishedVolume]++->*[Editor], [Person]*->*[PublishedVolume], [PublishedVolume]^[Book], [PrintPublication]^[InCollection], [Book]->*[InCollection] ." />*[Author], [AbstractAuthor]^[Author], [AbstractAuthor]*->1[Person], [AbstractAuthor]*->1[Affiliation], [Person]*->*[Publication], [Publication]^[PrintPublication], [PrintPublication]^[Article], [PrintPublication]^[InProceedings], [Publication]^[PublishedVolume], [PublishedVolume]^[Proceedings], [InProceedings]->[Proceedings], [AbstractAuthor]^[Editor], [PublishedVolume]++->*[Editor], [Person]*->*[PublishedVolume], [PublishedVolume]^[Book], [PrintPublication]^[InCollection], [Book]->*[InCollection] ." />*[Author], [AbstractAuthor]^[Author], [AbstractAuthor]*->1[Person], [AbstractAuthor]*->1[Affiliation], [Person]*->*[Publication], [Publication]^[PrintPublication], [PrintPublication]^[Article], [PrintPublication]^[InProceedings], [Publication]^[PublishedVolume], [PublishedVolume]^[Proceedings], [InProceedings]->[Proceedings], [AbstractAuthor]^[Editor], [PublishedVolume]++->*[Editor], [Person]*->*[PublishedVolume], [PublishedVolume]^[Book], [PrintPublication]^[InCollection], [Book]->*[InCollection] ." />

Eelco Visser: WebDSL: A Domain-Specific Language for Web Applications

Thu, 30 Apr 2009 09:38:28 +0000

I have been invited to give a talk at the Sixth International Workshop on Web Information Systems Modeling (WISM 2009), which will be held in June in Amsterdam (co-located with CAiSE 2009). Here's the abstract I wrote for the talk.

Abstract: In this talk I give an overview of the design and application of WebDSL, a domain-specific language for data centric web applications. WebDSL linguistically integrates the definition of data models, user interfaces, actions, access control rules, data validation rules, styling rules, and workflow definitions. While maintaining separation between these concerns through specialized sub-languages, linguistic integration ensures static consistency checking and correct code generation. The language allows developers to concentrate on the essential design of web applications, abstracting from accidental complexity, such as the details of data persistence. The combination of high-level and low-level constructs ensures high expressivity, while supporting customization to application requirements. The application of WebDSL is illustrated using the researchr.org application for bibliography sharing and reviewing.

Links:

Eelco Visser: Blog Abstraction (Twitter)

Mon, 13 Apr 2009 12:23:54 +0000

It was probably unavoidable. Tweetie app on iPod makes it easy. It seems I'm tweeting.

Blogging for the lazy. Let's see if I have more to say, or more frequently at least, on twitter than on this blog.

Eelco Visser: Creating Domain-Specific Languages with Stratego/XT

Mon, 06 Apr 2009 20:35:45 +0000

At the Code Generation 2009 conference, Lennart Kats and I will give a hands-on tutorial about building DSLs with Stratego/XT. The program says it thus:

Stratego/XT is a state-of-the art language and toolset for the development of domain-specific language implementations. In this session the participants learn to use Stratego/XT, by developing a generator for a small DSL for web applications generating PHP. The tutorial covers declarative syntax definition with SDF, code generation by model transformation, and model-to-model transformation by rewriting. The tutorial is based on a course in model-driven software development developed at Delft University of Technology.

NB Since this is a hands-on session places are strictly limited. Please let us know whether you plan to attend this session when you book your conference place. Places will be allocated on a first-come first-served basis.

Eelco Visser: Example-Driven Research

Thu, 05 Mar 2009 10:20:09 +0000

These days I am writing a book on ‘domain-specific language engineering’ for use in a master’s course at Delft University. The book is about the design and implementation of domain-specific languages, i.e. the definition of their syntax, static semantics, and code generators. But it also contains a dose of linguistic reflection, by studying the phenomenon of defining languages, and of defining languages for defining languages (which is called meta-modelling these days).

Writing chapters on syntax definition and modeling of languages, takes me back to my days as a PhD student at the University of Amsterdam. Our quarters were in the university building at theWatergraafsmeer, which was connected to the CWI building via a bridge. Since the ASF+SDF group of Paul Klint was divided over the two locations, meetings required a walk to the other end of the building. So, I would regularly wander to the CWI part of the building to chat. [While the third application we learned to use in our Unix course in 1989 was talk, with which one could synchronously talk with someone else on the internet (the first application was probably csh and the second email), face-to-face meetings were still the primary mode of communication; as opposed to the use of IRC to talk to one’s officemate.] Often I would look into Jan Heering’s office to say hi, and more often than not would end up spending the rest of the afternoon discussing research and meta-research.

One of the recurring topics in these conversations was the importance of examples. Jan was fascinated by the notion of ‘programming by example’, i.e. deriving a program from a bunch of examples of its expected behaviour, instead of a rigorous and complete definition for all cases. But the other use of examples was for validation, a word I didn’t learn until long after writing my thesis.

The culture of the day (and probably location?) was heavily influenced by mathematics and theoretical computer science. The game was the definition, preferably algebraically, of the artifacts of interest, and then, possibly proving interesting properties. The application minded would actually implement stuff. As a language engineer I was mostly interested in making languages with cool features. The motivation for these features was often highly abstract. The main test example driving much of the work on the ASF+SDF MetaEnvironment was creating an interactive environment for the Pico language (While with variable declarations). The idea being that once an environment for Pico was realized, creating one for a more realistic language would be a matter scaling up the Pico definition (mere engineering). Actually making a language (implementation) and using that to write programs would be a real test. To be fair, there were specifications of larger languages undertaken, such as ones of (mini-) ML [8] and Pascal [6]. As a student I had developed a specification of the syntax and static semantics of the object-oriented programming language Eiffel [16], but that was so big it was not usable at the Sun workstations we had at that time.

Time and again, Jan Heering would stress the importance of real examples to show the relevance of a technique and/or to discover the requirements for a design. While I thought it was a cool idea, I didn’t have examples. At least not to sell the design and implementation of SDF2, the syntax definition formalism that turned out to be the main contribution of my PhD thesis [20].
Continue reading "Example-Driven Research"

Eelco Visser: MODELS + GPCE + SLE in Denver

Wed, 25 Feb 2009 14:50:13 +0000

If you're doing research into domain-specific languages, model-driven engineering, or program generation, your agenda for the coming months is set. Early October the three main conferences on these topics are co-located in Denver. The deadlines are somewhat spread, so you should be able to submit a paper to each conference:

May 10: Model Driven Engineering Languages and Systems (MODELS'09)

May 18: Generative Programming and Component Engineering (GPCE'09)

July 10: Software Language Engineering (SLE 2009)

I'm looking forward to your submission, and to meeting you in Denver.

Karl Trygve Kalleberg: Domain-Specific Languages for Composable Editor Plugins

Sun, 01 Feb 2009 18:41:11 +0000

Lennart Kats, Eelco Visser and myself just got a paper accepted to LDTA'09. The paper is about declarative languages for describing programming editors. The main part of it is Lennart's work, but it's running on top of the Spoofax transformation infrastructure. The idea is simple: You don't want to fight with Java, complicated APIs and complicated XML when you implement an Eclipse-based editor for your DSL. Instead, you describe your language's grammar with SDF, provide some auxiliary information using our declarative editor languages, and Spoofax/IMP does the rest by generating the editor engine for you.

The abstract explains it in the usual academic style:

Modern IDEs increase developer productivity by incorporating many different kinds of editor services. These can be purely syntactic, such as syntax highlighting, code folding, and an outline for navigation; or they can be based on the language semantics, such as in-line type error reporting and resolving identifier declarations. Building all these services from scratch requires both the extensive knowledge of the sometimes complicated and highly interdependent APIs and extension mechanisms of an IDE framework, and an in-depth understanding of the structure and semantics of the targeted language.

This paper describes Spoofax/IMP, a meta-tooling suite that provides high-level domain-specific languages for describing editor services, relieving editor developers from much of the framework-specific programming. Editor services are defined as composable modules of rules coupled to a modular SDF grammar. The composability provided by the SGLR parser and the declaratively defined services allows embedded languages and language extensions to be easily formulated as additional rules extending an existing language definition. The service definitions are used to generate Eclipse editor plugins.

We discuss two examples: an editor plugin for WebDSL, a domain-specific language for web applications, and the embedding of WebDSL in Stratego, used for expressing the semantic rules of WebDSL.

Once I get bibtex-tools running on 64bit again, I'll link to the bib and pdf.

Karl Trygve Kalleberg: FOSDEM 2009

Sat, 17 Jan 2009 21:55:02 +0000

Like last year, I'm going to FOSDEM. Last year, I traveled from Paris with the express train (great experience). This year, it's back to planes again. Not looking forward to the security hysteria.

If you're interested in meeting me there, don't hesitate to fire off an e-mail.

Eelco Visser: Parse Table Composition

Fri, 19 Dec 2008 09:50:00 +0000

As mentionted before, we've been doing some real parsing research to better support parsers for extensible languages. Parse table composition provides separate compilation for syntax components such that syntax extensions can be provided as plugins to a compiler for a base language. Due to various distractions last Summer I seem to have forgotten to blog about the paper that Martin Bravenboer and I got accepted at the first international conference on Software Language Engineering (which Martin was looking forward too).

M. Bravenboer and E. Visser. Parse Table Composition. Separate Compilation and Binary Extensibility of Grammars. In D. Gasevic and E. van Wyk, editors, First International Conference on Software Language Engineering (SLE 2008). To appear in Lecture Notes in Computer Science, Heidelberg, 2009. Springer. [pdf]

Abstract: Module systems, separate compilation, deployment of binary components, and dynamic linking have enjoyed wide acceptance in programming languages and systems. In contrast, the syntax of languages is usually defined in a non-modular way, cannot be compiled separately, cannot easily be combined with the syntax of other languages, and cannot be deployed as a component for later composition. Grammar formalisms that do support modules use whole program compilation.

Current extensible compilers focus on source-level extensibility, which requires users to compile the compiler with a specific configuration of extensions. A compound parser needs to be generated for every combination of extensions. The generation of parse tables is expensive, which is a particular problem when the composition configuration is not fixed to enable users to choose language extensions.

In this paper we introduce an algorithm for parse table composition to support separate compilation of grammars to parse table components. Parse table components can be composed (linked) efficiently at runtime, i.e. just before parsing. While the worst-case time complexity of parse table composition is exponential (like the complexity of parse table generation itself), for realistic language combination scenarios involving grammars for real languages, our parse table composition algorithm is an order of magnitude faster than computation of the parse table for the combined grammars.

The experimental parser generator is available online.

Eric Bouwers: Visualizing the PHP grammar

Tue, 16 Dec 2008 21:36:20 +0000

A few weeks ago an e-mail from Didier Garcin popped up on the Stratego mailing list. He explained that he had written a python script that could visualize an abstract syntax signature. The script was also send to the list and I finally had some time to check it out.

It turned out that I had almost anything installed to use the script, only the pydot dependency needed some work. This was mostly because the script only seems to work with the 0.9.10 version of this library. After getting the script started it was really simple to generate the signature from the PHP-Front grammar.

So, here is the one for PHP4:

And the one for PHP5:

The first thing I notices where the big rectangles in both versions. These rectangles are the statements (smaller one) and expressions grouped together. What I also notices was that in both versions we see that the bottom of the graph (which corresponds with the smallest units in the language) looks the most complicated. This corresponds very well with the amount of effort put into the modeling of this part of the language.

Comparing both grammars to each other we can see that the latest versions is the most complicated one. Furthermore, if we compare both graphs to the graphs shown in this post we can see that the Java-graph appears to be the most similar one.

Actually, I do not think these images show anything, but please explain it to me when you think I just don't see it. Anyway, at least we have some nice pictures now :)

P.S. for those who are interested, more detailed images (in svg-format) are available here.

Eelco Visser: Talking about Parsing

Mon, 15 Dec 2008 21:11:00 +0000

Last Summer I attended the Code Generation 2008 conference in Cambridge to give a tutorial on WebDSL, as case study in domain-specific language engineering. The conference was an interesting change from the usual academic conferences I visit, in that the majority of the audience were from industry. It was good to see the interest in code generation in industry, but also disconcerting to observe the gap between academic research and industrial practice; but more about that some other time.

During the conference I was interviewed by Laurence Tratt for Software Engineering Radio about parsing. The interview podcast recently appeared as Episode 118.

It was a long time ago (1997) that I defended my PhD thesis, which was mostly about syntax definition and parsing. In particular, I introduced SDF2, which radically integrates lexical and context-free syntax, and the SGLR parsing algorithm for parsing arbitrary 'character-level' context-free grammars. Since finishing my thesis I have done quite a bit of `applied parsing research', using SDF and SGLR for applications such as meta-programming with concrete object syntax and DSL embedding, but I don't consider myself a hard-core parsing researcher any more. So I had to dig deep in my memory to talk about Noam Chomsky's language hierarchy, grammars as string rewrite systems, and parsing algorithms. I find the result a bit awkward to listen to, but people assure me that is because it is my own voice I'm listening too.

In the meantime my relation to parsing is changing again. While SDF/SGLR still provides the best approach to declarative definition of composite languages (in my opinion at least), it has some fundamental limitations which have never been addressed. A first step in addressing these limitations was taken in the SLE 2008 paper with Martin Bravenboer on parse table composition (see upcoming blog) to provide separate compilation for grammars. With a new PhD student starting in the new year, I hope to address other limitations such as the lack of error recovery.

Eelco Visser: Decorated Attribute Grammars

Fri, 12 Dec 2008 09:45:30 +0000

The paper "Decorated Attribute Grammars" by Lennart Kats, Tony Sloane and Eelco Visser has been accepted for presentation at the International Conference on Compiler Construction (CC 2009) to be held in March 2009 in York (UK). [pdf]

Abstract: Attribute grammars are a powerful specification formalism for tree-based computation, particularly for software language processing. Various extensions have been proposed to abstract over common patterns in attribute grammar specifications. These include various forms of copy rules to support non-local dependencies, collection attributes, and expressing dependencies that are evaluated to a fixed point. Rather than implementing extensions natively in an attribute evaluator, we propose attribute decorators that describe an abstract evaluation mechanism for attributes, making it possible to provide such extensions as part of a library of decorators. Inspired by strategic programming, they are specified using generic traversal operators. To demonstrate their effectiveness, we describe how to employ decorators in name, type, and flow analysis.

The ideas have been implemented in Aster, an extension of Stratego with reference attribute grammars.

Martin Bravenboer: Why the JVM Spec defines checkcast for interface types

Thu, 04 Dec 2008 15:03:48 +0000

I'm working on the specification of pointer analysis for Java using Datalog. Basically, a pointer analysis computes for each variable in a program the set of objects it may point to at run-time.

For this purpose I need to express parts of the JVM Spec in Datalog as well. As a simple example, the following Datalog rules define when a class is a subclass of another class.

/**
 * JVM Spec:
 * - A class A is a subclass of a class C if A is a direct 
 *   subclass of C
 */
Subclass(?c, ?a) <-
  DirectSubclass[?a] = ?c.

/**
 * JVM Spec:
 * - A class A is a subclass of a class C if there is a direct
 *   subclass B of C and class A is a subclass of B
 */
Subclass(?c, ?a) <-
  Subclass(?b, ?a),
  DirectSubclass[?b] = ?c.

As you can see, this is remarkably close to the original specification (quoted in comments). You can clearly see the relationship between the spec and the code, even if you are not familiar with Datalog.

Recently, I was working on the specification of the checkcast instruction. This instruction performs the run-time check if an object can be cast to some type. The JVM Spec for checkcast first defines some variables:

The following rules are used to determine whether an objectref that is not null can be cast to the resolved type: if S is the class of the object referred to by objectref and T is the resolved class, array, or interface type, checkcast determines whether objectref can be cast to type T as follows:

So, this basically says that we're checking the cast (T) S.

The first rule for this cast is straightforward:

If S is an ordinary (nonarray) class, then:

If T is a class type, then S must be the same class as T, or a subclass of T.

If T is an interface type, then S must implement interface T.

Well, if you're somewhat familiar with Java, or object-oriented programming, then this part is obvious. Again, the specification in Datalog is easy:

CheckCast(?s, ?s) <-
  ClassType(?s).

CheckCast(?s, ?t) <-
  Subclass(?t, ?s).

CheckCast(?s, ?t) -
  ClassType(?s),
  Superinterface(?t, ?s).

However, the next alternative in the specification is confusing:

If S is an interface type, then:

If T is a class type, then T must be Object.

If T is an interface type, then T must be the same interface as S or a superinterface of S.

The specification is crystal clear, but how can S ever be an interface type? S is the type of the object that is being cast, and how can an object ever have a run-time type that is an interface? Of course, the static type of an expression can be an interface, but we're talking about the run-time here!

I searched the web, which only resulted in a few hits. There was one question on a Sun forum years ago, where the one answer didn't make a lot of sense.

It turns out that this is indeed an `impossible' case. The reason why this item is in the specification, is because checkcast is recursively defined for arrays:

If S is a class representing the array type SC[], that is, an array of components of type SC, then:

...

If T is an array type TC[], that is, an array of components of type TC, then one of the following must be true:

...

TC and SC are reference types, and type SC can be cast to TC by recursive application of these rules.

So, if you have an object of type List[] that is cast to an Collection[], then the rules for checkcast get recursively invoked for the types S = List and T = Collection. Notice that List is an interface, but an object can have type List[] at run-time. If have not verified this with the JVM Spec maintainers, but as far as I can see, this is the only reason why the rule for interface types is there.

Just to show a little bit more of my specifications, here is the rule for the array case I just quoted from the JVM Spec:

CheckCast(?s, ?t) <-
  ComponentType[?s] = ?sc,
  ComponentType[?t] = ?tc,
  ReferenceType(?sc),
  ReferenceType(?tc),
  CheckCast(?sc, ?tc).

Isn't it beautiful how this exactly corresponds to the formal specification?

Unfortunately, even formal specifications can have errors, so I also specified a large testsuite that checks the specifications with concrete code. Here are some of the tests for CheckCast.

test Casting to self
  using database tests/hello/Empty.jar
  assert
    CheckCast("java.lang.Integer", "java.lang.Integer")

test Casting to superclasses
  using database tests/hello/Empty.jar
  assert
    CheckCast("java.lang.Integer", "java.lang.Number")
    CheckCast("java.lang.Integer", "java.lang.Object")

test Cast ArrayList to various superinterfaces
  using database tests/hello/Arrays.jar
  assert
    CheckCast("java.util.ArrayList", "java.util.List")
    CheckCast("java.util.ArrayList", "java.util.Collection")
    CheckCast("java.util.ArrayList", "java.io.Serializable")

test Cast class[] to implemented interface[]
  using database tests/hello/Arrays.jar
  assert
    CheckCast("java.util.ArrayList[]", "java.util.List[]")
    CheckCast("java.lang.Integer[]", "java.io.Serializable[]")

test Cast interface[] to superinterface[]
  using database tests/hello/Arrays.jar
  assert
    CheckCast("java.util.List[]", "java.util.Collection[]")

The tests are specified in a little domain-specific language for unit-testing Datalog that I implemented, initially for IRIS and later for LogicBlox. This tool is similar to parse-unit, a tool I wrote earlier for testing parsers in Stratego/XT. The concise syntax of a test encourages you to write a lot of tests. Domain-specific languages rock for this purpose!

Eelco Visser: WebWorkFlow: An Object-Oriented Workflow Modeling Language for Web Applications

Fri, 11 Jul 2008 19:39:08 +0000

The paper "WebWorkFlow: An Object-Oriented Workflow Modeling Language for Web Applications" by Zef Hemel, Ruben Verhaaf and Eelco Visser has been accepted for presentation at the conference on Model Driven Engineering Languages and Systems (MODELS 2008) to be held in Toulouse, France at the end of September 2008.

Abstract: Workflow languages are designed for the high-level description of processes and are typically not suitable for the generation of complete applications. In this paper, we present WebWorkFlow, an object-oriented workflow modeling language for the high-level description of workflows in web applications. Workflow descriptions define procedures operating on domain objects. Procedures are composed using sequential and concurrent process combinators. WebWorkFlow is an embedded language, extending WebDSL, a domain-specific language for web application development, with workflow abstractions. The extension is implemented by means of model-to-model transformations. Rather than providing an exclusive workflow language, WebWorkFlow supports interaction with the underlying WebDSL language. WebWorkFlow supports most of the basic workflow control patterns.

Eelco Visser: Heterogenous Coupled Evolution

Fri, 11 Jul 2008 19:35:05 +0000

The paper "Heterogenous Coupled Evolution" by Sander Vermolen and Eelco Visser has been accepted for presentation at the conference on Model Driven Engineering Languages and Systems (MODELS 2008) to be held in Toulouse, France at the end of September 2008.

Abstract: As most software artifacts, meta-models can evolve. Their evolution requires conforming models to co-evolve along with them. Coupled evolution supports this. Its applicability is not limited to the modeling domain. Other domains are for example evolving grammars or database schema. Existing approaches to coupled evolution focus on a single, homogeneous domain. They solve the co-evolution problems locally and repeatedly. In this paper we will present a systematic, heterogeneous approach to coupled evolution. It provides an automatically derived domain specific transformation language; a means of executing transformations at the top level; a derivation of the coupled bottom level transformation; and the ability to generically abstract from elementary transformations. The feasibility of the architecture is evaluated by applying it to data model evolution as well as grammar evolution.

Martin Bravenboer: New Conference on Software Language Engineering

Tue, 01 Jul 2008 08:02:25 +0000

This year is special. There is a new and exciting conference: the International Conference on Software Language Engineering (SLE). The deadline for submission of papers is July 14th, which is coming up soon! Before I start raving about the topics covered by this conference, here is the disclaimer: I'm on the program committee of this conference, and as such I believe it's my duty to advertise the conference.

Anyway, if done right, this conference has the potential to become a major and prestigious conference. The conference fills a clear gap: the topics of software language engineering do not exactly fit in major programming language conferences like OOPSLA, PLDI, POPL, and ECOOP. Nor do they fit exactly in the area of compiler construction (CC). CC does typically not accept more engineering or methodology-oriented papers. For OOPSLA and ECOOP the work more or less has to be in the context of object-oriented programming, for POPL it immediately has to be a principle (whatever that is), and for PLDI there are usually just a few slots available for papers that don't do something with memory management, garbage collection, program analysis, or concurrency. Personally, I've been pretty successful at getting papers in the area of software language engineering accepted at OOPSLA, but a full conference devoted to this topic is much better!

Another reason why I think that this conference has a lot of potential is that if I look at the list of topics of interest in the call for papers, then I can only think of one summary: everything that's fun! I'm convinced I'm not the only one who thinks these topics are fun. When talking to colleagues, I notice again and again most of us just love languages. The engineering of those languages is an issue for almost all computer scientists and many programmers in industry, and this conference will be the most obvious target for papers about this!

Also, the formalisms and used for the specification and implementation of (domain-specific) languages are still very much an open research topic. Standardization of languages is still far from perfect, as discussed by many posts on this blog. Also, new language implementation techniques are being proposed all the time, and extensible compilers for developing language extensions are more popular than ever. Not to mention the increasing interest in using domain-specific languages to help solve the software development problems we're facing.

Earlier in this post I wrote that this conference has major potential if done right. There are few risks. First, the conference has been started by two relatively small communities: ATEM and LDTA. I think the conference should attract a much larger community than the union of those two communities. I hope lots of people outside of the ATEM and LDTA communities will consider to submit a paper. Second, this year the conference is co-located with MODELS. Many programming language people are slightly allergic to model-driven engineering. I hope they will realize that this conference is not specifically a model-driven conference. Finally, the whole setup of the conference should be international and varied. I'm sorry to say that at this point I'm not entirely happy with the choice of keynote speakers. This nothing personal: I respect both keynote speakers, but the particular combination of the two speakers is a bit unfortunate. First, they are both Dutch. Second, neither of them is extremely well-known in the communities of OOPSLA, PLDI, or ECOOP. I hope that this will not affect the potential of this interesting conference.

Now go work on your submission!

Martin Bravenboer: Dubious Conferences: How do they threat people?

Tue, 03 Jun 2008 07:14:55 +0000

I just got a call for papers for the International e-Conference on Computer Science 2008 (IeCCS 2008). The IeCCS conference organizers and committee members are one of a kind! The submission deadline for papers is the 20th of June. Notification of acceptance is 25th of June. That's 5 days for reviewing. The camera ready deadline is 27th of June. That's 2 days for revising your paper. You've got to love the efficiency of these people! The 2007 edition of this conference has a program of 11 pages of accepted papers. When I review papers, I rarely get more than 2 done per day. If the IeCCS committee want to accept a similar number of papers this year, then they'd better make sure to get enough coffee (or tea, as advised in my Ph.D. thesis).

Now, if you are a computer science researcher such emails are hardly surprising. I delete several of them every week. Everybody is aware of conferences with questionable reviewing practices (see the SCIgen paper generator). What surprised me about the IeCCS call for papers is that there is actually a researcher on the committee who I vaguely know from when I was a student. So, I searched the web a bit to see how obvious the evidence is that the reviewing practices of IeCCS are questionable. Interestingly, I could find only one reference that mentions IeCCS as a conference where you'd better not submit to. It's an interesting presentation of somebody at the PSU.

It seems that lists of conferences with a dubious reputation (also known as fake conferences) are impossible to keep up. I've seen a few lists in the past, but they've all disappeared. What interests me is why those lists are taken down. The most well known list, by Arlindo Oliveira, was taken down after receiving threats by conference organizers. I've never quite understood that: how serious can such a threat be? Maybe they'll publish a random paper with my name? They'll put me on the program committee next year?

So well, here we go. Let's see what happens.

Notice: IeCCS is not fake. It is very real!

Eelco Visser: The Nix Build Farm: A Declarative Approach to Continuous Integration

Mon, 26 May 2008 20:44:43 +0000

The paper "The Nix Build Farm: A Declarative Approach to Continuous Integration" by Eelco Dolstra and Eelco Visser has been accepted for presentation at the International Workshop on Advanced Software Development Tools and Techniques co-located with ECOOP 2008 in

The paper is the first to come out of the 3TU CEDICT buildfarm project (which badly needs a webpage).

Abstract: There are many tools to support continuous integration (the process of automatically and continuously building a project from a version management repository). However, they do not have good support for variability in the build environment: dependencies such as compilers, libraries or testing tools must typically be installed manually on all machines on which automated builds are performed. The Nix package manager solves this problem: it has a purely functional language for describing package build actions and their dependencies, allowing the build environment for projects to be produced automatically and deterministically. We have used Nix to build a continuous integration tool, the Nix build farm, that is in use to continuously build and release a large set of projects.

Karl Trygve Kalleberg: Create-a-Project: Creating Stratego/XT projects the simple way

Sat, 24 May 2008 14:40:31 +0000

This last week, I spent some free cycles hacking together a small project instantiation tool for Stratego/XT. It makes setting up a fresh Stratego project really simple by automatically populating the project space with a default directory layout, build system files and some minimal program and syntax samples.

To create a project p0, all you have to do is:

$ crap --new-project p0

This creates all the files necessary for a complete GNU Autotools-based build system, including a sample Stratego program (src/xmpl.str):

p0/
   Makefile.am
   README.Developer
   README
   AUTHORS
   bootstrap
   p0.spec.in
   NEWS
   p0.pc.in
   configure.ac
   ChangeLog
   xmpl/
        Makefile.am
   syn/
       Makefile.am
   tests/
         Makefile.am
   src/
       Makefile.am
       xmpl.str

Once this is done, you can configure and compile the project,

$ ./bootstrap
$ ./configure
$ make all

install it,

$ make install

and even run the example transformation program:

$ echo "foo" | /usr/local/bin/xmpl
"Hello, World!"

The example program expects an input on stdin [or in a file specified by the -i switch], and will always produce the output string "Hello, World!".

The crap tool is part of the strategoxt-utils package. You can also download a stand-alone snapshot of crap.

More comprehensive documentation is available in the wiki. The tool is still very rough, so any suggestions for improvements and bug reports are very welcome.

Eelco Visser: OOPSLA'08 tutorial: Building Domain-Specific Languages for the Web

Tue, 13 May 2008 19:28:20 +0000

Just got word that my proposal for an OOPSLA tutorial has been accepted. So make sure to register for it if you are planning to attend the conference.

Abstract: Implementing web applications in an object-oriented language such as Java using state-of-the-art frameworks produces robust software, but involves a lot of boilerplate code. Domain-specific languages (DSLs) increase the productivity of software engineers by replacing such low-level boilerplate code by high-level models, from which code can be generated. This tutorial shows how to find domain-specific abstractions based on patterns in existing (reference) programs and build domain-specific languages to capture these abstraction using several DSLs for DSL engineering: SDF for syntax definition and Stratego/XT for code generation. The approach is illustrated using the design and implementation of WebDSL, a domain-specific language for web applications, which provides abstractions for data models, page definitions, access control, workflow, and styling. The tutorial will show how code generation by model transformation is an important technique for separation of concerns in DSL implementations for designing a DSL as a tower of abstractions, rather than as a monolithic language.

Eelco Visser: WebDSL: A Case Study in Domain-Specific Language Engineering

Sun, 11 May 2008 14:18:00 +0000

About fifteen months ago I announced my ``Domain-Specific Language Engineering'' project that would result in a tutorial for the GTTSE'07 summerschool. "This tutorial gives an overview of all aspects of DSL engineering: domain analysis, language design, syntax definition, code generation, deployment, and evolution, discussing research challenges on the way. The concepts are illustrated with DSLs for web applications built using several DSLs for DSL engineering: SDF for syntax definition, Stratego/XT for code generation, and Nix for software deployment." A rather bold statement, since at time I didn't have a DSL, yet. But I did manage to design and implement a first version of WebDSL before the summerschool in July 2007. I also wrote a paper for the participants proceedings. That version discussed the design process from analyzing programming patterns in Seam/JSF/Java to the design and implementation of the DSL using SDF and Stratego. (I never got around to the deployment part; Sander van der Burg has by now developed Nix expressions for building and deploying a WebDSL application on a web server.)

Now I have finished the final version of the paper for the proceedings to be published by Springer. The discussion of the WebDSL design and implementation has been much improved. But more importantly, the paper has two introductory sections about the 'domain-specific language engineering' process and three discussion sections evaluating WebDSL as a web engineering solution, discussing related DSL engineering approaches, and research challenges for language engineering. The resulting paper counts 85 pages (LNCS format) and 109 references. And still, it is only scratching the surface.

There is so much more to say about DSL design and implementation. The last couple of months I have been teaching a course on program transformation and generation in which we have explored paradigms for expressing analysis and transformation. During this time I have been contemplating how to structure the course entitled "model-driven software development" that I'll be teaching next year to a much larger group of master's students. I haven't resolved the issue yet. But my project for the next 10 months will be to write a comprehensive set of lecture notes (let's call it a book) on "domain-specific language engineering" covering methods and techniques for designing and implementing software languages, in particular, of course domain-specific languages. In all likelihood, languages for the web will again play a central role as examples.

E. Visser. WebDSL: A Case Study in Domain-Specific Language Engineering. In R. Laemmel, J. Saraiva, and J. Visser, editors, Generative and Transformational Techniques in Software Engineering (GTTSE 2007), Lecture Notes in Computer Science. Springer, 2008. Tutorial for International Summer School GTTSE 2007. [pdf]

Abstract: The goal of domain-specific languages (DSLs) is to increase the productivity of software engineers by abstracting from low-level boilerplate code. Introduction of DSLs in the software development process requires a smooth workflow for the production of DSLs themselves. This requires technology for designing and implementing DSLs, but also a methodology for using that technology. That is, a collection of guidelines, design patterns, and reusable DSL components that show developers how to tackle common language design and implementation issues. This paper presents a case study in domain-specific language engineering. It reports on a project in which the author designed and built WebDSL, a DSL for web applications with a rich data model, using several DSLs for DSL engineering: SDF for syntax definition and Stratego/XT for code generation. The paper follows the stages in the development of the DSL. The contributions of the paper are three-fold. (1) A tutorial in the application of the specific SDF and Stratego/XT technology for building DSLs. (2) A description of an incremental DSL development process. (3) A domain-specific language for web-applications with rich data models. The paper concludes with a survey of related approaches.

Eelco Visser: Stratego: 10 years

Thu, 08 May 2008 18:54:23 +0000

-------------------------------------------------------------------------
Dear author,

We are pleased to inform you that your paper entitled

     Building Program Optimizers with Rewriting [Strategies]

has been accepted for presentation at ICFP'98. In a next message, you
will receive reviews from the Program Committee that we hope you can
use to improve the final draft, which is due on July 14th. We will
also be sending you an ACM Copyright Release form, which must be
signed and returned by the same deadline.

Congratulations, and thank you for your submittal to ICFP'98.

  Paul Hudak
  Christian Queinnec
  co-Chairs, ICFP'98 Program Committee
-------------------------------------------------------------------------

The text of an email from June 21, 1998 that announced the acceptance at ICFP'98 of the first paper on Stratego written with Zino Benaissa and Andrew Tolmach. The implementation of the language for the paper marked the first version of the language and compiler. The idea of traversal strategies had been done before embedded in ASF+SDF. Just before the conference in September I managed to bootstrap the compiler. The language did not have a name in that paper yet. And the reviews were not very enthusiastic.

Eelco Visser: Declarative Access Control for WebDSL

Fri, 04 Apr 2008 20:27:50 +0000

The paper "Declarative Access Control for WebDSL: Combining Language Integration and Separation of Concerns" by Danny Groenewegen and Eelco Visser has been accepted for presentation at the International Conference on Web Engineering (ICWE'08), which will be held in July 2008 in Yorktown Heights, New York. [pdf]

I'm especially proud of this acceptance as it is (1) based on the Master's thesis work of Danny Groenewegen, and (2) the first paper about WebDSL to be accepted in the web engineering research community. (And also the first attempt; the other two papers featuring WebDSL appear in transformation venues.)

Abstract: In this paper, we present the extension of WebDSL, a domain-specific language for web application development, with abstractions for declarative definition of access control. The extension supports the definition of a wide range of access control policies concisely and transparently as a separate concern. In addition to regulating the access to pages and actions, access control rules are used to infer navigation options not accessible to the current user, preventing the presentation of inaccessible links. The extension is an illustration of a general approach to the design of domain-specific languages for different technical domains to support separation of concerns in application development, while preserving linguistic integration. This approach is realized by means of a transformational semantics that weaves separately defined aspects into an integrated implementation.

Eelco Visser: Code Generation by Model Transformation

Wed, 19 Mar 2008 11:14:57 +0000

The paper "Code Generation by Model Transformation" by Zef Hemel, Lennart Kats, and Eelco Visser was accepted for presentation at the International Conference on Model Transformation (ICMT'08).

Abstract: The realization of model-driven software development requires effective techniques for implementing code generators. In this paper, we present a case study of code generation by model transformation with Stratego, a high-level transformation language based on the paradigm of rewrite rules with programmable strategies that integrates model-to-model, model-to-code, and code-to-code transformations. The use of concrete object syntax guarantees syntactic correctness of code patterns, and supports the subsequent transformation of generated code. The composability of strategies supports two dimensions of transformation modularity. Vertical modularity is achieved by designing a generator as a pipeline of model-to-model transformations that gradually transforms a high-level input model to an implementation. Horizontal modularity is achieved by supporting the definition of plugins which implement all aspects of a language feature. We discuss the application of these techniques in the implementation of WebDSL, a domain-specific language for dynamic web applications with a rich data model.

Eelco Visser: Attribute Grammars in Stratego II

Tue, 18 Mar 2008 11:21:00 +0000

Today we had a follow up on last weeks discussion about an attribute grammar extension of Stratego. By now, Nicolas Pierron has created a proper extension of Stratego with attribute equations and made a translation to basic Stratego in combination with the Transformers run-time extension for attribute evaluation support. Next up is the port of the copy rules generator that makes writing attribute equations much less verbose. In the meantime Lennart Kats is working on a JastAdd style implementation and Tony Sloane on a Eli-style (static scheduling) implementation. With these implementations in place we will be able to do some proper exploration of the combination of attribute evaluation and rewriting (strategies). I can't wait to make an implementation of the WebDSL typechecker using the attribute extension. To be continued.

Karl Trygve Kalleberg: Proper navigation support for Spoofax

Fri, 14 Mar 2008 16:46:03 +0000

I finally figured out how to add proper navigation history support to Spoofax today. This one has been bugging me for quite some time. I remember spending far too much time diving through the documentation with the hopes of figuring out how this should be done properly. No luck.

Today I had a flash of inspiration, so I dug into the JDT code base. That code seemed to solve the same problem in a very complicated way, so I didn't want to copy their approach outright. Stymied, I started tracing exactly what happens with the navigation history when positions are placed into it. After a bit of fiddling around, I figured out that when I move the cursor, I should mark the position both before and after the cursor/focus moves to get the behaviour of JDT (which I tried to emulate). I've always only tried saving the editor location state either before I changed it, or afterwards. I also tried all kinds of alternative calls on the EditorPart hierarchy in vain. I now use ITextEditor.setHighlightRange() which appears to do the job, provided I call markInNavigatorHistory() "properly".

Anyway, the lesson is simple: if you call AbstractTextEditor.markInNavigationHistory(), remember to do it twice -- once before you change the editor/focus and once afterwards.

Karl Trygve Kalleberg: Porting Eclipse IMP from Eclipse 3.2 to 3.3

Thu, 13 Mar 2008 16:33:24 +0000

It's official: I'm the bootstrapper. My hacking life in the last few weeks have hardly been anything but bootsrapping. I've already said a few things about the Stratego compiler hacking. Since it takes ~3-4 hours for a full build of the Stratego compiler in the Delft buildfarm, I've had a couple of other projects to dive into in parallell. One of these has been the porting of Eclipse IMP from Eclipse 3.2 to 3.3.

In short, IMP is an IDE generator based on Eclipse. It provides set of plugins and wizards that makes the development of programming language environments (a lot) easier. The basic workflow when building an IDE for you favourite language with IMP is, (1) provide a grammar defined using the LPG grammar language, (2) use the IMP-provided wizards inside Eclipse to generate things like syntax highlighting support, outline support, code folding support, templates, text hovers, etc, then (3) fill in the skeletons provided by the generator. My personal view (subject to change without warning) of the generated code is that it's a guide to which parts of the Eclipse framework you need to extend in order to provide a given piece of functionality. Sort of a little helpful gnome pointing you in the right direction. In some cases, the generated code will actually do all you want, but more often than not, you will want to go beyond it.

That was the backgrounder on IMP. A major drawback of the current IMP releases is that they will only work on 3.2. Oh, and, of course, that IMP requires IMP to build IMP. Getting this beast ported to 3.3 wasn't as straightforward as I'd hoped. It took a few iterations. The first was getting it to build properly without any problems on my plain 3.2 installation. That took me several days. All kinds of subtle bugs surfaced, presumably because I have a different set of development habits than the IMPers.

Once those were patched and fixed upstream, I managed to bootstrap my first version on 3.2. An ensuing battle with race conditions in the startup code of various plug-ins followed. I hate static initializers, but apparently not everybody does. In a multi-plugin architecture where the order to plugin loading is not guaranteed, I cannot see how you can safely assume the order of static initializers across plugins, but those questions are not for me to ponder. I ripped them out, and replaced them with lazy initializers as far as possible, and that worked wonders. With that hurdle out of the way, it was all down hill: a couple of internal JFace and JDT classes had changed locations and APIs between 3.2 and 3.3, but it was quick enough to rewrite the offending code (another reason why depending on internal APIs is a bitch, though I realize that the features in question could not have been provided without doing so).

It's a huge disappointment to realize that my patches are only a couple of hundreds of lines. I felt like I had to rewrite the world, at places... Anyway, here's hoping to its inclusion in one of the pending releases. I've updated our sdf2imp tool to use the 3.3-based IMP, so we're already seeing a return on my investment:)

Eelco Visser: Attribute Grammars in Stratego

Tue, 11 Mar 2008 21:41:11 +0000

This morning I gave my very first lecture on attribute grammars, featuring Knuth's binary numbers AG (ripped from the jastadd.org site), and my own implementation of WebDSL data models in JastAdd. In the slides I explored the implementation of model of JastAdd by including the code it generates. While I had not added any JastAdd code since last week, the presentation further improved my understanding what is going in JastAdd. At the same time, I'm realizing I do not fully understand the design space we are considering, and the distinction between notational and essential differences between the various approaches.

In the afternoon, Nicolas Pierron, a student from the Transformers group at Epita who is doing an internship in Delft, showed the code of his implementation of WebDSL typechecking in the Transformers SDF attribute grammar extension, which is implemented by generating Stratego code with some native code hacks. The hacks basically implement thunks of deferred attribute evaluations for nodes in the tree with pointers to dependent attribute value thunks. Attributes are then evaluated on demand, and thus no static scheduling is done. This approach is essentially the same as that of the encoding of attribute grammars in a lazy functional language such as Haskell (a hobby of my former colleagues in Utrecht). If we would just add heap-bound closures to Stratego, implementing this pattern would become that much easier. The Transformers AG code operates on ATerms without sharing. Again, I'm not sure about the implications of this design choice. Are the Stratego libraries at all usable without maximal sharing? I guess I should ask Nicolas tomorrow. To be continued.

Karl Trygve Kalleberg: Stack tracing improvements

Tue, 11 Mar 2008 13:05:40 +0000

A limitation of my previous stack tracing patches was that io-wrap and io-stream-wrap did not properly report traces on failure. The reason for this is easy to spot if we look at how the error is handled (this is where execution flow ends up when you call io-wrap):

  option-wrap(opts, usage, about, announce, s) =
    parse-options(opts, usage, about)
    ; announce
    ; (s; report-success <+ report-failure)

  report-failure =
      report-run-time
    ; <fprintnl> (stderr(), [ (), ": rewriting failed"])
    ; <exit> 1

As you can imagine, even though the program now happily prints a stack trace when the main strategy exits with a failure, it will not be printed when exit is called.

I've introduced a couple of stack introspection functions for dealing with this: stacktrace-get-current-frame-name returns the name of the current frame s, stacktrace-get-all-frame-names returns a list of all frame names and, stacktrace-get-current-frame-index returns integer that holds the current depth of the stack. These are actually implemented by primitives in the Stratego Standard Library (SSL).

A caveat of these strategies is that calling them will of course alter the stack. Even in the wonderful world of computing, we're not entirely free of Heisenbergian effects, apparently. However, there's a simple workaround: call the primitives directly, since this bypasses the way the compiler registers the stack frames.

With this trick in hand, I rewrote the two above strategies to include proper stack tracing for io-wrap:

  option-wrap(opts, usage, about, announce, s) =
    parse-options(opts, usage, about)
    ; announce
    ; (s; report-success <+ prim("SSL_stacktrace_get_all_frame_names") ; report-failure)

  report-failure =
      ?stacktrace
    ; report-run-time
    ; <fprintnl> (stderr(), [ <whoami> (), ": rewriting failed, trace:"])
    ; <reverse ; map(<fprintnl> (stderr(), ["\t", <id>]))> stacktrace
    ; <exit> 1

Applying the modified io-wrap on the following sample program

 
  main = io-wrap(my-wrap(foo))

  my-wrap(s) = s

  foo = debug(!"foo") ; bar

  bar = debug(!"bar") ; fap ; zap

  fap = debug(!"fap") ; id

  zap = debug(!"zap") ; debug ; fail

gives

./prog: rewriting failed, trace:
        main_0_0
        io_wrap_1_0
        option_wrap_5_0
        lifted144
        input_1_0
        lifted145
        output_1_0
        lifted0
        my_wrap_1_0
        foo_0_0
        bar_0_0
        zap_0_0

Due to the compiler lifting inner strategies into freshly named, top-level strategies, the trace will contain some lifted* entries. Also, should you call strategies or rules which are compiled with older versions of the compiler, there will be "dark spots" in your trace. It won't be truncated -- only the frames due to the old library will be hidden.

Karl Trygve Kalleberg: Would you like a stack trace with your "rewriting failed"?

Mon, 10 Mar 2008 16:49:24 +0000

Prompted by my visit to EPITA, I hacked together some very basic support for stack traces in Stratego that might come in handy when a Stratego program fails.

Here's a simple Stratego program, called prog (which, if you look at it closely, will always fail):

  main = foo

  foo = bar

  bar = fap ; zap

  fap = id

  zap = fail

On the latest and greatest version of the compiler (build 17522 and later), you will get the following trace when this program is executed:

prog: rewriting failed, trace:
        main_0_0
        foo_0_0
        bar_0_0
        zap_0_0

There are a number of caveats with the tracing that I will try to get rid of, and, when there are only very hard problems left, explain myself out of, in a couple of future posts.

Eelco Visser: WebDSL in JastAdd

Thu, 28 Feb 2008 17:00:19 +0000

Earlier this week I was visiting the Programming Tools Group of Oege de Moor in Oxford. The visit was associated with the refactoring project in which I am an official collaborator. The project aims at studying the high-level description of refactorings. The starting point is to combine the strengths of program analysis with attribute grammars and program transformation with (strategic) rewriting based approaches. They are experimenting with the JastAdd-based Java compiler. For me the project is interesting as we are considering to integrate attribute grammars in Stratego. To start exploring this topic I've started writing an implementation of WebDSL in JastAdd together with Torbjörn Ekman, the main developer op JastAdd, who is a postdoc in Oxford. We got to implement a basic version of the data model sub-language complete with modules and code generation (using the StringTemplate library of Terence Parr). I'm planning to finish the implementation of the data model and use that as the basis for my lecture on JastAdd in the program transformation course. (And then hand the implementation to the students who can then add the implementation of (a subset of) the UI language.) We had further discussions about the relation of between JastAdd and Stratego; although quite different at first sight, there are interesting similarities (that I hadn't realised before). More to follow about that in the future.

Karl Trygve Kalleberg: Visit to EPITA

Wed, 27 Feb 2008 17:55:33 +0000

I visited Akim Demaille and his posse at EPITA today, and apparently there still is such as thing as free lunch (although, in my excitement over the good food, I kinda promised to help out with fixing some Stratego issues they are experiencing, so it was not entirely without entanglements).

I got to sit in on one of the bi-weekly status updates for the LRDE. The room numbered a little under 30 people, including students and faculty. They were kind enough to hold the meeting in English so that I could follow it. I found it surprising and very encouraging to have everybody report their progress (and, in a very few instances, lack thereof) in front of the entire lab. I've been missing this in many of the institutions I've been working at. It certainly increases the level of team feeling, and also makes it easier to uncover opportunities for collaboration between the various groups. For example, they all shared a lot of common infrastructure, including setups for newsgroups, a build farm, svn repos, etc.

I met two of the guys from the "previous" Transformers generation, Florian and Maxime. Florian was putting the finishing touches on a visualization tool for ambiguities in Transformers' attributed parse trees. It looked pretty sweet. Maxime was hacking a translator from a DSL for their Olena image processing library.

I also got to meet the new generation of Transformer students. I expect that I'll interact a lot more with them in the coming months, as they come to grips with Stratego.

Eelco Visser: WebDSL at Code Generation 2008

Thu, 21 Feb 2008 21:27:09 +0000

My proposal for a tutorial on 'WebDSL: A Case Study in Domain-Specific Language Engineering' has been accepted by the organizers of Code Generation 2008. This conference emerged from the codegeneration.net site, which collects information about code generation techniques and tools. As opposed to the conferences I usually visit, this one attracts quite a crowd from industry, I understand. Last year's event was quite a success, according to attendees I talked to, so I'm looking forward to event in general, and the opportunity to present Stratego/XT, SDF, and WebDSL to industry, in particular. Now think how to squeeze that into 75 min;)

Eelco Visser: Pull Deployment of Services

Wed, 13 Feb 2008 10:37:32 +0000

The NWO/EZ Jacquard Software Engineering Program has granted the project Pull Deployment of Services for an amount of 368K Euro which should pay for a PhD student (4 years) and a postdoc (3 years). In the project for which I am principal investigator, we collaborate with Merijn de Jonge from Philips Research and the buildfarm project at TU Delft in which software deployment expert Eelco Dolstra is postdoc. Here's the text from the proposal summary:

Hospitals are complex organizations, requiring the coordination of specialists and support staff operating complex medical equipment, involving large data sets, to take care of the health of large numbers of patients. The information technology infrastructure of hospitals is heterogeneous and may consist of thousands of electronic devices, ranging from workstations to medical equipment such as MRI scanners. These devices are connected by wired and wireless networks with complex topologies with different security and privacy policies applicable to different nodes. Software deployment in such a heterogeneous environment is inherently difficult. In order to make health-care professionals more effective and deployment and maintenance more tractable, the hospital information technology infrastructure is changing from a device-oriented to a service-oriented environment, in which the access to services is decoupled from the physical access to particular devices.

In this project, we propose a pull model for service deployment in which the components comprising a service are distributed over nodes in the network, depending on the network topology, properties of the application, and quality of service requirements. The goal of this project is to expand the state-of-the-art in software deployment to support pull deployment of services. In order to realize this goal we will conduct research in (1) modeling of services and network architectures, (2) technology for distributed deployment, and (3) tools for testing implementations of distributed services. We will build on our previous research in software deployment (Nix) and model-based software development (Stratego/XT). The project will be conducted in close collaboration with Philips as industrial partner and will consist of a series of experiments building prototype systems which implement service distribution scenarios of increasing complexity.

Karl Trygve Kalleberg: FOSDEM 2008

Sun, 10 Feb 2008 13:27:13 +0000

I'm going to FOSDEM again this year. A bunch of old friends will be coming, so the opportunity is too good to pass up. Also, since I'll be in Paris at the time around FOSDEM, travel is both fast and reasonably cheap. (Three cheers for high speed trains.)

If you're interested in meeting me there, don't hesitate to fire off an e-mail. There's no Gentoo room this year, so I'll be hanging around elsewhere. I'm bound to drop by the Free Java devroom, for sure:) Another gang I'm anxious to meet again are the Nix people.

Eelco Visser: Program Transformation & Generation

Tue, 05 Feb 2008 20:30:00 +0000

Today I started teaching a new master's course on Program Transformation & Generation at Delft University. The course studies techniques principles, techniques, applications of program transformation and generation. Using WebDSL as case study, several paradigms for implementing domain-specific languages will be studied, including term rewriting (Stratego), attribute grammars (Eli, JastAdd), and graph transformation. This is a departure from earlier courses I taught at Utrecht University about same subject. There I would spend a full quarter teaching on just Stratego/XT, which I felt was necessary to prepare master's students for a master project in this area. With the current state of documentation of Stratego/XT, it appears that such a in depth course is no longer necessary. At least, that is the experience with several (PhD) students who recently started developing Stratego applications succesfully without any prior training.

Eelco Visser: Generating Editors for Embedded Languages

Mon, 04 Feb 2008 14:15:00 +0000

The paper Generating Editors for Embedded Languages. Integrating SGLR into IMP by Lennart Kats, Karl Trygve Kalleberg, and Eelco Visser has been accepted by Language Descriptions, Tools, and Applications (LDTA'08) to be held in Budapest, Hungary in April 2008 as part of ETAPS'08. The paper reports on the succesful integration of the SGLR parser in the IMP framework for building language-specific Eclipse plugins. Through this integration the capability of SDF/SGLR to support language embedings is extended to the IDE. This project is a first step towards generation of full fledged IDEs from SDF/Stratego language definitions. From the abstract:

Integrated Development Environments (IDEs) increase productivity by providing a rich user interface and rapid feedback for a specific language. Creating an editor for a specific language is not a trivial undertaking, and is a cumbersome task even when working with an extensible framework such as Eclipse. The IMP framework relieves the IDE developer from a significant portion of the required work by providing various abstractions for this. For embedded % domain-specific languages, such as embedded regular expressions, SQL queries, or code generation templates, its LALR parser generator falls short, however. Scannerless parsing with SGLR enables concise, modular definition of such languages. In this paper, we present an integration of SGLR into IMP, demonstrating that a scannerless parser can be successfully integrated into an IDE. Given an SDF syntax definition, the \textsc{sdf2imp} tool automatically generates an editor plugin based on the IMP API, complete with syntax checking, syntax highlighting, outline view, and code folding. Using declarative domain-specific languages, these services can be customized, and using the IMP metatooling framework it can be extended with other features.