A Fresh Look at the Forth Dictionary
Wolf Wejgaard
Presented at the EuroFORTH conference 1995
Abstract
Forth obtains its power and productivity from the simple concept of a dictionary. Unfortunately, in conventional Forth the dictionary is only available during the test and run phase of a program. In this paper we show how a simple refactoring of Forth lets us make use of the dictionary also for design and development. In general this allows us to extend Forth systems with software engineering capabilities and it also lets us use a new dynamic method of development. The result is a further increase in productivity and a better handling of the complexity of large projects.
1. The Dictionary
"Forth words are organized into a structure called the dictionary. While the form of this structure is not specified by the Standard, it can be described as consisting of three logical parts: a name space, a code space, and a data space." (ANSI Forth Standard).
And, a dictionary is a "Lexicon, book listing the words of a language (alphabetically) and giving their meanings, pronunciations etc." (The Penguin English Dictionary)
The Forth Standard seems to suggest, that the meaning of Forth words is given by their code, for the interest and use of the CPU (who else needs code?). However, and fortunately, we find in many real Forth dictionaries an additional "logical part".
As figure 1 shows, the name list of real Forth systems includes a text field in addition to the code field. The text field points to a logical text space, which we would like to add with equal weight to the code and data spaces of the standard. The text space is in most systems represented physically by one or more text files or sequences of blocks, but there also exist Forth systems, which include the text space with the dictionary in the memory [1].
The text pointer allows us to easily VIEW (or LOCATE or SEE) the textual definition of a word. This is a very useful feature, but it is severely limited. We cannot look up the definitions of all words at all times, we can only SEE the words that have been compiled into code.
Thus the very useful text pointer only exists when the program has progressed to the test phase. Wouldn't it be a good idea to be able to access every word already in the design stage, independent of the code?
Figure 1: The logical structure of the dictionary of many real Forth systems. The name list contains pointers to both code and text.
2. Compiler or Editor?
Traditionally, in a Forth system the compiler builds the dictionary. Why? How about passing the duty along to the editor?
Let us pursue this idea. For every word that we enter into the program the editor immediately creates a dictionary entry. Thus we have the dictionary available from the very beginning of the program with the text field pointing to the word's definition. The compiler later merely adds the code pointer to the dictionary entry.
3. A new Forth feeling
Imagine what you could do, having a data structure available that points to the text of your program all the time -- independent of the load-ability of the words. Here are some suggestions:
Text management. Direct access to every word and hypertext lookup of definitions are evident. With the addition of further link fields to the dictionary we can present the program in a hierarchical structure that makes the program clear to understand and easy to work on. A browser shows both the large picture and the details. The structure makes it easy to move parts of the program around in order to keep the text clear and understandable. And, the road is wide open for adding software engineering functions to Forth.
Code management. The data structure offers new ways to manage the code. It lets us keep text and code in tune; after changes to the text of a word, its code can be recompiled and substituted in the target program. The state of the program is conserved over changes. We not only spare a full recompilation, we also need not restart and work our way to the place of interest. This provides a new quality of 'direct programming'. Code stripping is simple, you can have lean code in spite of comprehensive utility modules.
There are many more uses -- your imagination is the limit.
4. Fifth and Holon
At least two Forth systems exist in which the editor builds the dictionary: Fifth [1] and Holon [2]. These systems are different in concept, but both are of high practical value and suggest that it is worthwhile to have a fresh look on the Forth dictionary.
5. References
[1] Cliff Click and Paul Snow, Fifth: A Forth based Programming Environment, Proc. Rochester, 1986
[2] Wolf Wejgaard, A Taste of Direct Programming, Proc. FORML, 1994