GeL: a Lua binding for Gema

| Introduction | Usage | Functions | Examples |

1. Introduction   top next

 gema actions are enough for everyday usage but occasionally you may need more power to define a complex transformation.

  You might, also, want to use the superior matching capabilities of gema to drive your program.

  GeL is a Lua 5 library that:

• allows the execution of Lua function (and hence of a C function) in a gema action
• provides the powerful text matching mechanisms offered by gema as a set of Lua functions

  it has been tested with Lua 5.0.2.

  Refer to the detailed documentation to build gel togheter with gema, .

  1.1 Status

  GeL is not as mature as gema or Lua, please report any bugs you may find using SourceForge bug tracking subsystem (you don't need to log in as a SourceForge user) or sending an e-mail to Remo Dentato.

  1.2 Licence

GeL is released under the same licence of Gema. No specific reference needs to be made to GeL as long as the due reference to Gema and Lua have been made.

2. Usage   prev top next

  2.1 Gema like

  Gel may be used exactly as if it was gema, any valid gema script should work when executed by gel:

  gel [gemaopts] [-f rules.gema] [-l script.lua] [infile [outfile]]

. The only differences are:

•  The @lua{} function is available to execute lua scripts in actions
•  Text enclosed between "![" and "!]" is considered lua code and is immediately executed

  Rules specfied with '-f' are loaded, the lua script specified with '-l' is executed (possibily defining lua function to be used in some gema action) and the infile is translated to the outfile applying the rules defined in the default domain.

  All the gema options are recognized.

  2.2 Lua like

  Gel may be used as a Lua (non interactive) interpreter:

  gel [gemaopts] [-f rules.gema] -lua script.lua [args]

  Rules specfied with '-f' are loaded then the control is passed to the lua script specified with '-lua'. No input file is opened and it's a script responsability to begin the transformation process.

  The optional arguments args are passed to the script in the global table arg:

• arg.n will contain the number of arguments;
• arg[0] will contain the name of the script;
• arg[x] with x between 1 and arg.n, the arguments.

  All the gema options are recognized.

  2.3 Gua

  Gua is a stand-alone Lua interpreter linked against the gel library.

  gua [luaopts] [script.lua [args]]

  I added gua to this distribution to show how easy is to embed gel functions in your lua enabled programs.

 The only difference between gua.c and the original lua.c you can find in the Lua 5.0 distribution, is the addition of the function call gel_init(l) at line 386.
Actually there is another difference that it's not related to gel, I had to add a fflush(stderr) to the l_message() function to fix a problem with the mingw compiler.

  2.4 libgel

  All the gel function are available as static library named libgel.a on Unix or libgel.lib on Windows. The building process will automatically generate it.

3. Functions prev top next

  3.1 Gema

  Some new functions havr been added to gema:

@lua{string}

that executes string as a piece of lua code. Returns a value only if the lua code explicitly uses return:
 

• @lua{f()} returns nothing
• @lua{return f()} returns the first value returned by f().

 

@rules{group}

Reports the status of the nth group of rules. Two new escape sequences ( \Q and \q) have been added to create groups of rules; up to 8 groups can be created numbered from 0 to7.
To understand this new feature, imagine you want to use different rules to parse a block of text. A possible solution is to set a variable and then use @cmpn to have different behaviors with respect to the same match.
Instead, using \Q in a pattern means that that a match should fail if the specified group is disabled.
 

@rules{group;switch}

Modify the status of the nth group:
      0: turn off
      1: turn on
      x: toggle
 

@line{mark}

the @line function has been extendend to support the \M escape sequence
 

@column{mark}

the @column function has been extendend to support the \M escape sequence
 

  3.2 Lua

  Lua functions can be loaded in three ways:

• using the -l option in the command line;
• executing @lua{dofile("filename")} in an action;
• embedding the lua code in the gema rule file between "![" and "!]".

   The examples in the next section should clarify how to use those three methods.

   There are new lua functions to interact with gema environment:

gel.set(varname,value)

Set the value of varname (same as @set{}).
 

gel.get(varname)

Get the value of varname (same as @var{} in gema). Returns nil if the variable is unassigned.
 

gel.push(varname,value)

Use varname as a stack and push a value on top (same as @push{}in gema).
 

gel.pop(varname)

Use varname as a stack and pop the value on top (same as @pop{}in gema). It also returns the popped value.
 

gel.write(string)

Write a string at the current position in the gema output (only works if executed in a function called by gema with @lua{}).
 

gel.parsestring(string,[domain,[output]])

Translate the string using the rules defined into the named domain and write the result to the output file. The output parameter can be the name of a file to be created, or nil meaning that the result should be returned as the value of the function.
If output is "-" the translation will be written in the current gema output.
If domain is "" or nil the default domain will be used.
 

gel.parsefile(file,[domain,[output]])

Same as gel.parsestring but the text to be translated is taken from the specified file.
 

gel.parsestream(stream,[domain,[output]])

Same as gel.parsestring but the text to be translated is taken from the specified stream.
 

gel.streamfile(filename)

Create, from the specified file, a stream to be used with gel.parsestream()
 

gel.streamstring(string)

Create, from the given string, a stream to be used with gel.parsestream()
 

gel.streamclose(stream)

Close the given stream.
 

gel.streameof(stream)

Check if there is some text left in the stream.
 

gel.streamgetc(stream)

Get next char from the stream.
 

gel.streampeek(stream)

Returns te next char from the stream without advancing the stream.
 

gel.rulefile(file)

Read new patterns from the specified file. In gema it would be @define{@read{file}}
 

gel.rulestring(string)

Defines new patterns. Equivalent to @define{string}.
 

gel.ruleclear(domain)

Delete all patterns defined in the specified domain.

  4. Examples prev top

  Here you will find some short examples on:

• Definining lua functions
• Returning values from lua
• Accessing gema variables
• Parsing from Lua

  The best way to see GeL in action is to have a look at the example directory in the distribution package.

   4.1 Defining lua functions

  There are three methods for having lua functions defined:

1. Write the functions in a file and execute it through the command line -lua option
2. Write the functions in a file and execute in an action through the @lua{} function. This method also allows you to load a different set of lua functions depending on a match.
3. Embed the lua functions in the gema file using the new syntax "![" "!]"

  Here you will find the same task solved using the three methods.

  Consider the following task as an example of a function not available in gema (it could have been any other function not easily definable in gema):

• Find numbers in a file and create another file with every number replaced by its logarithm (if n > 0) or by the string "-infty" (if n <= 0).

  Let's assume that the lua function is stored in the inflog.lua file:

function inflog(x)
  if x>0 then return math.log(x)
  else return "-infty"
  end
end

  You can write a file inflog.gma as follows:

<N>=@lua{return inflog($0)}

  And execute it with (method 1):

gel -lua inflog.lua -f inflog.gma dat1.dat dat2.dat

  Or you could write inflog.gma as follows (method 2):

@lua{dofile("inflog.lua")}
<N>=@lua{return inflog($0)}

  And execute it with:

gel -f inflog.gma dat1.dat dat2.dat

  Or you could forget inflog.lua, write inflog.gel as follows (method 3):

![
function inflog(x)
  if x>0 then return math.log(x)
  else return "-infty"
  end
end
!]

<N>=@lua{return inflog($0)}

  And execute it with:

gel -f inflog.gel dat1.dat dat2.dat

  Note that the file extensions are totaly unimportant.

   4.2 Returning values from lua

  Normally return values of lua functions are discarded. You can have the first return value as result of the @lua{} function using an explicit return statement.

  Another method is to use the gel.write function.

  To show how those two methods are used, let's consider the following task:

• Collect identifiers and sourround them with "<id>" and "</id>" tags. If it's not the first time you encounter that identifers, add the attribute "first" to the id tag with the line where you first encountered it as value. At the end, write a sorted index of identifiers with the line where they first appear.

and a possible solution where both return and gel.write are used:

![
id_ln={}; n_id={}; table.setn(n_id,0)
 
function check_id(id,ln)
  local t=""
  if id_ln[id] then
    t=" first=\""..id_ln[id].."\""
  else
    id_ln[id]=ln
    table.insert(n_id,id)
  end
  return "<id"..t..">"..id.."</id>"
end
 
function sort_id()
  table.sort(n_id)
  gel.write("\n")
  for n,id in pairs(n_id) do
    gel.write(id.." "..id_ln[id].."\n")
  end
end
!]

<I>=@lua{return check_id("$0",@line)}
\Z=@lua{sort_id()}

   4.3 Accessing gema variables

  Gema has separate functions to manipulate variables, command line switches and command line parameters. Gel provides a function to get a value from lua and a function to set it. If the given name starts with "-" it is considered a parameter or a switch (in that order).

  The following table summarize the equivalence between Gel and Gema constructs.

              *Not yet implemented.

 4.4 Parsing from Lua

  Gel provides three lua functions to translate text according to gema rules:
gel.parsestring() and gel.parsefile()and gel.parsestream().

![
list_type = ""     -- Set when the start of list is found



function li_dot(dot)
  gel.write("<ld>")
  if list_type == "desc" then
    gel.parsestring(dot,"-","par")
  elseif list_type == "enum" then
    n=gel.get{"i"}
    gel.write(n)
    gel.set("i",n+1)
  else
    gel.write(dot)
  end
  gel.write("</ld>\n")
end
!]

list:\N\W[*]*\P\N\W[\G=<li>@lua{li_dot("$1")}@par{$2}
 

Gel was made by Remo Dentato and enahanced by Reuben Thomas.