Copyright © 2000-2004 Russell Smith.
Last modified Wednesday November 21 2001.

Lua-RPC

Lua is a scripting language that is simple, embeddable, and very flexible. Lua-RPC is a Lua library to allow simple remote procedure calling. That is, a Lua script running on one machine can call functions from a script that is running on another machine.

What's with the name? Lua ``procedures'' are actually called functions - so why didn't I call this library RFC? Well, RPC is also the name of another standard protocol that does a similar job (it is well known in the networking world), whereas RFC's are something quite different...

Download Lua-RPC

Lua-RPC versions: 0.15, 0.16. These are pre-beta releases. Lua-RPC is still under development. It should compile with an ANSI-C compiler under Unix and Windows. It works with Lua version 4.0.

License

Lua-RPC is distributed under the same license as Lua itself, except for the copyright. Lua-RPC is free software: it can be used for both academic and commercial purposes at absolutely no cost. However, if you use it a small credit in the documentation would be appreciated.

Lua-RPC is Copyright © 2001 Russell L. Smith. All rights reserved.

Want to help?

If you would like any of your modifications to Lua-RPC to become part of the public distribution, or if you just have some good ideas, then please write to me. Uphold the spirit of free software!

How to use it

In your embedded Lua application, include the header file 
    #include "luarpc.h"
and then call the following somewhere to register the Lua-RPC functions: 
    lua_RPClibopen (L)

In your Lua server-side (``slave'') script, you must listen for incoming RPC connections and service function requests. You can do this two ways. The simplest way is to call: 
    RPC_server (port_number)
which will listen for TCP connections on the given port number and service incoming requests automatically. This function will only return if there is an error. If you want to execute server-side Lua code as well as service function requests, do the following: 
    handle = RPC_listen (port_number)
    if not handle then
      -- there was an error
    end
    while 1 do
      -- see if there is a function request pending
      p = RPC_peek (handle)
      if p then
        -- process one function request
        RPC_dispatch (handle)
      else
        -- do some other stuff here
      end
    end
    -- stop the RPC server
    RPC_close (handle)

In your Lua client-side (``master'') script, call 
    handle = RPC_open (host_name, ip_port)
to open an RPC connection to the given host on the given port. The host name may also be an IP address written like "192.168.1.99". This function will return a handle to the RPC connection.

While the server connection is open, Lua functions in the remote script can be called like this: 
    return_value = handle.function_name (arg1, arg2, ...)
The syntax ``handle.name'' returns a userdata value that behaves like a function. When you call this value (by supplying some arguments), a function call request is made to the remote server, and the values are returned. The behavior is exactly like calling any other Lua function.

When you have finished calling functions in the remote script, call: 
    RPC_close (handle)
This will close the connection to the remote server. Any attempt to use the handle to call more functions will result in a runtime error.

The server only accepts connections from one client at a time. This means that a client can effectively ``lock'' the server simply by opening a connection to it. Thus client accesses to the server will be serialized. If you do not close the handle, other clients will not be able to access the server.

Error handling

Programming errors, such as supplying the wrong kinds of arguments to a function, are dealt with as normal Lua script errors. Other errors that are beyond the control of the local script are dealt with differently. These include:
  • Errors with the RPC network connection.
  • Errors that occurred during execution of the remote function.
When one of these errors occurs, one of two things occurs:
  1. If an error handling function has been registered by the user, it is called with an error message string, and then control is returned to the user's script.
  2. If no error handling function has been registered, the normal Lua error handling mechanism is used. This normally does not return control to the user's script.
When a network errors occurs the network connection is closed. Note that script errors in the remote function are propagated to the local script, which can sometimes cause confusion.

To register an error handling function, do this: 
    function error_handler (message_string)
      write ("Error: " .. message_string .. "\n")
    end

    RPC_on_error (error_handler)
To un-register the error handling function, do this: 
    RPC_on_error (nil)

Asynchronous function calls

The default remote function call waits for the return values from the server before control is returned to the client script. But often you want to start a server function that executes concurrently with the client script. To do this, use the RPC_async() function: 
    -- open a connection to the RPC server (the slave)
    handle = RPC_open (host,port)

    -- set the handle to "asynchronous" mode
    RPC_async (handle,1)

    -- now these function calls will return local control
    -- immediately, and they will be queued for processing
    -- on the server.
    handle.function1 (arg1, arg2, ...)
    handle.function2 (arg1, arg2, ...)

    -- set the handle to "synchronous" mode. now function
    -- calls will only return locally after they have
    -- finished executing on the server.
    RPC_async (handle,0)	-- use 0 or nil
Note that in asynchronous mode the return values are thrown away, so the function will return nil to the client. Server function errors will still be passed back to the client however - but as an error may not be received for some time, it may be associated with a later remote function call. This can be very confusing.

Notes

We could have a stateless scheme where function requests are sent as UDP packets. This would simplify things somewhat:
  • We don't need to open/close a handle any more, so the user code is simpler.
  • UDP is faster than TCP, so the function latency is reduced.
But this also has some annoying consequences:
  • We must guarantee delivery of all messages, so we have to make a robust ACKing / retransmission scheme, which is not a trivial undertaking.
  • Large function calls wont fit into a single UDP packet.
  • A stateless protocol gives us no way to ensure that we are getting exclusive use of the server - several clients could be talking to it simultaneously. This could be a problem in many applications where the server maintains an internal state. The TCP based protocol gives us a very convenient way to lock down the server while we are talking to it.

Problems

Currently circular references in tables are not handled well. If you provide such a structure to an Lua-RPC function call, you will get a stack overflow error.

The TO-DO list

  • Need a way to specify timeouts and such things, to make things robust.
  • Handle circular references in tables when dumping to a socket. Keep a table of tables already written?
  • Optimizations:
    • Encode shorter numbers with fewer bytes: s8,s16,s32,double - encode in type field.
    • Encode string lengths with fewer bytes (u8,u16,u32) - encoded in first byte.
    • Socket reading and writing stuff should use buffers (like FILEs), don't use system calls all the time.
  • Protocol for telling the client when the header or protocol version is bad.
  • Asynchronous client operation when no return arguments are expected.
  • Handle multi-threading on the server, so server can service more that one connection at a time.