Technical Note 11

Require revisited: Import

This LTN depends on "loadfile," introduced in Lua 5.0

Abstract

Lua 4.1 introduced the "require" function that loads and runs a file unless it already loaded. Lua 5.0 offers require as a built-in function in its base lib. The require command together with LTN 7 "Modules & packages" offers a basis for simple module support in Lua. This technical note proposes an improved version of require, dubbed "import." The proposed import scheme avoids direct access to the globals, corrects a globals related security loophole and handles cyclic module dependencies gracefully.

The problem

• The global name used by the package might already hold data (a name clash.)
• Users of the package must know to which global name the package assigns its interface.
• Metamethods set on the globals table might interfere with the loading of a module.

• Require relies on the LTN 7 system of global public interfaces as described above and provides no further package management.
• The require call in Lua 5.0 passes its own table of globals on to the required package, thereby offering the package a severe security loophole. Because require is a built-in C function, a "setglobals" call does not apply to require to prevent this.
• If modules require each other (i.e. are cyclically dependendent) then a require call will recurse indefinitely, resulting in a stack overflow.

The solution

• A package returns a "package install" function (PIF) that is in turn called by import.
• A table is passed to the PIF into which the package's public interface should be inserted. This table is then returned as the result of the import call. A package should no longer install a global interface.
• The package name and the full package path are passed as the second and third parameter to the PIF.
• Import imposes its caller's globals on the imported package.
• Import will report an error if a package is used before it is fully imported. This could happen during import of cyclic dependent packages. (A package is "used" if its public interface is accessed.) Packages can be cyclically dependendent without using each other during import. In this case import will succeed without error. (An example is given in section Explanation below.)

The import function could be implemented with the following Lua 5.0 code.

local imported = {}

local function package_stub(name)
  local stub = {}
  local stub_meta = {
    __index = function(_, index)
      error(string.format("member `%s' is accessed before package `%s' is fully imported", index, name))
    end,
    __newindex = function(_, index, _)
      error(string.format("member `%s' is assigned a value before package `%s' is fully imported", index, name))
    end,
  }
  setmetatable(stub, stub_meta)
  return stub
end

local function locate(name)
  local path = LUA_PATH
  if type(path) ~= "string" then
    path = os.getenv "LUA_PATH" or "./?.lua"
  end
  for path in string.gfind(path, "[^;]+") do
    path = string.gsub(path, "?", name)
    local chunk = loadfile(path)
    if chunk then return chunk, path end
  end
  return nil, path
end

function import(name)
  local package = imported[name]
  if package then return package end
  local chunk, path = locate(name)
  if not chunk then
    error(string.format("could not locate package `%s' in `%s'", name, path))
  end
  package = package_stub(name)
  imported[name] = package
  setglobals(chunk, getglobals(2))
  chunk = chunk()
  setmetatable(package, nil)
  if type(chunk) == "function" then
    chunk(package, name, path)
  end
  return package
end

Typical use of import is as follows:

-- import the complex package
local complex = import "complex"

-- complex now holds the public interface
local x = 5 + 3*complex.I

A package should be structured as follows:

-- first import all other required packages.
local a = import "a"
local b = import "b"

-- then define the package install function.
-- the PIF more or less contains the code of a
-- LTN 7 package.
local function pif(Public, path)

local Private = {}

function Public.fun()
  -- public function
end

-- etc.
end

-- return the package install function
return pif

Explanation

Import is almost backward compatible with require. Import will however not define the _REQUIREDNAME global during loading. An "old style" package that does not return a PIF will still be loaded and run but import returns an empty public interface. This will not impact old style code because require has no return values.

Here is an example of two packages mutually importing each other. Because neither one actually uses the other during import, this will not be a problem.

Package "a.lua":

local b = import "b"

local function pif(pub, name, path)

function pub.show()
  -- use a message from package b
  print("in " .. name .. ": " .. b.message)
end

pub.message = "this is package " .. name .. " at " .. path

end

return pif

Package "b.lua":

local a = import "a"

local function pif(pub, name, path)

function pub.show()
  -- use a message from package a
  print("in " .. name .. ": " .. a.message)
end

pub.message = "this is package " .. name .. " at " .. path

end

return pif

And some code importing and running both:

local a = import "a"
local b = import "b"

a.show() -- prints "in a: this is package b at ./b.lua"
b.show() -- prints "in b: this is package a at ./a.lua"

Weaknesses

Conclusion