Programming in Lua : 29.1

This first edition was written for Lua 5.0. While still largely relevant for later versions, there are some differences.
The fourth edition targets Lua 5.3 and is available at Amazon and other bookstores.
By buying the book, you also help to support the Lua project.

Programming in Lua

Part IV. The C API Chapter 29. Managing Resources

29.1 – A Directory Iterator

Previously, we implemented a dir function that returned a table with all files from a given directory. Our new implementation will return an iterator that returns a new entry each time it is called. With this new implementation, we will be able to traverse a directory with a loop like this one:

    for fname in dir(".") do  print(fname)  end

To iterate over a directory, in C, we need a DIR structure. Instances of DIR are created by opendir and must be explicitly released by a call to closedir. Our previous implementation of dir kept its DIR instance as a local variable and closed that instance after retrieving the last file name. Our new implementation cannot keep this DIR instance in a local variable, because it must query this value over several calls. Moreover, it cannot close the directory only after retrieving the last name; if the program breaks the loop, the iterator will never retrieve this last name. Therefore, to make sure that the DIR instance is always released, we store its address in a userdatum and use the __gc metamethod of this userdatum to release the directory structure.

Despite its central role in our implementation, this userdatum representing a directory does not need to be visible from Lua. The dir function returns an iterator function; this is what Lua sees. The directory may be an upvalue of the iterator function. As such, the iterator function has direct access to this structure, but Lua code has not (and does not need to).

In all, we need three C functions. First, we need the dir function, a factory that Lua calls to create iterators; it must open a DIR structure and put it as an upvalue of the iterator function. Second, we need the iterator function. Third, we need the __gc metamethod, which closes a DIR structure. As usual, we also need an extra function to make initial arrangements, such as to create a metatable for directories and to initialize this metatable.

Let us start our code with the dir function:

    #include <dirent.h>
    #include <errno.h>
    
    /* forward declaration for the iterator function */
    static int dir_iter (lua_State *L);
    
    static int l_dir (lua_State *L) {
      const char *path = luaL_checkstring(L, 1);
    
      /* create a userdatum to store a DIR address */
      DIR **d = (DIR **)lua_newuserdata(L, sizeof(DIR *));
    
      /* set its metatable */
      luaL_getmetatable(L, "LuaBook.dir");
      lua_setmetatable(L, -2);
    
      /* try to open the given directory */
      *d = opendir(path);
      if (*d == NULL)  /* error opening the directory? */
        luaL_error(L, "cannot open %s: %s", path,
                                            strerror(errno));
    
      /* creates and returns the iterator function
         (its sole upvalue, the directory userdatum,
         is already on the stack top */
      lua_pushcclosure(L, dir_iter, 1);
      return 1;
    }

A subtle point here is that we must create the userdatum before opening the directory. If we first open the directory, and then the call to lua_newuserdata raises an error, we lose the DIR structure. With the correct order, the DIR structure, once created, is immediately associated with the userdatum; whatever happens after that, the __gc metamethod will eventually release the structure.

The next function is the iterator itself:

    static int dir_iter (lua_State *L) {
      DIR *d = *(DIR **)lua_touserdata(L, lua_upvalueindex(1));
      struct dirent *entry;
      if ((entry = readdir(d)) != NULL) {
        lua_pushstring(L, entry->d_name);
        return 1;
      }
      else return 0;  /* no more values to return */
    }

The __gc metamethod closes a directory, but it must take one precaution: Because we create the userdatum before opening the directory, this userdatum will be collected whatever the result of opendir. If opendir fails, there will be nothing to close.

    static int dir_gc (lua_State *L) {
      DIR *d = *(DIR **)lua_touserdata(L, 1);
      if (d) closedir(d);
      return 0;
    }

Finally, there is the function that opens this one-function library:

    int luaopen_dir (lua_State *L) {
      luaL_newmetatable(L, "LuaBook.dir");
    
      /* set its __gc field */
      lua_pushstring(L, "__gc");
      lua_pushcfunction(L, dir_gc);
      lua_settable(L, -3);
    
      /* register the `dir' function */
      lua_pushcfunction(L, l_dir);
      lua_setglobal(L, "dir");
    
      return 0;
    }

This whole example has an interesting subtlety. At first, it may seem that dir_gc should check whether its argument is a directory. Otherwise, a malicious user could call it with another kind of userdata (a file, for instance), with disastrous consequences. However, there is no way for a Lua program to access this function: It is stored only in the metatable of directories and Lua programs never access those directories.