Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/openresty/lua-rds-parser

Resty DBD Stream (RDS) parser for Lua written in C
https://github.com/openresty/lua-rds-parser

Last synced: 3 months ago
JSON representation

Resty DBD Stream (RDS) parser for Lua written in C

Awesome Lists containing this project

README 

        
Name

====



lua-rds-parser - Resty-DBD-Stream (RDS) parser for Lua written in C



Status

======



This module is production ready.



Synopsis

========



```lua

local parser = require "rds.parser"



local res, err = parser.parse(rds)



if res == nil then

    error("failed to parse: " .. err)

end



print(res.errcode)

print(res.errstr)

print(res.insert_id)

print(res.affected_rows)



local rows = res.resultset

if rows then

    for i, row in ipairs(rows) do

        for col, val in pairs(row) do

            if val ~= parser.null then

                print(col .. ": " .. val)

            end

        end

    end

end

```



Description

===========



This Lua library can be used to parse the Resty-DBD-Stream formatted data

generated by [ngx_drizzle](https://github.com/openresty/drizzle-nginx-module#readme)

and [ngx_postgres](https://github.com/FRiCKLE/ngx_postgres/) into Lua

data structures. In the past, we have to use JSON as the intermediate data

format which is quite inefficient in terms of both memory and CPU time.



To maximize speed and minimize memory footprint, this library is implemented

in pure C.



Null values in RDS are turned into the light user data `parser.null` (or `ngx.null`)

where "parser" is the module object returned by Lua's "require".



JSON Serialization

==================



If you want to serialize the parsed result into JSON, please

use the [lua-cjson library](http://www.kyne.com.au/~mark/software/lua-cjson.php)

instead of lua-yajl, because lua-cjson is faster than lua-yajl

in many common cases, and more importantly,



```lua

parser.null == cjson.null ~= yajl.null

```



Using with HttpDrizzleModule

============================



To use with `ngx_drizzle`, here is a small example:



```nginx

upstream backend {

    drizzle_server 127.0.0.1:3306 protocol=mysql

                   dbname=ngx_test user=ngx_test password=ngx_test;

    drizzle_keepalive max=10 overflow=ignore mode=single;

}



server {

    ...



    location /mysql {

       drizzle_query $echo_request_body;

       drizzle_pass backend;

    }



    location /api {

       content_by_lua '

           local sql = "select * from cats"

           local resp = ngx.location.capture("/mysql", {

               method = ngx.HTTP_POST, body = sql

           })

           if resp.status ~= ngx.HTTP_OK or not resp.body then

               error("failed to query mysql")

           end



           local parser = require "rds.parser"

           local res, err = parser.parse(resp.body)

           if res == nil then

               error("failed to parse RDS: " .. err)

           end



           local rows = res.resultset

           if not rows or #rows == 0 then

               ngx.say("empty resultset")

               ngx.exit(0)

           end



           for i, row in ipairs(rows) do

               ngx.print("row ", i, ": ")

               for col, val in pairs(row) do

                   if val ~= parser.null then

                       ngx.print(col, "=", val, " ")

                   else

                       ngx.print(col, "=null ")

                   end

               end

               ngx.say()

           end

       ';

    }

}

```



On my machine, GET /api will yield



```

row 1: id=2 name=null

row 2: id=3 name=bob

```



of course, the actual output depends on the structure and contents of the

"cats" table in the mysql database.



You can use this Lua library with the ngx_postgres module in a similar way.



Installation

============



Build requirements



* Lua (http://www.lua.org/),

* or LuaJIT (http://www.luajit.org/)



Gnu make and gcc is required to build this module.



Linux/BSD/Solaris

----------------



```bash

gmake CC=gcc

gmake install CC=gcc

```



Mac OS X

--------



```bash

make LDFLAGS='-bundle -undefined dynamic_lookup' CC=gcc

make install

```



If your Lua or LuaJIT is not installed into the system, specify its

include directory like this:



```bash

make LUA_INCLUDE_DIR=/opt/luajit/include/luajit-2.0

```



You can specify a custom path for the installation target:



```bash

make install LUA_LIB_DIR=/opt/lualib

```



The `DESTDIR` variable is also supported, to ease RPM packaging.



TODO

====



* add support for option "compact" to generate a compact

Lua table for the "resultset" field.



Known Issues

============



* The endianness flag in RDS is not supported yet in this library,

and it will assume it's of the host's endian. So do not

try parsing the RDS stream that is generated by another

machine of a different endian.



Author

======



Yichun "agentzh" Zhang , CloudFlare Inc.



Copyright & License

===================



This module is licenced under the BSD license.



Copyright (C) 2011-2016, Zhang "agentzh" Yichun (章亦春) .



All rights reserved.



Redistribution and use in source and binary forms, with or without

modification, are permitted provided that the following conditions

are met:



* Redistributions of source code must retain the above copyright

notice, this list of conditions and the following disclaimer.



* Redistributions in binary form must reproduce the above copyright

notice, this list of conditions and the following disclaimer in the

documentation and/or other materials provided with the distribution.



THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS

"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT

LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR

A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT

HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,

SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED

TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR

PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF

LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING

NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS

SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.