Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/fons/cl-mongo
lisp interface to mongo db
https://github.com/fons/cl-mongo
Last synced: about 1 month ago
JSON representation
lisp interface to mongo db
- Host: GitHub
- URL: https://github.com/fons/cl-mongo
- Owner: fons
- License: mit
- Created: 2010-01-15T12:03:03.000Z (almost 15 years ago)
- Default Branch: master
- Last Pushed: 2022-02-05T03:49:00.000Z (almost 3 years ago)
- Last Synced: 2024-08-01T17:23:01.490Z (4 months ago)
- Language: Common Lisp
- Homepage: fons.github.com/cl-mongo
- Size: 197 KB
- Stars: 142
- Watchers: 19
- Forks: 31
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- License: COPYING
Awesome Lists containing this project
- awesome-mongodb - cl-mongo - Community Common Lisp interface (Libraries / Lisp)
- awesome-mongodb - cl-mongo - Community Common Lisp interface (Libraries / Lisp)
README
# cl-mongo
[](https://gitter.im/fons/cl-mongo?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
## Intro
[mongo is a scalable, high-performance, open source, schema-free, document-oriented database.](http://www.mongodb.org).
This is a common lisp interface that allows you to work with a mongo document database.
This is not a driver (yet) as specified in the mongo documents; There is still some
functionality I need to implement.
Except for gridfs most features have been implemented.
cl-mongo provides the ability to insert, update
and delete documents, indexing, searching using regexs etc.
In addition it supports javascript in the repl (using a commonly
available lisp to javascript compiler). This allows you to do use
mongodb's map-reduce from the lisp repl.
I developed this primarily using *sbcl* and to some extend *clozure
common lisp* on the linux and mac osx platforms.
This should also work on *clisp* and *allegro common lisp* and windows.
### multi-threading
The *do-query* function uses multithreading to process the results of
a batch as a query is in progress. This obviously requires that
multi-threading is supported. I've found that as of now (2010-10-23)
sbcl on mac osx doesn't support multithreading out of the box. So
you'd need to compile a new sbcl with multithreading enabled, which is
easy to do.
## Version
Version 0.9.0
I've added regression testing for all features currently supported by cl-mongo.
## Installation
Use asdf to install cl-mongo. cl-mongo depends on many other lisp
libraries (which in turn have their own dependecies). asdf is not a
dependency manager, so you would need to install all dependencies as
they come up as unavailable when you try to install cl-mongo.
On the other hand, cl-mongo is included in quicklisp, which should make life easier.
## Testing
The cl-mongo-test package contains regression tests for all the features currently supported.
cl-mongo-test is asdf-installable, but does not export it's tests.
To run the quick test, do this :
(use-package :cl-mongo-test)
(quick-test)
Quick test will connected to a locally running mongodb instance. It uses the "foo" collection
in the "test" database.
## A sample session
This connects to the test database on the local mongo server listening on its default port.
(use-package :cl-mongo)
(db.use "test")
Insert a key-value pair as the first document into collection "foo".
`(db.insert "foo" (kv "document" "one") )`
Pretty-print the documents returned by the find command. iter will ensure that the cursor is
fully iterated over.
(pp (iter (db.find "foo" :all)))`
{
"_id" -> objectid(4B5CF28970DFF196A75FE1F0)
"document" -> one
}
Create a document. A document is collection of key-value pairs and a unique identifier called "_id".
`(defvar *DOC* (make-document))`
Add various elements to the document.
(add-element "tag" "key" *DOC*)`
{ DOCUMENT
{
tag -> key
}
}
(add-element "array" (list 1 2 3 "hello") *DOC*)
{ DOCUMENT
{
tag -> key
1
2
3
hello
array -> NIL
}
}
Insert document into the database.
`(db.insert "foo" *DOC*)`
Print the current contents.
(pp (iter (db.find "foo" 'all)))
{
"_id" -> objectid(4B5CF28970DFF196A75FE1F0)
"document" -> one
}
{
"_id" -> objectid(8B508D5CBB5D451D961F046D)
"array" -> [ 1, 2, 3, hello,]
"tag" -> key
}
Bind variable `*DOC*` to the second document returned by the find command,
add an other element and save back to the collection.
(defvar *DOC* (cadr (docs (db.find "foo" :all))))`
(add-element "tags" (list 'good 'bad 'ugly) *DOC*)
(db.save "foo" *DOC*)
(pp (db.find "foo" :all))
{
"_id" -> objectid(4B5CF28970DFF196A75FE1F0)
"document" -> one
}
{
"_id" -> objectid(8B508D5CBB5D451D961F046D)
"tags" -> [ GOOD, BAD, UGLY,]
"tag" -> key
"array" -> [ 1, 2, 3, hello,]
}
Check the status of the server.
(db.use "admin")
(nd (db.run-command :serverstatus))
"ok" -> 1.0d0
"mem" ->
"mapped" -> 80
"virtual" -> 177
"resident" -> 3
"globalLock" ->
"ratio" -> 4.644753171959394d-5
"lockTime" -> 6493354.0d0
"totalTime" -> 1.39799764586d11
"uptime" -> 139799.0d0
## $ syntax
I've added various $.. macros which allow for a more declarative programming style. In stead of
doing something like :
(db.find "foo" (kv "k" (kv "$lte" 3)))
you can use :
(db.find "foo" ($<= "k" 3))
To declare a unique ascending index on "k" in collection "foo" , you can say :
($index "foo" :unique :asc "k")
## What's missing
At least the following is missing :
* Request id/ Response id are left 0 in the header.
* GridFS
* ......
## API documentation
I use Edi Weitz's [documentation template](http://weitz.de/documentation-template/)
to generate an api description based on embedded comments.
## News
2010-10-23
I've improved the read performance several orders of magnitude. This
also reduces the memory foot pront quite a bit.
In addition I've added a multi-threaded reader called do-query.
CL-MONGO - api reference
Abstract
The code comes with
a MIT-style
license so you can basically do with it whatever you want.
Download shortcut: http://github.com/fons/cl-mongo.
Contents
-
Download
-
The CL-MONGO dictionary
-
$
-
$!=
-
$!in
-
$+
-
$-
-
$/
-
$<
-
$<=
-
$>
-
$>=
-
$add-to-set
-
$all
-
$em
-
$exists
-
$in
-
$inc
-
$index
-
$map-reduce
-
$mod
-
$not
-
$pop-back
-
$pop-front
-
$pull
-
$pull-all
-
$push
-
$push-all
-
$set
-
$unset
-
$where
-
*mongo-default-db*
-
*mongo-default-host*
-
*mongo-default-port*
-
*repo-root*
-
add-element
-
cwd
-
date-time
-
db.add-user
-
db.auth
-
db.collections
-
db.collections
-
db.count
-
db.create-collection
-
db.delete
-
db.distinct
-
db.ensure-index
-
db.eval
-
db.find
-
db.indexes
-
db.indexes
-
db.insert
-
db.iter
-
db.next
-
db.run-command
-
db.save
-
db.sort
-
db.stop
-
db.update
-
db.use
-
defjs
-
defsrvjs
-
do-query
-
doc-id
-
docs
-
document
-
generate-readme
-
get-element
-
ht->document
-
install-js
-
iter
-
jsdef
-
kv
-
make-document
-
mapdoc
-
mongo
-
mongo
-
mongo-close
-
mongo-registered
-
mongo-show
-
mongo-swap
-
mr.gc
-
mr.gc.all
-
mr.p
-
nd
-
now
-
nwd
-
pp
-
remove-js
-
ret
-
rm
-
rm-element
-
show
-
time-zone
-
with-mongo-connection
-
Acknowledgements
Download
CL-MONGO together with this documentation can be downloaded from http://github.com/fons/cl-mongo. The
current version is 0.1.0.
The CL-MONGO dictionary
[Macro]
$ &rest args => result
[Macro]
$!= &rest args => result
[Macro]
$!in &rest args => result
[Macro]
$+ arg &rest args => result
[Macro]
$- arg &rest args => result
[Macro]
$/ regex options => result
[Macro]
$< &rest args => result
[Macro]
$<= &rest args => result
[Macro]
$> &rest args => result
[Macro]
$>= &rest args => result
[Macro]
$add-to-set &rest args => result
[Macro]
$all &rest args => result
[Macro]
$em array &rest args => result
[Macro]
$exists &rest args => result
[Macro]
$in &rest args => result
[Macro]
$inc &rest args => result
[Macro]
$index collection &rest args => result
[Macro]
$map-reduce collection map reduce &key query limit out keeptemp finalize verbose => result
Run map reduce on the mongo server. map and reduce are either the names of the
javascript functions, created with defjs or defsrvjs or are function definitions in javascript.
The keywords refer to option available for map reduce in mongo. This returns a result summary document.
When using :keeptemp t without specificing :out the collection is mr.<collection>
[Macro]
$mod &rest args => result
[Macro]
$not &rest args => result
[Macro]
$pop-back &rest args => result
[Macro]
$pop-front &rest args => result
[Macro]
$pull &rest args => result
[Macro]
$pull-all &rest args => result
[Macro]
$push &rest args => result
[Macro]
$push-all &rest args => result
[Macro]
$set &rest args => result
[Macro]
$unset &rest args => result
[Macro]
$where &rest args => result
[Special variable]
*mongo-default-db*
database opened by the default connection
[Special variable]
*mongo-default-host*
host for the default connection.
[Special variable]
*mongo-default-port*
port for the default connection.
[Special variable]
*repo-root*
root of the repository; used for documentation generation
[Generic function]
add-element key value document => result
add element with key and value to a document
[Function]
cwd &key mongo => result
Show the current database.
[Function]
date-time second minute hour day month year &optional time-zone => result
Generate a time stamp the mongo/bson protocol understands.
[Generic function]
db.add-user username password &key mongo readonly => result
Add a user to the database.
[Generic function]
db.auth username password &key mongo => result
authenticate a user with a password
[Generic function]
db.collections &key mongo => result
[Function]
db.collections &key mongo => result
Show all the collections in the current database.
[Generic function]
db.count collection selector &key mongo => result
Count all the collections satifying the criterion set by the selector.
:all can be used to return a count of
all the documents in the collection.
[Generic function]
db.create-collection collection &key => result
create a collection
[Generic function]
db.delete collection object &key mongo => result
Delete a document from a collection. The *document* field is used to identify the document to
be deleted.
You can enter a list of documents. In that the server will be contacted to delete each one of these.
It may be more efficient to run a delete script on the server side.
[Generic function]
db.distinct collection key &key mongo => result
Return all the distinct values of this key in the collection
[Generic function]
db.ensure-index collection keys &key drop-duplicates unique mongo asc => result
Create an index specified by the keys in a collection
[Generic function]
db.eval code &rest rest => result
run javascript code server side
[Generic function]
db.find collection kv &key selector limit skip options mongo => result
Find documents in the collection using the selector specified by kv.
Methods take two keywords. ':limit' sets the maximum number of documents returned. The default is 1.
':skip' sets the number of documents to skip in this query. It's default is 0.
Since the default value of the limit is one, db.find by default is the equivalant of *findOne* in the
mongo documentation.
[Generic function]
db.indexes &key mongo => result
[Function]
db.indexes &key mongo => result
Return all indexes in the database.
[Generic function]
db.insert collection document &key mongo => result
Insert a document in a collection. A document is typically generated by `(make-document)`,
but it can also be a hash table, a key-value pair or kv list (see the kv functions).
[Generic function]
db.iter result &key limit mongo => result
next document iteration
[Generic function]
db.next collection cursor-id &key limit mongo => result
Executes the next call on the iterator identified by cursor-id.
[Generic function]
db.run-command cmd &key arg mongo collection index => result
Run a database command on the server. See the mongo documentation for a list of commands.
For most commands you can just uses the key-value shown in the mongo documentation.
[Generic function]
db.save collection document &key mongo => result
Save a document to the collection. If the document has a unique `_id` value (i.e. if it's generated
by `(make-document)` ) it will be 'upserted' (that is: it will be inserted if the document
doesn't exist). If the document a hash table or a kv set, it will be inserted.
In other words this a a helper-function build around *db.insert* and *db.update*.
[Macro]
db.sort collection query &rest args => result
sort macro : Takes the same arguments and keywords as db.find but converts the query
so it works as a sort. use the :field keyword to select the field to sort on.
Set :asc to nil to reverse the sort order
[Generic function]
db.stop cursor &key mongo => result
Stop iterating and clean up the iterator on the server by making a server call.
[Generic function]
db.update collection selector new-document &key mongo upsert multi => result
In a collection update the document(s) identified by the selector statement.
This method has two keywords. ':upsert' : If t insert the document if the document cannot be
found in the collection. ':multi' : Update all documents identified by the selector.
[Generic function]
db.use db &key mongo => result
Use a database on the mongo server. Opens a connection if one isn't already
established. (db.use -) can be used to go to a previosuly visited database,
similar to cd -.
[Macro]
defjs name args declaration* statement* => result
Define client side javascript. Works like defun; body is in lisp, with parenscript
additions, like return. So (defjs hello (x y) (return (+ x y))) defines an adder.
macro creates a lisp function which sends the javascript function over to the mongo
server to be evaluated. Result is processed and returned to the reader.
This will execute 10 times on the server :
(mapcar (lambda (x) (hello 10 x)) (list 1 2 3 4 5 6 7 8 9 10))
[Macro]
defsrvjs name args declaration* statement* => result
Creates a function which stores and executes javascript on the server. The first time
the function is called the javascript function is stored on the server. Subsequent calls will
call out to the server.
Works like defun; the function body is defined in lisp, with parenscript additions. Since
the body of the function already resides on the server this should have less impact on
the network. Use :install t to reinstall.
[Function]
do-query coll &key map-fn reduce-fn initial-value query mongo limit selector => result
Performs a multi-threaded query on a mongo database. coll is the collection name. The reduce-fn keyword is used to specify a function which
will be called for each member of a batch of data returned from mongodb.
The reduce-fn function is executed while the query for the next batch is in progress. The default for reduce-fn is the identity function.
The reduction keyword is used to specify a function which is executed when the database queries have finished.
It's default implementation is to return a hash table, with the mongodb id as the hash key. The query, mongo, limit and selector keywords are used in the same way as for db.find.
[Function]
doc-id doc => result
return the unique document id
[Function]
docs result => result
Stop the iterator (if any) and return the list of documents returned by the query.
Typical ue would be in conjunction with db.find like so (docs (iter (db.find 'foo' 'll)))
[Standard class]
document
document
Document class. A document consists of key/value pairs stored in a internal hash table plus
an internally generated unique id.
Accessors are : 'elements' which returns the internal hash table;
'_id' which returns the unique id and '_local_id' which if true means that
the document was generated by the client (as opposed to having been read from the server).
[Function]
generate-readme &key path => result
This function generates a README.md file with the latest api description.
The :path keyword specifies the location. It expects a sub-directory <path>/doc.
Api documentation is generated on the fly, by extracting comments from the classes,
generics and fuctions exported in the packages file.
The resulting file is <path>/doc/index.html. <path>/README.md is generated by
appending the api documentation to <path>/doc/readme-base.md.
:path or *REPO-ROOT* are typically set to the root of the repository.
[Generic function]
get-element key document => result
Get an element identified by key from the document.
[Function]
ht->document ht => result
Convert a hash-table to a document.
[Macro]
install-js name => result
Allows server based javascripts the be installed without being run.
[Function]
iter result &key mongo max-per-call => result
Exhaustively iterate through a query. The maximum number of responses
per query can be specified using the max-per-call keyword.
[Macro]
jsdef name => result
Return the body of the javascript function; otherwise nill.
[Generic function]
kv a &rest rest => result
This a helper function for key-value pairs and sets of key-value pairs.
In a key-value pair like (kv key value) the key has to be a string and the
value something which is serializable.
key-value pairs can be combined using kv as well : (kv (kv key1 val1) (kv key2 val2)).
This combination of key-value pairs is equivalent to a document without a unique id.
The server will assign a unique is if a list of key-value pairs is saved.
[Function]
make-document &key oid size => result
Constructor. key ':oid' is a user supplied unique id. An internal id will be generated if none
is supplied.
[Function]
mapdoc fn document => result
[Standard class]
mongo
Encapsulates the connection to the mongo database.
Each connection is a added to a global registry.
[Generic function]
mongo &key host port db name host port db name => result
This method returns the connection referred to by
the name identifier from the connection registry. The connection name is unique.
If no connection with that name exists, a new connection with the supplied or default
host, port and db parameters will be created. The default host is localhost;
the default port is 27017; the default db is admin.
[Generic function]
mongo-close name => result
Close the connection to the mongo database.
The name should uniquely identify the connection to close.
This is either a mongo object or the name the object is bound to
in the connection registry. To close all open connections use the special symbol 'all
[Generic function]
mongo-registered name => result
Return a conection registered by this name or nil..
[Function]
mongo-show => result
Show all registered connections and their session id
[Generic function]
mongo-swap left right => result
Swap the names of the left and right connections. Typical use would be
`(mongo-swap :default :alt)`. After the function call :default will refer to the connection
previously referred to as :alt. A connection named :default is returned by `(mongo)` and is the default used in the api. The connections are returned in the order they were passed in (but with the names
swapped between them). To re-open a connection you can say
`(mongo-close (mongo-swap :default (mongo :host <newhost> :portid <portid> :name :temp)))`
and a new default connection is registered.
[Function]
mr.gc &key mongo => result
remove the temporary collections created by map-reduce
[Function]
mr.gc.all &key mongo => result
remove the all collections created by map-reduce, temporary as well as permanent
[Function]
mr.p results => result
show the contents of the results collection map-reduce created
[Function]
nd result &key stream => result
Pretty-print for non-document responses, like the response to a database command.
[Function]
now => result
Return the current date and time in bson format.
[Function]
nwd => result
Show the database set by the `(db.use -)` command
[Generic function]
pp result &key nd stream => result
Pretty-print the results returned from a query. To be used on the repl.
This will format each server reply as if it were a document. This is obviously
ok in mosty cases. See nd for an alternative.
[Macro]
remove-js name => result
[Function]
ret result => result
return value bound to retval in a return document. Used for db.count, db.distinct, functions etc..
[Function]
rm collection query &key mongo => result
Delete all the documents returned by a query. This is not an efficient
way of deleting documents as it invloves multiple trips to the server.
Mongo allows for execution of java-script on the server side, which provides
an alternative. Typical use would be (rm (iter (db.find 'foo' (kv 'key' 1)))),
which deletes all documents in foo, with field key equal to 1.
[Generic function]
rm-element key document => result
Remove element identified by key from a document
[Generic function]
show things &key msg nl order => result
Print a list of things. Things can be users, databases,
collections in the current database,
the profile and more. Things is a keyword so (show :users) will show all users.
[Function]
time-zone => result
Set the time zone appropriate for the current environment.
[Macro]
with-mongo-connection args &rest body => result
Creates a connection to a mongodb, makes it the default connection
and evaluates the body form.
args uses the same keyword set as mongo (:db :localhost :port)
args is passed on to make-mongo when the connection is created.
Acknowledgements
This documentation was prepared with DOCUMENTATION-TEMPLATE.
$Header: /usr/local/cvsrep/documentation-template/output.lisp,v 1.14 2008/05/29 08:23:37 edi Exp $