Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/y10k/rims

Ruby IMap Server
https://github.com/y10k/rims

daemon-process imap-server mailbox rims ruby

Last synced: 8 months ago
JSON representation

Ruby IMap Server

Awesome Lists containing this project

README 

          
RIMS

====



RIMS is Ruby IMap Server.

This gem provides a complete IMAP server by itself.  The IMAP

server can run as a daemon, mailboxes are provided and messages

can be delivered to them.



Installation

------------



Add this line to your application's Gemfile:



```ruby

gem 'rims'

```



And then execute:



    $ bundle



Or install it yourself as:



    $ gem install rims



Simple Usage

------------



Type following to show usage.



    $ bundle exec rims help

    usage: rims command options

    

    commands:

        help               Show this message.

        version            Show software version.

        server             Run IMAP server.

        daemon             Daemon start/stop/status tool.

        post-mail          Post mail to any user.

        imap-append        Append message to IMAP mailbox.

        mbox-dirty-flag    Show/enable/disable dirty flag of mailbox database.

        unique-user-id     Show unique user ID from username.

        show-user-mbox     Show the path in which user's mailbox data is stored.

    

    command help options:

        -h, --help



Tutorial

--------



Something to need for RIMS setup are following:



* IP address of your server to run RIMS.

* At least one pair of username and password.

* Your e-mail client that can use IMAP.



In this tutorial, IP address is `192.168.56.101`, username is `foo`,

and password is `bar`.



### First step



Let's try to start RIMS. Type following on your console. And some

messages are shown at console.



    $ bundle exec rims server -u foo -w bar

    I, [2015-01-24T21:02:37.030415 #24475]  INFO -- : start server.

    I, [2015-01-24T21:02:37.035052 #24475]  INFO -- : open socket: 0.0.0.0:1430

    I, [2015-01-24T21:02:37.036329 #24475]  INFO -- : opened: [AF_INET][1430][0.0.0.0][0.0.0.0]

    I, [2015-01-24T21:02:37.036569 #24475]  INFO -- : process ID: 24475

    I, [2015-01-24T21:02:37.037105 #24475]  INFO -- : process privilege user: toki(1000)

    I, [2015-01-24T21:02:37.037401 #24475]  INFO -- : process privilege group: toki(1000)



Add e-mail account to your e-mail client:



* Username is `foo`.

* IMAP server is `192.168.56.101`. This may be replaced to your server

  hostname or IP address.

* IMAP port number is `1430`. This is default of RIMS.

* IMAP authentication password is `bar`.



If setup is success, empty mailbox named INBOX is shown at new mail

account of your e-mail client.



Last, type Ctrl+C on your console to stop server.



### Configuration file



Password at command line parameter is insecure because password is

peeped from another user using `ps aux`. Username and password should

be written at configuration file.



RIMS configuration file format is YAML. Type following in file of

`config.yml` and save.



```yaml

user_list:

  - { user: foo, pass: bar }

```



And start RIMS with `-f config.yml` option.



    $ bundle exec rims server -f config.yml

    I, [2015-01-26T23:20:24.573462 #6106]  INFO -- : start server.

    I, [2015-01-26T23:20:24.574507 #6106]  INFO -- : open socket: 0.0.0.0:1430

    I, [2015-01-26T23:20:24.581892 #6106]  INFO -- : opened: [AF_INET][1430][0.0.0.0][0.0.0.0]

    I, [2015-01-26T23:20:24.582044 #6106]  INFO -- : process ID: 6106

    I, [2015-01-26T23:20:24.596335 #6106]  INFO -- : process privilege user: toki(1000)

    I, [2015-01-26T23:20:24.596985 #6106]  INFO -- : process privilege group: toki(1000)



If setup is success, empty mailbox named INBOX is shown at mail

account of your e-mail client.



Last, type Ctrl+C on your console to stop server.



### Mail delivery to mailbox



In this section, the way that you deliver mail to mailbox on RIMS is

described. Prepare a sample mail text file that is picked from your

e-mail client. The sample mail file is named `mail.txt` on description

of this section.



Simple way is that you use IMAP APPEND command. `rims` tool has IMAP

APPEND command. Type following on your console.



    $ bundle exec rims imap-append -v -n 192.168.56.101 -o 1430 -u foo -w bar mail.txt

    store flags: ()

    server greeting: OK RIMS vX.Y.Z IMAP4rev1 service ready.

    server capability: IMAP4REV1 UIDPLUS AUTH=PLAIN AUTH=CRAM-MD5

    login: OK LOGIN completed

    append: OK  APPEND completed



The option of `-v` is verbose mode. If you don't need display

information, no verbose option exists. If mail delivery is success,

you will see that message appears in INBOX on your e-mail client.



The advantage of IMAP APPEND is to be able to use it by any IMAP mail

server as well as RIMS. The disadvantage of IMAP APPEND is that it

requires your password. This may be insecure.



Special user is defined to deliver mail to any user's mailbox on RIMS.

By special user, it is possible to deliver mail without your password.

The disadvantage of special user is that it can be used only in RIMS.



At first, you prepare a special user to deliver mail. Type following

in configuration file. And start RIMS.



```yaml

user_list:

  - { user: foo, pass: bar }

  - { user: "#postman", pass: "#postman" }

```



And type following on your console.



    $ bundle exec rims post-mail -v -n 192.168.56.101 -o 1430 -w '#postman' foo mail.txt

    store flags: ()

    server greeting: OK RIMS vX.Y.Z IMAP4rev1 service ready.

    server capability: IMAP4REV1 UIDPLUS AUTH=PLAIN AUTH=CRAM-MD5

    login: OK LOGIN completed

    append: OK  APPEND completed



The option of `-v` is verbose mode. If you don't need display

information, no verbose option exists. If mail delivery is success,

you will see that message appears in INBOX on your e-mail client.



### IMAP well known port and server process privilege



Default port number of RIMS is 1430. IMAP protocol well known port

number is 143. If RIMS opens server socket with 143 port, it needs to

be root user process at unix. But RIMS doesn't need to be root user

process as IMAP server.



To open server socket with well known 143 port at RIMS:



1. RIMS starts at root user.

2. RIMS opens server socket with 143 port by root user privilege.

3. RIMS calls setuid(2). And privilege of process is changed from root

   user to another.

4. RIMS starts IMAP server with another's process privilege.



For example, port number is `imap2` (it is service name of well known

port of 143), process user privilege is `toki` (uid 1000), and process

group privilege is `toki` (gid 1000). Type following in configuration

file.



```yaml

user_list:

  - { user: foo, pass: bar }

  - { user: "#postman", pass: "#postman" }

imap_port: imap2

process_privilege_user: toki

process_privilege_group: toki

```



And type following on your console.



    $ sudo bundle exec rims server -f config.yml

    [sudo] password for toki: 

    I, [2015-01-31T21:32:30.069848 #9381]  INFO -- : start server.

    I, [2015-01-31T21:32:30.070068 #9381]  INFO -- : open socket: 0.0.0.0:imap2

    I, [2015-01-31T21:32:30.070374 #9381]  INFO -- : opened: [AF_INET][143][0.0.0.0][0.0.0.0]

    I, [2015-01-31T21:32:30.070559 #9381]  INFO -- : process ID: 9381

    I, [2015-01-31T21:32:30.070699 #9381]  INFO -- : process privilege user: toki(1000)

    I, [2015-01-31T21:32:30.070875 #9381]  INFO -- : process privilege group: toki(1000)



### Daemon



If RIMS server is started from console terminal, RIMS server process

is terminated on closing its console terminal.  At unix, server

process has to be started as daemon process for the server to keep

running its service.



RIMS server can start as daemon process. Type following on your

console.



    $ sudo bundle exec rims daemon start -f config.yml



`sudo` is required for well known 143 port (see previous section).

Daemon process is started quietly and prompt of console terminal is

returned immediately. But daemon process is running at background.

To see background daemon process, type following on your console.



    $ ps -elf | grep rims

    5 S root      3191  1720  0  80   0 - 23026 wait   21:10 ?        00:00:00 ruby /home/toki/rims/vendor/ruby/2.2.0/bin/rims daemon start -f config.yml

    5 S toki      3194  3191  0  80   0 - 26382 inet_c 21:10 ?        00:00:00 ruby /home/toki/rims/vendor/ruby/2.2.0/bin/rims daemon start -f config.yml



RIMS daemon is two processes. 1st root process is controller process.

2nd process that isn't root is server process. RIMS daemon doesn't

display messages and errors at console. You should see log files to

verify normal running of RIMS daemon.



To see log of controller process, watch syslog at system directory.

Type following on your console.



    $ tail -f /var/log/syslog

    Feb  1 21:10:00 vbox-linux rims-daemon[3191]: start daemon.

    Feb  1 21:10:00 vbox-linux rims-daemon[3191]: run server process: 3194



To see log of server process, watch imap.log at local directory. Type

following on your console.



    $ tail -f imap.log

    I, [2015-02-01T21:10:00.989859 #3194]  INFO -- : start server.

    I, [2015-02-01T21:10:00.990084 #3194]  INFO -- : open socket: 0.0.0.0:imap2

    I, [2015-02-01T21:10:00.990989 #3194]  INFO -- : opened: [AF_INET][143][0.0.0.0][0.0.0.0]

    I, [2015-02-01T21:10:00.991393 #3194]  INFO -- : process ID: 3194

    I, [2015-02-01T21:10:00.991615 #3194]  INFO -- : process privilege user: toki(1000)

    I, [2015-02-01T21:10:00.991703 #3194]  INFO -- : process privilege group: toki(1000)



RIMS daemon process can be controlled from command line tool. Defined

operations are start, stop, restart and status. Start operation is

already described.



Stop operation:



    $ sudo bundle exec rims daemon stop -f config.yml



Restart operation:



    $ sudo bundle exec rims daemon restart -f config.yml



Status operation:



    $ sudo bundle exec rims daemon status -f config.yml

    daemon is running.



    $ sudo bundle exec rims daemon status -f config.yml

    daemon is stopped.



Server Configuration

--------------------



Server options on start may be described at config.yml file.

Config.yml is a YAML format file and its contents are explained

at later.



To start server with config.yml file, type following.



    $ bundle exec rims server -f a_server_directory/config.yml



### Config.yml Parameters





  
base_dir


  
This parameter describes a base directory of server. Mailbox

  data is located at inside of base directory. Default is a parent

  directory of config.yml file. Explicit description of this parameter

  is interpreted as a relative path from a parent directory of

  config.yml file.



  
log_stdout


  
This parameter describes a severity level of logging messages

  that is written to standard output.  See description of Logger class

  for more detail of logging. This parameter is one value selected

  from DEBUG, INFO, WARN, ERROR or FATAL. If QUIET value is chosen,

  standard output logging is disabled. Default is INFO. The danger is

  that password may be embedded in message on user authentication in

  DEBUG logging level.



  
log_file


  
This parameter describes a path of log file. Default is

  "imap.log" under the base_dir. Explicit description of this

  parameter is interpreted as a relative path from a base_dir.



  
log_level


  
This parameter describes a severity level of logging

  messages. See description of Logger class for more detail of

  logging. This parameter is one value selected from DEBUG, INFO,

  WARN, ERROR or FATAL. Default is INFO. The danger is that password

  may be embedded in message on user authentication in DEBUG logging

  level.



  
log_shift_age


  
This parameter describes a number of old rotated log files to

  keep or periodical log rotation. Decimal number is interpreted as a

  number of files to keep. Periodical log rotation is one value

  selected from daily, weekly or monthly. Default is none. See

  description of Logger.new class method for more detail of log

  rotation.



  
log_shift_size


  
This parameter describes a maximum log file size on log file

  rotation. Default is none. See description of Logger.new class

  method for more detail of log rotation.



  
key_value_store_type


  
This parameter describes a type of key-value store. Key-value

  store is used to save a mailbox data. This parameter is only one

  value of GDBM, and it is default value.



  
use_key_value_store_checksum


  
This parameter decides whether to use checksum. This parameter

  is boolean, true or false. If this parameter is true, a mailbox data

  is saved with its checksum to an entry of key-value store, and a

  checksum is checked on loading a mailbox data from an entry of

  key-value store. Default is true.



  
hostname


  
This parameter describes a hostname of server. Default is the

  name displayed by hostname(1) command.



  
username


  
This parameter describes a name of mailbox user. This parameter

  and the next password parameter are the pair. If there are two or

  many users, user_list parameter should be used instead of this

  parameter. At least one user is need to start a server.



  
password


  
This parameter describes a password of mailbox user. This

  parameter and the previous username parameter are the pair. If there

  are two or many users, user_list parameter should be used instead of

  this parameter. At least one user is need to start a server.



  
user_list


  
This parameter describes many users of mailbox. The value of

  this parameter is a sequence of maps. A map in the sequence should

  have two entries, the two entries are user and pass. user entry

  describes name of a user. pass entry describes password of a

  user. At least one user is need to start a server.



  
imap_host


  
This parameter describes hostname or IP address of a server

  socket to listen(2) and accept(2). Default is 0.0.0.0.



  
imap_port


  
This parameter describes IP port number or service name of a

  server socket to listen(2) and accept(2). Default is 1430.



  
mail_delivery_user


  
This parameter describes a special user to deliver mail to any

  user. Password definition of this special user is same to a normal

  user. Default is "#postman".



  
process_privilege_user


  
This parameter describes a privilege user name or ID for server

  process.  When server process starts on root user, setuid(2) is

  called and server process privilege user is changed from root user

  to this parameter user.  Default is 65534 (typical user ID of

  nobody) and should be changed.



  
process_privilege_group


  
This parameter describes a privilege group name or ID for server

  process.  When server process starts on root user, setgid(2) is

  called and server process privilege group is changed from root group

  to this parameter group.  Default is 65534 (typical group ID of

  nogroup) and should be changed.





Mailbox Data

------------



Mailbox data exists under the base directory. Next picture is a

overview of mailbox data filesystem under the base directory.



    a_base_directory

    |

    +-- mailbox.2/

        |

        +-- 2c/

            |

            +-- 26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae/

                |

                +-- message

                |

                +-- meta

                |

                +-- mailbox_1

                |

                +-- mailbox_2

                |

                ...



There is a MAILBOX\_DATA\_STRUCTURE\_VERSION directory under first

inside of the base directory.  When mailbox data structure will be

changed in future, MAILBOX\_DATA\_STRUCTURE\_VERSION directory will be

changed too.  Now latest version is "mailbox.2".



There are user directories under the MAILBOX\_DATA\_STRUCTURE\_VERSION

directory.  A user is identified by unique user ID.  Unique user ID is

a SHA256 HEX digest of a username.  For example, type following to

display a "foo" user's unique user ID.



    $ bundle exec rims unique-user-id foo

    2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae 



First two characters of unique user ID is used as a bucket directory.

Unique user ID substring from 3rd character to last exists as a user

directory under the bucket directory.  Shortcut tool to search a two

level directory of a user under a base directory exists.  For example,

type following to display a "foo" user's directory.



    $ bundle exec rims show-user-mbox a_base_directory foo

    a_base_directory/mailbox.2/2c/26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae



There are three types of files under the user directory.  Three types

are message, meta and mailbox.  Each file is key-value store.  Only

one type of key-value store is available now, it is GDBM.  A GDBM

key-value store file has a filename suffix, the suffix is ".gdbm".

Mailbox data does not depend on a specific type of key-value store.



### Message key-value store



Message key-value store file preserves all messages of a user.  Per

one user, only one file exists about this file type.  Contents of

message key-value store is simple.  A key is a message ID, and a value

is message text.  A message ID is a unique number of a message in RIMS

internal.  For example, type following to see overview of contents at

a message key-value store.



    $ bundle exec rims debug-dump-kvs --dump-size --no-dump-value a_base_directory/mailbox.2/2c/26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae/message

    "2": 21938 bytes

    "1": 126014 bytes

    "4": 22928 bytes

    "0": 6326 bytes

    "3": 65168 bytes



### Meta key-value store



Meta key-value store file preserves all meta data of message and

folders.  Per one user, only one file exists about this file type.

Contents of meta key-value store is complex, and only outline of

content is described here.





  
dirty


  
Dirty flag. If enabled, this flag exists. If disabled, this flag

  doesn't exists. This flag will be enabled on updating mailbox

  data.



  
cnum


  
A change number of mailbox data. If mailbox data is modified,

  this number is increased.



  
msg_id


  
The next number of message ID. Message ID is unique number of

  message in RIMS internal.



  
uidvalidity


  
The next number of uidvalidity. Uidvalidity is unique number of

  mailbox in IMAP.



  
mbox_set


  
Mailbox set of a user.



  
mbox_id2name-#


  
Mapping from mailbox ID to mailbox name.



  
mbox_name2id-#


  
Mapping from mailbox name to mailbox ID.



  
mbox_id2uid-#


  
The next uid at a mailbox. Uid is unique number of message at a

  mailbox in IMAP.



  
mbox_id2msgnum-#


  
Number of messages at a mailbox.



  
mbox_id2flagnum-#-#


  
Number of flags at a mailbox.



  
msg_id2date-#


  
Mapping from message ID to internal date of a message. Internal

  date is a message attribute in IMAP.



  
msg_id2flag-#


  
Set of flags at a message.



  
msg_id2mbox-#


  
Mapping from message ID to mailbox's uid.





For example, type following to see overview of contents at a meta key-value store.



    $ bundle exec rims debug-dump-kvs a_base_directory/mailbox.2/2c/26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae/meta

    "msg_id2mbox-3": 25 bytes: {1=>#}

    "msg_id2date-2": 49 bytes: 2013-11-08 13:34:10 +0900

    "mbox_id2uid-1": 1 bytes: "6"

    "msg_id2mbox-1": 25 bytes: {1=>#}

    "msg_id2flag-2": 6 bytes: "recent"

    "mbox_id2msgnum-1": 1 bytes: "5"

    "uidvalidity": 1 bytes: "2"

    "mbox_id2flagnum-1-recent": 1 bytes: "5"

    "msg_id2date-0": 49 bytes: 2013-11-08 06:47:50 +0900

    "cnum": 1 bytes: "6"

    "msg_id2date-4": 49 bytes: 2013-11-08 11:57:28 +0900

    "msg_id2mbox-2": 25 bytes: {1=>#}

    "mbox_set": 1 bytes: "1"

    "msg_id2flag-0": 6 bytes: "recent"

    "msg_id2flag-4": 6 bytes: "recent"

    "msg_id2date-1": 49 bytes: 2013-11-08 19:31:03 +0900

    "msg_id": 1 bytes: "5"

    "mbox_id2name-1": 5 bytes: "INBOX"

    "msg_id2mbox-0": 25 bytes: {1=>#}

    "mbox_name2id-INBOX": 1 bytes: "1"

    "msg_id2mbox-4": 25 bytes: {1=>#}

    "msg_id2flag-1": 6 bytes: "recent"

    "msg_id2date-3": 49 bytes: 2013-11-08 12:47:17 +0900

    "msg_id2flag-3": 6 bytes: "recent"



### Mailbox key-value store



Mailbox key-value store file preserves key-value pairs of uid and

message ID.  Per one user, plural files exist about this type of file

because plural mailboxes are allowed at one user.  Mailbox key-value

store filenames are "mailbox\_1", "mailbox\_2", ...  And 1,2,... are

mailbox ID.  Contents of mailbox key-value store is simple.  A key is

a uid, and a value is message ID.  A uid is a unique number of a

message in a mailbox in IMAP.  A message ID is a unique number of a

message in RIMS internal.  For example, type following to see overview

of contents at a mailbox key-value store.



    $ bundle exec rims debug-dump-kvs a_base_directory/mailbox.2/2c/26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae/mailbox_1

    "2": 1 bytes: "1"

    "5": 1 bytes: "4"

    "1": 1 bytes: "0"

    "4": 1 bytes: "3"

    "3": 1 bytes: "2"



Contributing

------------



1. Fork it

2. Create your feature branch (`git checkout -b my-new-feature`)

3. Commit your changes (`git commit -am 'Add some feature'`)

4. Push to the branch (`git push origin my-new-feature`)

5. Create new Pull Request