Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mkoeppe/cl-bibtex

A compatible re-implementation of the BibTeX program in Common Lisp, with a BST-to-CL compiler. By Matthias Köppe
https://github.com/mkoeppe/cl-bibtex

Last synced: 23 days ago
JSON representation

A compatible re-implementation of the BibTeX program in Common Lisp, with a BST-to-CL compiler. By Matthias Köppe

Awesome Lists containing this project

README 

        
A BibTeX system in Common Lisp

******************************



CL-BibTeX is a replacement for the BibTeX program, written in

Common Lisp.  The aim is to enable the user to format bibliographic

entries using Common Lisp programs, rather than using the stack

language of BibTeX style files.  



It is released under the terms of version 2.1 of the GNU Lesser

General Public License as published by the Free Software Foundation

(see file COPYING) or any later version, as clarified by the preamble

found in COPYING-preamble.txt.



(Note: run-lisp is included from CLOCC and has its own license.)



Development now takes place on GitHub at



    https://github.com/mkoeppe/cl-bibtex



A project home page is at



    http://www.nongnu.org/cl-bibtex/



(The old development pages at

http://savannah.nongnu.org/projects/cl-bibtex/ are obsolete.)



There is a mailing list `[email protected]'.

You can subscribe to the list at 



    http://lists.nongnu.org/mailman/listinfo/cl-bibtex-users



Reasons

=======



A powerful tool is needed for dealing with citations in scientific

documents.  BibTeX is good for formatting bibliographies, but

customizing the format is a pain because it requires writing/changing

a program in BAFLL (BibTeX Anonymous Forth-Like Language -- 

Drew McDermott in comp.lang.lisp), which looks like this:



FUNCTION {tie.or.space.connect}

{ duplicate$ text.length$ #3 <

    { "~" }

    { " " }

  if$

  swap$ * *

}



Other citation-related processing like producing a list of authors and

citation indices is usually done with a combination of BibTeX, TeX

macros, and ad-hoc scripting with Perl or AWK.



The new CL-BibTeX system aims to replace all this.



Status

======



CL-BibTeX is a compatible and stable re-implementation of the BibTeX

program, which was written by Oren Patashnik in the WEB language.



It consists of the following components:



 * A reader for BibTeX style files (.bst),

 * a reader and a writer for bibliography databases (.bib),

 * a reader for .aux files,

 * an implementation of BibTeX's built-in functions (like for parsing and

   formatting names, etc.),

 * an interpreter for the stack language of BibTeX style files.



Thus CL-BibTeX allows to both use existing BibTeX style files,

and to write new BibTeX styles in Common Lisp.  



In addition to this, CL-BibTeX contains:



 * A compiler that transforms BibTeX style files into comprehensible

   Common Lisp programs (using type analysis).



   The resulting Common Lisp programs can serve as a starting point

   for the customization of the style.



Future directions of CL-BibTeX's development can be found in the TODO

file.



Where to start

==============



Compile and load all Lisp files, using ASDF or bibtex.system.



The function BIBTEX is like the bibtex program.  Pass the file name

STEM; BIBTEX will read STEM.aux, interpret a BST file, and write

STEM.bbl:



	(bibtex-compiler:bibtex "example")



In fact, when the STEM.aux file requests a bibliography style, BIBTEX

first looks whether a Lisp function that implements the style has been

defined using DEFINE-BIBTEX-STYLE.  Otherwise, it tries to find (using

kpathsearch) and load a Lisp file named "STYLE.lbst"; it is supposed

to define the style using DEFINE-BIBTEX-STYLE.  Finally, BIBTEX tries

to find "STYLE.bst" and interpret it.



A BibTeX style implemented in Lisp is supposed to read all

bibliography files in *BIB-FILES* and to write the formatted

bibliography to the stream *BBL-OUTPUT*.  The package BIBTEX-RUNTIME

contains useful routines for reading AUX files, formatting names, etc.

An important function is READ-ALL-BIB-FILES-AND-COMPUTE-BIB-ENTRIES.  



You can invoke the BST-to-Common-Lisp compiler with the function

COMPILE-BST-FILE:



	(bibtex-compiler:compile-bst-file (kpathsea:find-file "amsabbrv.bst")

					  "amsabbrv.lbst")



The resulting Lisp file is a readable Common Lisp version of the BST

file, which can be run on the AUX file to produce a BBL file, without

using the BST interpreter:



        (compile-file "amsabbrv.lbst")

        (load "amsabbrv.lbst")            ; calls DEFINE-BIBTEX-STYLE



	(bibtex-compiler:bibtex "example")	    ; now uses the Lisp style



There is also a little shell script "bibtex" for invoking CL-BIBTEX

from the shell; it uses the "run-lisp" script from CLOCC.



Restrictions of the compiler

============================



The BST->CL compiler works by analyzing the type of the wizard-defined

functions (i.e., those defined by a FUNCTION command in the BST

file).  The compiler tries to find out how many "literals" (values) a

function pops from the stack and how many it pushes and determines

their types (BOOLEAN, INTEGER, STRING).



* If a wizard-defined function tries to pop a function literal that it

  hasn't pushed, or leaves a function literal on the stack, you lose.

  (In other words, you can't define higher-order BST functions.)



* The two branches of an IF$ function must deliver the same net number

  of values.  (As a special exception, the type of IF$ within a WHILE$

  body which occurs in the FORMAT.NAMES function of the standard

  styles is also supported.  This is reported with a warning message.)



On the compilation of a language that no-one can read or write

==============================================================



The language of BibTeX style files (BAFLL) is expected to be read or

written by "wizards" only.  (This is how the original BibTeX calls the

authors of BibTeX style files.)  In fact, when casual users try to

customize a BibTeX style file, they often come up with slightly broken

styles that appear to work but which fail in corner cases.  The reason

is that programs written in BAFLL can hardly be read, written, or even

debugged. 



This causes an extra difficulty for the compiler, because broken

corner cases tend to misbehave on the stack, rendering the function

unanalyzable, hence uncompilable.  The compiler now contains a

heuristic that tries to fix these situations, but it may fail.  In

this case, you should try to understand the error messages, fix the

BibTeX style file accordingly, and try compiling it again.



Implementation dependencies

===========================



It does not run on some 0.7.? version of SBCL because it does not like

(COPY-READTABLE NIL).  SBCL 0.7.9 seems to be fine.



I noticed that CMUCL 3.0.8 18c+ does not like (PEEK-CHAR T STREAM NIL

#\Space); it won't skip over whitespace when the EOF character is

whitespace?  (I don't use this any more in CL-BibTeX.)



CLISP 2.30:

1. When I invoke COMPILE-BST-FILE, CLISP says "Lisp stack

   overflow. RESET"; this seems to be caused by the pprint-dispatch

   function for DEFINE-BIBTEX-STYLE.

2. The format directive ~<...~:>  (logical block) does not seem to work

   properly.

(I have used reader conditionalization to make CL-BibTeX work with CLISP.)



Have fun!



 -- Matthias Koeppe