{"id":22889196,"url":"https://github.com/rocq-archive/coq-serapi","last_synced_at":"2025-04-04T14:05:06.441Z","repository":{"id":37866307,"uuid":"59719510","full_name":"rocq-archive/coq-serapi","owner":"rocq-archive","description":"Coq Protocol Playground with Se(xp)rialization of Internal Structures.","archived":false,"fork":false,"pushed_at":"2024-11-13T11:15:38.000Z","size":6104,"stargazers_count":128,"open_issues_count":5,"forks_count":40,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-03-28T13:05:52.316Z","etag":null,"topics":["coq","json","machine-learning-api","proof-assistant","protocol","serialization","sexp"],"latest_commit_sha":null,"homepage":"","language":"Coq","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rocq-archive.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-05-26T04:22:15.000Z","updated_at":"2024-11-13T15:06:52.000Z","dependencies_parsed_at":"2023-10-05T10:50:09.328Z","dependency_job_id":"3929665c-d55f-4bf3-8270-735b3da445c7","html_url":"https://github.com/rocq-archive/coq-serapi","commit_stats":{"total_commits":863,"total_committers":27,"mean_commits":"31.962962962962962","dds":"0.17497103128621094","last_synced_commit":"652a8cc88ecf79bf435fcb96363468bb75813a71"},"previous_names":["rocq-archive/coq-serapi","ejgallego/coq-serapi"],"tags_count":73,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rocq-archive%2Fcoq-serapi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rocq-archive%2Fcoq-serapi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rocq-archive%2Fcoq-serapi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rocq-archive%2Fcoq-serapi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rocq-archive","download_url":"https://codeload.github.com/rocq-archive/coq-serapi/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247187538,"owners_count":20898327,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coq","json","machine-learning-api","proof-assistant","protocol","serialization","sexp"],"created_at":"2024-12-13T21:07:10.743Z","updated_at":"2025-04-04T14:05:06.412Z","avatar_url":"https://github.com/rocq-archive.png","language":"Coq","funding_links":[],"categories":[],"sub_categories":[],"readme":"**Note**: Coq SerAPI has now stopped development, the 0.20 release for\nCoq 8.20 will be the last managed by us. Coq SerAPI has been succeeded\nby [coq-lsp](https://github.com/ejgallego/coq-lsp/), which solves many\nlongstanding issues and feature requests.\n\nSee https://github.com/ejgallego/coq-serapi/issues/252 for more\ninformation. The `serlib` component of this repository now lives in\nthe `coq-lsp` repository..\n\nNew maintainers for SerAPI are **much welcome** ! Please see\nhttps://github.com/coq-community/manifesto/issues/160 if you would\nlike to help keeping SerAPI maintained.\n\nWe'd like to thanks all the people that have contributed in one way or\nanother to SerAPI after all these years, without you neither SerAPI or\n`coq-lsp` would have been possible.\n\n## SerAPI: Machine-Friendly, Data-Centric Serialization for Coq\n\n[![Build Status][action-badge]][action-link]\n[![Zulip][zulip-badge]][zulip-link]\n\n[action-badge]: https://github.com/ejgallego/coq-serapi/actions/workflows/ci.yml/badge.svg?branch=v8.16\n[action-link]: https://github.com/ejgallego/coq-serapi/actions/workflows/ci.yml?query=branch%3Av8.16\n\n[zulip-badge]: https://img.shields.io/badge/Zulip-chat-informational.svg\n[zulip-link]: https://coq.zulipchat.com/#narrow/stream/256331-SerAPI\n\nTo install with opam:\n```\n$ opam install coq-serapi\n$ sertop --help\n```\n\nAlternatively, if you use Nix:\n```\n$ nix-shell -p coq_8_13 coqPackages_8_13.serapi\n$ sertop --help\n```\n\nSerAPI is a library for machine-to-machine interaction with the [Coq proof assistant](https://coq.inria.fr),\nwith particular emphasis on applications in IDEs, code analysis tools, and machine learning.\nSerAPI provides automatic serialization of Coq's internal\n[OCaml](https://ocaml.org) datatypes from/to [JSON](https://www.json.org) or\n[S-expressions](https://en.wikipedia.org/wiki/S-expression) (sexps).\n\nSerAPI is a proof-of-concept and should be considered\nalpha-quality. However, it is fully functional and supports, among\nother things, asynchronous proof checking, full-document parsing, and\nserialization of Coq's core datatypes. SerAPI can also be run as\n[WebWorker](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers)\nthread, providing a self-contained Coq system inside the\nbrowser. Typical load times in Google Chrome are less than a second.\n\nThe main design philosophy of SerAPI is to **make clients' lives easy**,\nby providing a convenient, robust interface that hides most\nof the scary details involved in interacting with Coq.\n\nFeedback from Coq users and developers is very welcome and _intrinsic_\nto the project. We are open to implementing new features and exploring\nnew use cases.\n\n### Documentation and Help:\n\n- [Protocol Documentation](http://ejgallego.github.io/coq-serapi/coq-serapi/Serapi/Serapi_protocol/)\n- [interface file](serapi/serapi_protocol.mli)\n- [SerAPI's FAQ](notes/FAQ.md)\n- [technical report](https://hal-mines-paristech.archives-ouvertes.fr/hal-01384408)\n- [issue tracker](https://github.com/ejgallego/coq-serapi/issues)\n- [Zulip chat](https://coq.zulipchat.com/#narrow/stream/256331-SerAPI)\n- [Gitter chat](https://gitter.im/coq-serapi/Lobby) channel (legacy)\n- [mailing list](https://x80.org/cgi-bin/mailman/listinfo/jscoq)\n\n**API WARNING:** _The protocol is experimental and may change often_.\n\n### Quick Overview and Install:\n\nSerAPI can be installed as the [OPAM](https://opam.ocaml.org) package `coq-serapi`\nor the [Nix](https://nixos.org) package `coqPackages_8_13.serapi`.\nSee [build instructions](notes/build.md) for manual installation. The experimental\n[in-browser version](https://x80.org/rhino-hawk) is also online.\n\nSerAPI provides an interactive \"Read-Print-Eval-Loop\" `sertop`, a\nbatch-oriented compiler `sercomp`, and a batch-oriented tokenizer `sertok`.\nSee the manual pages and `--help` pages of each command for more details.\n\nTo get familiar with SerAPI we recommend launching the `sertop` REPL,\nas it provides a reasonably human-friendly experience:\n\n```\n$ rlwrap sertop --printer=human\n```\n\nYou can then input commands. `Ctrl-C` will interrupt a busy Coq\nprocess in the same way it interrupts `coqtop`.\n\nThe program `sercomp` provides a command-line interface to some\nkey functionality of SerAPI and can be used for batch processing\nof Coq documents, e.g., to serialize Coq source files from/to lists of\nS-expressions of Coq vernacular sentences. See `sercomp --help` for some\nusage examples and an overview of the main options. The program `sertok`\nprovides similar functionality at the level of Coq source file tokens.\n\n### Protocol Commands:\n\nInteraction with `sertop` is done using _commands_, which can be optionally tagged in the form of `(tag cmd)`; otherwise, an automatic tag will be assigned.\nFor every command, SerAPI **will always** reply with `(Answer tag Ack)` to indicate that the command was successfully parsed and delivered to Coq, or with a `SexpError` if parsing failed.\n\nThere are three categories of [commands](serapi/serapi_protocol.mli#L147):\n\n- **Document manipulation:** `Add`, `Cancel`, `Exec`, ...: these commands instruct Coq to perform some action on the current document.\n  Every command will produce zero or more different _tagged_ [answers](serapi/serapi_protocol.mli#52), and  a final answer `(Answer tag Completed)`, indicating that there won't be more output.\n\n  SerAPI document commands are an evolution of the OCaml STM API, [here](https://github.com/ejgallego/jscoq/blob/master/etc/notes/coq-notes.md) and [here](https://github.com/siegebell/vscoq/blob/master/CoqProtocol.md) you can find a few informal notes on how it works. We are working on a more detailed specification, for now you can get some more details in the issue tracker.\n\n- **Queries:** `(Query ((opt value) ...) kind)`:\n\n  Queries stream Coq objects of type `kind`. This can range from options, goals and hypotheses, tactics, etc.\n  The first argument is a list of options: `preds` is a list of conjunctive filters, `limit` specifies how many values the query may return.\n  `pp` controls the output format: `PpSer` for full serialization, or `PpStr` for \"pretty printing\". For instance:\n  ```lisp\n  (tag (Query ((preds (Prefix \"Debug\")) (limit 10) (pp PpSexp)) Option))\n  ```\n  will stream all Coq options that start with \"Debug\", limiting to the first 10 and printing the full internal Coq datatype:\n  ```lisp\n  (CoqOption (Default Goal Selector)\n     ((opt_sync true) (opt_depr false) (opt_name \"default goal selector\")\n     (opt_value (StringValue 1))))\n  ...\n  ```\n  Options can be omitted, as in: `(tag (Query ((limit 10)) Option))`, and\n  currently supported queries can be seen [here](serapi/serapi_protocol.mli#L118)\n\n- **Printing:** `(Print opts obj)`: The `Print` command provides access to the Coq pretty printers.\n  Its intended use is for printing (maybe IDE manipulated) objects returned by `Query`.\n\n### Roadmap and Developer organization:\n\nSerAPI is organized in branches corresponding to upstream Coq\nversions; usually, branch v8.XX is compatible with Coq 8.XX, and\ncorresponds to SerAPI 0.XX. These branches are stable and can be\nrelied upon.\n\nThe branch `main` tracks Coq `master` branch, and it is not a stable\nbranch; force pushes and random rebases can happen there; handle with\ncare!\n\nWe are working on fixing this problematic setup, which is that way as\nin the past such branch used to be \"private\", but now that SerAPI is\nin Coq's CI the development workflow has changed, with developer\nsubmitting PRs to it.\n\nThese days, most work related to SerAPI is directly happening over\n[Coq's upstream](https://github.com/coq/coq) itself. The main\nobjective is to improve the proof-document model; building a rich\nquery language will be next. See the [roadmap issue]() in our bug\ntracker for more information about roadmap and the [Developer\nInformation](#Developer-information) section for more details on the\ndevelopment setup.\n\n### Clients and Users:\n\nSerAPI has been used in a few contexts already, we provide a few\npointers here, feel free to add your own!\n\n- [jsCoq](https://github.com/ejgallego/jscoq) allows you to run Coq in\n  your browser. jsCoq is the predecessor of SerAPI and will shortly be\n  fully based on it.\n- [mCoq](https://github.com/EngineeringSoftware/mcoq) is a tool for mutation analysis\n  of Coq projects, based on serializing and deserializing Coq code via SerAPI.\n  See the accompanying [tool paper](https://users.ece.utexas.edu/~gligoric/papers/JainETAL20mCoqTool.pdf),\n  and the [research paper](https://users.ece.utexas.edu/~gligoric/papers/CelikETAL19mCoq.pdf)\n  which describes and evaluates the technique.\n- [elcoq](https://github.com/cpitclaudel/elcoq), an emacs technology\n  demo based on SerAPI by [Clément Pit-Claudel](https://github.com/cpitclaudel). `elcoq` is not fully\n  functional but illustrates some noteworthy features of SerAPI.\n- [PeaCoq](https://github.com/Ptival/PeaCoq), a Coq IDE for the\n  browser, has an experimental branch that uses SerAPI.\n- [GrammaTech's Software Evolution Library (SEL)](https://grammatech.github.io/sel/)\n  provides tools for programmatically modifying and evaluating software. SEL operates\n  over multiple software representations including source code in\n  several languages and compiled machine code. Its Coq module uses\n  SerAPI to serialize Coq source code into ASTs, which are parsed into\n  Common Lisp objects for further manipulation. GrammaTech uses this\n  library to synthesize modifications to software so that it satisfies\n  an objective function, e.g., a suite of unit tests.\n  SerAPI support was added to SEL by Rebecca Swords.\n- [CoqGym](https://github.com/princeton-vl/CoqGym) is a Coq-based learning environment for theorem proving.\n  It uses SerAPI to interact with Coq and perform feature-extraction. Its author notes:\n\n  \u003e CoqGym relies heavily on SerAPI for serializing the internal structures of Coq.\n  \u003e I tried to use Coq's native printing functions when I started with this project,\n  \u003e but soon I found SerAPI could save a lot of the headaches with parsing Coq's output.\n  \u003e Thanks to SerAPI authors, this project wouldn't be possible (or at least in its current form) without SerAPI.\n\n  See also the [paper](https://arxiv.org/abs/1905.09381) describing CoqGym.\n- [Proverbot9001](https://github.com/UCSD-PL/proverbot9001) is a proof search system based on machine\n  learning techniques, and uses SerAPI to interface with Coq.\n  See also the [paper](https://arxiv.org/abs/1907.07794) describing the system.\n- [Roosterize](https://github.com/EngineeringSoftware/roosterize) is a tool for\n  suggesting lemma names in Coq projects based on machine learning.\n  See the [paper](https://arxiv.org/abs/2004.07761) describing the technique and tool.\n  Additional paper with demo: https://arxiv.org/abs/2103.01346 .\n- The [paper](https://arxiv.org/abs/2006.16743) _Learning to Format Coq Code Using Language Models_\n  implements a Coq code formatter.\n- [MathComp corpus](https://github.com/EngineeringSoftware/math-comp-corpus) is a machine learning\n  dataset based on the [Mathematical Components](https://math-comp.github.io/) family of Coq projects,\n  and includes several machine-readable representations of Coq code generated via SerAPI.\n  The dataset was used to train and evaluate the Roosterize tool.\n- A Python interface for SerAPI can be found at [PyCoq](https://github.com/IBM/pycoq)\n- A direct Python interface to Coq, using `serlib` can be found at https://github.com/ejgallego/pyCoq\n- SerAPI is being used to improve the Coq regression proof\n  selection tool [iCoq](https://cozy.ece.utexas.edu/icoq/),\n  see the [paper](https://users.ece.utexas.edu/~gligoric/papers/CelikETAL17iCoq.pdf).\n- SerAPI is being used in additional software testing and machine learning projects; we will\n  update this list as papers get out of embargo.\n\n### Quick Demo (not always up to date):\n\n```lisp\n$ rlwrap sertop --printer=human\n(Add () \"Lemma addn0 n : n + 0 = n. Proof. now induction n. Qed.\")\n  \u003e (Answer 0 Ack)\n  \u003e (Answer 0 (Added 2 ((fname \"\") (line_nb 1) (bol_pos 0) (line_nb_last 1) (bol_pos_last 0) (bp 0) (ep 26))\n  \u003e            NewTip))\n  \u003e ...\n  \u003e (Answer 0 (Added 5 ... NewTip))\n  \u003e (Answer 0 Completed)\n\n(Exec 5)\n  \u003e (Answer 1 Ack)\n  \u003e (Feedback ((id 5) (route 0) (contents (ProcessingIn master))))\n  \u003e ...\n  \u003e (Feedback ((id 5) (route 0) (contents Processed)))\n  \u003e (Answer 1 Completed)\n\n(Query ((sid 3)) Goals)\n  \u003e (Answer 2 Ack)\n  \u003e (Answer 2\n  \u003e  (ObjList ((CoqGoal ((fg_goals (((name 5) (ty (App (Ind ...))))\n                         (bg_goals ()) (shelved_goals ()) (given_up_goals ()))))))\n  \u003e (Answer 2 Completed)\n\n(Query ((sid 3) (pp ((pp_format PpStr)))) Goals)\n  \u003e (Answer 3 Ack)\n  \u003e (Answer 3 (ObjList ((CoqString\n  \u003e   \"\\\n  \u003e    \\n  n : nat\\\n  \u003e    \\n============================\\\n  \u003e    \\nn + 0 = n\"))))\n  \u003e (Answer 3 Completed)\n\n(Query ((sid 4)) Ast)\n  \u003e (Answer 4 Ack)\n  \u003e (Answer 4 (ObjList ((CoqAst ((((fname \"\") (line_nb 1) (bol_pos 0) (line_nb_last 1)\n  \u003e                                (bol_pos_last 0) (bp 34) (ep 50)))\n  \u003e ...\n  \u003e            ((Tacexp\n  \u003e              (TacAtom\n  \u003e                (TacInductionDestruct true false\n  \u003e ...\n  \u003e (Answer 4 Completed)\n\n(pp_ex (Print ((sid 4) (pp ((pp_format PpStr)))) (CoqConstr (App (Rel 0) ((Rel 0))))))\n  \u003e (Answer pp_ex Ack)\n  \u003e (Answer pp_ex(ObjList((CoqString\"(_UNBOUND_REL_0 _UNBOUND_REL_0)\"))))\n\n(Query () (Vernac \"Print nat. \"))\n  \u003e (Answer 6 Ack)\n  \u003e (Feedback ((id 5) (route 0) (contents\n  \u003e    (Message Notice ()\n  \u003e    ((Pp_box (Pp_hovbox 0) ...)\n  \u003e (Answer 6 (ObjList ()))\n  \u003e (Answer 6 Completed)\n\n(Query () (Definition nat))\n  \u003e (Answer 7 Ack)\n  \u003e (Answer 7 (ObjList ((CoqMInd (Mutind ....)))))\n  \u003e (Answer 7 Completed)\n```\n\n### Technical Report:\n\nThere is a brief [technical report](https://hal-mines-paristech.archives-ouvertes.fr/hal-01384408)\ndescribing the motivation, design, and implementation of SerAPI. If you are using\nSerAPI in a project, please cite the technical report in any related publications:\n\n```bibtex\n@techreport{GallegoArias2016SerAPI,\n  title = {{SerAPI: Machine-Friendly, Data-Centric Serialization for Coq}},\n  author = {Gallego Arias, Emilio Jes{\\'u}s},\n  url = {https://hal-mines-paristech.archives-ouvertes.fr/hal-01384408},\n  institution = {MINES ParisTech},\n  year = {2016},\n  month = Oct,\n}\n```\n\n## Developer Information\n\n### Technical Details\n\nSerAPI has four main components:\n\n- `serapi`, an extended version of the current IDE protocol;\n- `serlib`, a library providing automatic de/serialization of most Coq data structures using `ppx_conv_sexp`. This should be eventually incorporated into Coq itself. Support for `ppx_deriving_yojson` is work in progress;\n- `sertop`, `sertop_js`, toplevels offering implementations of the protocol;\n- `sercomp`, `sertok`, command-line tools providing access to key features of `serlib`.\n\nBuilding your own toplevels using `serlib` and `serapi` is encouraged.\n\n### Advanced Use Cases\n\nWith a bit more development effort, you can also:\n\n- Use SerAPI as an OCaml library. The low-level serialization library\n  [`serlib/`](/serlib) and the higher-level SerAPI protocol in\n  [`serapi/serapi_protocol.mli`](/serapi/serapi_protocol.mli) can be\n  linked standalone.\n\n- Use SerAPI's web worker [JavaScript Worker](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers)\n  from your web/node application. In this model, you communicate with SerAPI using\n  the typical `onmessage/postMessage` worker API. Ready-to-use builds\n  may be found at\n  [here](https://github.com/ejgallego/jscoq-builds/tree/serapi), we\n  provide an example REPL at: https://x80.org/rhino-hawk\n\nWe would also like to provide a [Jupyter/IPython kernel](https://github.com/ejgallego/coq-serapi/issues/17).\n\n### Developer/Users Mailing List\n\nSerAPI development is mainly discussed [on GitHub](https://github.com/ejgallego/coq-serapi)\nand in the [Gitter channel](https://gitter.im/coq-serapi/Lobby).\nYou can also use the jsCoq mailing list by subscribing at:\nhttps://x80.org/cgi-bin/mailman/listinfo/jscoq\n\nThe mailing list archives should also be available at the Gmane group:\n`gmane.science.mathematics.logic.coq.jscoq`. You can post to the list\nusing nntp.\n\n## Acknowledgments\n\nSerAPI has been developed at the\n[Centre de Recherche en Informatique](https://www.cri.ensmp.fr) of\n[MINES ParisTech](http://www.mines-paristech.fr/) (former École de\nMines de Paris) and was partially supported by the\n[FEEVER](http://www.feever.fr) project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frocq-archive%2Fcoq-serapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frocq-archive%2Fcoq-serapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frocq-archive%2Fcoq-serapi/lists"}