Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/roualdes/testbank
A database manager for test questions.
https://github.com/roualdes/testbank
Last synced: 23 days ago
JSON representation
A database manager for test questions.
- Host: GitHub
- URL: https://github.com/roualdes/testbank
- Owner: roualdes
- License: bsd-3-clause
- Created: 2016-10-18T20:39:54.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2024-02-24T22:42:21.000Z (9 months ago)
- Last Synced: 2024-04-13T23:54:20.853Z (7 months ago)
- Language: JavaScript
- Homepage:
- Size: 1.76 MB
- Stars: 1
- Watchers: 3
- Forks: 0
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# TestBank
TestBank is hosted at `https://testbank.roualdes.us/`. To generate an instance
of exercise `0Y7x`, run
```
curl https://testbank.roualdes.us/0Y7x
```
Rerun the above command. Notice that there is a random component to
the exercise. This is the crux of TestBank: exercise templates with
random components generated from code run in a
[Jupyter](https://jupyter.org/) backend. Right now, only
[Python3](https://www.python.org/) and [R](https://www.r-project.org/)
are available.
## Examples
The directory `examples` contains two primary examples. First, an
approximate exam from my MATH 314 course at [Chico
State](https://www.csuchico.edu/). The second a working example of how
one could leverage TestBank with
[check50](https://cs50.readthedocs.io/check50/). Each subfolder has
its own README.
The files `exampels/add.r` and `examples/add.json` are intended to
demonstrate the file structure necessary for adding an exercise to
TestBank's databases.
## Data
For now, let's simplify things and store exercises in .json files. I'll use
[lowdb](https://github.com/typicode/lowdb) to interact with the JSON
files. There will be one file for exercises and one file for tags.
The last bit will be how each exercise is returned to the end user.
### Exercise schema
```
{
id: "a unique ID, 4 characters long; [0-9a-zA-Z]{4}", # generated by TestBank
language: python | r,
exercise: "code to produce an exercise",
tags: ["tag1", ..., "tagT"] # planned but not implemented,
}
```
### Tags
If a user searches the database with a complex query such as `tags: tag1 AND (tag2 OR tag3)`, then the following secondary database should
make for more efficient searching.
```
{
tag1: [id1, ..., idA],
tag2: [id1, ..., idB],
...
tagT: [id1, ..., idT]
}
```
After collecting the queried tags' arrays, simple set operations
should enable direct replacement of AND with `&&` and OR with `||`.
### Output schema
Each exercise will be returned from the TestBank server as a JSON
object with one of the two following structures. If an exercise
is requested, the user will receive a JSON object with the following schema
```
{
id: "a unique ID",
seed: 1234,
context: "the context of this exercise",
questions: ["partA", ..., "partZ"],
random: {X: x, μ: m, σ: s} # Associative Array specific to language
# eg R => list(), Python => dict() or {}
}
```
The order of the following elements of an exercise's output schema is not important.
- `id` string. A string of 4 characters, [0-9a-zA-Z]{4}, that uniquely
identifies each exercise. TestBank will generate these autmatically
upon insertion of an exercise into its database.
- `seed` int. A seed for the (pseudo) random number generation which is
constrained to be in `[1, min(R int, or Numpy np.uint32)] = [1, 2147483647]`. If no seed is specified with each request for an
exercise (or its solution), TestBank will randomly chose a seed.
- `context` string. A string the sets up the exercise's context. If no
follow up questions are involved in an exercise, the `context` can
contain the question/prompt.
- `questions` array of strings. The `questions` array holds parts,
say A, B, C, D, and E, of an exercise as strings. While the
`questions` key is required, the value may be an empty array..
- `random` Object, aka associative array. The `random` object holds named
elements of the randomly generated components of an exercise. While
the `random` key is required, the value may be an empty associative
array
If a solution is requested, the user will receive a JSON object with
the following schema
```
{
id: "a unique ID",
seed: 1234,
solutions: 0.314 || "0.314" || ["partA", ..., "partZ"],
random: {X: x, μ: m, σ: s} # Associative Array specific to language
# eg R => list(), Python => dict() or {}
}
```
Again, the order is not important. `solutions` is the only new key.
- `solutions` number, string, or array of strings. If questions is an
array, then `solutions` should be an array containing answers to
each part of the exercise's `questions`. If questions is an empty
array, then `solutions` can be a number or a string, appropriately
matching the prompt of `context`.
## Writing exercises
To write an exercise, you must at least use the following structure,
inclusive of the custom [Mustache](https://github.com/janl/mustache.js) tags
for the exercise's ID:
```
id = '#< ID >#'
... code to produce Output schema
```
These custom Mustache tags have two benefits. First, they help
prevent my text editor from getting confused while developing TestBank
exercises, at least within some fairly standard data science
programming languages, R, Python, Julia. Second, they simplify writing LaTeX
within Python strings, since Python's string formatting insists on
double curly braces (otherwise used in Mustache) to get single curly
braces (used in LaTeX) into a string.
Examples exist in the
[examples](https://github.com/roualdes/testbank/tree/master/examples)
directory.
### Extra Mustache tags
If you want to provide a seed and/or solutions, and keep the solution
hidden from the (not yet developed) website, use Mustache tags that allow TestBank to
selectively ignore the exercise and/or the solution.
```
... code necessary for both exercise and solution
id = '#< ID >#'
seeed = #< SEED >#
#< #exercise >#
... code to produce exercise Output schema
#< /exercise >#
#< #solution >#
... code to produce solution Output schema
#< /solution >#
```
It is possible to ignore the seed template entirely. Simply don't put
the seed template within the exercise's code.
The solution template has two goals. First, to enable a request for a
specific exercise to return only the solution. Second, when a website
that permits searching through TestBank's database is developed, the
solution template is intended to allow for hidden solutions. Thus any
exercise could be searched, found, and read without displaying the
solution.
Examples exist in the
[examples](https://github.com/roualdes/testbank/tree/master/examples)
directory.
## Check an exercise before entering it into the database
The goal is to make entering exercises into the database as automatic
as possible, while simultaneously addressing _security_ of the TestBank
sever, _stability_ of the kernel, and _response_ time.
For instance, if you wanted to add the problem `examples/add.r` (with
associated meta data file at `examples/add.json`) follow these steps.
1. Eyeball code in `add.r` for malicious content.
2. From this project's root directory, run
```
test/exercise.sh examples/add.json
```
If the tests above pass, insert the exercise to TestBank's database
with the `upsert` command from TestBank's CLI.
The directory
[test](https://github.com/roualdes/testbank/tree/master/test) contains
more details on tests.
## TestBank command line interface (CLI)
TestBank's GitHub repository comes with a command line interface,
`cli.js`, which attempts to help with testing exercises and entering
exercises into the database. The `node` commands below must be run
from the root directory of this repository.
### Testing
To test whether or not an exercise will run (after the Mustache tags
are entered), consider the file
[examples/add.r](https://github.com/roualdes/testbank/tree/master/examples/add.r).
At the command line, with
[littler](http://dirk.eddelbuettel.com/code/littler.html)
installed/aliased as `lr` run
```
$ lr -e "$(node cli.js test examples/add.json exercise)"
```
If the above command runs just fine, then there's a hope that your
code will run on the TestBank kernel after being entered into the
TestBank database.
To test a Python solution, run
```
$ node cli.js test examples/ex01.json solution | python3
```
To ensure that the output is a correctly formatted JSON object, run
```
$ node cli.js test examples/ex01.json solution | python3 | python3 -m json.tool
```
To ensure that the output, and correctly formatted, JSON object
appropriately follows the Output schema run
```
$ node cli.js test examples/ex01.json solution | python3 | python3 -m
json.tool | bash test/schema.sh
```
The script `exercise.sh` in directory
[test](https://github.com/roualdes/testbank/tree/master/test) will run
all of these steps for you, and for both the exercise and solution
section of the code, by running
```
test/schema.sh examples/add.json
```
### Inserting an exercise into TestBanks's database
To insert an exercise into TestBank's database, use the command line
interface as
```
$ node cli.js insert ex.json
```
Where `ex.json` is JSON file for an exercise which provides some meta
information about the exercise to be entered. See examples in the
[examples](https://github.com/roualdes/testbank/tree/master/examples)
directory for more information.
## TODO
[] Add to README something about requesting solutions based on
evenness of seed.
[] Enable more complete/robust testing of the database and TestBank
backend.
[] Develop a database searchable landing/home page for TestBank.
[] Add HTML, online homework system-esque, example. Maybe
[bootswatch](https://bootswatch.com/)'s theme
[flatly](https://bootswatch.com/flatly/)?
[] Authorization? I'm thinking anybody can see questions and the code
that generates them. Anybody can request a solution. But it should
necessitate some form of authorization to see the code the produces
the solutions. If seeds are used appropriately, then just knowing the
answer is 0.532 for a specific seed won't help much. And if no
randomization is appropriate for a question, then no likely no
solution code is appropriate.
[] Figure out versions of dependencies.
[] Insert (possible) FAQs as exercises in the database. Should double
as helping to explain how to use TestBank and show off some of the
features, like showing/hiding solutions and ignoring the SEED.
[] GZIP databases.
[] Restart Jupyter kernel's when/if they crash.
## Development
To work on TestBank, the following dependencies are necessary. There
is an [Ansible](https://www.ansible.com/) playbook named `ansible.yml`
that can set up an Ubuntu 18 machine, if you prefer.
- [Node.js](https://nodejs.org/)
- all of the Node.js packages within the file `package.json`, which
can be installed with `npm install` from within this directory.
- [Python3](https://www.python.org/)
- [R](https://www.r-project.org/)
The following packages within each language's ecosystem
- For Python use [pip](https://pip.pypa.io/en/stable/) to obtain
- [jupyterlab](https://jupyterlab.readthedocs.io/en/stable/)
- [numpy](https://www.numpy.org/)
- [scipy](https://www.scipy.org/)
- [pandas](https://pandas.pydata.org/)
- For R use the function `install.packages()` to obtain
- [IRkernel](https://github.com/IRkernel/IRkernel)
- After installation, in R (not RStudio), within a Terminal that
has access to the Python envirnoment containing jupyterlab, run
```
library(IRkernel)
IRkernel::installspec()
```
- [dplyr](https://dplyr.tidyverse.org/)
- [tidyverse](https://ggplot2.tidyverse.org/)
- [jsonlite](https://cran.r-project.org/web/packages/jsonlite/index.html)
- [pander](https://rapporter.github.io/pander/)
And for tests one needs,
- [littler](http://dirk.eddelbuettel.com/code/littler.html)
- This can not be installed as an alias, since some scripts
necessitate `lr` being on the path.
- [bash](https://www.gnu.org/software/bash/)
- [jq](https://stedolan.github.io/jq/)
This is my best attempt at a complete list, but I'm still iffy about
versions of everything. Let me know what I've missed.
PRs welcome.
## License
License: Open source, [BSD (3-clause)](https://opensource.org/licenses/BSD-3-Clause).