https://github.com/csev/sakai-scripts

A Repo of my scripts that I use to make my life as a Sakai developer easier
https://github.com/csev/sakai-scripts

Last synced: 6 days ago
JSON representation

A Repo of my scripts that I use to make my life as a Sakai developer easier

• Host: GitHub
• URL: https://github.com/csev/sakai-scripts
• Owner: csev
• Created: 2014-12-30T16:09:28.000Z (almost 10 years ago)
• Default Branch: master
• Last Pushed: 2024-10-17T23:31:56.000Z (26 days ago)
• Last Synced: 2024-10-20T10:51:00.814Z (24 days ago)
• Language: Shell
• Size: 4.42 MB
• Stars: 16
• Watchers: 7
• Forks: 35
• Open Issues: 2
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
Sakai-Scripts

=============

Some scripts that I use to make my life as a Sakai developer easier.

This is a set of scripts to set up a developer instance of Sakai on

your system or set up a nighly build.  Sakai can run on a 2GB RAM system

but is a lot more comfortable with a 4GB or more app server.

The outlie of this document is that you jump to the setup for Docker,

Ubuntu, Mac, or WSL and do some pre-steps and then continue with the

common steps, below.

Getting started on ubuntu in Docker

======================================

A quick way to test all this if you have this is to just make a ubuntu in your docker:

    docker run -it -p 8080:8080 ubuntu

You might want to give your docker images 4GB - ubuntu + Sakai does not fit into 2GB.

Getting Started on Ubuntu Linux

===============================

* Install git using the command

        apt-get update

        apt-get -y install git curl unzip zip vim

* Check out sakai-scripts into some folder and then install the rest of the pre-reqs

        cd

        git clone https://github.com/csev/sakai-scripts

        cd sakai-scripts

        bash ubuntu.sh

        source ~/.sdkman/bin/sdkman-init.sh

* If this is a fresh setup and you have no database you can do the following:

        apt-get -y install mariadb-server

        apt-get -y install mariadb-client

        service mysql start

Continue with the common steps below.

Getting Started on Mac (as needed)

==================================

* Install git using the command

        xcode-select --install

* Make sure you have Java-11 JDK (not JRE) Installed 

        java -version

        javac -version

* If you need a JDK-11 is is easiest to use using https://sdkman.io/

    

        curl -s "https://get.sdkman.io" | bash

        source ~/.sdkman/bin/sdkman-init.sh

        sdk install java 11.0.16-tem

    

* Make sure you have Maven (mvn) 3.8 or later installed. If you have Homebrew installed, you can use

        sdk install maven

        source ~/.sdkman/bin/sdkman-init.sh

 

* Check out sakai-scripts into some folder.   I tend to make a folder

named "dev" in my home directory:

        cd

        mkdir dev  (if needed)

        cd dev

        git clone https://github.com/csev/sakai-scripts

* If you need a database, install MAMP or something else if you have not already done so

Continue working with the Common steps below.

Getting Started with WSL for Sakai Developers

=============================================

Install WSL and Ubuntu.  There are lots of good sources on the Internet

on how to do this but there are some sticking points where you might get

stuck:

* Make sure Virtualization is enabled in your BIOS - it usually is off to

be more secure - but if it is not on WSL2 will break with lots of cryptic

messages - just boot into the BIOS and enable it.

* Go into Start -> Control Panel -> Programs -> Programs and Features and enable `Virtual

Machine Platform`, `Hyber-V` (Win11), and `Windows Subsystem for Linux` - The WSL option

might only show up after you install WSL.

You can install WSL and Ubuntu from the Windows Application Store or

using the `wsl` command in the command line (Start -> cmd).  

    wsl --install

You may need to restart your system, then do Start -> WSL - the first time

after install you will need to make a Linux User and Password.  Don't forget these.

Once you can log into WSL and see a prompt like:

    csev@WSLSakaiger:~$

You are in.  We are going to install Tsugi in order to get Apache, PHP, MySQL,

and PHPMyAdmin installed into your WSL.

    sudo bash

    cd /root

    git clone https://github.com/tsugiproject/tsugi-build.git

    cd tsugi-build

    bash ubuntu/build-dev.sh

Once the software has been installed you need to configure the installation.  This will be

a localhost only setup with no access to port 3306 from the Internet.

    cd /root

    cp tsugi-build/ubuntu/ubuntu-env-dev.sh ubuntu-env.sh

    source ubuntu-env.sh

    bash /usr/local/bin/tsugi-dev-configure.sh return

    

Then in a Windows browser navigate to:

    http://localhost/phpMyAdmin

Login as `root` / `root` And verify there is a `tsugi` database.  You can use this to do

MySQL maintenance and queries - because the port is shared between WSL and Windows, this

can be done from Windows.

Navigate to 

    http://localhost/tsugi/admin

The admin unlock is `tsugi` - you should see the Tsugi Administration Console.  This verifies

the PHP, MySQL, Tsugi, PHPMyAdmin, and Apache installs are working and accessible from Windows.

Our next step is to install the Sakai scripts and dependencies.

    sudo bash

    cd /root

    git clone https://github.com/csev/sakai-scripts

    cd sakai-scripts

    bash ubuntu.sh

    source ~/.sdkman/bin/sdkman-init.sh

Verify that Java is installed with:

    java --version

It should respond with something like:

    openjdk 11.0.12 2021-07-20

Continue with the common steps below.  Note we are doing all the steps

below in WSL as root.

Common Steps For Ubuntu, WSL, and Mac

=====================================

You can either run your own fork of Sakai, if you want to make PR's or run your

own branches.   But if you are just getting started you can just checkout the Sakai

repo and run it - skip straight to "Compiling and Running Sakai".

Using Your Fork

---------------

Skip this section if you just want to checkout the main Sakai repository and

get it running.

* If you want to run your own fork, create an account on github 

and go to https://github.com/sakaiproject/sakai and "fork" a copy into

your github account

* If you want to push commits to your fork, set up git with the folowing commands

        git config --global user.name "John Doe"

        git config --global user.email [email protected]

* Open a teminal and navigate to the sakai-scripts directory and:

        cp config-dist.sh config.sh

* Edit config.sh with a text editor of your choice

    * Start at "Settings start here"

    * Change "sakaiproject" in the GIT REPO  variable to be your github account if you want to run your fork

    * If you are not running MAMP or MariaDB, edit the MySQL root password, urls, ports, etc

Compiling and Running Sakai

---------------------------

(If you are in WSL make sure to be runnning as root with an `sudo bash`)

* Make a copy of the ditributed `sakai.properties` and edit with the editor of your choice.

    cp sakai-dist.properties sakai.properties

* If you don't already have a database set up, make sure your MariaDB, MAMP or XAMPP is running

and run `bash db.sh` to create a database. This script lists the databases so you should

see `sakai23` as one of the databases if things work.  **Note**: If you run

`bash db.sh`  more than once it will drop and re-create the database.

        Active MySQL Databases:

        Database

        information_schema

        mysql

        performance_schema

        sakai23

        You will need add / update these lines in your sakai.properties

        [email protected].....

        [email protected].....

        [email protected]....

At the end of `db.sh` it prints out three lines that need to insert into `sakai.properties`.

Make sure to edit your copy of `sakai.properties` and edit/update the database strings before creating the Tomcat

* Run `bash na.sh` to set up the Tomcat

* Run `bash co.sh` to check the Sakai source code out (which repo is checked out

is in `config.sh`)

* Run `bash qmv.sh` to compile it all - the first time you do this it will take

a long time and use a lot of network bandwidth as it downloads a bunch of

library code in maven.   The first full compile might take 20+ minutes - later

full compiles will be about 2-3 minutes and partial compiles are 30 seconds or

less.  If the compile fails due to a download - just run `bash qmv.sh` again

* Run `bash start.sh` to start the Tomcat

* Run `bash tail.sh` to watch the logs (Press CTRL-C to stop the tail).  Watch for tracebacks as the `tail`

goes by.   You can always take a look at `apache-tomcat-*version*/logs/catalina.out` if the tail output

goes by too quickly.  One the first start up on a fresh database, you might want to take a look at the

tables in the `sakai23` database to see Sakai add all the tables on the first start.

    Sakai is up when you see a message in the log that looks as follows:

        ... AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]

* Navigate to http://localhost:8080/portal

* To shut Sakai down, in the `tail.sh` terminal window, press `CTRL-C` and run bash `stop.sh` to shutdown Tomcat

* Copy `smv.sh` into a folder in your PATH and set execute permission

so you can recompile any sub-folder in Sakai that has a pom.xml

by typing "smv.sh".  The `smv.sh` command simply looks through your

path to find the correct value for `maven.tomcat.home` and sets up some

JAVA environment variables.

Description of the Scripts

==========================

Here are the scripts to be run in roughly the following order:

ubuntu.sh

---------

This runs a bunch of apt-get commands to load the pre-requisites for Sakai

for Ubuntu.  If you are running something other than Ubuntu, you can look

at this and use whatever package manager that is on the system to

install the necessary pre-requisites.  

config-dist.sh

--------------

This is a configuration script that sets a few parameters.  If you want/

need to tweak the parameters simply copy this to config.sh so git does

not try to check it in.  If there is a config.sh, it will be used instead

of config-dist.sh.

Generally you need to change the git repo to be checked out and the

root mysql password for non-MAMP MySQL connections.

But there is a lot of flexibility here if you are setting up nightly build

servers.   Changing things like the HTTP port, shutdown port, and location

of the Tomcat logs in config.sh allows you check these scripts out

into several directories and make different nightly builds.  If you get

really tricky, you can override sakai-dist.properties by copying it to

sakai.properties (which git will ignore) and change lots of things like

switching from MySql to Oracle.  

Note that db.sh does not understand oracle so if you are running oracle,

you might want to change MYSQL\_COMMAND to be "cat" so it does not try

to run mysql to create an Oracle database.

db.sh

-----

This sets up a MySql database with an account and password based on

the configuration

co.sh

-----

Checkout all of the Sakai source into the folder trunk.   This checks out

the main Sakai source and a few projects that Sakai depends on in case you

want to peruse the source of those projects.

db.sh

-----

Check to see if a database is there and create it if necessary.

na.sh

-----

Download and create a fresh instance of Tomcat, patch configurations,

and set up a sakai.properties

qmv.sh

------

Compile all of the Sakai source bypassing unit tests.   This saves a lot

of time in the initial compile.   The mv.sh will compile with unit test.

Once things work and you need to go out for an errand, you can run a mv.sh

to compile with all unit tests.

start.sh

--------

Start tomcat.  Actually this script calls stop.sh at the begining to

shut down any running tomcat on the configured port so it can aso be

used as a "restart" to bounce Tomcat.

stop.sh

-------

Stop tomcat

tail.sh

-------

Tail the tomcat log.

Using Sakai

===========

And at this point, you should be able to navigate a browser to:

    http://localhost:8080/portal

If you need to recompile, the steps to shut down Sakai are to first

abort your tail command with a CTRL-C and then run stop.sh

Usually to keep things clean, I remove all the files in the Tomcat logs

folder before I restart Sakai.

Developer Re-Compiles

=====================

smv.sh

------

This is for developers working on one or more of the elements of

Sakai.  This can be run deep in the code tree at any maven node

that can compile (i.e. has a pom.xml).   You can actually put this in your

~/bin path and then just type ./smv.sh to do a re-compile into

your Tomcat.

A surprising amount of Sakai development can be done without

rebooting your Sakai.  If you are working on tool code, generally you can

recompile without a reboot.  Changes to APIs in shared or components/services

generally require a Tomcat bounce.

Setting up Nightly Builds

=========================

There is a lot of flexibility inherent in creating config.sh and/or

sakai.properties that lets you set up a bunch of nightlies running

on the same server and then having crontab entries to run things

at whatever period(s) you desire.  You can set up different databases,

different TCP ports, log to some Apache web space, even use Oracle

instead of MySql with some effort.  The script nightly.sh is pretty

much ready to handle the job once the configuration is in place.

You can test all the scripts separately as you are configuring things

and then nightly.sh knits them together.

nightly.sh

----------

This script just calls the other scripts in the right order.  The script

does a checkout, shuts down any tomcat (killing if necessary), creates a new

empty database, creates a fresh tomcat, checks out the code you want to compile,

compiles the code and starts Tomcat.   Since the script stops and starts the

right Tomcat automatically, you can run this script over and over interactively

or in a cron job.