https://github.com/gtesei/patterns_for_continuous_integration_docker_jenkins
Patterns of Continuous Integration for Data Science Python Projects
https://github.com/gtesei/patterns_for_continuous_integration_docker_jenkins
coverage coveralls docker jenkins mkdocs pypi pytest python readthedocs
Last synced: 3 months ago
JSON representation
Patterns of Continuous Integration for Data Science Python Projects
- Host: GitHub
- URL: https://github.com/gtesei/patterns_for_continuous_integration_docker_jenkins
- Owner: gtesei
- License: bsd-3-clause
- Created: 2018-10-12T22:40:13.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2018-10-19T21:15:03.000Z (over 6 years ago)
- Last Synced: 2025-01-11T13:53:16.805Z (4 months ago)
- Topics: coverage, coveralls, docker, jenkins, mkdocs, pypi, pytest, python, readthedocs
- Language: Python
- Homepage:
- Size: 460 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Patterns for Continuous Integration with Docker using Jenkins
__Full story:__ [Patterns of Continuous Integration for Data Science Python Projects](https://medium.com/@gtesei/the-ci-arsenal-that-any-open-source-machine-learning-python-project-should-consider-e34759dd66c4)
## Install Jenkins via Docker
see https://github.com/jenkinsci/docker/blob/master/README.md
### Dockerfile
```docker
FROM jenkins/jenkins:lts
USER root
RUN apt-get update && \
apt-get -y install apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
$(lsb_release -cs) \
stable" && \
apt-get update && \
apt-get -y install docker-ce
RUN apt-get install -y docker-ce
RUN usermod -a -G root jenkins
RUN usermod -a -G sudo jenkins
USER jenkins
```
### Build Docker Image
```sh
docker build -f Dockerfile_Jenkins -t my-jenkins .
```
### Start Container
```sh
docker run -p 8080:8080 -p 50000:50000 --name myjenkins -v /var/run/docker.sock:/var/run/docker.sock my-jenkins
```
### Enter in Docker Container and Execute
```sh
docker exec -it myjenkins bash
```
__Note__: use option ```-u 0``` to enter as root
```sh
git config --global http.sslverify false
```
## Using credentials with Jenkins
Necessary for
* Docker Hub account
* Coveralls account (option)
see https://jenkins.io/doc/book/using/using-credentials/
* From the Jenkins home page (i.e. the Dashboard of the Jenkins classic UI), click Credentials > System on the left.
* Under System, click the Global credentials (unrestricted) link to access this default domain.
## Creating Jenkins Pipeline for a Git Project
#### New Item >> Enter pipeline name and select Pipeline >> OK
#### Pipeline >> Pipeline Script from SCM >> Git >> Enter Git URL >> Save
## Jenkinsfile
```groovy
pipeline {
agent none
stages {
stage('install-build-test-coveralls') {
agent {
docker {
image 'python:3.6.1'
args '-u root:sudo'
}
}
environment {
COVERALLS_REPO_TOKEN = credentials('d79070bd-2e2b-4a91-ba27-e1ecd68897ad')
}
steps {
/*** INSTALL ***/
sh "python --version"
sh "pip install --upgrade pip"
sh "pip install -r requirements-dev.txt"
sh "pip install pytest pytest-cov"
sh "pip install coveralls"
sh "pip install coverage"
/*** BUILD ***/
sh "pip install -e ."
/*** TEST ***/
sh "py.test --doctest-modules --cov"
/*** COVERALLS ***/
sh "echo $COVERALLS_REPO_TOKEN"
/* sh "coverage run setup.py test" */
sh "py.test --doctest-modules --cov"
sh "coverage report -m"
sh "COVERALLS_REPO_TOKEN=$COVERALLS_REPO_TOKEN_PSW coveralls"
}
}
stage('docker') {
agent any
environment {
IMAGE_NAME = 'gtesei/hello_docker_jenkins'
DOCKER_REGISTRY = credentials('3897e886-55e9-491f-95f9-bf0280b72966')
}
steps {
sh "docker pull $IMAGE_NAME || true"
sh 'docker build --pull --cache-from "${IMAGE_NAME}:develop" --tag "$IMAGE_NAME" . || true'
sh 'docker login -u $DOCKER_REGISTRY_USR -p $DOCKER_REGISTRY_PSW'
sh 'docker tag $IMAGE_NAME "${IMAGE_NAME}:develop"'
sh 'docker push "${IMAGE_NAME}:develop"'
}
}
}
}
```