{"id":18368706,"url":"https://github.com/radanalyticsio/tensorflow-build-s2i","last_synced_at":"2025-04-06T17:31:50.075Z","repository":{"id":72027183,"uuid":"110710382","full_name":"radanalyticsio/tensorflow-build-s2i","owner":"radanalyticsio","description":"S2I image  for building tensorflow binaries","archived":false,"fork":false,"pushed_at":"2019-03-25T19:04:53.000Z","size":43589,"stargazers_count":5,"open_issues_count":1,"forks_count":14,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-22T04:02:09.550Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/radanalyticsio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-14T15:46:33.000Z","updated_at":"2020-03-08T15:53:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"ff5f1de6-1799-469d-9a9b-8a8d339cb58d","html_url":"https://github.com/radanalyticsio/tensorflow-build-s2i","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radanalyticsio%2Ftensorflow-build-s2i","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radanalyticsio%2Ftensorflow-build-s2i/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radanalyticsio%2Ftensorflow-build-s2i/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radanalyticsio%2Ftensorflow-build-s2i/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/radanalyticsio","download_url":"https://codeload.github.com/radanalyticsio/tensorflow-build-s2i/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247522415,"owners_count":20952544,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T23:27:06.920Z","updated_at":"2025-04-06T17:31:50.067Z","avatar_url":"https://github.com/radanalyticsio.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tensorflow BUILD S2I\n\n## About\n\nThis S2I respository has [template](https://docs.openshift.org/latest/dev_guide/templates.html) files used for building tensorflow wheel files for:\n* `Centos7`\n* `Fedora27`\n* `Fedora28`\n* `RHEL7.5`\n\nNOTE: for `RHEL7.5` you need a system with RHEL Subscription enabled.\n\n##### For CPU\nBuilding Tensorflow from source on Linux can give better performance.\nThis is because \n\u003e ...the default TensorFlow wheel files target the broadest range of hardware to make TensorFlow accessible to everyone.\n\u003e If you are using CPUs for training or inference, it is recommended to compile TensorFlow with all of the optimizations available for the CPU in use.\n\u003e To install the most optimized version of TensorFlow, build and install from source. \n\u003e If there is a need to build TensorFlow on a platform that has different hardware than the target, \n\u003e then cross-compile with the highest optimizations for the target platform....\n\nFor example:\n\n```--copt=-mavx --copt=-mavx2 --copt=-mavx512f --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 ```\n\nusing above options in `CUSTOM_BUILD` will build the package with optimizations for `FMA, AVX and SSE`.\n\nBuild and install from source is fraught with errors due to `gcc` compatability issues, mismatch between CUDA platform and cuDNN libraries with TensorFlow, `python` versions, `bazel` versions , unsupported optimizations on the target CPU etc.\n\nThe templates and Dockerfiles available here provide a flexible approach to create wheel files for different combinations of OS, `python` version, `bazel` version etc by specifying them as PARAMS(`--param=`) in the templates. \nThe wheel files created from these templates are available at [AICoE/tensorflow-wheels](https://github.com/AICoE/tensorflow-wheels/releases).\nAnd the instructions to use with pipfile are given here [AICoE's TensorFlow Artifacts](https://index-aicoe.a3c1.starter-us-west-1.openshiftapps.com/).\n\n##### For GPU: TODO\nGPU is not yet supported.\n\n\n## Bazel build options\n* `TF_NEED_JEMALLOC`: = 1\n* `TF_NEED_GCP`: = 0\n* `TF_NEED_VERBS`: = 0\n* `TF_NEED_HDFS`: = 0\n* `TF_ENABLE_XLA`: = 0\n* `TF_NEED_OPENCL`: = 0\n* `TF_NEED_CUDA`: = 1\n* `TF_NEED_MPI`: = 0\n* `TF_NEED_GDR`: = 0\n* `TF_NEED_S3`: = 0\n* `TF_CUDA_VERSION`: = 9.0\n* `TF_CUDA_COMPUTE_CAPABILITIES`: = 3.0,3.5,5.2,6.0,6.1,7.0\n* `TF_CUDNN_VERSION`: = 7\n* `TF_NEED_OPENCL_SYCL`:= 0\n* `TF_NEED_TENSORRT`:= 0\n* `TF_CUDA_CLANG`:= 0\n* `GCC_HOST_COMPILER_PATH`:= /usr/bin/gcc\n* `CUDA_TOOLKIT_PATH`:= /usr/lib/cuda\n* `CUDNN_INSTALL_PATH`:= /usr/lib/cuda\n* `TF_NEED_KAFKA`:=0\n* `TF_NEED_OPENCL_SYCL`:=0\n* `TF_DOWNLOAD_CLANG`:=0\n* `TF_SET_ANDROID_WORKSPACE`:=0\n* `TF_NEED_IGNITE`:=0\n* `TF_NEED_ROCM`:=0\n\nHere is the default build command used to build tensorflow. \n* `CUSTOM_BUILD`:=`bazel build --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2  --cxxopt='-D_GLIBCXX_USE_CXX11_ABI=0' --cxxopt='-D_GLIBCXX_USE_CXX11_ABI=0' --local_resources 2048,2.0,1.0 --verbose_failures //tensorflow/tools/pip_package:build_pip_package`\n\nFollowing should be left blank for a build job.\n* `TEST_LOOP`:=\n* `BUILD_OPTS`:=\n\n\n\n## Usage\n\n### To create a wheel file\n\n*set some environment values for convenience*\n```\n# valid values are 2.7,3.6,3.5\nPYTHON_VERSION=3.6\n\n# git token and repo\nexport GIT_TOKEN=\nexport GIT_RELEASE_REPO=\n```\n\n#### 1. Create the templates\n```\noc create -f tensorflow-build-image.json\noc create -f tensorflow-build-job.json\noc create -f tensorflow-build-dc.json\n```\n\n#### 2. Create Tensorflow build image\n```\noc new-app --template=tensorflow-build-image \\\n--param=APPLICATION_NAME=tf-rhel75-build-image-${PYTHON_VERSION//.} \\\n--param=S2I_IMAGE=registry.access.redhat.com/rhscl/s2i-core-rhel7  \\\n--param=DOCKER_FILE_PATH=Dockerfile.rhel75  \\\n--param=PYTHON_VERSION=$PYTHON_VERSION \\\n--param=BUILD_VERSION=2 \\\n--param=BAZEL_VERSION=0.22.0\n```\nThe above command creates a tensorflow builder image `APPLICATION_NAME:BUILD_VERSION` for specific OS.\n\nThe values for `S2I_IMAGE` are :\n- Fedora26- `registry.fedoraproject.org/f26/s2i-core`\n- Fedora27- `registry.fedoraproject.org/f27/s2i-core`\n- Fedora28- `registry.fedoraproject.org/f28/s2i-core`\n- RHEL7.5- `registry.access.redhat.com/rhscl/s2i-core-rhel7`\n- Centos7- `openshift/base-centos7`\n\nThe values for `DOCKER_FILE_PATH` are :\n- Fedora26- `Dockerfile.fedora26`\n- Fedora27- `Dockerfile.fedora27`\n- Fedora28- `Dockerfile.fedora28`\n- RHEL7.5- `Dockerfile.rhel75`\n- Centos7- `Dockerfile.centos7`\n\n\n*OR*\n\nImport the template `tensorflow-build-image.json` into your namespace from Openshift UI.\nAnd then deploy from UI with appropriate values.\n\n#### 3. Create Tensorflow wheel for CPU using the build image\n\n```\noc new-app --template=tensorflow-build-job  \\\n--param=APPLICATION_NAME=tf-rhel75-build-job-${PYTHON_VERSION//.} \\\n--param=BUILDER_IMAGESTREAM=tf-rhel75-build-image-${PYTHON_VERSION//.}:2  \\\n--param=PYTHON_VERSION=$PYTHON_VERSION  \\\n--param=BAZEL_VERSION=0.22.0 \\\n--param=GIT_RELEASE_REPO=$GIT_RELEASE_REPO  \\\n--param=GIT_TOKEN=$GIT_TOKEN\n```\nNOTE: `BUILDER_IMAGESTREAM = APPLICATION_NAME:BUILD_VERSION` from step 2.\n\n*OR*\n\nImport the template `tensorflow-build-job.json` into your namespace from Openshift UI.\nAnd then deploy from UI with appropriate values.\nTensorflow wheel files will be pushed to `$GIT_RELEASE_REPO` using the token `$GIT_TOKEN`.\n(NOTE: This will ONLY work if the oauth token has scope of \"repo\".\nYou can generate Personal API access token at https://github.com/settings/tokens. Minimal token scope is \"repo\".)\n\n### To create a DEV environment for debugging build issues :\n```\noc new-app --template=tensorflow-build-dc  \\\n--param=APPLICATION_NAME=tf-rhel75-build-dc-${PYTHON_VERSION//.} \\\n--param=BUILDER_IMAGESTREAM=tf-rhel75-build-image-${PYTHON_VERSION//.}:2  \\\n--param=PYTHON_VERSION=$PYTHON_VERSION  \\\n--param=TEST_LOOP=y\n```\nNOTE: `BUILDER_IMAGESTREAM = APPLICATION_NAME:BUILD_VERSION` from step 2. \n\nSee [Usage example](https://github.com/thoth-station/tensorflow-build-s2i/blob/master/Developing.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradanalyticsio%2Ftensorflow-build-s2i","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fradanalyticsio%2Ftensorflow-build-s2i","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradanalyticsio%2Ftensorflow-build-s2i/lists"}