TensorFlow binaries do not work on RHEL6 because they require GLIBC 2.17. The system version is 2.12. Therefore you need to build Tensorflow. (Having another glibc on the system is just painful.) That still requires a few tricks.

First of all create a new conda environment, using anaconda2:

conda create -n tensorflow pip python=2.7

Get hold of tensorflow from:

git clone https://github.com/tensorflow/tensorflow tensorflow
cd tensorflow
git checkout r1.5

In order to build TF, need bazel. Again, do this with Conda:

conda install -c conda-forge bazel

This will install bazel v0.8.1. That is the reason for using TF r1.5; later version rquire bazel > 0.10

cd tensorflow

And go through all the options. Said no to everything I think — building with CPU support only. My system did not have CUDA tools installed and I do not have root access.

bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

This is likely to throw up errors at some point about clock_gettime. In order to fix it do

bazel build --config=opt --linkopt=-lrt //tensorflow/tools/pip_package:build_pip_package

Also, in bazel-tensorflow/external/protobuf_archive/BUILD add -lrt to the following:

LINK_OPTS = select({
    ":android": [],
    "//conditions:default": ["-lpthread", "-lm", "-lrt"],

If you are lucky, this will build it successfully. Now build the python wheel package using:

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

And then install with

pip install /tmp/tensorflow_pkg/tensorflow-1.6.0-py2-none-any.whl

And that should be it! Test the installation in python interpreter with

import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()

The following webpage has some useful pointers for TF: