Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

@peidaqi Thank for the code. However, when I run your code, the inference time (prediction phase) seems not to be reduced. Do you have any idea about this ?

Below is the time logging in the prediction function:

-- Your run_classifier.py
time 1: 2019-03-04 10:28:27.399931
time 2: 2019-03-04 10:28:27.399948
time 3: 2019-03-04 10:28:37.916306

-- Original run_classsifier.py
time 1-oringial: 2019-03-04 10:33:06.684124
time 2-oringial: 2019-03-04 10:33:06.684171
time 3-oringial: 2019-03-04 10:33:17.847752

        import datetime
        print("time 1: ", datetime.datetime.now())
        result = estimator.predict(input_fn=predict_input_fn)
        print("time 2: ", datetime.datetime.now())
        output_predict_file = os.path.join(FLAGS.output_dir, "test_results.tsv")
        with tf.gfile.GFile(output_predict_file, "w") as writer:
            num_written_lines = 0
            tf.logging.info("***** Predict results *****")
            for (i, prediction) in enumerate(result):
                probabilities = prediction["probabilities"]
                if i >= num_actual_predict_examples:
                    break
                output_line = "\t".join(
                    str(class_probability)
                    for class_probability in probabilities) + "\n"
                writer.write(output_line)
                num_written_lines += 1
        assert num_written_lines == num_actual_predict_examples
        print("time 3: ", datetime.datetime.now())

@ntson2002 Not sure what you mean by not reduced, but it seems to me the time difference in your reports show the new run_classifer.py is indeed faster - thought only by a small margin.
Time 3 - Time 2:
New run_classifier - 10.516358
Original run_classifier - 11.163581

estimator.predict returns a generator so I imagine most time would be spent with the Python facilities and loops. Also your code has lots of I/O per loop that takes lots of time, compared to the one-off loading of the pre-trained model, which could make the difference insignificant.

I'd suggest testing with estimator.eval and remove all those I/O operations. After all, this is only a small fix, and if you have fast SSDs (seems to be in your case?) the performance gain would not be very big.

@googlebot I signed it!