Hello everyone,
I have only found a few posts about this in the forums - all with almost no response, or unsatisfactory response. I trained a model using autoML in vertex AI for hundreds of $$ expecting to be able to run the model at the edge using tflite-runtime - as is advertised as the most effective method for lightweight inference.
Problem is both of my exported tflite models I created: 1 in June, 1 today are not working when trying to inference using tflite_runtime.interpreter. I get an ABORT failure. The really odd thing is efficientdet_tflite0 that I pulled from official Google github repo works with my script just fine. There seems to be something majorly wrong here that we can’t use the Google stack from beginning to end as advertised.
Has anyone solved this issue??
I am back with an update after significant troubleshooting for anyone here who is having this issue. The problem is that tflite-runtime only supports a SUBSET of operations available in tensorflow. In my specific case I have narrowed down my hypothesis to 2 sources of incompatibility.
I haven’t done detailed stack tracing to step through execution, but based on my analysis of the model architecture compared with the tflite methods available my model has tfl.pseudo_qconst2 while the docs show tfl.pseudo_qconst … not sure if this is the smoking gun, but this appears to be the only opp not available in tflite but present in my model for now…
The other issue I am trying to contend with is the model metadata not matching what is expected in tflite-runtime. My model export from GCP appears to have metadata but based on my research tflite-runtime does not support parsing metadata. I think this is more likely the culprit.
This is unfortunately spelled out at this URL:
Final update - I am giving up on tflite-runtime. 
I really wanted to use it. The promise of tiny binaries and low overhead were a mirage in the desert. I buckled and converted my model to onnx and have opted for onnx-runtime instead. I hope this is helpful for someone down the line!!
