Dealing with this professionally for DNNs. It just doesn't work. The large, important DNN models are so complicated, the toolchains for optimized execution don't do sane things unless you do some sort of vertical integration. The community tried with things like TVM, Halide, ONNX and others .. it is just crazy if you don't have a fully opinioned pipeline. Just my personal opinion.