clang errors out linking lto objects
riscv64-yoe-linux-musl-ld: /tmp/lto-llvm-d497c5.o: can't link soft-float modules with double-float modules
This is something needs to be addressed in clang for riscv
as of now disable lto for rv32/rv64 when using clang
Signed-off-by: Khem Raj <raj.khem@gmail.com>