Hello,
This is not possible in standard SAP, you could use BAdi WORKORDER_UPDATE method AT_SAVE with your custom logic to read open production order for material if more than 3 orders are open then terminate conversion with an error message.
Best Regards,
R.Brahmankar