Thousands of CBD workers will start returning to their offices from Monday, after Premier Daniel Andrews gave the green light to up to 50 per cent of desks being filled.