Vic 'ghost town' braces for fire danger