Hi,
I have a backup environment that runs on windows server 2003 (Master) and Windows server 2008 (Media servers) running RMAN based backups for Linux and Unix clients.
Recently the backups started failing with "cannot connect on socket" on these clients. The backups run at night and we don' troubleshoot the issues at night. When i look at them in the morning, i see that i can access client host properties fine from the master server admin console.
I have 2 similar issues for the past month. in both of them i see a pattern. just the first set of parallel jobs from just one node(2 node oracle cluster) fails. both of the failed policies run NBU 6.5 client. I am aware that this is way old software and i need to upgrade it. I am already on the track for upgrade for next week. But I am not sure if that is the reason why it is failing.
We pay for symantec support and I have engaged symantec for more than 3 weeks now to troubleshoot this one. they came back saying that socket's backend is file descriptor and we are running out of file descriptors at the start time of backups and thats what is causing these failures. our Linux and Unix experts(I am a windows system admin) see no evidence for the file descriptors to run out.
At this point I am at nowhere to go. Anyone who might have seen the same issue or a similar one, can you help me here please.
Specifics:
Master server : Windows server 2003 eNTERPRISE EDITION NBU 7.5.0.4
Media server : Windows server 2008 enterprise edition NBU 7.5.0.4
Client 1 : Redhat 5.8 NBU 6.5
Client 2 : redhat 6.5 NBU 6.5
both clients run oracle RAC cluster