FMS hangs periodically around backup time

[Charles Delfs]

Interesting case:

- FMS 13 on windows Server 2008

- Occasionally (apparently near same time a back up runs (not progressive) FMS goes unresponsive, unable to open files, unable to access via XML CWP and WD also.

- FMS is accessed via Clients, WebDirect and XML CWP at the same time.

- Windows logs do not show any funky activity

- I have seen

It 'Appears' when a script is running (previously scheduled) and a backup run at the same time FMS hangs.

Restarting fmsServer and AdminServer fixes the issue.

Has anyone encountered an issue like this?

Charles Delfs

[Steven H. Blackwell]

How much RAM is allocated to cache on this machine?  The symptoms you describe can be associated with insufficient RAM allocation.

 

Steven

[Wim Decorte]

same question about the # of processors and cores.

 

Are any of the FMS stats in the stats log off?  High "wait time per call", or "elapsed time per call"?

 

Is the backup schedule set to verify the backup?

 

Does the scheduled script log any errors in the FMS event log?

[Charles Delfs]

same question about the # of processors and cores.

 

Are any of the FMS stats in the stats log off?  High "wait time per call", or "elapsed time per call"?

 

Is the backup schedule set to verify the backup?

 

Does the scheduled script log any errors in the FMS event log?

Wim:

- 6 Core CPU

- Nothing 'looks' off stats wise

- Verify Backup is disabled

 

I have seen this with 2 different scripts, one script times out when it runs over time (as if it hangs) it is a simple looping counter that resets some values, and is stable on its own.

 

The other script I have seen this happen on (when backup runs) has a large amount of logging and I can see that it is not doing anything odd or complicated when it crashes. In fact I can see slight variations in timing where it crashes. I had changed the schedule time so they do not overlap but that is not a solution.

When it does hang I see a single core running at about 17% as if something got miessed and something is still running.

 

Thanks for the reply!

 

Charles

 

 

 

 

How much RAM is allocated to cache on this machine?  The symptoms you describe can be associated with insufficient RAM allocation.

 

Steven

This server has 16Gig Ram and currently 6G has been assigned for caching.

[Wim Decorte]

I've seen sub-optimal performance with cache set too high for what it needs to be.  Try scaling it back to default 800MB of FMS12 and monitor both the cache hit % and the performance.

 

When you say "crash", is it a real crash with the crash reported in the Windows system / event log?

 

Do you have the "force abort" setting enabled for the script schedule for when it goes past its allowed run-time?

[Charles Delfs]

I've seen sub-optimal performance with cache set too high for what it needs to be.  Try scaling it back to default 800MB of FMS12 and monitor both the cache hit % and the performance.

 

When you say "crash", is it a real crash with the crash reported in the Windows system / event log?

 

Do you have the "force abort" setting enabled for the script schedule for when it goes past its allowed run-time?

Thanks Wim,

The crash is more like a hang,

- The event log shows the script that was running has been forced to abort, but it appears it does not.

- CPU stays elevated as if something is happening,

- Users are unable to log in and you cannot see any DB's in the open remote dialogue box.

- FMS admin panel: Unable to disconnect servers or close any DB's

 

To recover from the issue FMS must be stopped and restarted.

[Wim Decorte]

any .DMP files in the FMS logs folder?

[Charles Delfs]

any .DMP files in the FMS logs folder?

Unfortunately not, just a .dmp from several months ago. Nothing current related to this issue.

 

CD