Cisco Cisco Process Orchestrator 3.0 User Guide

Page of 242
 
5-17
Cisco Process Orchestrator User Guide
OL-30196-01
Chapter 5      Managing High Availability and Resiliency
  Handling Restarts and Failures
  •
Other activities can have side effects and cannot be rerun without consequence. For example, if a 
CLI adds some entry for a device to a file, running the CLI again will cause two entries for the 
device. In these cases, Process Orchestrator shows these processes as failed after the restart; the 
operator must examine the failures, then rerun the failed processes. 
  •
The Web HTTP Request has a “Restart when interrupted” checkbox. Click this option if the HTTP 
Request is performing a read operation, in which case it should be restartable, but do not click it 
when performing a write operation because you could accidentally create multiple entries.
Process Orchestrator offers process Start Points that process authors can use to give operators restart 
points when a process needs to be restarted. If a volatile activity is run in Process Orchestrator and has 
not returned a status and Process Orchestrator shuts down, when Process Orchestrator restarts, this 
activity will not resume and the results of what the activity was supposed to do are lost.
Adapters determine whether to declare activities as restartable. The Process Orchestrator server also 
allows activities to save their state as they run, which allows long-running activities to be restarted. For 
example, a Create Approval activity might save a state that the approval request task has been created 
but the activity is blocked waiting on an approval. 
Process Orchestrator's writing of state is transactional. Process Orchestrator will not proceed to the next 
step until the state change is committed to the database. There should be no time gap where process state 
can be lost. This persistence has a cost. With every step in every process, Process Orchestrator writes to 
the database to update states such as when the activity starts, stops, etc. Where this performance hit is 
inappropriate, Process Orchestrator provides a setting to turn restartability off for a process. Many 
processes do not need to restart after a shutdown or failure; it might often be acceptable to rerun the 
process at the next scheduled interval or on the next instance of the trigger. For example, many SAP 
diagnostic processes fall into this category. In the case of SAP diagnostics, Process Orchestrator's SAP 
automation pack process definitions have restartability turned off on analysis processes. If the server 
fails, the state of the SAP server with respect to alerts might not be the same after restart anyway. It is 
often best to not restart these processes but to instead to relearn the state of the SAP system. Also, by 
disabling restartability, the performance impact of maintaining state is reduced.
Handling Reporting Data Queues During Restarts and Failures
To improve performance, reporting data is held in queues and is occasionally written to the database in 
bulk rather than an item at a time. During a friendly shutdown, the queue is flushed so that the reporting 
data is fully synchronized to the reporting database. In the event of a failure, the data in the queue will 
be present on restart and will be sent whenever the Process Orchestrator Server starts up; no data will be 
lost. However, if an unexpected failure occurs for which there is a need to reinstall the Process 
Orchestrator server on a different computer or restore the computer hosting the Process Orchestrator 
server from a backup, reporting data from these queues is lost. 
Handling Scheduled Triggers During Restarts and Failures
In a Process Orchestrator high availability environment, schedules that are monitored on server A will 
move to Server B when server A fails. There is a small window (of a few seconds) during which a 
schedule could be missed while server B is detecting server A’s failure.