Cisco Cisco Process Orchestrator 3.0 User Guide
5-17
Cisco Process Orchestrator User Guide
OL-30196-01
Chapter 5 Managing High Availability and Resiliency
Handling Restarts and Failures
•
Other activities can have side effects and cannot be rerun without consequence. For example, if a
CLI adds some entry for a device to a file, running the CLI again will cause two entries for the
device. In these cases, Process Orchestrator shows these processes as failed after the restart; the
operator must examine the failures, then rerun the failed processes.
CLI adds some entry for a device to a file, running the CLI again will cause two entries for the
device. In these cases, Process Orchestrator shows these processes as failed after the restart; the
operator must examine the failures, then rerun the failed processes.
•
The Web HTTP Request has a “Restart when interrupted” checkbox. Click this option if the HTTP
Request is performing a read operation, in which case it should be restartable, but do not click it
when performing a write operation because you could accidentally create multiple entries.
Request is performing a read operation, in which case it should be restartable, but do not click it
when performing a write operation because you could accidentally create multiple entries.
Process Orchestrator offers process Start Points that process authors can use to give operators restart
points when a process needs to be restarted. If a volatile activity is run in Process Orchestrator and has
not returned a status and Process Orchestrator shuts down, when Process Orchestrator restarts, this
activity will not resume and the results of what the activity was supposed to do are lost.
points when a process needs to be restarted. If a volatile activity is run in Process Orchestrator and has
not returned a status and Process Orchestrator shuts down, when Process Orchestrator restarts, this
activity will not resume and the results of what the activity was supposed to do are lost.
Adapters determine whether to declare activities as restartable. The Process Orchestrator server also
allows activities to save their state as they run, which allows long-running activities to be restarted. For
example, a Create Approval activity might save a state that the approval request task has been created
but the activity is blocked waiting on an approval.
allows activities to save their state as they run, which allows long-running activities to be restarted. For
example, a Create Approval activity might save a state that the approval request task has been created
but the activity is blocked waiting on an approval.
Process Orchestrator's writing of state is transactional. Process Orchestrator will not proceed to the next
step until the state change is committed to the database. There should be no time gap where process state
can be lost. This persistence has a cost. With every step in every process, Process Orchestrator writes to
the database to update states such as when the activity starts, stops, etc. Where this performance hit is
inappropriate, Process Orchestrator provides a setting to turn restartability off for a process. Many
processes do not need to restart after a shutdown or failure; it might often be acceptable to rerun the
process at the next scheduled interval or on the next instance of the trigger. For example, many SAP
diagnostic processes fall into this category. In the case of SAP diagnostics, Process Orchestrator's SAP
automation pack process definitions have restartability turned off on analysis processes. If the server
fails, the state of the SAP server with respect to alerts might not be the same after restart anyway. It is
often best to not restart these processes but to instead to relearn the state of the SAP system. Also, by
disabling restartability, the performance impact of maintaining state is reduced.
step until the state change is committed to the database. There should be no time gap where process state
can be lost. This persistence has a cost. With every step in every process, Process Orchestrator writes to
the database to update states such as when the activity starts, stops, etc. Where this performance hit is
inappropriate, Process Orchestrator provides a setting to turn restartability off for a process. Many
processes do not need to restart after a shutdown or failure; it might often be acceptable to rerun the
process at the next scheduled interval or on the next instance of the trigger. For example, many SAP
diagnostic processes fall into this category. In the case of SAP diagnostics, Process Orchestrator's SAP
automation pack process definitions have restartability turned off on analysis processes. If the server
fails, the state of the SAP server with respect to alerts might not be the same after restart anyway. It is
often best to not restart these processes but to instead to relearn the state of the SAP system. Also, by
disabling restartability, the performance impact of maintaining state is reduced.
Handling Reporting Data Queues During Restarts and Failures
To improve performance, reporting data is held in queues and is occasionally written to the database in
bulk rather than an item at a time. During a friendly shutdown, the queue is flushed so that the reporting
data is fully synchronized to the reporting database. In the event of a failure, the data in the queue will
be present on restart and will be sent whenever the Process Orchestrator Server starts up; no data will be
lost. However, if an unexpected failure occurs for which there is a need to reinstall the Process
Orchestrator server on a different computer or restore the computer hosting the Process Orchestrator
server from a backup, reporting data from these queues is lost.
bulk rather than an item at a time. During a friendly shutdown, the queue is flushed so that the reporting
data is fully synchronized to the reporting database. In the event of a failure, the data in the queue will
be present on restart and will be sent whenever the Process Orchestrator Server starts up; no data will be
lost. However, if an unexpected failure occurs for which there is a need to reinstall the Process
Orchestrator server on a different computer or restore the computer hosting the Process Orchestrator
server from a backup, reporting data from these queues is lost.
Handling Scheduled Triggers During Restarts and Failures
In a Process Orchestrator high availability environment, schedules that are monitored on server A will
move to Server B when server A fails. There is a small window (of a few seconds) during which a
schedule could be missed while server B is detecting server A’s failure.
move to Server B when server A fails. There is a small window (of a few seconds) during which a
schedule could be missed while server B is detecting server A’s failure.