Data retention & information governance

Industry regulations or internal policies may require you to retain data for a minimum period of time, or delete data after a set amount of time. Catalytic has built in data retention features to help you meet information governance requirements.

Retention policies

By default, Catalytic saves and makes all instance data available to you; no files, tables or other objects are deleted automatically.

There are two options for managing data deletion:

  • Use a scheduled policy: for larger data retention initiatives, apply a data retention policy per Workflow to delete past instances. Learn more about scheduled policies.
  • Use data deletion actions: If you have a process where only select instances must be deleted, such as if a user selects “Delete my data”, or where data must be deleted in only specific situations, but not for all instances, use the data deletion actions. Learn more about data deletion actions.

Delete completed run data after a set duration

You can apply a scheduled data retention policy to automatically delete old data. Data retention policies are applied per Workflow from the Workflow settings page in the Security section.

  1. Access the Workflow Settings page by selecting Workflows from the top navigation bar at any time.
  2. From there, select a Workflow, then select

    screen readers look here
  3. At the bottom, in the Security section, select
  4. Enter a number of days, then select . The policy immediately goes into effect.

    screen readers look here

After you apply the policy, completed run data is retained for a set duration, at which point it is permanently deleted. Deleted data cannot be recovered. The shortest possible policy is 1 day.

💡Tip: Throughout this article there are references to runs using terms like “scheduled for deletion” or “in progress”, for more information on these states, see Definitions of different run states. Runs may also be referred to as “instances”.

Definitions of different run states

To understand how a new policy may affect your Workflow’s runs, consider 5 following “states” a run can be in:

  • Future: A run that the Workflow starts at some point in the future.
  • In progress: The run has started, but is not finished.
  • Completed: The run has finished or completed, and may or may not be scheduled for deletion.
  • Deleted: The run has finished, and was deleted after a set duration due to a data retention policy.
  • Scheduled for deletion: The run has finished and because of an active data retention policy, is scheduled to be deleted after a set duration—but that duration has not elapsed. It is said to be “scheduled” to be deleted.

What happens when you add a new data retention policy

When applying a policy to a Workflow for the first time, the policy is applied to all currently in progress and future runs. The policy does not affect any completed runs.

When a new policy is applied:

  • In progress runs inherit the new policy and will be deleted after the set duration. See example.
  • Completed runs are not affected. See example.
  • Future runs will always have the new policy and will be deleted after the set duration.
Example: Applying a new policy, and the affect on in progress runs

In this example, a run is in progress when a new policy is applied.

screen readers look here
  1. A run starts on July 5.
  2. Before it completes, a ⚠️ 7 day data retention policy is applied to the parent Workflow. The policy applies to all in progress runs.
  3. The run completes on July 12, and is scheduled to be deleted on July 19 based on the policy.
  4. On July 19, 7 days later, the run data is deleted.
Example: Applying a new policy and the affect on completed runs

In this example, a run has already completed by the time a new policy is applied.

screen readers look here
  1. A run starts on July 5.
  2. The run completes on July 12, there is no previous data retention policy so the run is not scheduled for deletion.
  3. On July 14, after the run has completed, a ⚠️ 7 day data retention policy is applied to the parent Workflow. The policy does not apply to any completed runs, and run data is not deleted.

How a retention policy works with already completed runs

Completed runs are unaffected by data retention policies. The only way to delete previously completed run data is using the Data deletion actions. See Conditionally delete past run data for more information.

What happens when you overwrite an existing data retention policy

If a new policy replaces an existing policy, in progress runs inherit the new policy. Completed runs scheduled for deletion also inherit the new policy.

💡Tip: There is a 1 hour buffer on any changes to an active policy. If you accidentally overwrite a policy, you have 1 hour to set it back to the original policy before the new policy goes into effect.

When a new policy overwrites an existing data retention policy:

  • In progress runs inherit the new policy and will be deleted after the new set duration. See example.
  • Runs scheduled for deletion inherit the new policy. Runs that are older than a new policy will be deleted after 1 hour. See example.
  • Completed runs are not affected.
  • Future runs will always have the new policy and will be deleted after the set duration.

⚠️ Heads-up: Completed runs scheduled for deletion that are older than a new policy will be deleted 1 hour after the policy is applied. See the Overwriting an existing policy and the affect on completed runs scheduled for deletion example for more info on this.



Example: Overwriting an existing policy and the affect on in progress runs

In this example, a run is in progress when an existing policy is overwritten.

screen readers look here
  1. A ⚠️ 7 day data retention policy is applied on July 2.
  2. A run starts on July 5.
  3. Before it completes, a new ⚠️ 2 day data retention policy is applied. The new policy applies to all in progress runs and all completed runs scheduled for deletion. The run inherits the new policy.
  4. The run completes on July 12, and is scheduled to be deleted 2 days later on July 14.
Example: Overwriting an existing policy and the affect on completed runs scheduled for deletion

In this example, a run has already completed and is scheduled for deletion when the existing policy is overwritten.

screen readers look here
  1. A run starts on July 5.
  2. Before it completes, a ⚠️ 7 day data retention policy is applied to the parent Workflow. The policy applies to all in progress runs.
  3. The run completes on July 12, and is scheduled to be deleted on July 19 based on the policy.
  4. On July 17, 2 days before the scheduled deletion, a new ⚠️ 2 day data retention policy is applied. The new policy applies to all in progress runs and all completed runs scheduled for deletion.
  5. The run is deleted in 1 hour. Because it had a previous policy, it inherits the new policy. And because more than 2 days had already elapsed, it will be deleted.

How to remove a data retention policy

  1. Access the Workflow Settings page by selecting Workflows from the top navigation bar at any time.
  2. From there, select a Workflow, then select
  3. At the bottom, in the Security section, select
  4. Delete any entry, and leave the field empty. Then select . The existing policy will be overwritten.
screen readers look here

What happens when you remove an existing data retention policy

If an active policy is removed, all completed instances that are scheduled to be deleted will not be deleted.

💡Tip: There is a 1 hour buffer on any changes to an active policy. If you accidentally remove a policy, you have 1 hour to set it back to the original policy before the new policy goes into effect.

When an existing data retention policy is removed:

  • In progress runs will not be deleted upon completion.
  • Runs scheduled for deletion will not be deleted.
  • Future runs will not be deleted.
  • Deleted runs cannot be recovered and are permanently deleted.

It is recommended to only remove an active policy when necessary. If a policy is accidentally removed, runs scheduled for deletion will not be deleted. Removing a data retention policy can interfere with scheduled run deletion.

⚠️ Heads-up: Always check to make sure you comply with any relevant regulations (GDPR) before removing an active policy.

Conditionally delete past run data

If you have a process where only select instances must be deleted, such as if a user selects “Delete my data”, or where data must be deleted in only specific situations, but not for all instances, use the Data Deletion: Delete run action to delete a completed run and its data, including related tasks, files, and data tables.

Use this action as part of a GDPR or privacy compliance process to manage stored data. For larger data retention initiatives, applying a data retention policy is the easiest way to manage data and delete all past instances.

Save specific files or tables from deletion

Use the Data Deletion: Save tables or save files action to save individual objects from deletion from a scheduled data retention policy, or the Data Deletion: Delete run action.

screen readers look here

The save files and tables actions can preserve important objects that are part of the process, but are not necessary or advisable to delete.

This action must be inside the same process as the objects to be saved. It is not possible to save tables or files from past completed instances, or instances from other Workflows.

Permissions

Any user with build permissions on the Workflow can set, overwrite, or remove a data retention policy.

Get help with a problem or question

If something’s not working as expected, or you’re looking for suggestions, check through the options below.

How do I remove a data retention policy?

To remove a data retention policy, delete any entry and leave the field blank. Then select .

screen readers look here

See What happens when removing an existing data retention policy for how this affects a process.

Can I recover deleted runs?

Unfortunately, deleted runs cannot be recovered. This is strictly enforced in order to meet any stringent data retention requirements within an organization or government.

The best thing you can do is immediately remove the data retention policy.

How can I tell if my process has an active data retention policy?

Check the Workflow Settings page for any Workflow, then open the Security section. If there is any number, there is an active policy.

screen readers look here

How to access the Workflow Settings page

  1. Access the Workflow Settings page by selecting Workflows from the top navigation bar at any time.
  2. From there, select a Workflow, then select
  3. At the bottom, in the Security section, select